path: root/doc
diff options
authorNicolás Reynolds <>2012-04-08 12:01:26 -0300
committerNicolás Reynolds <>2012-04-08 12:01:26 -0300
commit7db00e8e126f890cca5f742238ef438b7d418d28 (patch)
treeb38944b09a2d3f17651c5fe9648328abe02f1604 /doc
parentebd8317901120abc9deab59532172074aa57d770 (diff)
treepkg documentation
Diffstat (limited to 'doc')
1 files changed, 126 insertions, 0 deletions
diff --git a/doc/treepkg.markdown b/doc/treepkg.markdown
new file mode 100644
index 0000000..2808599
--- /dev/null
+++ b/doc/treepkg.markdown
@@ -0,0 +1,126 @@
+# TreePKG
+`treepkg` is a tool for recursively building packages from an ABS tree. It has
+been tested while building packages for the mips64el port and it has proven to
+be useful.
+It was written having in mind the experience of `fullpkg`, which converted to
+`fullpkg-ng` and then splitted into `fullpkg-build` and `fullpkg-find`. It's
+aim is to simplify algorithms implemented on the fullpkg family, while solving
+some design issues that made fullpkg miss some packages sometimes.
+## Requirements
+`treepkg` needs the help of `toru-path` for "indexing" an ABS tree. `toru-path`
+stores a plain text database of "pkgname:path" pairs, where pkgname is replaced
+by the "pkgbase", "pkgname", and "provides" fields of a PKGBUILD, followed by
+the path of the current PKGBUILD.
+This information is then used by `treepkg` to know where to find the PKGBUILD
+of a package. The fullpkg family needed to guess this by traversing the full
+ABS tree, and therefore it was unable to find pkgnames that differ from
+So, to use `treepkg` you need to run `toru-path` after the ABS tree update.
+> Currently `toru-path` doesn't remove duplicated or updated pairs, but it
+> picks the last ones and only processes new PKGBUILDs. This means `toru-path`
+> works correctly but it's database will grow up slowly.
+## How does it work
+`treepkg` must be run from the path of the PKGBUILD you're going to build (this
+may change over time). This will be DEPTH=0 and it will create a temporary
+directory to work on in the format /tmp/pkgbase-treepkg-random-string. Inside
+this directory, it will copy the needed PKGBUILDs prefixed with their depth
+number. The first package will always be copied to 000\_pkgbase.
+From then on, treepkg sources the PKGBUILD and runs itself over all pkgnames on
+the depends and makedepends array, only if it detects their versions aren't
+already built or deprecated by newer ones, using the `is_built` utility.
+While processing this info, it will increase the depth of the packages. It'll
+also write a CSV file with the knowledge it acquires from the ABS tree (useful
+for debugging). This file is called BUILDORDER.
+When this process ends (no more needed dependencies are found), the temporary
+work dir is traversed in inverse order (from higher depths to 0) running the
+FULLBUILDCMD from libretools.conf and then the HOOKLOCALRELEASE variable, which
+*must* provide a way to `repo-add` the packages to an available repository, so
+the next build will find and install the newer version using pacman.
+For instance, having this as the first pacman repository (on /etc/pacman.conf):
+ [stage3]
+ Server = /var/cache/pacman/pkg
+And this on /etc/makepkg.conf:
+ PKGDEST=/var/cache/pacman/pkg
+Your HOOKLOCALRELEASE script should look like this:
+ # Get needed vars
+ source /etc/makepkg.conf
+ source /etc/libretools.conf
+ source PKGBUILD
+ unset build package check
+ fullver=$(full_version ${epoch:-0} ${pkgver} ${pkgrel})
+ pkgs=()
+ # Generate canonical package paths
+ msg "Adding packages to [stage3]..."
+ for name in ${pkgname[@]}; do
+ msg2 "${name} ${fullver}"
+ pkgs+=("${PKGDEST}/${name}-${fullver}-*.pkg.tar.*")
+ done
+ # Add the packages to a local
+ repo-add ${PKGDEST}/stage3.db.tar.gz ${pkgs[@]}
+ # Stage the packages for later releasing
+ librestage $1
+> Note the first HOOKLOCALRELEASE argument is the remote repository name (core,
+> extra, etc.)
+There's a special case when a dependency depends on another that was put on
+a depth level lower than itself. In this case the build order will be wrongly
+assumed and you may end up with broken packages.
+To explain it with an example:
+ ghostscript (0) - fontconfig (1)
+ \ cups (1) - fontconfig (ignored)
+The second time fontconfig appears, it will be ignored. In this case cups will
+build against an fontconfig version that will be outdated by the fontconfig
+version at depth 1. In this cases, `treepkg` will detect it cached the
+dependency on a lower depth, and will "bury" it to a depth higher than the
+current one. Thus this will become the build path:
+ ghostscript (0) - fontconfig (buried)
+ \ cups (1) - fontconfig (2)
+## Tips
+`treepkg` accepts two arguments: 1) the temporary work dir and 2) the next
+depth level it should use (if current equals 0, it'll pass 1). You don't need
+to pass this arguments when running it manually, they're used internally to
+automatically construct the build path.
+But if a build failed, `treepkg` will cancel itself immediately informing you
+where the leftovers files where left. If you pass this path to `treepkg` as the
+first argument, it will resume the build, skipping to the last package being
+You can take the opportunity given by this to modify the build path or the
+PKGBUILDs, without having to re-run `treepkg` twice. For instance you can
+remove a package from the build order, or move it manually, or update the
+PKGBUILD that made `treepkg` fail in the first place.
+You don't probably want to mess with the second argument though.