librefetch(8) -- downloads or creates a liberated source tarball ================================================================ ## SYNOPSIS `librefetch` [] []
`librefetch` `-`[`g`|`S`|`M`|`h`] ## DESCRIPTION `librefetch` is a program to streamline creation of custom source tarballs for `PKGBUILD(5)` files. If a URL mentioned in the `source` array in a `PKGUILD` is in a location that Parabola uploads "custom" source tarballs to (or configured locations), and no file is at that URL, librefetch will automatically create it for you. This works because a post-install script for the package configures `librefetch` as the download agent for `https://` URLs in `makepkg.conf`; allowing it to jump in and create a file if need be. Because of this, it is almost never necessary to call `librefetch` manually. The post-install script also configures `librefetch` as the download agent for `libre://` URLs, for compatability with PKGBUILDs that used a previous version of librefetch. There are 5 modes: * `download`: Download the tarball from the configured mirror. * `create`: Create the tarball from a `PKGBUILD`/`SRCBUILD`. * `checksums`: Generate integrity checks for source files. * `srcbuild`: Print the effective build script. * `makepkg`: Print the effective makepkg script. * `help`: Print `librefetch` usage information. The normal mode of operation is `download` mode. If `download` mode fails, it may choose to try `create` mode. ## OPTIONS * `-C`: Force `create` mode (don't download) * `-D`: Force `download` mode (don't create) * `-p` : Use an alternate build script for `create` mode (instead of `PKGBUILD`). If an `SRCBUILD` file exists in the same directory, it is used instead. * `-g` | `--geninteg`: Use `checksums` mode: Generate integrity checks for source files. * `-S` | `--srcbuild`: Use `srcbuild` mode: print the effective build script. * `-M` | `--makepkg`: Use `makepkg` mode: print the effective makepkg script. * `-h` | `--help`: Use `help` mode: Show useage information. Other options, if they are documented in `makepkg -h`, are passed to the modified copy of makepkg created during `create` mode. ## DOWNLOAD MODE If begins with the string `libre://`, it is replaced with the first value in `MIRRORS`, as configured in `librefetch.conf(5)`; this is for compatability with PKGBUILDs that used a previous version of librefetch. It uses `DOWNLOADER`, as configured in `librefetch.conf` to attempt to download the source tarball from that URL. If that fails, and following conditions are met, it proceeds to `create` mode: * The `-D` flag has not been specified to inhibit `create` mode. * The `` begins with one of the values in `MIRRORS`. The latter requirement allows librefetch to be used as a generic HTTP(S) download agent, that can automatically create files from whitelisted locations. ## CREATE MODE The principle of `create` mode is that a special `PKGBUILD(5)`, called a `SRCBUILD(5)`, installs source files to `$pkgdir`, and the resulting "package" is then used as a source tarball. The `SRCBUILD` exists in the same directory as the `PKGBUILD`. It can be created manually, or generated on-the-fly from the `PKGBUILD`. Extra steps are taken to ensure that as long as the same directory contents go in, an identical tarball will come out--the checksum of the file should not change based on when it is built or who builds it. The `SRCBUILD` is either created, or sanitized if it already exists, then fed to a modified version of `makepkg(8)`. The purpose of the modified `makepkg` is so that the resulting tarball doesn't contain package metadata, doesn't end with a .pkg file extension, and always produces an identicle tarball. When this documentation speaks of a file being modified, it is a temporary copy of the file that is modified, your original file will remain intact. ## SRCBUILD GENERATION As explained in the `CREATE MODE` section, in `create` mode, this program generates an `SRCBUILD` file. For debugging purposes, this file can be printed instead of executed with `srcbuild` mode. ### PRE-EXISTING SRCBUILD The use of `SRCBUILD` files pre-dates `librefetch`. By convention, they set `PKGDEST` and `PKGEXT` in `package()` in order to modify the behavior of `makepkg`. Because a modified version of `makepkg` is used, this interferes with the correct behavior. To compensate for this, lines containing "`PKGDEST=`" or "`PKGEXT=`" are deleted from the `SRCBUILD`. The general idea is that `build()` makes any modifications to the source, then `package()` copies it from `$srcdir` to `$pkgdir`. ### SRCBUILD FROM PKGBUILD Possibly more elegant than having a separate `SRCBUILD` file is having an `mksource()` function in the main `PKGBUILD`. This results in less boilerplate and fewer files to edit. Note that this only happens if a file named `SRCBUILD` doesn't already exist; when migrating a package from a manually created `SRCBUILD` to this method, the `SRCBUILD` must be deleted (or renamed) for this to work. The dynamically created `SRCBUILD` is created by copying `PKGBUILD` to a temorary file, then re-setting variables and re-defining functions. Following is a table of the translations. Variables source = mksource noextract = mknoextract *sums = mk*sums (md5, sha1, sha256, sha384, sha512) depends = checkdepends = makedepends = mkdepends Functions prepare() { :; } build() { mksource; } check() { :; } package() { cp -a "$srcdir"/*/ "$pkgdir/"; } The `mksource()` function does not need to be defined. If it isn't defined, then no transformations will be made to the source between it being extracted to `$srcdir` and copied to `$pkgdir`. In summary: * Set `mksource=()` and `mkmd5sums=(`) to act as `source=(`) and `md5sums=()` * Declare a `mksource()` function to make modifications to the source, if nescessary. Other changes: * `pkgname` is set to `pkgbase`, or the first element of the `pkgname` array. * `options=()` is set have `makepkg` avoid making changes to `$pkgdir`. The exact change is: options=(!strip docs libtool staticlibs emptydirs !zipman purge !upx) * `PURGE_TARGETS=()` has vcs directories added to it: PURGE_TARGETS=(.bzr/ .cvs/ .git/ .hg/ .svn/ .makepkg/) ### MAKEPKG MODIFICATIONS The following modifications are made to makepkg: * Allow us to manipulate the output file (`$pkg_file`) * Do not include metadata in the output file (`${comp_files[@]}`) * Force 'ustar' tar format, don't allow it to upgrade to 'pax' to store extended file attributes. * Don't symlink the resulting file into the current directory. * `PURGE_TARGETS` interprets an item as a directory if it ends with a slash ("/"). * Timestamps in `$pkgdir` are reset to "1990-01-01 0:0:0 +0", so that the resulting tarball will be the same, regardless of when it was created. * Sort the files included in the tarball; normally the order of files in a tarball is essentially random (even if it tends to be the same when re-created on the same machine). * append `-libre` to `$srcdir` * append `-libre` to `$pkgbasedir` (which becomes `$pkgdir`) * Don't check if the package has already been built. For debugging purposes, this modified makepkg can be printed instead of executed with `makepkg` mode. Before it is run in create mode, `PKGEXT`, `PKGDEST`, and `pkg_file` are set as environment variables. ## CONFIGURATION See `librefetch.conf(5)` for details on configuring librefetch using the `librefetch.conf` file. ## SEE ALSO librefetch.conf(5), makepkg(8), PKGBUILD(5), SRCBUILD(5)