diff options
Diffstat (limited to 'HACKING')
| -rw-r--r-- | HACKING/chroot-tools.md | 3 | ||||
| -rw-r--r-- | HACKING/librefetch.md | 25 | ||||
| -rw-r--r-- | HACKING/libs.md | 15 | ||||
| -rw-r--r-- | HACKING/licensing.md | 8 | ||||
| -rw-r--r-- | HACKING/mksource.md | 92 | ||||
| -rw-r--r-- | HACKING/testing.md | 23 | ||||
| -rw-r--r-- | HACKING/translations.md | 108 |
7 files changed, 255 insertions, 19 deletions
diff --git a/HACKING/chroot-tools.md b/HACKING/chroot-tools.md new file mode 100644 index 0000000..e50aa11 --- /dev/null +++ b/HACKING/chroot-tools.md @@ -0,0 +1,3 @@ +Unfortunately, `makechrootpkg.sh` is GPLv2 ONLY. This means that +everything that uses it must be held to GPLv2+ instead of GPLv3+. I'm +calling this anything that gets loaded into the same process as it. diff --git a/HACKING/librefetch.md b/HACKING/librefetch.md new file mode 100644 index 0000000..f4ac4f1 --- /dev/null +++ b/HACKING/librefetch.md @@ -0,0 +1,25 @@ +`librefetch` respects `SOURCE_DATE_EPOCH`[1] for reproducible builds, +and has had this behavior from the start (though the variable name +wasn't there from the start, that came later). But how can this be!? +If you asked h01ger (the guy heading up reproducible builds in Debian +and Fedora), tar needs to support `--clamp-mtime` to do this, and that +flag wasn't in any tar implementation when librefetch came into +existence. The flag not yet being in upstream GNU tar was for a long +time the biggest blocker to most of upstream Debian being +reproducible! + +While it is a nice flag that I'm surprised wasn't added years ago, +it's trivial to just adjust the input files first: + + find . -exec touch --no-dereference --date=DATE -- {} +` + +Of course, `--clamp-mtime`/`--mtime` is now in upstream GNU tar. So +librefetch supporting this is no longer impressive. But, librefetch +still isn't using `--mtime`. Why? Well, because it uses libarchive +`bsdtar`, not GNU tar, and bsdtar still doesn't have the flag. + +I wrote this document mostly because I expect that sometime in the +future I'll think to myself "I should simplify the code and just use +`--mtime`", not remembering the differences between tars. + +[1]: https://reproducible-builds.org/specs/source-date-epoch/ diff --git a/HACKING/libs.md b/HACKING/libs.md new file mode 100644 index 0000000..8bebaf6 --- /dev/null +++ b/HACKING/libs.md @@ -0,0 +1,15 @@ +Special stuff about hacking ih the /src/lib directory: + + - Everything should be GPLv2 AND GPLv3 compatible. No GPLv3 only. + - Name a file `libre${NAME}` if it should be executable directly, or + `${name}.sh` if it should only be available to be sourced. + - When printing a message that is internal to /src/lib, and not part + of the programm calling the library; prefix the print command with + `_l`. `_l()` is defined in `common.sh` (and `librelib`, since it + cannot use any libraries itself). + - When changing the message functions, be aware that some are + duplicated in: + * /src/chroot-tools/chcleanup + * /src/chroot-tools/distcc-tool + * /src/lib/librelib + And that they probably need to be updated as well. diff --git a/HACKING/licensing.md b/HACKING/licensing.md index 6d17e31..ec83689 100644 --- a/HACKING/licensing.md +++ b/HACKING/licensing.md @@ -6,14 +6,12 @@ New code should (please) be licensed GPLv2+. I say v2 instead of v3 because some code from Arch is GPLv2 (no "or any later version"), and having to worry about which programs can be combined is a huge pain. -Copyright statements should look like +Copyright statements for most code, should be of the form: # Copyright (C) YEARS NAME <EMAIL> -for most code, for 3rd-party code that has been imported, indent it a -bit: +For 3rd-party code, which has been imported, indent it a bit: # Copyright (C) YEARS NAME <EMAIL> -Always put a line with `# License:` specifying the license of that -file. +Always put a `# License:` line specifying the license of that file. diff --git a/HACKING/mksource.md b/HACKING/mksource.md new file mode 100644 index 0000000..01d759d --- /dev/null +++ b/HACKING/mksource.md @@ -0,0 +1,92 @@ +This is the internal libremakepkg program flow for a PKGBUILD with mksource(). + +<pre> +Given: + * a PKGBUILD with a mksource() function and a repo.parabola.nu URL in source() + * /etc/makepkg.d/librefetch.conf is sourced by makepkg.conf + * the checksum has be generated with: `librefetch -g`, and added to mksha256sums() + * mk*sums() is populated with a checksum for each mksource() (use: `librefetch -g`) + * *sums() is populated with 'SKIP' for each source() (add it before publishing the PKGBUILD) + +/usr/bin/libremakepkg + -> msg 'Starting pre-build activities...' + -> msg 'Downloading sources...' + -> /usr/lib/libretools/chroot/makechrootpkg.sh::download_sources() + +/usr/lib/libretools/chroot/makechrootpkg.sh::download_sources() + -> /usr/bin/makepkg --allsource (w/ special ENV and in-chroot makepkg.conf) + +/usr/bin/makepkg --allsource ($SOURCEONLY == 2): + -> msg "Making package:" + -> /usr/share/makepkg/source.sh::download_sources('allarch') + +/usr/share/makepkg/source.sh::download_sources(): + -> msg "Retrieving sources..." + -> (/usr/bin/librefetch is download agent) + +/usr/bin/librefetch::doit() + -> 'download' mode ($src == PKGBUILD::source() , $inmirror == true) + -> if found + -> return to /usr/bin/makepkg + -> if not found + -> 'create' mode + -> /tmp/<SPECIAL>/makepkg -p /tmp/<SPECIAL>/PKGBUILD + +/tmp/<SPECIAL>/makepkg (SOURCEONLY == 0): + -> msg "Making source:" + -> msg "Checking source dependencies..." + -> /tmp/<SPECIAL>/source.sh::download_sources('allarch') + +/tmp/<SPECIAL>/source.sh::download_sources(): + -> msg "Retrieving sources..." + -> (/usr/bin/librefetch is download agent) + +/usr/bin/librefetch::doit() + -> 'download' mode ($src == PKGBUILD::mksource() , $inmirror == false) + -> return to /tmp/<SPECIAL>/makepkg + +/tmp/<SPECIAL>/makepkg (post-return from /usr/bin/librefetch): + -> NOTE: all standard PKGBUILD functions are stubbed out, + with the only purpose to run the real PKGBUILD::mksource() + the standard functions in the real PKGBUILD will run later in the chroot + -> msg "Validating source files" + -> msg "Extracting sources..." + -> msg "Starting prepare()" + -> msg "Starting build()" + -> msg "Starting mksource()" + -> PKGBUILD::mksource() + -> msg "Starting check()" + -> msg "Starting package()" + -> msg "Creating package" + -> msg "Finished making:" + +NOTE: here we rewind back to /usr/lib/libretools/chroot/makechrootpkg.sh::download_sources() + to get/create the source-ball signature + +/usr/bin/librefetch::doit() + -> 'download' mode + -> if not found + -> 'create' mode ($src == PKGBUILD::source() , $inmirror == true) + -> /usr/bin/librefetch::create_signature() + +/usr/bin/librefetch::create_signature() + -> msg "Signing package..." + -> /usr/share/makepkg/integrity.sh::check_source_integrity() + -> /usr/share/makepkg/integrity/verify_checksum.sh + -> msg "Validating source files" + -> /usr/share/makepkg/integrity/verify_signature.sh + -> msg "Verifying source file signatures with gpg" + +NOTE: here we rewind back to /tmp/<SPECIAL>/makepkg + +/tmp/<SPECIAL>/makepkg: + -> msg "Source package created:" + -> exit + +NOTE: then we rewind back to /usr/lib/libretools/chroot/makechrootpkg.sh::download_sources() + which returns to /usr/bin/libremakepkg + +/usr/bin/libremakepkg: + -> msg "Starting to build the package" + -> NOTE: build package in the normal way +</pre> diff --git a/HACKING/testing.md b/HACKING/testing.md index 8dee485..3b38d40 100644 --- a/HACKING/testing.md +++ b/HACKING/testing.md @@ -1,18 +1,13 @@ Testing ======= -Please write unit tests for new things. Tests can be run with `make -check`, which just runs `./testenv roundup` in the `test/` directory. -Relatedly, you need the `roundup` (the `sh-roundup` package on -Parabola) tool to run the tests. `./testenv` can be given -`--no-network` and/or `--no-sudo` to dissable tests that require those -things. Make can be made to pass those things in by setting -`TESTENVFLAGS`. If you don't dissable either, I *strongly* recommend -setting TMPDIR to somewhere on a btrfs partition before running the -tests; otherwise the chroot tests will take forever. I mean, they +Please write unit tests for new things. Tests can be run with `make check`, +which just runs `./testenv bats cases` in the `test/` directory. You will need +the `bats` tool (the `bats` Parabola package) in order to run the tests. `bats` +can be given `--filter-tags !network,!sudo,!btrfs` to disable tests which +require those things. `make check` can be made to pass those flags in by +setting `BATS_FLAGS`. If `!btrfs` is not given, then TMPDIR *must* be on btrfs +partition (or else the tests that need btrfs will fail). If neither `!network` +nor `!sudo` is given, then I *strongly* recommend setting TMPDIR to somewhere on +a btrfs partition; otherwise the chroot tests will take forever. I mean, they already take long on btrfs, but without it... _dang_. - -I also recommend having the `haveged` daemon running. That's good -general advice, but also: some of the tests make GPG keys, this -"should" take on the order of 1 second, but can take several minutes -if you don't have `haveged` running. diff --git a/HACKING/translations.md b/HACKING/translations.md new file mode 100644 index 0000000..085a18c --- /dev/null +++ b/HACKING/translations.md @@ -0,0 +1,108 @@ +Translations for programs are provided in files with the `.po` suffix. These are +created by copying PO-template (`.pot`) files, and filling in the missing +"msgstr" values. + +To add a translation, you'll first (1) need to create the `.pot` files if you +don't already have them, and then (2) copy them and (3) fill them in. + + +## 1. Create the `.pot` files + +If you are working from a release source tarball, you can skip this step (the +`.pot` files are included in the tarball); if you are working from git, read on. + +libretools' .pot files are not tracked in git; they are created by running +`make`. You'll need a checkout of both 'devtools-par' and 'libretools' to run +`make` if building from git. The 'devtools-par' and 'libretools' source root +directories should have the same parent directory. + + $ git clone https://git.parabola.nu/packages/libretools.git/ + $ git clone https://git.parabola.nu/packages/devtools-par.git/ + $ cd libretools + $ make po/files.generate + $ cd po/ + $ ls + gitget.pot + librelib.pot + libretools.pot + +NOTE: If you have modified the libretools source code, that may cause errors +during translation; especially if you have modified any of the translatable +strings. The failure will be evident by the error message: + + msgcmp: found N fatal errors + +If you see this error, refer to the "Updating translations" section below. + + +## 2. Create the `.po` Files + +Create a folder under `po/` with the two-letter language code[^1] you want to +add. Then, copy the `.pot` files to that folder, changing the file extension to +`.po`. Finally, link the `subdir.mk` file as the Makefile for that directory. +For example: + + $ mkdir es + $ for file in *.pot; do cp $file es/${file%t}; done + $ ln -s ../subdir.mk es/Makefile + +[^1]: See the langauge code table: <http://www.lingoes.net/en/translator/langcode.htm> + + +## 3. Fill the `.po` Files + +From there you can open each .po file in your favorite text editor, and fill in +the `msgstr` values with translations of the associated `msgid` keys. + + +## 4. Synchronizing/Updating Translations + +Normally, the translations are prepared as part of the dist make target +(the same command used to generate the release source-ball). Eg: + + $ make dist + +However, if you have modified the libretools source code significantly or +modified any translatable strings, it is likely that the `make` command will +fail, due to the translation files being mis-aligned with the latest changes. +You will need to run `msgmerge` to merge the newly generated .pot templates +into the existing .po files, as indicated by the `msgcmp` command traces in +the build log. + +Eg: If the following error message appeers after this log trace: + + msgcmp --use-untranslated --use-fuzzy po/es/libretools.po po/libretools.pot + .... + msgcmp: found 42 fatal errors + +then run: + + $ msgmerge po/es/libretools.po po/libretools.pot + +Notice any red (fuzzy) lines. Those are relatively minor mis-matches, which the +tool was able to reconcile, but may need correction. Also notice any green, +commented-out, 'msgid'/'msgstr' line pairs. Those are unresolved mis-matches, +which definitely need correction. They may correspond to a new 'msgid'/'msgstr' +line pair in the `msgmerge` output, with the new text as 'msgid', and an empty +'msgstr'; but the changes were too significant for the tool to determine the +correlation. The standard output of `msgmerge` is the raw text for a new +replacemnt .po file. Next, direct the output to a file, make the corrections +to the replacemnt .po file, using the `msgmerge` output as a guide; then replace +the failed .po file with the new corrected file. + + $ msgmerge po/es/libretools.po po/libretools.pot > po/es/libretools.po-merged + $ mv po/es/libretools.po{-merged,} + +Alternatively, a merge tool such as `meld`, is especially helpful for this task. +A merge tool should highlight the same merge conflicts, shown in the the +`msgmerge` output, as a side-by-side comparison, allowing you to merge the +changes from the generated '-merged' file, into the tracked .po file, in-place. +In order to minimize the possible merge conflict next time, it is best to merge +all of the changes, especially the comments with line numbers. + + $ meld po/es/libretools.po{-merged,} + +Once `make clean && make` is able to complete successfully, commit the new .po +file to the VCS. + + $ git commit -m 'sync translations' po/es/libretools.po |