| Age | Commit message (Collapse) | Author |
|---|
|
it RW
A key aspect of libremakepkg is that it tries to be strict about many
things, in order to catch issues.
One issue is that sources are downloaded during build(), meaning that
they're missing from the .src.pkg.tar sourceball. So, by default
libremakepkg runs build() with networking disabled, to catch this
issue. If there is a problematic package, we have an -N flag to
enable networking, as an escape hatch; as we only have finite packager
time/effort.
One issue is when a package can't be rebuilt from the .src.pkg.tar
sourceball. If the PKGBUILD modifies itself, then it won't match
what's in the sourceball. This is what the
libremakepkg.bats:"libremakepkg does not run pkgver" test demonstrates
and tests-for; this failing demonstration testcase was added in
044b4e1 (test: libremakepkg: Add some failing tests [ci-skip],
2018-07-31, Luke Shumaker <lukeshu@lukeshu.com>). We solved by
mounting the $startdir read-only in 646ac02 (libremakepkg,chcleanup:
Be stricter about network access, 2018-08-03, Luke Shumaker
<lukeshu@lukeshu.com>). However, it turns out that this caused issues
for a few packages. So, this protection was reverted in a6f6ac4
(libremakepkg: fix building packages requring a rw startdir,
2019-05-17, Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>). This
is bad, it potentially lets many issues slip through undetected.
Instead, handle it like we do networking: Have the protection on by
default, say "PLEASE don't turn this off", but recognize the increased
cost in time and efforts and so provide the `-W` flag as an escape
hatch.
|
|
This documents all the contributions of the last few years, based on
the Git history.
|
|
Several packages require a read-write startdir:
- Some packages have a pkgver that is computed dynamically
through a pkgver function. This is the case for many packages
using git repositories. At the end of the package build, the
pkgver is automatically updated in the PKGBUILD, however,
without that fix that fails with libremakepkg as the PKGBUILD
was set read-only.
- Some packages like linux-libre are modifying the install=
script. This is done by creating a temporary install script
in the startdir that is then modified with sed. Once this is
done that install script is then dynamically selected.
As this also require to have read-write access to the
startdir to be read-write it fails to build the package if
it's not the case.
In both cases it's possible to modify the PKGBUILDs to
workaround the issue, however the Arch Linux distribution has
a read-write startdir, and modifying each affected packages
would significatively increase the cost (in time and efforts)
of maintaining Parabola.
This was broken by the commit 646ac0258c3295943778142468aadfe5b04ad6d1
("libremakepkg,chcleanup: Be stricter about network access").
Thanks to elibrokeit on the #parabola IRC channel on Freenode
for helping me identifying the cause of read-only mounts in
libremakepkg.
Signed-off-by: Denis 'GNUtoo' Carikli <GNUtoo@cyberdimension.org>
|
|
"Ignore space change" might be helpful when viewing this diff.
|
|
"What is the alignment of the libremakepkg options section based on?", you
may ask. You'll see in the next commit!
|
|
I don't know what I was thinking when I wrote it that way.
For one, the "<&0" bit is a no-op. It redirects stdin to be stdin. Maybe
I had meant to write "<&-" and close it?
For two, just write it normally. It's simple enough, don't make it look
like magic.
|
|
That file hasn't been created since 646ac025 (2018-08-04).
|
|
expect input
If stdin is a TTY, systemd-nspawn will create a proxy PTY, and pump the
data from real stdin through the proxy PTY. Because the PTY acts as a
buffer, that means that it eats stdin, EVEN IF THE PROGRAM DOESN'T ACTUALLY
READ IT. This is surprising to the user.
We can't know if the command to `librechroot run` is going to read from
stdin, but for most uses of librechroot we do know. So, set stdin to
/dev/null for those cases.
|
|
|
|
Objectives:
- Once the source package has been created, never run makepkg with
networking enabled again, so that we can be 100% sure that the source
package has all of the sources that we need.
- Don't let makepkg edit the PKGBUILD (eg. by evaluating pkgver())
User-facing changes:
- libremakepkg now disables networking during prepare()
- libremakepkg no longer runs pkgver()
- chcleanup (and thus `librechroot clean-pkgs`) now also installs packages
Technical description of changes:
- In devtools (and not really in this commit):
* No longer split /chrootprepare off of /chrootbuild. The point of
splitting it was that we'd have prepare() run from /chrootprepare
and the rest later run from /chrootbuild, so that we could leave
networking enabled during prepare() but not the rest. Now that we're
disabling networking during prepare(), that's pointless.
* Have download_sources create a source package (rather than having
/chrootbuild create the source package at the same time that we create
the binary package). We adjust the caller to temporarily set
SRCPKGDEST to a temporary directory, so that we can get the file and
control the filename when we move it in to the real SRCPKGDEST.
- Have chcleanup install the dependencies. If we used the traditional
`makepkg --sync` to install the dependencies, then we'd be breaking the
"no networking for makepkg after source package creation" objective.
chcleanup already has all of the logic necessary to accomplish this.
If there's a discrepancy in chcleanup behavior and makepkg behavior, and
makepkg thinks it needs to install something, then that just means we'll
have to fix the bug in chcleanup, instead of letting it go for more than
a year (*cough* https://labs.parabola.nu/issues/1311 *cough*).
- Use files extracted from the source package (rather than files found in
the current directory) to build the package.
- We mount the temporary directory containing the extracted source package
files read-only, to be sure that makepkg doesn't modify the PKGBUILD.
This is necessary because --holdver only disables pkgver() if it's a VCS
package.
|
|
|
|
|
|
libremakepkg defined a cleanup() function, which overrode
common.sh:cleanup(), which meant that common.sh:die() exited with a
'0' status.
|
|
Do this by syncing makepkg_args with makechrootpkg's
default_makepkg_args; recent devtools have makechrootpkg not copy the
keyring in to the chroot.
This incorporates 3 commits from Arch devtools:
7ca4eb82d (2017-05-02): add --holdver
0cbc179d2 (2017-07-13): use long options; `-s`→`--syncdeps`, `-L`→`--log`
75fdff181 (2017-07-13): add --skipinteg
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Only make this change in places where it shouldn't make a difference, and
something weird has to be going on for the subshell to fail. This is on
par with checking the return value of malloc. We don't need tests for each
of these failure cases.
|
|
These were found with the help of shellcheck.
|
|
|
|
|
|
|
|
|
|
|
|
Packages should be buildable without any extra incantations, and it was
more of a source of mistakes than anything else.
|
|
|
|
|
|
|
|
|
|
|
|
I can't figure out a reliable way to test this, as it depends on the host
pacman cache.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Feature request: https://labs.parabola.nu/issues/458
|
|
I'm a fan of systemd-nspawn. I really am. It handles a lot of things the
"right" way, so you don't have to.
However, it handles PTYs and signals in an incredibly stupid way. Working
around one is f---ed up by the other being wrong.
Before I implemented the change that this reverts, this is what happens:
systemd-nspawn> I need to create a PTY and proxy between it and stdio.
I can specify a terminal size when doing this--I should
set it to reflect the terminal I am connected to.
systemd-nspawn> /dev/stdout: Are you a TTY?
/dev/stdout> systemd-nspawn: No.
---- The correct thing to do here would be to either not set the size,
---- or set it to 80×24
systemd-nspawn> /dev/stdin: Are you a TTY?
/dev/stdin> systemd-nspawn: Yes.
systemd-nspawn> /dev/stdin: Great, what size are you?
/dev/stdin> systemd-nspawn: I am X×Y.
systemd-nspawn: libc: Please give me a PTY that is X×Y.
---- Why the f--- would you do this? The size only effects *output*.
---- If you don't know the characteristics of the output, set
---- reasonable defaults! Don't base it on the characteristics of the
---- input!
My solution was to make /dev/stdin not be a TTY by closing it. This had
the desired effect, but tripped up the signal handling.
Here is how signals were handled before:
systemd-nspawn> /dev/stdin: Are you a TTY?
/dev/stdin> systemd-nspawn: Yes.
systemd-nspawn> /dev/stdin: Great, please don't send SIGINT on C-c.
I will handle it myself.
*user presses C-c, systemd reads it*
systemd-nspawn> child processes: here's a SIGINT.
Here's what happens when I close, or do other things to make stdin not be a
TTY:
systemd-nspawn> /dev/stdin: Are you a TTY?
/dev/stdin> systemd-nspawn: No.
systemd-nspawn> /dev/stdin: Oh, OK.
*user presses C-c, /dev/stdin interprets it*
/dev/stdin> systemd-nspawn: Here's a SIGINT.
systemd-nspawn> *dies, abandoning child processes*
The clear correct thing to do would have been to *handle* SIGINT, and pass
it along to the children.
|
|
Besides preventing interactive use, this prevents the program from knowing
the size of the terminal.
|
|
|
|
There are two downsizes to using sed; it line-buffers, and (the existing
pattern) doesn't handle '\r'.
So, now I've included a Perl helper program to do a better job.
I'd originally written it in C, and while that version was faster (the Perl
version stutters occasionally), it required recompilation for different
architectures. I could have gotten around that with tcc, but I didn't want
to add any dependencies. Which is why I settled on Perl. It's part of
group=(base), and it is required by packages in group=(base-devel), so I
can count on it always being there.
It doesn't handle every movement character, just '\n' and '\r', but that's
enough for curl and pacman.
|
|
Conflicts:
src/abslibre-tools/createworkdir
|
|
|
