The system tests can be very I/O intensive, because many of them copy
OCI images from the test suite's image cache directory to its local
container/storage store, create containers, and then delete everything
to run the next test with a clean slate. This makes them slow.
The runtime environment tests, which includes the environment variable
tests, are particularly slow because they don't skip the I/O even when
testing error handling. This makes them a good target for
optimizations.
The environment variable tests query the values of different environment
variables from different containers without changing their state.
Therefore, a lot of disk I/O can be avoided by creating these containers
only once for all the tests.
This can reduce the time needed to run the environment variable tests
from almost 26 minutes to almost 9 minutes.
https://github.com/containers/toolbox/pull/1646
The XDG_CACHE_HOME environment variable is supposed to default to
$HOME/.cache [1], just as it did in the test suite, and this location is
meant to be used as a cache for 'normal' use by the user. Test suites
generally don't qualify as 'normal' use.
One expects that deleting the cache shouldn't affect 'normal' use other
than degrading performance. However, deleting these temporary files
used by the test suite will cause actual breakage. Even if the user
doesn't manually delete the cache, two concurrent invocations of the
test suite can do so or lead to other unexpected collisions, because the
paths are constant across multiple invocations.
Therefore, it's better to limit the scope of the test suite's temporary
files within the sandbox offered by Bats [2]. The sandbox is clearly
labelled as being used by Bats, is unique for each invocation, and Bats
takes care of cleaning everything up once it has finished running.
Note that there's no need for the system-test-storage sub-directory
under BATS_SUITE_TMPDIR. So it was left out.
[1] https://specifications.freedesktop.org/basedir-spec/latest/
[2] https://bats-core.readthedocs.io/en/stable/writing-tests.htmlhttps://github.com/containers/toolbox/pull/1645
The p11-kit-modules package in Ubuntu provides p11-kit-client.so, but
the /etc/pkcs11/modules directory that's necessary to configure p11-kit
to use p11-kit-client.so is not created by any package.
It's better to ensure that the /etc/pkcs11/modules directory exists in
the image, instead of having the Toolbx container's entry point create
it at runtime, because it can be a confirmation that p11-kit was built
to read the module configuration from this location.
This should have been part of commit aa8507730d.
https://github.com/containers/toolbox/issues/626
The /etc/pkcs11 directory and /etc/pkcs11/pkcs11.conf.example file are
created by the p11-kit package in Arch Linux, and the lib11-kit package
provides p11-kit-client.so. However, the /etc/pkcs11/modules directory
that's necessary to configure p11-kit to use p11-kit-client.so is not
created by any package.
It's better to ensure that the /etc/pkcs11/modules directory exists in
the image, instead of having the Toolbx container's entry point create
it at runtime, because it can be a confirmation that p11-kit was built
to read the module configuration from this location.
This should have been part of commit 259de86c8f.
https://github.com/containers/toolbox/issues/626
It's been a while since it's been necessary to read the ID field from
os-release(5) outside this package or the VARIANT_ID field anywhere at
all. Therefore, it's time to adjust the code to reflect this reality.
Fallout from 8caa7cd828https://github.com/containers/toolbox/pull/1642
The system tests can be very I/O intensive, because many of them copy
OCI images from the test suite's image cache directory to its local
container/storage store, create containers, and then delete everything
to run the next test with a clean slate. This makes them slow.
The runtime environment tests, which includes the D-Bus tests, are
particularly slow because they don't skip the I/O even when testing
error handling. This makes them a good target for optimizations.
The D-Bus tests check if methods can be called across the user or
session and system D-Bus instances from different containers without
changing their state. Therefore, a lot of disk I/O can be avoided by
reating these containers only once for all the tests.
This can reduce the time needed to run the D-Bus tests from almost 10
minutes to almost 5 minutes.
https://github.com/containers/toolbox/pull/1641
This will reduce the size of the src/pkg/utils/utils.go file and make it
easier to specify which part of the code base is maintained by whom.
https://github.com/containers/toolbox/pull/1639
This will reduce the size of the src/pkg/utils/utils.go file and make it
easier to specify which part of the code base is maintained by whom.
https://github.com/containers/toolbox/pull/1639
This will reduce the size of the src/pkg/utils/utils.go file and make it
easier to specify which part of the code base is maintained by whom.
https://github.com/containers/toolbox/pull/1639
This will reduce the size of the src/pkg/utils/utils.go file and make it
easier to specify which part of the code base is maintained by whom.
https://github.com/containers/toolbox/pull/1639
The system tests can be very I/O intensive, because many of them copy
OCI images from the test suite's image cache directory to its local
container/storage store, create containers, and then delete everything
to run the next test with a clean slate. This makes them slow.
The runtime environment tests, which includes the networking tests, are
particularly slow because they don't skip the I/O even when testing
error handling. This makes them a good target for optimizations.
The networking tests check the behaviour and configuration of the
network in different containers without changing their state.
Therefore, a lot of disk I/O can be avoided by creating these containers
only once for all the tests.
This can reduce the time needed to run the networking tests from almost
15 minutes to almost 6 minutes.
https://github.com/containers/toolbox/pull/1637
The libp11-kit package was added to the arch-toolbox image to ensure the
presence of p11-kit-client.so. Currently, the package is already pulled
in by various dependencies, like the gnutls and p11-kit packages.
Therefore, it doesn't increase the size of the base image, but serves as
a safeguard against any inadvertent changes.
A subsequent commit will use this to give Toolbx containers access to
the certificates from certificate authorities on the host. This commit
was kept separate from the changes to toolbox(1) to ensure that the
arch-toolbox image is ready before that happens.
https://github.com/containers/toolbox/issues/626
The system tests can be very I/O intensive, because many of them copy
OCI images from the test suite's image cache directory to its local
container/storage store, create containers, and then delete everything
to run the next test with a clean slate. This makes them slow.
The runtime environment tests, which includes the group and user tests,
are particularly slow because they don't skip the I/O even when testing
error handling. This makes them a good target for optimizations.
The group and user tests check the group and user configuration in
different containers without changing their state. Therefore, a lot of
disk I/O can be avoided by creating these containers only once for all
the tests.
This can reduce the time needed to run the group and user tests from
almost 22 minutes to almost 5 minutes.
https://github.com/containers/toolbox/pull/1635
The system tests can be very I/O intensive, because many of them copy
OCI images from the test suite's image cache directory to its local
container/storage store, create containers, and then delete everything
to run the next test with a clean slate. This makes them slow.
The tests for toolbox(1) invocations forwarded to the host, which
includes the help tests, are particularly slow because they never skip
the I/O. This makes them a good target for optimizations.
The help tests for toolbox(1) invocations forwarded to the host use the
same default Toolbx container to invoke toolbox(1) from without changing
its state. Therefore, a lot of disk I/O can be avoided by creating the
default container only once for all those tests.
This can reduce the time needed to run the help tests from almost 7
minutes to a few seconds.
https://github.com/containers/toolbox/pull/1635
Now that there's a website at https://containertoolbx.org/ it makes more
sense to link to it instead of the code repository. The website is a
superset of the code repository and contains a lot more useful
information for someone who is not familiar with the Toolbx project.
https://github.com/containers/toolbox/pull/1632
When the fmt.Fprintf() [1] function is used to write to a
strings.Builder [2] instance, it uses the io.Writer [3] interface, which
is the strings.Builder.Write() method. This method is practically the
same as the strings.Builder.WriteString() method, other than the fact
that the former accepts a slice of bytes and the latter accepts a
string. So, the difference is the initial call to fmt.Fprintf().
Therefore, unless format verbs [4] are needed to build the string,
fmt.Fprintf() can be replaced with strings.Builder.WriteString(). It
reduces one function call and is shorter to type.
Fallout from the following:
* e390f15469
* 7542f5fc86
* e58992066f
* 8dd2f8e80a
* 063bdf965f
[1] https://pkg.go.dev/fmt#Fprintf
[2] https://pkg.go.dev/strings#Builder
[3] https://pkg.go.dev/io#Writer
[4] https://pkg.go.dev/fmthttps://github.com/containers/toolbox/pull/1632
This will prevent any silly bug in getting the initialization stamp path
from breaking the communication protocol between the 'enter' or 'run'
commands on the host and the Toolbx container's entry point process.
https://github.com/containers/toolbox/pull/1633
This is meant to reduce the size of the initContainer() function that
implements the heart of the 'init-container' command.
The debug log and error message were tweaked to match the name of the
function and for consistency with the configureRPM() function.
https://github.com/containers/toolbox/pull/1631
The runtime directory is needed a few times during the course of
commonly used Toolbx commands. It's used at start-up for all commands
except 'completion' and 'init-container' to synchronize the invocation
of 'podman system migrate'. The entry point (ie., 'init-container')
uses it to read the generated Container Device Interface specification
and create the initialization stamp file. The 'enter' and 'run'
commands use it to write the CDI specification and twice to detect the
creation of the initialization stamp file.
Since the runtime directory is always the same within a process, there's
no need to repeatedly go through all the steps of parsing the user and
group IDs, creating the directory, setting its ownership, and logging
the name of directory. Once the directory is successfully created, it's
path can be cached and returned for subsequent use.
In case an error occurred while setting up the runtime directory,
subsequent attempts to get it will go through all the steps again. This
doesn't matter much in practice because toolbox(1) can't continue in the
absence of a working runtime directory.
https://github.com/containers/toolbox/pull/1624
... for CVE-2024-0135 or GHSA-9v84-cc9j-pxr6, CVE-2024-0136 or
GHSA-vcfp-63cx-4h59, and CVE-2024-0137 or GHSA-frhw-w3wm-6cw4.
The src/go.sum file was updated with 'go mod tidy'.
https://github.com/containers/toolbox/pull/1614
Debarshi Ray