The POSIX shell implementation used to read and respect the HOME
environment variable. It broke when the Go implementation started using
user.Current() [1], because it uses getpwuid_r(3), which only uses the
GNU Name Service Switch's (or NSS') password database and ignores HOME.
This created a strange situation where the toolbox(1) binary ignored the
HOME environment variable, while the profile.d/toolbox.sh shell start-up
snippet and Podman read and respected it.
Restoring the HOME environment variable will enable users to create
Toolbx containers with different home directories from the host for
increased isolation, and to isolate the host's HOME from the system
tests.
Fallout from e8d7f25e83
[1] https://pkg.go.dev/os/user#Currenthttps://github.com/containers/toolbox/issues/1044https://github.com/containers/toolbox/issues/1564
This solves the common issue where /dev/vboxusb/* is owned by group
"vboxusers", causing volume mounts to fail when running `toolbox enter`.
Same is true for any other mapped dirs with supplemental group owners.
Signed-off-by: Michael Winters <1688867+mwinters0@users.noreply.github.com>
A subsequent commit will show the debug logs and errors from a Toolbx
container's entry point as part of the 'enter' and 'run' commands. To
test this behaviour, it will be necessary to intentially fail the entry
point.
Moreover, container start-up is a concurrent operation. If the entry
point fails too early, then it will be caught by the 'podman inspect'
right after the 'podman start' before the inotify(7) watches are put in
place. Otherwise, it will be handled by the timeout. Therefore, it
will be necessary to shake out any bugs arising out of unexpected races.
To address this, two environment variables have been introduced:
* TOOLBX_DELAY_ENTRY_POINT
* TOOLBX_FAIL_ENTRY_POINT
The TOOLBX_DELAY_ENTRY_POINT environment variable can be set to a
positive integer during the 'create' command to add a delay, in terms of
seconds, when the Toolbx container's entry point is started by 'enter'
and 'run'.
Similarly, if the TOOLBX_FAIL_ENTRY_POINT environment variable is set to
a positive integer during the 'create' command, the entry point will
later fail during 'enter' and 'run'. The error message will have only
one line if its value is one, else it will have two.
https://github.com/containers/toolbox/issues/750
Currently, it's not possible to create a Toolbx container from an image
without a name:
$ podman build --squash images/fedora/f39
STEP 1/21: FROM registry.fedoraproject.org/fedora:39
STEP 2/21: ARG NAME=fedora-toolbox
STEP 3/21: ARG VERSION=39
...
--> 2f9bdf11c8d4
2f9bdf11c8d4d7674dfb17d8edcfd13475d8636077f1a6208ecd616de77d7f80
$ toolbox create --image 2f9bdf11c8d4
Error: empty RepoTag for image 2f9bdf11c8d4
The image's fully qualified name is fetched from its RepoTags for purely
cosmetic reasons to show a precise human-readable name in the debug logs
and 'podman inspect --type container'. Therefore, there's no reason to
fail the creation of a Toolbx container in the absence of it.
Note that an image without a name will have an empty RepoTags array in
the JSON returned by 'podman inspect --format json --type image'. It's
different from not having a RepoTags array at all in the JSON, which may
or may not be indicative of a more serious problem and will continue to
fail the creation of the Toolbx container as before.
https://github.com/containers/toolbox/pull/1486
It's less of a user-facing operation, and more of a backend one. A
subsequent commit will improve the error handling when getting the fully
qualified name of an image from its RepoTags to handle cases where the
'create' command is used with an image without a name.
https://github.com/containers/toolbox/pull/1486
This is meant to make the project more searchable on the Internet. More
and more people have been pointing out that "toolbox" is terribly
difficult to search for, and it's impossible to find any decent
Internet real estate by that name.
Some exceptions:
* The code repository is still https://github.com/containers/toolbox.
It will be renamed after giving a heads-up to other contributors.
* The name of the binary is still 'toolbox'. The name is embedded
into existing Toolbx containers as their entry point, which is bind
mounted from the host operating system when the containers are
started. Trivially renaming the binary will prevent these
containers from starting.
* For similar reasons, the TOOLBOX_PATH environment variable is still
the same.
* For similar reasons, the profile.d file to be read by the shell on
start-up is still called toolbox.sh.
* The label used to identify Toolbx containers and images is still
called com.github.containers.toolbox. There are many existing
Toolbx containers, and many Toolbx images beyond the control of the
Toolbx project that use this label to identity themselves. Simply
renaming the label will prevent these containers and images from
being recognized.
* The names of the built-in Toolbx images still retain the word
'toolbox'. Images under the new name need to be published on the
OCI registries and the toolbox(1) binary needs to be taught to
handle both old and new names, wherever necessary, for backwards
compatibility.
* The stamp file used to identify Toolbx containers is still called
/run/.toolboxenv because it's used by various external programs and
users to identify Toolbx containers.
* The OSC 777 escape sequence to track and preserve the user's current
Toolbx container [1] still emits 'toolbox' as the name of the
container runtime. Changing the escape sequence can break terminal
emulation applications, like Prompt [2], that consume it. Hence, it
needs to be done carefully.
* The runtime directories at /run/toolbox, when used as root, and
$XDG_RUNTIME_DIR/toolbox, when used rootless, weren't renamed.
When used as root, /run/toolbox is embedded into existing Toolbx
containers as a bind mount from the host. Trivially renaming the
path will prevent these containers from starting.
Secondly, both these paths are used to synchronize container
start-up. If the paths are trivially renamed, and the toolbox(1)
binary is updated and used without stopping all existing containers,
then it won't be able to enter containers that were already started.
Strictly speaking, this scenario isn't supported, since updates are
always expected to be "offline" [3]. However, it's worth noting
because solving the previous problem might also address this.
* The configuration file for RPM is still called
/usr/lib/rpm/macros.d/macros.toolbox.
[1] https://gitlab.freedesktop.org/terminal-wg/specifications/-/issues/17
[2] https://gitlab.gnome.org/chergert/prompt
[3] https://www.freedesktop.org/software/systemd/man/latest/systemd.offline-updates.htmlhttps://github.com/containers/toolbox/issues/1399
It doesn't make sense to show the image download prompt when the
standard input or output stream is redirected to something other than a
terminal device.
During such non-interactive use, there's no way for the user to see the
prompt and the size of the image and then make a decision based on them.
The decision has to be made differently and earlier. The user will
either never download or always download or will use 'skopeo inspect'
to decide for themself.
Secondly, when the input and output are not connected to a terminal, the
terminal escape sequences and the terminal-specific ioctl(2) requests
used to show the prompt won't work anyway.
Some changes by Debarshi Ray.
https://github.com/containers/toolbox/pull/1428
It takes 'skopeo inspect' a few seconds to fetch the image size from the
remote registry, and while that happens the user can't interact with the
image download prompt:
$ toolbox create
Image required to create toolbox container.
<wait for a few seconds>
Download registry.fedoraproject.org/fedora-toolbox:39 (359.8MB)? [y/N]:
This feels awkward because it's not clear to the user what's going on
during those few seconds. Moreover, while knowing the image size can be
convenient at times, for example when disk space and network bandwidth
are limited, it's not always important.
It will be better if 'skopeo inspect' ran in the background, while
waiting for the user to respond to the image download prompt, and once
the image size has been fetched, the image download prompt can be
updated to include it.
So, initially:
$ toolbox create
Image required to create toolbox container.
Download registry.fedoraproject.org/fedora-toolbox:39 ( ... MB)? [y/N]:
... and then once the size is available:
$ toolbox create
Image required to create toolbox container.
Download registry.fedoraproject.org/fedora-toolbox:39 (359.8MB)? [y/N]:
If skopeo(1) is missing or too old, then the prompt can continue without
the size, as it did before:
$ toolbox create
Image required to create toolbox container.
Download registry.fedoraproject.org/fedora-toolbox:39 [y/N]:
The placeholder for the missing image size (ie., ' ... MB') was chosen
to have seven characters, so that it matches the most common sizes. The
human-readable representation of the image size is capped at four valid
numbers [1]. Unless it's a perfect round number like 1KB or 1.2MB, it
will likely use all four numbers and the decimal point, which is five
characters. Then two more for the unit, because it's very unlikely that
there will be an image that's less than 1KB in size and will be shown in
bytes with a B. That makes it seven characters in total.
Updating the image download prompt with the results of 'skopeo inspect'
is vulnerable to races. At the same time as the terminal's cursor is
being moved to the beginning of the current line to overwrite the
earlier prompt with the new one, the user can keep typing and keep
moving the cursor forward. This competition over the cursor can lead to
awkward outcomes.
For example, the prompt can overwrite the characters typed in by the
user, leaving characters in the terminal's input buffer waiting for the
user to hit ENTER, even though they are not visible on the screen.
Another example is that hitting BACKSPACE can end up deleting parts of
the prompt, instead of stopping at the edge.
This is solved by putting the terminal device into non-canonical mode
input and disabling the echoing of input characters, while the prompt is
being updated. This prevents input from moving the terminal's cursor
forward, and from accumulating in the terminal's input buffer even if
it might not be visible. Any input during this interim period is
discarded and replaced by '...', and a fresh new prompt is shown in the
following line.
In practice, this race shouldn't be too common. It can only happen if
the user is typing right when the prompt is being updated, which is
unlikely because it's only supposed to be a short 'yes' or 'no' input.
The use of the context.Cause and context.WithCancelCause functions [2]
requires Go >= 1.20. Bumping the Go version in src/go.mod then requires
a 'go mod tidy'. Otherwise, it leads to:
$ meson compile -C builddir --verbose
...
/home/rishi/devel/containers/git/toolbox/src/go-build-wrapper
/home/rishi/devel/containers/git/toolbox/src
/home/rishi/devel/containers/git/toolbox/builddir src/toolbox
0.0.99.4 cc /lib64/ld-linux-x86-64.so.2 false
go: updates to go.mod needed; to update it:
go mod tidy
ninja: build stopped: subcommand failed.
[1] https://pkg.go.dev/github.com/docker/go-units#HumanSize
[2] https://pkg.go.dev/contexthttps://github.com/containers/toolbox/issues/752https://github.com/containers/toolbox/issues/1263
A subsequent commit will use this to ensure that the user can still
interact with the image download prompt while 'skopeo inspect' fetches
the image size from the remote registry. Initially, the prompt will be
shown without the image size. If the user responds before the size is
fetched, then the pending 'skopeo inspect' will be cancelled.
https://github.com/containers/toolbox/issues/752https://github.com/containers/toolbox/issues/1263
A subsequent commit will use this to ensure that the user can still
interact with the image download prompt while 'skopeo inspect' fetches
the image size from the remote registry.
To do this, at some point, the terminal device will be put into
non-canonical mode input and the echoing of input characters will be
disabled to retain full control of the cursor position. Unfortunately,
this will require access to the full termios(3) struct that isn't given
by golang.org/x/term, and, hence, the code needs to be written using the
underlying termios(3) API.
This future code will have enough overlap with the IsTerminal API from
golang.org/x/term that it doesn't make sense to use a separate module
(ie., golang.org/x/term) for it.
https://github.com/containers/toolbox/issues/752https://github.com/containers/toolbox/issues/1263
This uses 'skopeo inspect' to get the size of the image on the registry,
which is usually less than the size of the image in a local
containers/storage image store after download (eg., 'podman images'),
because they are kept compressed on the registry. Skopeo >= 1.10.0 is
needed to retrieve the sizes [1].
However, this doesn't add a hard dependency on Skopeo to accommodate
size-constrained operating systems like Fedora CoreOS. If skopeo(1) is
missing or too old, then the size of the image won't be shown, but
everything else would continue to work as before.
Some changes by Debarshi Ray.
[1] Skopeo commit d9dfc44888ff71a6
https://github.com/containers/skopeo/commit/d9dfc44888ff71a6https://github.com/containers/skopeo/issues/641https://github.com/containers/toolbox/issues/752
Signed-off-by: Nieves Montero <nmontero@redhat.com>
The --monitor-host option was added to the 'init-container' command in
commit 8b84b5e460 to accommodate Podman versions older than 1.2.0
that didn't have the '--dns none' and '--no-hosts' options for
'podman create'. These options are necessary to keep the Toolbx
container's /etc/resolv.conf and /etc/hosts files synchronized with
those of the host.
Note that Podman 1.2.0 was already available a few months before
commit 8b84b5e460 introduced the --monitor-host option. The
chances of someone using an older Podman back then was already on the
decline, and it's very unlikely that a container created with such a
Podman has survived till this date.
Commit b6b484fa79 raised the minimum required Podman version to
1.4.0, and made the '--dns none' and '--no-hosts' options a hard
requirement. The minimum required Podman version was again raised
recently in commit 8e80dd5db1 to 1.6.4. Therefore, these days,
there's no need to separately use the --monitor-host option of
'init-container' for newly created containers to indicate that the
Podman version wasn't older than 1.2.0.
Given all this, it's time to stop using the --monitor-host option of
'init-container', and assume that it's always set. The option is still
accepted to retain compatibility with existing Toolbx containers.
For containers that were created with the --monitor-host option, a
deprecation notice will be shown as:
$ podman start --attach CONTAINER
Flag --monitor-host has been deprecated, it does nothing
...
https://github.com/containers/toolbox/pull/617
So far the minimum required Podman version was 1.4.0, based on what used
to be available in RHEL 7. These days, Podman 1.6.4 is old enough to be
in RHEL 7.9. Hence it's time to bump the baseline.
https://github.com/containers/toolbox/pull/1253
It's better not to use the global flag variables beyond the top-level
RunE functions, because sometimes the lower-level functions are re-used
from other files within the 'cmd' package. In this case,
createContainer(), and hence pullImage(), is also used in src/cmd/run.go
to implement the 'run' command. However, the 'run' command doesn't have
a --authflags option.
Since the default value of the flag is the zero value of the type, which
is a NOP in the code, it's likely that the code was still correct, but
it will be better to maintain some discipline here to highlight the
inputs needed by the lower-level functions. Otherwise, things can get
tangled up.
Fallout from ecd1ced719https://github.com/containers/toolbox/pull/1240
Podman creates a private cgroup namespace for containers on cgroups v2
by default. The host's cgroupfs is mounted at /sys/fs/cgroup giving an
inconsistent view of the cgroups. Toolbx doesn't intend to provide a
segregated security domain. So, there is no need for a cgroup namespace
and Toolbx containers can just use the host's namespace.
Having a private cgroup namespace for containers isn't currently causing
problems for any use-case, but it did come in handy when investigating
how to run a non-nested display server from within a Toolbx container on
a virtual terminal. Since this requires a change to the 'podman create'
arguments, it's not going to have an effect on existing containers, and
re-creating containers is annoying for users. So, it might be better to
get ahead of the curve and do it preemptively.
https://github.com/containers/toolbox/issues/992
Signed-off-by: Sebastian Wick <sebastian.wick@redhat.com>
Currently, the container name and release are only validated if they
were specified as command line options. Neither the value of release
in the configuration file nor the container name generated from an
image are validated.
There's also a lot of repeated code in the command front-ends to
validate the container name and release. This opens the door for
mistakes. Any adjustment to the code must be repeated elsewhere, and
there are subtle interactions and overlaps between the validation code
and the code to resolve container and image names.
It's worth noting that the container and image name resolution happens
for both the command line and configuration file options, and generates
the container name from the image when necessary.
Therefore, validating everything while resolving cleans up the command
front-ends and increases the coverage of the validation.
This introduces the use of sentinel error values and custom error
implementations to identify the different errors that can occur while
resolving the container and images, so that they can be appropriately
shown to the user.
https://github.com/containers/toolbox/pull/1101
Cobra (the CLI library) has an advanced support for generating shell
completion. It support Bash, Zsh, Fish and PowerShell. This offering
covers the majority of use cases with some exceptions, of course.
The generated completion scripts have one behavioral difference when
compared to the existing solution: flags (--xxx) are not shown by
default. User needs to type '-' first to get the completion.
https://github.com/containers/toolbox/pull/840
Co-authored-by: Ondřej Míchal <harrymichal@seznam.cz>
Using a non-supported distribution via `--distro` resulted in a silent
fallback to the Fedora image which confuses users. When a faulty release
format was used with `--release` a message without any hint about the
correct format was shown to the user.
This separates distro and release parsing into two chunks that have
greater control over error reporting and provides more accurate error
reports for the user.
Fixes https://github.com/containers/toolbox/issues/937https://github.com/containers/toolbox/pull/977
The Toolbox repository was moved to the 'containers' organization some
time ago already[0]. Containers marked with the label:
com.github.debarshiray.toolbox=true
will remain supported but new containers will not be created with it.
https://github.com/containers/toolbox/pull/510
[0] de5e5df9b7
pkg/utils has been in Go Toolbox since its birth. Along the way it
accumulated a number of functions where a few of them are purely CLI
related. Since the majority of functions in the package are related to
some "deeper" functionality in Toolbox, it makes more sense to move the
selected few to package cmd. This will make pkg/utils a bit leaner and
create a dedicated space for cmd utility functions to live in.
In the process the error creation functions no longer require the
executableBase argument to be passed to them.
https://github.com/containers/toolbox/pull/819
A subsequent commit will add support for configuration files, which can
override the default toolbox image. Since this override affects all
commands, it effectively ends up adding a fourth option to the 'enter'
command, other than the existing options to change the distribution,
release and container. This makes it a lot more difficult to reason
when only 'toolbox enter --release N' is enough to enter the created
container.
https://github.com/containers/toolbox/pull/828https://github.com/containers/toolbox/pull/851
Having the entire host file system hierarchy mounted inside a toolbox
container gives the containers a more complete environment that's
resilient against future changes in the layout of the file system
hierarchy and the need for giving access to new paths to support new
use-cases. Otherwise, one would have to create a new container to get
access to any path that lies outside the /boot, /etc, /run, /tmp, /usr
and /var directories.
As a nice side-effect, this also simplifies the bind mount handling
code.
https://github.com/containers/toolbox/pull/827
The regexp.MatchString [1] API returns an error only when the regular
expression is faulty, and the boolean return value tells if a match was
found. In this case, the regular expression is baked into the code as a
string literal. So, unless there's a programmer error, it should always
be valid.
Fallout dd947016b3
[1] https://golang.org/pkg/regexp/#MatchStringhttps://github.com/containers/toolbox/pull/825
This builds upon commit eedfdda535, which added more information
to the error messages presented to the user by including the errors
thrown by the lower layers of the code.
However, if the errors are being thrown by external modules, or are
coming from functions that are too many layers below, then it is
difficult to guarantee their contents. They might be duplicating
information added by the upper layers, or in extreme cases might even
contain JSON blobs, simply because it made sense for the API contracts
of the functions generating the errors.
Therefore, it's better to put them in the debug logs to retain control
over what gets displayed to the user under normal (ie., non-debug)
operation.
https://github.com/containers/toolbox/pull/809
The spinner needs to be explicitly stopped before showing the example
'enter' command for using the container. Otherwise, it gets misprinted:
$ toolbox create foo
Creating container foo: / Created container: foo
Enter with: toolbox enter foo
A comment was added to highlight this, since it might not be obvious at
first sight.
Due to such potential quirks, it might be better to keep the spinner
somewhat tightly encapsulated with the code that necessitates it, which
in this case is 'podman create'. For instance, we already need to be
careful to avoid enclosing the pullImage function with a spinner
because it carries it's own.
The code lying between the 'podman pull' and the 'podman create' is so
light that a human user isn't able to discern the absence of a
spinner. So, it seems worth leaning towards ease of understanding and
avoiding potential traps.
This reverts commit 3aaa1d30f1.
https://github.com/containers/toolbox/pull/746
Michael Winters
Debarshi Ray
Ondřej Míchal
Nieves Montero
Sebastian Wick
Oliver Gutierrez
Anthony Rabbito