gvproxy and win-sshproxy have capabilities to serve this type of enpoint.
This change only adds one additional API enpoint publishing by appending
proxy command lines.
Signed-off-by: Arthur Sengileyev <arthur.sengileyev@gmail.com>
Podman machine list now supports a new option, --all-providers, which lists all machines from all providers.
Signed-off-by: Ashley Cui <acui@redhat.com>
Example:
DefaultEnvironment="VAR1=word1 word2" VAR2=word3 "VAR3=word 5 6"
Sets three variables "VAR1", "VAR2", "VAR3".
Double quote is not escaped, as there is no chance it appears in a proxy value. User can still espace it if really necessary
Signed-off-by: Philippe Martin <phmartin@redhat.com>
Fix one minor issue with vfkit error handling. First checking if err !=
nil OR errors.Is() is pointless as the err != is already true.
Second nilerr complains because we return nil when we hit an error
branch, in this case this is correct because an error means VM is
stopped.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
It qemu cannot be compiled anyway so make sure we do not try to compile
parts where the typechecker complains about on windows.
Also all the e2e test files are only used on linux as well.
pkg/machine/wsl also reports some error but to many for me to fix them
now. One minor problem was fixed in pkg/machine/machine_windows.go.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
By enabling UserKnownHostsFile=/dev/null, and CheckHostIP=no
options to the defaults we prevent the user from adding the host key
multiple times and from flakes that can raise Remote Host Id change.
Resolves: https://github.com/containers/podman/issues/23505
Signed-off-by: Nicola Sella <nsella@redhat.com>
Read stderr from ssh-keygen before calling wait(), since cmd.Wait() closes cmd.StderrPipe() after it exits, causing a read-on-closed-pipe error.
Signed-off-by: Ashley Cui <acui@redhat.com>
When removing the gvproxy pid file, under CI conditions we could hit a
case where the PID file being removed seemed to have an open handle on
it still. I could not find anything in podman that left an open handle
and gvproxy would have quit before this, so we think it is likely
another process holding it. I could not find root cause with CI because
I could not trip the flake.
this new code allows windows (specifically hyperv bc WSL does not use
GVProxy) to ignore an ERROR_SHARING_VIOLATION.
Signed-off-by: Brent Baude <bbaude@redhat.com>
This adds generic utility to convert file system path into URL structure.
Instead of string manipulation it uses URL parsing and building routines.
Appending absolute path to `unix:///` URL out of the box correctly
handles URL format on Windows platform, where filepath should be prepended
by additional `/` before drive letter.
Signed-off-by: Arthur Sengileyev <arthur.sengileyev@gmail.com>
When the file is empty it is possible our code panics as bar.ProxyReader
returns nil when the bar is finished which is the case for 0 size as it
doesn't have to read anything from there. However as this happens on
different goroutines it is race and most of the time still works.
To fix this simply skip the progress bar setup for empty files.
While at it fix the deprecated argument in the tests.
Fixes#23281
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
In case of timeouts actually log the command again and make sure to send
SIGABRT to the process as go will create a useful stack strace where we
can see where things are hanging. It also kill the process unlike the
default Eventually().Should(Exit()) call the leaves the process around.
The output will be captured by default in the log so we just see the
stack trace there.
And while at it bump the timout up to 10 mins, we are hitting hard
flakes in CI where machine init takes longer than 5 mins for unknown
reasons but this seems to be good enough.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Currently all podman machine rm errors in AfterEach were ignored.
This means some leaked and caused issues later on, see #22844.
To fix it first rework the logic to only remove machines when needed at
the place were they are created using DeferCleanup(), however
DeferCleanup() does not work well together with AfterEach() as it always
run AfterEach() before DeferCleanup(). As AfterEach() deletes the dir
the podman machine rm call can not be done afterwards.
As such migrate all cleanup to use DeferCleanup() and while I have to
touch this fix the code to remove the per file duplciation and define
the setup/cleanup once in the global scope.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
On linux and macos the connections are stored under the home dir by
default so it is not a problem there but on windows we first check
the APPDATA env and use this dir as config storage. This has the problem
that it is not cleaned up after each test as such connections might leak
into the following test causing failues there.
Fixes#22844
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The driver is now hardcoded again, and there can only be
one type of mounts at a time (which one changes over time)
Revert "Make it possible to select the volume driver"
This reverts commit 6630e5cf66.
Signed-off-by: Anders F Björklund <anders.f.bjorklund@gmail.com>
Close loophole that would allow you to assign more memory than the
system has to a podman machine
Fixes: #18206
Signed-off-by: Brent Baude <bbaude@redhat.com>
The logs are not verbose if `--debug` is not set, and very useful to
have if gvproxy exits unexpectedly.
Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
This PR is a couple of small fixes so that our CI would be capable of running the machine test suite on the libkrun provider.
RUN-2172
Signed-off-by: Brent Baude <bbaude@redhat.com>
Podman machine reset now removes and resets machines from all providers availabe on the platform.
On windows, if the user is does not have admin privs, machine will only reset WSL, but will emit a warning that it is unable to remove hyperV machines without elevated privs.
Signed-off-by: Ashley Cui <acui@redhat.com>
One problem on FCOS is that the root directory is immutable, as such in
order to mount arbitrary paths from the host we must make it mutable
again and create these dir on boot in order to be able to mount there.
The current logic was racy as it used one unit for each path and they
all did chattr -i /; mkdir -p $path; chattr -i / and systemd can run
these units in parallel. That means it was possible for another unit to
make / immutable before the unit could do the mkdir. I pointed this out
on the original PR[1] but we never followed up on it...
Now this here changes several things. First have one unit that does the
chattr -i / (immutable-root-off.service), it is hooked into
remote-fs-pre.target which means it is executed before the network
mounts (virtiofs) are done.
Then we have another unit that does chattr +i /
(immutable-root-on.service) which turn the immutable root back on after
remote-fs.target which means all mount are done at this point.
Additionally the automount unit is removed because it does not add any
value for us and it was borken anyway as it used the virtiofs tag as
path so systemd just ignored it.
[1] https://github.com/containers/podman/pull/20612#discussion_r1384846241Fixes#22569
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
The current timeout was not long enough. Systemd default is 90s so we
should wait for at least that long. Also it really doesn't make sense to
throw an error we saying we failed waiting for stop. We should hard
terminate the VM in case a graceful shutdown did not happen.
Fixes#22515
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Stop copying the pre-pulled uncompressed machine disk into the individual test dir. The machine pull code already makes a copy of the disk into the test's HOMEDIR/.local/share/containers/podman/machine, and works off that copy.
Before the change: TESTDIR/<image> is copied to TESTDIR/podman_test/<image> by the test, and then podman machine copies the image to TESTDIR/podman_test/.local/share/containers/podman/machine/provider/<image>
After the change: TESTDIR/<image> is copied to TESTDIR/podman_test/.local/share/containers/podman/machine/provider/<image> by podman machine
The image that is actually run is at TESTDIR/podman_test/.local/share/containers/podman/machine/provider/<image> in both instances.
Signed-off-by: Ashley Cui <acui@redhat.com>
We do a soft stop via systemd to allow graceful shutdown behavior.
Hoewever for unknown reason we are hitting such a case in CI right now.
Regardless of the CI issue we should always to the hard terminate in
such case so only log the timeout as warning.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
First of some commands ignored cmd.Wait() error which means it was
impossible to notice any command errors. And others only returned
the wait error as it which when a command fails is just
`exit status <code>` which is not helpful at all.
This commit should add proper error wrapping with stderr to get useful
strings back hopefully.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
1. Prefer using the built-in functions of `golang.org/x/sys/windows` rather than using `syscall` to directly call system APIs
2. Reduce bitwise operations to improve code maintainability
3. Minimize defining your own Flags, and prioritize using those already defined in `golang.org/x/sys/windows`
Signed-off-by: Kevin Cui <bh@bugs.cc>
Switch to using virtiofs by default, and delete the 9p code.
This is structured as a separate patch to make it easier
to revert if need be.
Signed-off-by: Colin Walters <walters@verbum.org>
I'm hitting a bug with 9p when trying to transfer large files.
In RHEL at least 9p isn't supported because it's known to have a
lot of design flaws; virtiofsd is the supported and recommended
way to share files between a host and guest.
Add a new hidden `PODMAN_MACHINE_VIRTFS` environment
variable that can be set to `virtiofs` to switch to virtiofsd.
Signed-off-by: Colin Walters <walters@verbum.org>
When running with "log-level=debug" and libkrun as machine provider,
spawn a Terminal to execute "krunkit" to enable users to have full
access to the VMs console for debugging purposes.
Users obtain an interactive, text console with scrollback. It's possible
to interact with both the kernel and GRUB2. To obtain even additional
debugging information, users can add "console=hvc0" to the linux kernel
command line through GRUB2 (it may be worth considering extending the
initial configuration of the VM to add that argument by default).
Signed-off-by: Sergio Lopez <slp@redhat.com>
Previously, the mac podman-machine tests installed rosetta before
executing any tests. As a best-practice (and because the Macs in CI are
shared) tests should never permanently modify the system. As of this
commit, the system setup script used for the CI Macs does the rosetta
installation. Remove the test setup code that installed rosetta and
add a CI-level confirmation that it's been pre-installed.
Signed-off-by: Chris Evich <cevich@redhat.com>
In GetDefaultDevices(), make sure MachineConfig has an AppleHypervisor
instance before attempting to access it. This fixes a SIGSEGV when
running with libkrun as machine provider.
Signed-off-by: Sergio Lopez <slp@redhat.com>
Use specV1.AnnotationTitle instead of hard-coding its value when we go
looking for podman-machine images.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
Adds provider detection for LibKrun in `pkg/provider/platform_darwin.go`.
Changes the macOS version check to 13 instead of 11, and now uses the `semver`
package to verify current version.
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
Extends the `pkg/machine/provider` package to add an API which includes
provider detection based on the host operating system.
Signed-off-by: Jake Correnti <jakecorrenti+github@proton.me>
AppleHV accepts a max 36 bytes for mount tags. Instead of using the fully qualified path for the mount tag, SHA256 the path, and truncate the shasum to 36 bytes.
Also correctly escape dashes in mounted paths.
Signed-off-by: Ashley Cui <acui@redhat.com>
Cache cleanups only happen if there is a cache miss, and we need to pull a new image
For quay.io/podman/machine-os, we remove all old images from the cache dir. This means we will delete any file that exists in the cache dir; this should be safe to do since the machine pull code should be the only thing touching this cache dir. OCI machine images will always have a different manifest, and won’t be updated with the same manifest, so if the version moves on, there isn’t a reason to keep the old version in the cache, it really doesn’t change.
For Fedora (WSL), we use the cache, so we go through the cache dir and remove any old cached images, on a cache miss. We also switch to using ~/.local/share/containers/podman/machine/wsl/cache as the cache dir rather than ~/.local/share/containers/podman/machine/wsl. Both these behaviors existed in v4.9, but are now added back into 5.x.
For generic files pulled from a URL or a non-default OCI image, we shouldn’t actually cache, so we delete the pulled file immediately after creating a machine image. This restores the behavior from v4.9.
For generic files from a local path, the original file will never be cleaned up
Unsure how to test, so:
[NO NEW TESTS NEEDED]
Signed-off-by: Ashley Cui <acui@redhat.com>
This PR adds libkrun support to podman machine. This is an experimental feature and should not be marketed yet. Before we unmark the experimental status on this function, we will need to have full CI support and a full podman point release has pased.
This work relies on the fact that vfkit and libkrun share a reasonably (if not perfectly) same API. The --log-level debug option will not show a GUI screen for boots as krun is not capable of this.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Non-Linux systems, such as BSD kernels, constrain xatter updates
according to file permissions. This is in contrast to Linux selinux
attr writes, which are governed by an selinux policy. By dafault this
policy apllows users to relabel files owned by themselves even if file
perms would otherwise disallow write.
This results in robust container relabeling results on Linux, and
fragile results everywhere else. Therefore, change the mac policy to
force the nfs_t context on all files, and ignore all relabel
events.
As a side-effect, this will disallow any ability to store custom
selinux constants on files. However, this is of limited use in
a machine context, since files in these volumes are externally
managed on systems which do not support SELinux.
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
There has been various issues with vfkit exiting with
"Error: vfkit exited unexpectedly with exit code 1"
Among other reasons, this can be caused by vfkit being
built without the com.apple.security.virtualization
entitlement, and this can also happen when running
vfkit.x86_64 on Apple silicon hardware.
At the moment, the vfkit logs are not available, so there is no easy way
to know what's happening. This PR redirects vfkit stdout/stderr to
podman's log when --log-level debug is used.
Signed-off-by: Christophe Fergeau <cfergeau@redhat.com>
creating vsocks in windows requires admin privileges. there could be
some workarounds made in the future,but the general deal has
always been, you need to be admin. lets enforce this with an error
until those work-arounds can be implemented.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Windows is not guaranteed to have the SSH feature installed, so prefer the use
of the built-in ssh client for all operations other than podman machine ssh,
which requires terminal pty logic. This restores previous behavior in 4.x.
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
this pr fixes two hyperv bugs. previous podman 5 versions of hyperv
failed to actually remove the vm from hyperv when machine rm -f was
called.
also fixes an annoying bug where removal of the hyperv ignition entries
were failing because this can only be done (with the current api) when
the vm is running. new api in latest libhvee fixes this.
Signed-off-by: Brent Baude <bbaude@redhat.com>
I am really not sure why the caller even should have the option to set
this. We should always use the correct isolation type based on the
privileges the server runs under never the client. podman-remote build
seems to send the default based on its local privs which was wrong as
well. To fix this I also changed the client to send the default if the
isolation flag is not set.
Fixes#22109
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
We need to take another lock to prevent concurrent starts from different
machines.
I manually tested it by starting three VM in parallel with:
podman machine start & podman machine start test1 & podman machine start test2
I also added a CI test that seems to work as expected (failed with the
old binary, worked with the new)
Before this patch I was able to start more than VM, with this patch it
now only starts one of them and the other ones will fail to start with
a proper error.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
This function is not used, it has been refactored in the general
starting good higher up the stack.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Currently we first read the conf and then lock, this is racy because
while we wait for the lock another process might change the state so
the only way to have the actual current state is to read the file
while holding the lock.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
- Fixes conflicts such as removal of second machine deleting a socket of a
the first machine while it's running
- Move API socket into runtime directory for consistency
- Add API and gvproxy sockets to removal list
- Cleanup related logic
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
we are having second thoughts about *requiring* a policy.json on podman
machine hosts. we are concerned that we need to work out some more use
cases to be sure we do not make choices now that limit us in the near
term future. for example, should the policy files be the same for
container images and machine images? And should one live on the host
machine and the other live in the machine?
therefore, if a policy.json *is* present in the correct location, we will use and honor it; however, if it does not, we will allow the machine image to be pulled without a policy.
Signed-off-by: Brent Baude <baude@redhat.com>
Co-authored-by: Paul Holzinger <45212748+Luap99@users.noreply.github.com>
Signed-off-by: Brent Baude <bbaude@redhat.com>
1. Added the xz decompression unit tests
2. Removed the xz implementation to use the one from c/images
3. Removed the specific macos gzip, zstd compressor and use
the generic compressor but with SparseWriter if GOOS == darwin
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
Adding the final machine endpoint as quay.io/podman/machine-os in the
Podman code. As a reminder, we decided we would set this in containers
conf once things settle down and this code would then be removed.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Move the writes into the shim level to make sure they happen while we
hold the machine lock to prevent any race conditions reading/writing the
file.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
First make sure we check that a given VM exist when holding the VM lock
for it. The check in cmd/podman/machine/init.go is a nice quick out but
not enough to ensure that 2 processes to not create the same VM at the
same time. The only way to ensure this is by holding the lock and
checking if the VM config file exists.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Arthur Sengileyev
openshift-merge-bot[bot]
Ashley Cui
Mario Loriedo
Philippe Martin
Daniel J Walsh
Paul Holzinger
Nicola Sella
Brent Baude
Chris Evich
Anders F Björklund
Christophe Fergeau
Kevin Cui
Colin Walters
Sergio Lopez
Shion Tanaka
Nalin Dahyabhai
Gabriel Parreiras
Jake Correnti
Brent Baude
Giuseppe Scrivano
fanqiaojun
Jason T. Greene
Brent Baude
Miloslav Trmač