There is one user in the code and one in the tests.
First the test is testing an external function that doesn't seem to make
a lot of sense.
Second when we pull the machine image and renmae doesn't work it would
mean it is on a different disk. That should never happen.
The oci pull is to
~/.local/share/containers/podman/machine/<provider>/cache/<sha>/...
and then it gets moved to
~/.local/share/containers/podman/machine/qemu/cache/<sha>.<extension>
These can under normal circumstances never be on different disk as it
would mean the pulled sha is known ahead of time and then that the blob
path is a mount point which is unrealistic and not something we have to
support.
It also removes three transitive dependencies.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
Fix new issues found by usetesting, mainly we should use t.TempDir() in
test which makes the code better as this will be removed on test end
automatically so no need for defer or any error checking.
Also fix issues reported by exptostd, these mainly show where we can
switch the imports to the std maps/slices packages instead of the
golang.org/x/exp/... packages.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
When the file is empty it is possible our code panics as bar.ProxyReader
returns nil when the bar is finished which is the case for 0 size as it
doesn't have to read anything from there. However as this happens on
different goroutines it is race and most of the time still works.
To fix this simply skip the progress bar setup for empty files.
While at it fix the deprecated argument in the tests.
Fixes#23281
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
1. Added the xz decompression unit tests
2. Removed the xz implementation to use the one from c/images
3. Removed the specific macos gzip, zstd compressor and use
the generic compressor but with SparseWriter if GOOS == darwin
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
Neither of the SparseWriter users actually _wants_ the underlying
WriteSeeker to be closed; so, don't.
That makes it clear where the responsibility for closing the file
lies, and allows us to remove the reliance on the destinations
reliably returning ErrClosed.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
While working on #21592 we figured out that the
the full VM File was loaded in memory when detecting
the file format, but only a few bytes are needed.
This commit address that.
[NO NEW TESTS NEEDED]
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
Added some tests to verify that files extractions works
with different compression format.
Created a decompressor interface with 2 main methods:
reader(): returns an io.Reader for the specific compression algorithm
copy(): extracts the compressed file into the file provided as param
Created 5 decompressor types:
- gzip: extract gzip files
- xz: extract xz files
- zip: extract zip files
- generic: extract any other file using github.com/containers/image/v5/pkg/compression
- uncompressed: only do a copy of the file
Minor fix to the progress bar instances: added a call to bar.Abort(false)
that happens before Progress.Wait() to avoid that it hangs when a bar is
not set as completed although extraction is done.
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
... instead of using a multi-variable state machine.
The net effect of this code is exactly the same as the previous implementation,
except:
- the operation after Write() returns an error might differ
- If the file ends with zeroes, we don't Seek(-1), and
we don't create a hole at all if it is too small, preferring
to save a syscall.
But this formulation is hopefully easier to prove correct.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
- Add more test cases
- Test that we create the expected (large) holes;
don't enforce anything for the <zerosThresholt ones.
O)nly changes test code, should not change behavior.
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
given that we are moving to building our own machine images, we have
decided to use zstd compression as it is superior in speed to the
alternatives. as such, this pr adds zstd to our machine code; and also
has to account for dealing with sparseness on darwin; which the default
zstd golang library does not.
[NO NEW TESTS NEEDED]
Signed-off-by: Brent Baude <bbaude@redhat.com>
Also addresses a number of issues:
- StopHostNetworking isn't plumbed, win-sshproxy leaks on hyperv
- Wait api and print output doesn't work properly on Windows
- API forwarding doesn't work on WSL
- Terminal corruption with after start/stop on Windows
- Gvproxy is forcefully killed vs gracefully quit
- Switching rootful/rootless does not update /var/run/docker.sock on the guest
- File already closed error on init
- HyperV backend is publishing Unix sockets when it should be named pipes
- User-mode networking doesn't always work
- Stop state outside of lock boundaries
- WSL blocks parallel machined (should be supported)
[NO NEW TESTS NEEDED]
Signed-off-by: Jason T. Greene <jason.greene@redhat.com>
Added the module github.com/crc-org/crc/ as a dependency.
Updated `decompress.go` and `copy_test.go` in compression
so that `CopySparse` from crc-org/crc/v2/pkg/os is used instead
of the local version in `copy.go`.
Deleted `copy.go` that is not used anymore.
Signed-off-by: Mario Loriedo <mario.loriedo@gmail.com>
Moving from Go module v4 to v5 prepares us for public releases.
Move done using gomove [1] as with the v3 and v4 moves.
[1] https://github.com/KSubedi/gomove
Signed-off-by: Matt Heon <mheon@redhat.com>
gz by definition is not able to preserve the sparse nature of files. using some code from the crc project and gluing it together with our decompression code, we can re-create the sparseness of a file. one downside is the operation is a little bit slower, but i think the gains from the sparse file are well worth it in IO alone.
there are a number of todo's in this PR that would be ripe for quick hitting fixes.
[NO NEW TESTS NEEDED]
Signed-off-by: Brent Baude <baude@redhat.com>
In #20538, I was asked to consider refactoring the new OCI pull code
from within the generic machine directory. This is something I had
tried when originally coding it but it became apparent that a much
larger refactor to prevent circular deps was needed. Because I did not
want to pollute the initial PR with that refactor, I asked for the PR to
merge first. This is the refactor that needed to be done.
Signed-off-by: Brent Baude <bbaude@redhat.com>
Paul Holzinger
Mario Loriedo
openshift-merge-bot[bot]
Miloslav Trmač
Brent Baude
Giuseppe Scrivano
Jason T. Greene
Matt Heon
Brent Baude