It turns out only counting the layers size is not sufficient for
`podman systemd df` as it excludes the size of the manifests, configs
and potentially attached meta data in the storage.
Instead, sum the image sizes but distract redundant layers. That indeed
gives the expected result and does not yield negative results for
reclaimable space.
Remove the unrelease LayersDiskUsage function again to avoid expensive
recalculation of the layer tree. We are still below 1.0, and I am
convinced the total image size belongs into DiskUsage.
NOTE: The DiskUsage function does not have test coverage in libimage.
This should be addressed at some point but in the interest of
time I leverage podman's system tests.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Used `go fmt` rules to migrate away from deprecated functions, for
instance `gofmt -w -s -r 'ioutil.TempDir(a, b) -> os.MkdirTemp(a, b)'`
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Add an API to query the sum of the layer sizes. This data is needed to
fix containers/podman/issues/16135.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Images runtime used the OCI archive transport with an empty system context,
so such environment variables from user like TMDDIR were ignored.
Signed-off-by: Mikhail Khachayants <tyler92@inbox.ru>
As of now NormalizePlatform ignores normalizing or setting default
variants for specific arch types thus producing empty variants in
results even when default variants were expected for such cases.
Example consider: `linux/armhf` -> `{os:linux, arch: arm, variant: v7}`
Signed-off-by: Aditya R <arajan@redhat.com>
When checking the platform of an image, take into account that it may be
corrupted.
Partially-fixes: containers/podman/issues/15853
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Wrap in a `nil` check to make sure that consumers not using events are
not bothered with log messages. It's probably worth moving the check
into the function but I do not want start Yak shaving in a quick fix.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Podman adds an Error: to every error message. So starting an error
message with "error" ends up being reported to the user as
Error: error ...
This patch removes the stutter.
Also ioutil.ReadFile errors report the Path, so wrapping the err message
with the path causes a stutter.
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
As reported in containers/podman/issues/15485, Docker sends untag events
prior to removing the image. Follow that example for compatibility
reasons.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Trying to remove an image that is in use by containers is a confusing
experience for users who may not be aware of "external" containers which
are not displayed in `podman ps` by default (see containers/podman/issues/15006).
Add some context to the error from containers/storage to guide the user
into listing external containers and force-removing the image.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
ManifestInspect should contain all known formats for a valid manifest
list as of now only supported formats are `OCIv1` and `Docker` so
inspect should support fields from `OCIv1` format as well. Following
commit adds a new field to inspect i.e `Annotations` from `OCIv1`.
Example output from podman
```console
podman manifest inspect test
{
"schemaVersion": 2,
"mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
"manifests": [
{
"mediaType": "application/vnd.docker.distribution.manifest.v2+json",
"size": 528,
"digest": "sha256:9b2a28eb47540823042a2ba401386845089bb7b62a9637d55816132c4c3c36eb",
"platform": {
"architecture": "amd64",
"os": "linux"
},
"annotations": {
"annotationTest1": "annotationTest2"
}
}
]
}
```
Closes: https://github.com/containers/podman/issues/15069
Signed-off-by: Aditya R <arajan@redhat.com>
there was some eerily similar code in c/common and in podman for
creating filter functions for various types. Move some of it here
and add support for the label!= filter in libnetwork only. Libimage already supports it.
after this merges, will file a PR with the fix for containers within podman as well
see containers/podman#14182
Signed-off-by: Charlie Doern <cdoern@redhat.com>
`podman manifest add` uses `ManifestList.Add(` but of now `Add(` does
not locks while adding instances to the list thus causing race scenarios
where storage is not reloaded and overrided by another invocation of the
command.
Following problem is solved in two steps
* Add -> LockByInstance: Acquire a fs lock by instance ID so other
invocation waits until this invocation completes its write.
* Add -> LockByInstance -> reload: Reload instance digests from storage
just after acquiring lock to make sure we are not overriding any just
written instance.
Reproducer: https://github.com/containers/podman/issues/14667#issue-1277034660
Closes: https://github.com/containers/podman/issues/14667
[NO NEW TESTS NEEDED]
[NO TESTS NEEDED]
This needes integration tests so its hard to verify race in CI.
Signed-off-by: Aditya R <arajan@redhat.com>
`github.com/pkg/errors` is deprecated since quite some time so we now
use the native error wrapping for more idiomatic golang.
Signed-off-by: Sascha Grunert <sgrunert@redhat.com>
Use containerd's platform package for platform checks. While the OCI
image spec requires the platform values to conform with GOOS and GOARCH
definitions of Go' runtime package, the values of uname are used by
convention. Supporting these values silences annoying false-positive
warnings.
Fixes: #containers/podman/issues/14669
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Do not check the platform of an image when it was looked up by it's ID.
In that case we must assume that the user/tool knows what they are
doing.
Further make the warnings configurable via a new `PlatformPolicy` field
in the `ImageLookupOptions`. By default, the error will only be printed
on the debug logs. User can opt-in to display the error on the warning
level. Not all code paths should warn. For instance, when inspecting
an image. This way, consumers of libimage can opt-in. The policy can
later on be extended to error out instead of logging.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Check the platform when looking up images locally. When the user
requested a custom platform and a local image doesn't match, the
image will be discarded. Otherwise a warning will be emitted.
Also refactor the code to make it more maintainable in the future.
Fixes: containers/podman/issues/12682
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Warn when the platform of a pulled image does not match the
user-specified platform. The checks are only performed if the user
requested a custom platform.
Do not error out and warn only since there are many images in the wild
that claim to be of another architecture. An error would break existing
workloads; we did that once and had to revert immediately.
Fixes: containers/podman/issues/14293
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
While in theory it would be optimal if the shortnames package returned
such an error, there is no need for it in practice since others error
sources such as an invalid input name would already error out before.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Remove the TODO item for VirtualSize. I had a look at Docker's source
where the VirtualSize also equals the Size. So there's nothing we can
or should do to remain compatible.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
TODO() may mistakenly indicate there's something we need to change in
the future which is not the case.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
To have tools using libimage be able to auto-complete the search filters
on the CLI, move the consts and vars to a new `libimage/define` package.
The new package prevents pulling in all the low-levels libraries.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Close the reader of an Docker archive to make sure that artifacts in
TMPDIR are removed.
Closes: github.com/containers/podman/issues/14287
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
gofumpt is more opinionated version of gofmt. One can use it from their
$EDITOR, or IDE, or gopls to better format the source code.
Previously, commit b951b72412 already formatted all the code with
gofumpt, but since then a couple of things crept in.
Fix these (with gofumpt v0.3.1) and enable the gofumpt linter.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Wherever possible slices should be allocated with the correct capacity to
avoid unnecessary memory allocations.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
unparam and exportloopref already work without changes.
For revive I had to silence many naming issues. I decided to silence them
instead of changing the name because I didn't want to break any code.
Signed-off-by: Paul Holzinger <pholzing@redhat.com>
if image_copy_tmp_dir is set in containers.conf it needs to be used in
the systemcontext for BigFilesTemporaryDir value.
Fixes: https://github.com/containers/podman/issues/14091
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
This one:
libimage/manifests/manifests.go:387:10: elseif: can replace 'else {if cond {}}' with 'else if cond {}' (gocritic)
} else {
^
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
gofumpt is a stricter version of gofmt, basically making the code more
readable, and fixing the gocritic's octalLiterar warnings like this one:
pkg/util/util_supported.go:26:17: octalLiteral: use new octal literal style, 0o722 (gocritic)
return (perm & 0722) == 0700
^
Generated by gofumpt -w .
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
> libimage/manifests/manifests.go:408:3: S1033: unnecessary guard around call to delete (gosimple)
> if _, needToDelete := l.instances[instanceDigest]; needToDelete {
> ^
Indeed, we can just call delete right away.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
Error checking is important. In these two cases, though, we don't have a
way to return an error, so make it explicit that we ignore the error.
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
`image-spec` already supports variant and if `ImportOptions` contains
any configured `Variant` it should be set on the imported image as well.
What does this fix:
Allows `podman import --variant <something> some.tar image-name` to
become functional from `no-op`
Ref: https://github.com/opencontainers/image-spec/blob/main/specs-go/v1/config.go#L93
Signed-off-by: Aditya R <arajan@redhat.com>
Add a field to `RemoveImages` that would ingore if a specified image
does not exist and not throw an error.
The intended use case is adding a `podman rmi --ignore` flag.
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
When looking up an image by a short name that prefixes another image's
ID, the one matching the short name should be returned.
This means that we need to do a final lookup in the storage with the
specified name (without normalization) to continue matching short IDs.
Since it's common that users of libimage (e.g., Buildah) internally
refer to images by full ID, let's make sure that we check for that
first. This way, we'll match full IDs on first lookup and keep the
expected performance.
Note that a name starting with `sha2556:` must be followed by a 64-byte
hex value; something we didn't check for before.
Fixes: containers/podman/issues/12761
Signed-off-by: Valentin Rothberg <vrothberg@redhat.com>
Valentin Rothberg