Commit Graph

1245 Commits

Author SHA1 Message Date
Miloslav Trmač cbb7c078af Don't use reference.TrimNamed().String()
reference.TrimNamed(r).String() is equivalent to r.Name(), just more expensive.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač a758135b2c Simplify verboseName now that isNamed is always true
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač 5cb0e19782 Make verboseName accept only a non-nil reference.Named
All existing users pass a reference.Named, and check for nil before
calling this function.

This only changes the interface; the implementation will be simplified
presently.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač 88567cf09e Add a test for verboseName
Before touching the internals of reference representation, make sure
the expected values of verboseName are known so that we don't
break any users by touching the internals.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač 04e1018848 Fix ValidatePolicyConfigurationScope
Re-enable the failing tests, and rewrite the implementation to match expected inputs.

Notably this re-enables the @ID form with no name.

Like docker.Transport, this does no validation of the name part; using
the docker/reference parses is not correct because they are not intended to
accept e.g. hostname-only "docker.io"

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač b8969b7aba Rewrite TestTransportValidatePolicyConfigurationScope
For some unfanthomable reason I wrote the test to accept non-canonical
inputs to ParseReference, instead of the canonicalized prefixes returned by
PolicyConfigurationIdentity and PolicyConfigurationNamespace.

So, replace the expected cases by the actually returned forms, up to
the full name:tag@digest@ID, and the relevant prefixes.

This only changes the tests; the cases which would fail are commented out
for now.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač 225e91a1ff Re-add the :tag or @digest form at start of PolicyConfigurationNamespaces
... if PolicyConfigurationIdentity includes @id.  At least the :tag form is clearly useful.

This does not handle the name:tag@digest case (which _can_ happen), where the tag is
currently recorded inside s.name but not s.tag; the possible code handling it would
be very non-obviously pointing out this difference.  For now, only leave a FIXME in the test.
Maybe it should be handled by refusing such input instead; e.g. that's what
docker.Transport does.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač cce388188e Add more test cases, along with a few FIXMEs
This does not change the implementation at all yet.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač c395f15bc8 Make TestStorageReferenceDockerDeference table-driven
... using validReferenceTestCases.

The cases in that table are a superset of the two previously used cases.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-06-04 16:50:17 +02:00
Miloslav Trmač 7112396df8
Merge pull request #455 from agonzalezro/wrap-nil
errors.Wrapf doesn't return error when err is nil
2018-06-02 00:24:34 +02:00
Álex González 8dd79ce245 errors.Wrapf doesn't return error when err is nil
In this commit you can see how at the beginning of the function
`getManifestDescriptor` it was being checked for an error was different than
nil and returning in that case, however, few lines later that error (remember,
with nil value) was being wrapped. Since the original error was nil, the
wrapping was as well returning a nil.

A regression tests was added.

This commit should close the following issue in skopeo as per @mtrmac
suggestion: github.com/projectatomic/skopeo/issues/496

Signed-off-by: Álex González <agonzalezro@gmail.com>
2018-06-01 15:58:30 +02:00
Antonio Murdaca 74806b702b
Merge pull request #462 from rhatdan/spelling
Fix spelling mistakes in Makefile
2018-06-01 15:24:28 +02:00
Daniel J Walsh 6ba33e8cbd Fix spelling mistakes in Makefile
Signed-off-by: Jonathan Boulle <jonathanboulle@gmail.com>
Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-06-01 09:12:16 -04:00
Miloslav Trmač a6bfd83990
Merge pull request #461 from giuseppe/pinpoint-ostree-thread
ostree: use the same thread for ostree operations
2018-05-30 18:14:38 +02:00
Giuseppe Scrivano a60973bec9
ostree: use the same thread for ostree operations
Since https://github.com/ostreedev/ostree/pull/1555, locking is
enabled by default in OSTree.  Unfortunately it uses thread-private
data and it breaks the Golang bindings.  Force the same thread for the
write operations to the OSTree repository.

Signed-off-by: Giuseppe Scrivano <gscrivan@redhat.com>
2018-05-30 17:11:16 +02:00
Miloslav Trmač ad33f7b73f
Merge pull request #433 from rhatdan/registries.conf
Add functions to return Registries paths, tools can use them to help users debug
2018-05-26 06:18:19 +02:00
Miloslav Trmač c89900df25
Merge pull request #446 from mgoltzsche/docker-dependency-update
Updated docker dependencies.
2018-05-26 05:38:51 +02:00
Daniel J Walsh 305627e9be Add functions to return Registries paths
Tools can use them to help users debug

We are seeing lots of issues with people taking podman and buildah and just
building them and installing them without greating registries.conf file.

They end up doing

buildah from fedora
or
podman from fedora

And we report that fedora does not exist.

I would like to make podman/buildah smarter so that they could state something like
podman from fedora
fedora does not exist.  Your /etc/containers/registries.conf and /etc/contaienrs/registries.d directory are empty.

Signed-off-by: Daniel J Walsh <dwalsh@redhat.com>
2018-05-23 04:58:15 -04:00
Max Goltzsche a1116a8ee4 Updated docker dependencies.
Closes #445. Added containerd/continuity dep since it is required by docker code that is required by this library.

Signed-off-by: Max Goltzsche <max.goltzsche@gmail.com>
2018-05-22 23:57:48 +02:00
Miloslav Trmač 81bf68ddd4
Merge pull request #459 from mtrmac/c-storage-dep
Add github.com/syndtr/gocapability to vendor.conf
2018-05-22 19:14:23 +02:00
Miloslav Trmač e23bcd87dc Add github.com/syndtr/gocapability to vendor.conf
This is a newly required dependency of containers/storage.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-05-22 18:40:56 +02:00
Miloslav Trmač 3143027065
Merge pull request #458 from umohnani8/transport
Allow image to be saved without a reference name
2018-05-15 21:13:26 +02:00
umohnani8 a5264ad5c4 Allow image to be saved without a reference name
The docker-archive and oci-archive transport should allow the
destination of the image to be valid without the reference part also.
Format is transport:path[:reference] where reference is optional.
This is for the case where a user just wants to save/push an image with
the image ID only and not the name.
Creates archives with empty repotags.

Signed-off-by: umohnani8 <umohnani@redhat.com>
2018-05-15 12:27:42 -04:00
Miloslav Trmač 90bd556419
Merge pull request #454 from vrothberg/https-fallback
sysregistriesv2: fallback to https for v1 backwards compat
2018-05-14 17:07:07 +02:00
Valentin Rothberg 3ee1a4d36d sysregistriesv2: fallback to https for v1 backwards compat
To be backwards compatible with the v1 config format, fallback to using
https when no URI scheme is specified (e.g., "docker.io").

Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
2018-05-14 08:22:36 +02:00
Miloslav Trmač db00e23581
Merge pull request #457 from mtrmac/docker-err-ignored
Clarify that err != nil && !os.IsNotExist(err) does not need to be handled
2018-05-12 04:13:25 +02:00
Miloslav Trmač e4bce45ad4 Clarify that err != nil && !os.IsNotExist(err) does not need to be handled
... and that this is not an oversight, per the conversation in
https://github.com/containers/image/pull/453 .

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-05-12 03:32:31 +02:00
Miloslav Trmač a1a9391830
Merge pull request #456 from nlewo/pr-tempDir
tmpdir: unixTempDirForBigFiles can be overridden at build time
2018-05-07 19:35:40 +02:00
Antoine Eiche 907702eea8 tmpdir: unixTempDirForBigFiles can be overridden at build time
This commit exposes a variable `unixTempDirForBigFiles` to allow some
Linux distributions to use `/tmp`.
On NixOS for instance, `/tmp` is not a tmpfs and Nix sandboxed builds
do not allow to write to `/var/tmp`, only to `/tmp`.

Note the `tmpdir.TemporaryDirectoryForBigFiles()` is now also used in
the `storage` package to create temporary directories.

Signed-off-by: Antoine Eiche <lewo@abesis.fr>
2018-05-07 18:23:47 +02:00
Miloslav Trmač d8718b1c5a
Merge pull request #450 from vrothberg/multitags
docker-archive: multiple tag support
2018-05-07 16:39:11 +02:00
Valentin Rothberg d03510da5b
Merge branch 'master' into multitags 2018-05-02 08:47:04 +02:00
Miloslav Trmač a74645804c
Merge pull request #301 from mtrmac/daemon-identity
Implement policy configuration scopes in docker-daemon:
2018-05-01 22:36:17 +02:00
Miloslav Trmač 13f5a40990 Implement policy configuration scopes in docker-daemon:
This is necessary for verification / policy enforcement of in-daemon
images.

It is awkward in that an image may have no name and be usable only by an
ID; such IDs could have a policy identity but they can’t really be
namespaced.

For now, implement policy configuration scopes only for named image
references; ID-only references use the root scope.  This allows users to
make ID-only images untrusted or to reject them, forcing users to use
image names if they want the policy to approve.  This gives a fairly
natural semantics, equivalent to docker: policies.  ID-like policy
configuration scopes are forbidden.  If truly necessary, we can add
support for single-ID policy IDs into the policy in the future, but
that’s unlikely to be too useful—IDs change over time, so such a policy
would not be likely to be persistent; and a single-use policy built for
a single image can just as well use the universal "" scope.

(Note that docker-daemon: and docker: namespaces are still separate in
the policy—images from any source can be loaded into the daemon via
(docker load) / copy to docker-daemon:, so there is in general no
expectation that the two namespaces are equal.  Though a
special-purpose tool may well want to create an in-memory policy by
loading the system-wide one and grafting the docker:
PolicyTransportScopes map to docker-daemon: as well.)

[In many cases an image will have an unique RepoTags/RepoDigests value
which can be used to get _a_ name from the ID, probably the right guess.
For now we keep the ImageReference implementation dumb, though
inspecting the image by ID and giving the reference a tagged/digested
name is at least plausible in principle.  Special-purpose tools
can script this without having to wait for daemonReference to add this.]

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-05-01 22:07:18 +02:00
Miloslav Trmač 8a78b63e5d
Merge pull request #441 from novas0x2a/follow-tags
tags is a paginated api, need to follow Link headers
2018-05-01 22:03:47 +02:00
Valentin Rothberg 20f59851be
Merge branch 'master' into multitags 2018-05-01 14:27:02 +02:00
Mike Lundy 07a0a49a3e rework docker GetRepositoryTags
1) docker tags api is paginated, so follow Link headers
2) an ImageSource (and thus a docker.Image) requires a tag to exist, so
   this formerly could not operate on repos without a `latest` tag. This
   reworks the function as a package function that takes an
   ImageReference instead of a source, and uses the tag for policy
   enforcement (but doesn't get upset if the tag is not provided, since
   it defaults to "latest").
3) a backwards-compat function is provided that matches the old
   function.

Signed-off-by: Mike Lundy <mike@fluffypenguin.org>
2018-04-30 16:10:00 -07:00
Miloslav Trmač 88423e35d5
Merge pull request #451 from umohnani8/auth
Don't fail if XDG_RUNTIME_DIR is set to a non-existent path
2018-04-30 20:11:39 +02:00
umohnani8 fb7ec3915d Don't fail if XDG_RUNTIME_DIR is set to a non-existent path
If the image is a public image, credentials are not needed to pull it,
try pulling the image even if XDG_RUNTIME_DIR is set to a non-existent path.

Signed-off-by: umohnani8 <umohnani@redhat.com>
2018-04-30 13:07:39 -04:00
Antonio Murdaca d4e2aadc9e
Merge pull request #429 from vrothberg/mirror-support
introduce sysregistriesv2
2018-04-30 14:32:38 +02:00
Valentin Rothberg 633422b620 introduce sysregistriesv2
Introduce sysregistriesv2, which changes the format of the
`registries.conf` TOML configuration.  Instead of having different lists
to specify search registries, blocked and insecure ones, all data is
encapsulated into one registry type.  The registry type allows to
specify a list of mirrors, which can be used in the endpoint lookup to
serve, for instance, as pull through caches for the associated registry.

An example configuration may look as follows:

```toml
[[registry]]
url = "https://registry.com"
prefix = "another-registry.com"

[[registry.mirror]]
url = "http://registry-mirror.com"
insecure = true
```

The upper example shows the configuration for `https://registry.com`.
The prefix is used for matching images, and to translate one namespace
to another.  If `prefix="example.com/bar"`, `url="https://example.com/foo/bar"`
and we pull from `example.com/bar/myimage:latest`, the image can
effectively be pulled from `example.com/foo/bar/myimage:latest`. If no
prefix is specified, it defaults to the specified URL.

Ease migration from sysregistries v1 to v2 by also loading
configurations in the v1 TOML format.  Throw an error in case a config
tries to mix both formats.  This allows a smoother migration for
developers, maintainers and the user, who can switch to the new config
format once all tools have been updated to v2.

Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
2018-04-30 07:55:24 +02:00
Valentin Rothberg 739fbe0b9a docker-archive: multiple tag support
Add support to create a docker-archive with more than one RepoTag,
enabling users such as skopeo, to create or copy multitag archives.
Support for other transports can be added in the future.

Fixes: https://github.com/containers/image/issues/447
Signed-off-by: Valentin Rothberg <vrothberg@suse.com>
2018-04-27 15:54:25 +02:00
Miloslav Trmač e51d350816
Merge pull request #444 from agonzalezro/remove-deps-from-vendor
Remove unused dependencies from vendor.conf
2018-04-18 21:46:39 +02:00
Álex González 392c1d34fe Remove unused dependencies from vendor.conf
Signed-off-by: Álex González <agonzalezro@gmail.com>
2018-04-14 19:11:58 +02:00
Miloslav Trmač 25826a7965
Merge pull request #439 from umohnani8/transport
Add more information to error returned for oci
2018-04-11 20:52:14 +02:00
umohnani8 4895b93da0 Add more information to error returned for oci
Add what the syntax of the oci and oci-archive transport should be if an error
is returned when preparing the destination.

Signed-off-by: umohnani8 <umohnani@redhat.com>
2018-04-11 09:30:26 -04:00
Miloslav Trmač 490d44adfc
Merge pull request #440 from mtrmac/context-cleanups
context.Context cleanups
2018-04-10 19:49:43 +02:00
Miloslav Trmač 8968eb0bc3 Remove a few unused context.context parameters from private functions
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-04-10 19:12:04 +02:00
Miloslav Trmač 2f122a2760 Add TODO notes for places which do a lot of work on the local filesystem
It would be nice if those could be canceled, although so far we seem not
to have any users which would benefit.

Many would be easily handled with a cancelable variant of io.Copy,
but a few might be much more involved.

Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-04-10 19:12:04 +02:00
Miloslav Trmač ec9050f766 Fix the example in doc.go
Signed-off-by: Miloslav Trmač <mitr@redhat.com>
2018-04-10 19:12:04 +02:00
Antonio Murdaca e5b7a01f77
Merge pull request #431 from novas0x2a/context-everywhere
Put context.Context arguments on almost everything
2018-04-10 19:04:18 +02:00