As reported in #11294, the Docker daemon will execute contains it
shouldn't run in the event that a requested tag is present in an image's
ID. This leads to the wrong image being started up silently.
This change reduces the risk of such a collision by using the short ID
iff the actual revOrTag looks like a short ID (not that it necessarily
is).
Signed-off-by: Richard Burnison <rburnison@ebay.com>
This makes `registry.Service` a first class type and does not use jobs
to interact with this type.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Update registry package to use the v2 registry api from distribution. Update interfaces to directly take in digests.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When buffering to file add support for compressing the tar contents. Since digest should be computed while writing buffer, include digest creation during buffer.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
This patch causes `The image you are pulling has been verified` status
message to be produced also when the repository is pulled for the first
time.
Signed-off-by: Michal Minar <miminar@redhat.com>
When pushing to the official registry, v2 push will be attempted first. If the v2 endpoint is unavailable, a push to v1 will occur. If a v2 push is started and failed, the push operation will fail. Pulling non-official images the official v2 registry is also enabled to pull the newly push images. When a pull of a v2 image fails, v1 will always be attempted.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Add ability to refer to an image by repository name and digest using the
format repository@digest. Works for pull, push, run, build, and rmi.
Signed-off-by: Andy Goldstein <agoldste@redhat.com>
Update pull code to consider any layer download or new tag as an update.
Update hello-world frozen image to be explicitly tagged as frozen, to not interfere with pull tests. The hello-world is used by pull tests because of its small size and there is no other official image with such a size.
fixes#11383
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Manifest is now generated during a v2 push, not relying on previously generated hashes. When pushing a layer, the hash is directly calculated from the tar contents which will be pushed. Computing the hash on push ensures that the hash contents always match what is seen by the registry. This also mitigates issues with tarsum differences and permits using pure SHA digests.
Additionally the new manifest function is moved to the unit tests since it is no longer called outside the tests.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The call to io.Copy was not being checked for errors at a critical point in the
pull logic. This patch will log the error value if not nil.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
pushV2Image() calls TarLayer() which returns an archive. One needs to
Close() the archive once done otherwise it will leave mounted devices
if devicemapper graph driver is being used.
Signed-off-by: Derek McGowan <derek@mcgstyle.net>
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
manifest code calls TarLayer() and gets archive. This archive needs to
be closed once caller is done using it to release the resrouces held
by archive. For the devicemapper graphdriver, archive keeps a device
mounted (device which is backing the layer). If archive.Close() is not
called, that device remains mounted and later deletion of device fails
leading to various other issues.
Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
The trust graph currently pulls the keys needed to verify official images. For non-official images this graph check is useless. To avoid hitting the key statement url when pulling from private v2 registries, restrict the check to official image pulls.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Pushing by image ID is not allowed in the Docker CLI and not supported by the registry. An unnamed image also cannot be pushed to a private registry, since no endpoint is specified and it will default to the hub. The hub also does not support this use case, therefore removing the code path is the best solution.
The ability to push a layer without a name is unsupported by the v2 registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Only show the verification message if all the tarsum checks pass and the image manifest is verified.
No longer return an error when a tarsum verification fails, just reset the verification flag.
Tarsum verification is less meaningful without a verified manifest and therefore it should not cause an error.
Updated the verified image test to pull an image which expected to have a verified manifest and contents.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
It didn't work on v2 anyways. And an image with a lot of aliases was slow to
fetch.
Docker-DCO-1.1-Signed-off-by: Jessie Frazelle <princess@docker.com> (github: jfrazelle)
On pull treat an unavailable v2 registry as a non-error fallback.
On push only show v2 error message in debug mode.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
No longer push to the official v2 registry when it is available. This allows pulling images from the v2 registry without defaulting push. Only pull official images from the v2 official registry.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
When requesting a token, the basic auth header is always being set even
if there is no username value. This patch corrects this and does not set
the basic auth header if the username is empty.
Also fixes an issue where pulling all tags from a v2 registry succeeds
when the image does not actually exist on the registry.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The checksum is now being stored in a separate file beside the image
JSON file.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Before the V2 registry changes, images with no config could be pushed.
This change fixes a regression that made those images not able to be
pushed to a registry.
Signed-off-by: Euan Kemp <euank@euank.com>
When progress reader closes it overwrites the progress line with the full progress bar, replaces the completed message.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
The v2 session code will no longer update the indexEndpoint value, therefore it is not necessary to save and restore the value for use with v1.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
While the v2 pull operation is writing the body of the layer blob to disk
it now computes the tarsum checksum of the archive before extracting it to
the backend storage driver. If the checksum does not match that from the
image manifest an error is raised.
Also adds more debug logging to the pull operation and fixes existing test
cases which were failing. Adds a reverse lookup constructor to the tarsum
package so that you can get a tarsum object using a checksum label.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This refactors the starting work by the prior commits to make this safe
for access. A maximum of 5 worker go routines are started to lookup
images on the endpoint. Another go routine consumes the images that are
required to be pushed into a map for quick lookups. The map is required
because the pushing of the image json and layer have to be done in the
correct order or the registry will explode in fire.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Passing RepositoryInfo to ResolveAuthConfig, pullRepository, and pushRepository
Moving --registry-mirror configuration to registry config
Created resolve_repository job
Repo names with 'index.docker.io' or 'docker.io' are now synonymous with omitting an index name.
Adding test for RepositoryInfo
Adding tests for opts.StringSetOpts and registry.ValidateMirror
Fixing search term use of repoInfo
Adding integration tests for registry mirror configuration
Normalizing LookupImage image name to match LocalName parsing rules
Normalizing repository LocalName to avoid multiple references to an official image
Removing errorOut use in tests
Removing TODO comment
gofmt changes
golint comments cleanup. renaming RegistryOptions => registry.Options, and RegistryServiceConfig => registry.ServiceConfig
Splitting out builtins.Registry and registry.NewService calls
Stray whitespace cleanup
Moving integration tests for Mirrors and InsecureRegistries into TestNewIndexInfo unit test
Factoring out ValidateRepositoryName from NewRepositoryInfo
Removing unused IndexServerURL
Allowing json marshaling of ServiceConfig. Exposing ServiceConfig in /info
Switching to CamelCase for json marshaling
PR cleanup; removing 'Is' prefix from boolean members. Removing unneeded json tags.
Removing non-cleanup related fix for 'localhost:[port]' in splitReposName
Merge fixes for gh9735
Fixing integration test
Reapplying #9754
Adding comment on config.IndexConfigs use from isSecureIndex
Remove unused error return value from isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
Adding back comment in isSecureIndex
Signed-off-by: Don Kjer <don.kjer@gmail.com>
If .dockerignore mentions either then the client will send them to the
daemon but the daemon will erase them after the Dockerfile has been parsed
to simulate them never being sent in the first place.
an events test kept failing for me so I tried to fix that too
Closes#8330
Signed-off-by: Doug Davis <dug@us.ibm.com>
There has been a lot of discussion (issues 4242 and 5262) about making
`FROM scratch` either a special case or making `FROM` optional, implying
starting from an empty file system.
This patch makes the build command `FROM scratch` special cased from now on
and if used does not pull/set the the initial layer of the build to the ancient
image ID (511136ea..) but instead marks the build as having no base image. The
next command in the dockerfile will create an image with a parent image ID of "".
This means every image ever can now use one fewer layer!
This also makes the image name `scratch` a reserved name by the TagStore. You
will not be able to tag an image with this name from now on. If any users
currently have an image tagged as `scratch`, they will still be able to use that
image, but will not be able to tag a new image with that name.
Goodbye '511136ea3c5a64f264b78b5433614aec563103b4d4702f3ba7d4d2698e22c158',
it was nice knowing you.
Fixes#4242
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
This commit is patch for following comment
// TODO: This method should return the errors instead of masking them and returning false
Signed-off-by: Daehyeok Mun <daehyeok@gmail.com>
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
When we use the engine/env object we can run into a situation where
a string is passed in as the value but later on when we json serialize
the name/value pairs, because the string is made up of just numbers
it appears as an integer and not a string - meaning no quotes. This
can cause parsing issues for clients.
I tried to find all spots where we call env.Set() and the type of the
name being set might end up having a value that could look like an int
(like author). In those cases I switched it to use env.SetJson() instead
because that will wrap it in quotes.
One interesting thing to note about the testcase that I modified is that
the escaped quotes should have been there all along and we were incorrectly
letting it thru. If you look at the metadata stored for that resource you
can see the quotes were escaped and we lost them during the serialization
steps because of the env.Set() stuff. The use of env is probably not the
best way to do all of this.
Closes: #9602
Signed-off-by: Doug Davis <dug@us.ibm.com>
I was confused earlier when I did:
```
docker push localhost.localdomain:1234/foo
```
Because docker told me:
```
No such id: localhost.localdomain:1234/foo
```
I actually had buried in my mind the solution to this, but the error
message
confused me because I had recently had some fun trying to get the
registry
working and therefore thought it was telling me that I didn't have an
account
on the registry.
This pull request makes it unambiguous that the error is that the
specified
image is unknown.
/cc @cpuguy83
Docker-DCO-1.1-Signed-off-by: Peter Waller <p@pwaller.net> (github: pwaller)
Therer is a bug in the 'skip' decision when exporting a repository
(`docker save repo`)
Only the layers of the first image are included in the archive (the
layers of the next images are missing)
Signed-off-by: Anthony Baire <Anthony.Baire@irisa.fr>
Now, newly created/imported layers will have the checksum of
the layer diff computed and stored in the image json file.
For now, it is not an error if the computed checksum does not
match an existing checksum, only a warning message is logged. The
eventual goal is to use the checksums in the image JSON to verify
the integrity of the layer contents when doing `docker load` or
`docker pull`, and error out if it does not match.
Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
The argument specified the json data to save to disk when registering
a new image into the image graph. If it is nil, then the given image
is serialized to json and that is written by default. This default
behavior is sufficient if the given image was originally deserialzed
from this jsonData to begin with which has always been the case.
Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
Since V2 registry does not yet implement mirrors, when mirrors are given automatically fallback to V1 without checking V2 first.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Fix issue with restoring the tag store and setting static configuration
from the daemon. i.e. the field on the TagStore struct must be made
internal or the json.Unmarshal in restore will overwrite the insecure
registries to be an empty struct.
Signed-off-by: Michael Crosby <michael@docker.com>
Conflicts:
graph/pull.go
graph/push.go
graph/tags.go
As this feature requires more testing it is much safter to get the
underlying changes into the codebase first then enable the feature in
another release after proper testing and verification can be done.
Signed-off-by: Michael Crosby <crosbymichael@gmail.com>
Add support for pulling signed images from a version 2 registry.
Only official images within the library namespace will be pull from the
new registry and check the build signature.
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
Using repo tag in the status message for better usability, as per review comments
Added documentation and Changed code to print Status after downloads are complete
Addresses #2404
Signed-off-by: Srini Brahmaroutu <srbrahma@us.ibm.com>
Now that the archive package does not depend on any docker-specific
packages, only those in pkg and vendor, it can be safely moved into pkg.
Signed-off-by: Rafe Colton <rafael.colton@gmail.com>
When user passes an invalid tag to `docker build`
(i.e. `docker build -t abcd:A0123456789B0123456789C0123456789 .`), check the
tag first and terminate-early so user can specify the tag again
Docker-DCO-1.1-Signed-off-by: Daniel, Dao Quang Minh <dqminh89@gmail.com> (github: dqminh)
Currently inspect output just shows Size info, and that usally be
very small and even 0 which is confusing.
Fixes#8016
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
Some graphdrivers are Differs and type assertions are made
in various places throughout the project. Differ offers some
convenience in generating/applying diffs of filesystem layers
but for most graphdrivers another code path is taken.
This patch brings all of the logic related to filesystem
diffs in one place, and simplifies the implementation of some
common types like Image, Daemon, and Container.
Signed-off-by: Josh Hawn <josh.hawn@docker.com>
Adds support for a --registry-mirror=scheme://<host>[:port]
daemon flag. The flag may be present multiple times. If
provided, mirrors are prepended to the list of endpoints used
for image pull. Note that only mirrors of the public
index.docker.io registry are supported, and image/tag resolution
is still performed via the official index.
Docker-DCO-1.1-Signed-off-by: Tim Smith <timbot@google.com> (github: timbot)
Now from a single invocation of `docker save`, you can specify multiple
images to include in the output tar, or even just multiple tags of a
particular image/repo.
```
> docker save -o bundle.tar busybox ubuntu:lucid ubuntu:saucy fedora:latest
> tar tf ./bundle.tar | wc -l
42
> tar xOf ./bundle.tar repositories
{"busybox":{"latest":"2d8e5b282c81244037eb15b2068e1c46319c1a42b80493acb128da24b2090739"},"fedora":{"latest":"58394af373423902a1b97f209a31e3777932d9321ef10e64feaaa7b4df609cf9"},"ubuntu":{"lucid":"9cc9ea5ea540116b89e41898dd30858107c1175260fb7ff50322b34704092232","saucy":"9f676bd305a43a931a8d98b13e5840ffbebcd908370765373315926024c7c35e"}}
```
Further, this fixes the bug where the `repositories` file is not created
when saving a specific tag of an image (e.g. ubuntu:latest)
document multi-image save and updated API docs
Docker-DCO-1.1-Signed-off-by: Vincent Batts <vbatts@redhat.com> (github: vbatts)
Graph.Get function already returns nil when the image doesn't exist so
following comment is obsolete.
FIXME: return nil when the image doesn't exist, instead of an error
Signed-off-by: Daehyeok.Mun <daehyeok@gmail.com>
renaming this struct to more clearly be session, as that is what it
handles.
Splitting out files for easier readability.
Signed-off-by: Vincent Batts <vbatts@redhat.com>
This is part of the ongoing effort to remove the deprecated server/
package, and generally cleanup and simplify the codebase.
Signed-off-by: Solomon Hykes <solomon@docker.com>
- Directories should have u+x (0600 -> 0700)
- Don't chown files to root because we don't have to
Docker-DCO-1.1-Signed-off-by: Kato Kazuyoshi <kato.kazuyoshi@gmail.com> (github: kzys)
functions to pkg/parsers/kernel, and parsing filters to
pkg/parsers/filter. Adjust imports and package references.
Docker-DCO-1.1-Signed-off-by: Erik Hollensbe <github@hollensbe.org> (github: erikh)
Alexander Morozov
Brian Goff
Alexander Morozov
Antonio Murdaca
Srini Brahmaroutu
David Calavera
Doug Davis
Simei He
Ma Shimiao
Tibor Vass
Richard Burnison
Hu Keping
David Young
Doug Davis
Phil Estes
Shishir Mahajan
José Tomás Albornoz
Michael Crosby
Derek McGowan
Lei Jitang
Peter Choi
Michal Minar
bobby abbott
Josh Hawn
Andy Goldstein
Arnaud Porterie
Jessie Frazelle
Darren Shepherd
Dan Walsh
Srini Brahmaroutu
Doug MacEachern
Steve Francia
Michael Crosby
Vivek Goyal
Jessica Frazelle
Tibor Vass
guoxiuyan
Euan
Stephen J Day
Vincent Batts
Ian Babrou
Victor Vieux
Ian Calvert
Don Kjer
Daehyeok Mun
unclejack
Peter Waller
Anthony Baire
Michael Crosby
Burke Libbey
Vincent Batts
Rafe Colton
Victor Vieux
Daniel, Dao Quang Minh
Qiang Huang
Tim Smith
Josiah Kiehl
unclejack
Satoshi Amemiya
Solomon Hykes
Kato Kazuyoshi
Erik Hollensbe