This fix tries to fix the issue in #21848 where `docker stats` will not correctly
display the container stats in case the container reuse another container's
network stack.
The issue is that when `stats` is performed, the daemon will check for container
network setting's `SandboxID`. Unfortunately, for containers that reuse another
container's network stack (`NetworkMode.IsConnected()`), SandboxID is not assigned.
Therefore, the daemon thinks the id is invalid and remote API will never return.
This fix tries to resolve the SandboxID by iterating through connected containers
and identify the appropriate SandboxID.
A test case for `stats` remote API has been added to check if `stats` will return
within the timeout.
This fix fixes#21848.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
When user try to restart a restarting container, docker client report
error: "container is already active", and container will be stopped
instead be restarted which is seriously wrong.
What's more critical is that when user try to start this container
again, it will always fail.
This error can also be reproduced with a `docker stop`+`docker start`.
And this commit will fix the bug.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
This fix tries to fix the discrepancy between API and CLI on hostname
validation. Previously, the hostname validation was handled at the
CLI interface in runconfig/opts/parse.go and return an error if the
hostname is invalid. However, if an end user use the remote API to
pass the hostname, the error will not be returned immediately.
Instead the error will only be thrown out when the container creation
fails. This creates behavior discrepancy between API and CLI.
In this fix, the hostname validation was moved to
verifyContainerSettings so the behavior will be the same for API and
CLI.
After the change, since CLI does not handle the hostname validation
any more, the previous unit tests about hostname validation on CLI
in runconfig/opts/parse_test.go has to be updated as well because
there is no validation at this stage. All those unit tests are moved
to integration test TestRunTooLongHostname so that the hostname
validation is still properly covered as before.
Note: Since the hostname validation moved to API, the error message
changes from `invalid hostname format for --hostname:` to
`invalid hostname format:` as well because `--hostname` is passed
to CLI only.
This fix fixes#21595.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
This fix tries to add an additional syslog-format of `rfc5424micro` which follows
the same as rfc5424 except that it use microsecond resolution for timestamp. The
purpose is to solve the issue raised in #21793 where log events might lose its
ordering if happens on the same second.
The timestamp field in rfc5424 is derived from rfc3339, though the maximium
resolution is limited to "TIME-SECFRAC" which is 6 (microsecond resolution).
The appropriate documentation (`docs/admin/logging/overview.md`) has been updated
to reflect the change in this fix.
This fix adds a unit test to cover the newly introduced format.
This fix fixes#21793.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
When container is automatically restarted based on restart policy,
docker events can't get "start" event but only get "die" event, this is
not consistent with previous behavior. This commit will add "start"
event back.
Signed-off-by: Zhang Wei <zhangwei555@huawei.com>
This adds support to the Windows graph driver for ApplyDiff on a base
layer. It also adds support for hard links, which are needed because the
Windows base layers double in size without hard link support.
Signed-off-by: John Starks <jostarks@microsoft.com>
Previously, Windows only supported running with a OS-managed base image.
With this change, Windows supports normal, Linux-like layered images, too.
Signed-off-by: John Starks <jostarks@microsoft.com>
Fixes an issue that prevents nano server images from loading properly. Also updates logic for custom image loading to avoid preventing daemon start because an image failed to load.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Overlay tests were failing when /var/tmp was an overlay mount with a misleading message.
Now overlay tests will be skipped when attempting to be run on overlay.
Tests will now use the TMPDIR environment variable instead of only /var/tmp
Fixes#21686
Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
On aufs, auplink is run before the Unmount. Irrespective of the
result, we proceed to issue a Unmount syscall. In which case,
demote erros on auplink to warning.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
Since the layer store was introduced, the level above the graphdriver
now differentiates between read/write and read-only layers. This
distinction is useful for graphdrivers that need to take special steps
when creating a layer based on whether it is read-only or not.
Adding this parameter allows the graphdrivers to differentiate, which
in the case of the Windows graphdriver, removes our dependence on parsing
the id of the parent for "-init" in order to infer this information.
This will also set the stage for unblocking some of the layer store
unit tests in the next preview build of Windows.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
This else case was lost in the migration from native execdriver to OCI
implementation via runc. There is no need to have external setkey when
--net=host.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com>
Kernel has no limit for memory reservation, but in different
kernel versions, the default behavior is different.
On kernel 3.13,
docker run --rm --memory-reservation 1k busybox cat /sys/fs/cgroup/memory/memory.soft_limit_in_bytes
the output would be 4096, but on kernel 4.1, the output is 0.
Since we have minimum limit for memory and kernel memory, we
can have this limit for memory reservation as well, to make
the behavior consistent.
Signed-off-by: Qiang Huang <h.huangqiang@huawei.com>
This vendors in new spec/runc that supports
setting readonly and masked paths in the
configuration. Using this allows us to make an
exception for `—-privileged`.
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
These fields are needed to specify the exact version of Windows that an
image can run on. They may be useful for other platforms in the future.
This also changes image.store.Create to validate that the loaded image is
supported on the current machine. This change affects Linux as well, since
it now validates the architecture and OS fields.
Signed-off-by: John Starks <jostarks@microsoft.com>
btrfs-progs-4.5 introduces device delete by devid
for this reason btrfs_ioctl_vol_args_v2's name was encapsulated
in a union
this patch is for setting btrfs_ioctl_vol_args_v2's name
using a C function in order to preserve compatibility
with all btrfs-progs versions
Signed-off-by: Julio Montes <imc.coder@gmail.com>
- Refactor generic and path based cleanup functions into a single function.
- Include aufs and zfs mounts in the mounts cleanup.
- Containers that receive exit event on restore don't require manual cleanup.
- Make missing sandbox id message a warning because currently sandboxes are always cleared on startup. libnetwork#975
- Don't unmount volumes for containers that don't have base path. Shouldn't be needed after #21372
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
This makes Windows behavior consistent with Linux -- the entry point must
be an executable, not an executable and set of arguments.
Signed-off-by: John Starks <jostarks@microsoft.com>
This pull request added a `SecurityOptions` field in the `GET /info`
output to show if there is `apparmor`, `seccomp`, or `selinux` suport.
The API changes are updated in the documentation and the update in
`GET /info` is covered by the test case in `TestInfoApi`.
This pull request fixes#20909.
Signed-off-by: Yong Tang <yong.tang.github@outlook.com>
Fix unmount issues in the daemon crash and restart lifecycle, w.r.t
graph drivers. This change sets a live container RWLayer's activity
count to 1, so that the RWLayer is aware of the mount. Note that
containerd has experimental support for restore live containers.
Added/updated corresponding tests.
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
runc expects a systemd cgroupsPath to be in slice:scopePrefix:containerName
format and the "--systemd-cgroup" option to be set. Update docker accordingly.
Fixes 21475
Signed-off-by: Anusha Ragunathan <anusha@docker.com>
This change include filter `name` and `driver`,
and also update related docs to reflect that filters usage.
Closes: #21243
Signed-off-by: Kai Qiang Wu(Kennan) <wkqwu@cn.ibm.com>
Following a journal log almost always requires a descriptor to be
allocated. In cases where we're running out of descriptors, this means
we might get stuck while attempting to start following the journal, at a
point where it's too late to report it to the client and clean up
easily. The journal reading context will cache the value once it's
allocated, so here we move the check earlier, so that we can detect a
problem when we can still report it cleanly.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com>
When we set up to start following a journal, if we get error results
from sd_journal_get_fd() or sd_journal_get_events() that prevent us from
following the journal, report the error instead of just mysteriously
failing.
Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.
In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Instead of implementing refcounts at each graphdriver, implement this in
the layer package which is what the engine actually interacts with now.
This means interacting directly with the graphdriver is no longer
explicitly safe with regard to Get/Put calls being refcounted.
In addition, with the containerd, layers may still be mounted after
a daemon restart since we will no longer explicitly kill containers when
we shutdown or startup engine.
Because of this ref counts would need to be repopulated.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
Now that the namespace sharing code via runc is vendored with the
containerd changes, we can disable the restrictions on container to
container net and IPC namespace sharing when the daemon has user
namespaces enabled.
Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
This allows a user to specify explicitly to enable
automatic copying of data from the container path to the volume path.
This does not change the default behavior of automatically copying, but
does allow a user to disable it at runtime.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
This adds support for the passthrough on build, push, login, and search.
Revamp the integration test to cover these cases and make it more
robust.
Use backticks instead of quoted strings for backslash-heavy string
contstands.
Signed-off-by: Aaron Lehmann <aaron.lehmann@docker.com>
This cleans up some of the use of the filepoller which makes reading
significantly more robust and gives fewer changes to fallback to the
polling based watcher.
In a lot of cases, if the file was being rotated while we were adding it
to the watcher, it would return an error that the file doesn't exist and
would fallback.
In some cases this fallback could be triggered multiple times even if we
were already on the fallback/poll-based watcher.
It also fixes an open file leak caused by not closing files properly on
rotate, as well as not closing files that were read via the `tail`
function until after the log reader is completed.
Prior to the above changes, it was relatively simple to cause the log
reader to error out by having quick rotations, for example:
```
$ docker run --name test --log-opt max-size=10b --log-opt max-files=10
-d busybox sh -c 'while true; do usleep 500000; echo hello; done'
$ docker logs -f test
```
After these changes I can run this forever without error.
Another fix removes 2 `os.Stat` calls when rotating files. The stat
calls are not needed since we are just calling `os.Rename` anyway, which
will in turn also just produce the same error that `Stat` would.
These `Stat` calls were also quite expensive.
Removing these stat calls also seemed to resolve an issue causing slow
memory growth on the daemon.
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
so that the user knows what's not in the container but should be.
Its not always easy for the user to know what exact command is being run
when the 'docker run' is embedded deep in something else, like a Makefile.
Saw this while dealing with the containerd migration.
Signed-off-by: Doug Davis <dug@us.ibm.com>
Changes how the Engine interacts with Registry servers on image pull.
Previously, Engine sent a User-Agent string to the Registry server
that included only the Engine's version information. This commit
appends to that string the fields from the User-Agent sent by the
client (e.g., Compose) of the Engine. This allows Registry server
operators to understand what tools are actually generating pulls on
their registries.
Signed-off-by: Mike Goelzer <mgoelzer@docker.com>
Signed-off-by: John Howard <jhoward@microsoft.com>
Signed-off-by: John Starks <jostarks@microsoft.com>
Signed-off-by: Darren Stahl <darst@microsoft.com>
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
Docker logs was only closing the logger when the HTTP response writer received a close notification, however in non-follow mode the writer never receives a close. This means that the daemon would leak the file handle to the log, preventing the container from being removed on Windows (file in use error). This change explicitly closes the log when the end of stream is hit.
Signed-off-by: Stefan J. Wernli <swernli@microsoft.com>
Lei Jitang