Commit Graph

1170 Commits

Author SHA1 Message Date
Arnaud Porterie 7d193ef1f3 Add builtin nodes discovery
Use `pkg/discovery` to provide nodes discovery between daemon instances.

The functionality is driven by two different command-line flags: the
experimental `--cluster-store` (previously `--kv-store`) and
`--cluster-advertise`. It can be used in two ways by interested
components:

1. Externally by calling the `/info` API and examining the cluster store
   field. The `pkg/discovery` package can then be used to hit the same
   endpoint and watch for appearing or disappearing nodes. That is the
   method that will for example be used by Swarm.
2. Internally by using the `Daemon.discoveryWatcher` instance. That is
   the method that will for example be used by libnetwork.

Signed-off-by: Arnaud Porterie <arnaud.porterie@docker.com>
2015-09-25 14:52:09 -07:00
Jess Frazelle ff92f45be4 Merge pull request #16567 from calavera/context_per_request
Define a context per request.
2015-09-25 13:01:51 -07:00
Morgan Bauer 93bd57b0b2
refactor create to not expose internal data structures
- use existing exposed type

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-25 09:35:18 -07:00
David Calavera 27c76522de Define a context per request.
Avoid creating a global context object that will be used while the daemon is running.

Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.

This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-24 18:44:53 -04:00
David Calavera de41640435 Merge pull request #16228 from duglin/ContextualizeEvents
Add context.RequestID to event stream
2015-09-24 14:16:22 -07:00
Doug Davis 26b1064967 Add context.RequestID to event stream
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:

```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.

Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.

I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.

ping @icecrime @calavera @crosbymichael

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 11:56:37 -07:00
Ma Shimiao 19613feddf client/search: small logic fix
Signed-off-by: Ma Shimiao <mashimiao.fnst@cn.fujitsu.com>
2015-09-24 11:11:03 +08:00
Brian Goff ebe7ef9fc2 Merge pull request #16411 from duglin/DaemonErrors
Move more 'daemon' errors to the new error package
2015-09-23 21:20:56 -04:00
Brian Goff 40e8560412 Merge pull request #16415 from HuKeping/dockerinfo
Format output of docker info
2015-09-23 21:15:23 -04:00
Jess Frazelle ddd0c47412 Merge pull request #16497 from MHBauer/attach-refactor-new
refactor attach to not use internal data structures
2015-09-23 16:52:39 -07:00
Jess Frazelle d71ae45772 Merge pull request #16484 from HuKeping/history-format
Format the output of docker history
2015-09-23 16:52:21 -07:00
Jess Frazelle c0c941627a Merge pull request #16289 from cpuguy83/11957_fix_stdin_block_after_container_exit
Ensure stdin does not block after container stop
2015-09-23 15:29:48 -07:00
Doug Davis 0a734182eb Move more 'daemon' errors to the new error package
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-23 09:51:45 -07:00
Morgan Bauer 8aef1a33eb
refactor attach to not use internal data structures
- refactor to make it easier to split the api in the future

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-23 08:55:21 -07:00
Hu Keping 141e91c480 Format output of docker info
Format those info which will only be displayed when daemon is
in debug mode.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-23 14:35:13 +08:00
Phil Estes 9dc0973655 Merge pull request #16331 from calavera/server_middlewares
Organize server pre-func logic in middlewares.
2015-09-22 16:10:42 -05:00
Hu Keping 6353f3a7b5 Format the output of docker history
Once there have '\t' in the "CREATED BY" section, it will possibly
cause a messy output of docker history.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-22 21:27:20 +08:00
David Calavera a7e23cab8d Merge pull request #15864 from dmcgowan/notary-offline
Support content trust offline
2015-09-21 21:11:36 -07:00
Alexandre Beslic f97e6f2a37 Merge pull request #16419 from HuKeping/comments
Comment: add description for some code which hard to understand
2015-09-21 20:44:54 -07:00
Hu Keping 3b443abec2 Comment: add description for some code which hard to understand
Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-22 09:34:49 +08:00
Evan Hazlett 0c71c19239 Merge pull request #16404 from duglin/FixRenameError
Fix 'rename' error msg and error checking
2015-09-21 15:51:02 -04:00
Tibor Vass d572bab4a0 Merge pull request #16437 from runcom/invalid-logger-address
daemon: logger: error out on daemon start if invalid logger address
2015-09-21 14:53:47 -04:00
David Calavera 0fea04d27e Organize server pre-func logic in middlewares.
It defines global middlewares for every request.
This makes the server slightly more composable.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-21 14:27:06 -04:00
Zhang Kun 7cf343d106 add docker server version to /info
Signed-off-by: Zhang Kun <zkazure@gmail.com>
2015-09-21 21:15:32 +08:00
Antonio Murdaca 76b3b54314 api: server: container: fix wrong comment
This comment was wrongly referring to the old job mechanism and it
wasn't clear what it was trying to document.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-09-21 00:46:16 +02:00
Sally O'Malley e41753678d docker restarts running OR stopped containers, docs edit rm "running"
Signed-off-by: Sally O'Malley <somalley@redhat.com>
2015-09-20 09:16:27 -04:00
Doug Davis 848792c42e Fix 'rename' error msg and error checking
`docker rename foo ''` would result in:
```
usage: docker rename OLD_NAME NEW_NAME
```
which is the old engine's way of return errors - yes that's in the
daemon code.  So I fixed that error msg to just be normal.

While doing that I noticed that using an empty string for the
source container name failed but didn't print any error message at all.
This is because we would generate a URL like: ../containers//rename/..
which would cause a 301 redirect to ../containers/rename/..
however the CLI code doesn't actually deal with 301's - it just ignores
them and returns back to the CLI code/caller.

Rather than changing the CLI to deal with 3xx error codes, which would
probably be a good thing to do in a follow-on PR, for this immediate
issue I just added a cli-side check for empty strings for both old and
new names. This way we catch it even before we hit the daemon.

API callers will get a 404, assuming they follow the 301, for the
case of the src being empty, and the new error msg when the destination
is empty - so we should be good now.

Add tests for both cases too.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-18 11:12:22 -07:00
Brian Goff 3dbfb3d38c Ensure stdin does not block after container stop
Fixes #11957
Fixes #12319

Also removes check for Darwin when the stdin reader is closed as it
doesn't appear to block any more.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-09-18 13:48:07 -04:00
Antonio Murdaca 41646cb4e3 Merge pull request #16389 from HuKeping/stats-zero
Reset data of stats in docker cli when container stopped
2015-09-18 18:59:51 +02:00
Hu Keping da67833e63 Reset data of stats in docker cli when container stopped
When use `docker stats` to minitor a running container and then stop it,
there are some fields need to be reset to zero. Otherwise it will keep
displaying the data it received last time.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-18 15:24:13 +08:00
Doug Davis a283a30fb0 Move api/errors/ to errors/
Per @calavera's suggestion: https://github.com/docker/docker/pull/16355#issuecomment-141139220

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-17 11:54:14 -07:00
Jess Frazelle 828e4ac45a Merge pull request #16355 from duglin/DaemonErrors
Convert some "daemon" static error strings to the new errocode package format
2015-09-17 11:48:37 -07:00
David Calavera cafee2eb45 Merge pull request #16361 from MHBauer/stats-refactor
refactor stats to not use internal data structures
2015-09-17 08:28:07 -07:00
Antonio Murdaca ffaee8fe78 Merge pull request #16336 from mountkin/fix-hijack
error should be checked earlier in the hijack function
2015-09-17 09:38:48 +02:00
Morgan Bauer 2d5d606fd3
refactor stats to not use internal data structures
- refactor to make it easier to split the api in the future
 - addition to check the existing test case and make sure it contains
   some expected output

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-16 17:28:52 -07:00
Doug Davis f7d4b4fe2b Convert some "daemon" static error strings to the new errocode package format
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-16 16:16:42 -07:00
Sebastiaan van Stijn 259a0fb16c Merge pull request #15786 from HuKeping/stats-network
Refactor the statistics of network in docker stats
2015-09-16 20:29:16 +02:00
Shijiang Wei f2d978ccf9 error should be checked earlier in the hijack function
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
2015-09-16 20:28:07 +08:00
Madhav Puri 54240f8da9 Support for passing build-time variables in build context
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.

- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.

- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.

Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
2015-09-16 03:31:15 -07:00
Brian Goff e91f2c26ce Merge pull request #15069 from duglin/UseErrorPackage
Use the new error package
2015-09-15 09:28:53 -04:00
Hu Keping d3379946ec Refactor the statistics of network in docker stats
For now docker stats will sum the rxbytes, txbytes, etc. of all
the interfaces.

It is OK for the output of CLI `docker stats` but not good for
the API response, especially when the container is in sereval
subnets.

It's better to leave these origianl data to user.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-15 15:40:34 +08:00
David Calavera 7ce270d4ea Merge pull request #15931 from vdemeester/api-client-ps-coverage
Add more unit tests on api/client/ps package
2015-09-14 14:02:49 -07:00
Doug Davis 628b9a41b0 Use the new error package
This is the first step in converting out static strings into well-defined
error types.  This shows just a few examples of it to get a feel for how things
will look. Once we agree on the basic outline we can then work on converting
the rest of the code over.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-14 13:11:01 -07:00
Derek McGowan 5e11cd43aa Ignore ping errors in notary repository setup
Notary is capable of acting in offline mode, making use of cache TUF data.
When ping is not successful, notary should still be attempted without error.

Signed-off-by: Derek McGowan <derek@mcgstyle.net> (github: dmcgowan)
2015-09-14 11:08:25 -07:00
Nalin Dahyabhai e611a189cb Add log reading to the journald log driver
If a logdriver doesn't register a callback function to validate log
options, it won't be usable.  Fix the journald driver by adding a dummy
validator.

Teach the client and the daemon's "logs" logic that the server can also
supply "logs" data via the "journald" driver.  Update documentation and
tests that depend on error messages.

Add support for reading log data from the systemd journal to the
journald log driver.  The internal logic uses a goroutine to scan the
journal for matching entries after any specified cutoff time, formats
the messages from those entries as JSONLog messages, and stuffs the
results down a pipe whose reading end we hand back to the caller.

If we are missing any of the 'linux', 'cgo', or 'journald' build tags,
however, we don't implement a reader, so the 'logs' endpoint will still
return an error.

Make the necessary changes to the build setup to ensure that support for
reading container logs from the systemd journal is built.

Rename the Jmap member of the journald logdriver's struct to "vars" to
make it non-public, and to make it easier to tell that it's just there
to hold additional variable values that we want journald to record along
with log data that we're sending to it.

In the client, don't assume that we know which logdrivers the server
implements, and remove the check that looks at the server.  It's
redundant because the server already knows, and the check also makes
using older clients with newer servers (which may have new logdrivers in
them) unnecessarily hard.

When we try to "logs" and have to report that the container's logdriver
doesn't support reading, send the error message through the
might-be-a-multiplexer so that clients which are expecting multiplexed
data will be able to properly display the error, instead of tripping
over the data and printing a less helpful "Unrecognized input header"
error.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
2015-09-11 16:50:03 -04:00
Brian Goff bc48194ded Merge pull request #16215 from HuKeping/save
Refactor CLI docker save
2015-09-11 13:52:59 -04:00
Hu Keping 22be1e443f Refactor CLI docker save
Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-11 15:31:55 +08:00
David Calavera 0e50d946a2 Signal to stop a container.
Allow to set the signal to stop a container in `docker run`:
- Use `--stop-signal` with docker-run to set the default signal the container will use to exit.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-10 19:56:05 -04:00
Rozhnov Alexandr e9e68fa2d2 Added tags list to /images/:id/json api.
It closes #10139.

Signed-off-by: Rozhnov Alexandr <nox73@ya.ru>
2015-09-10 11:55:01 +03:00
David Calavera 6efe45990d Merge pull request #15953 from duglin/DaemonContext
Add a "context" to the api/server/* code
2015-09-09 17:28:38 -07:00