Commit Graph

435 Commits

Author SHA1 Message Date
Jess Frazelle ff92f45be4 Merge pull request #16567 from calavera/context_per_request
Define a context per request.
2015-09-25 13:01:51 -07:00
Morgan Bauer 93bd57b0b2
refactor create to not expose internal data structures
- use existing exposed type

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-25 09:35:18 -07:00
David Calavera 27c76522de Define a context per request.
Avoid creating a global context object that will be used while the daemon is running.

Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.

This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-24 18:44:53 -04:00
Doug Davis 26b1064967 Add context.RequestID to event stream
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:

```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.

Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.

I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.

ping @icecrime @calavera @crosbymichael

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 11:56:37 -07:00
Brian Goff ebe7ef9fc2 Merge pull request #16411 from duglin/DaemonErrors
Move more 'daemon' errors to the new error package
2015-09-23 21:20:56 -04:00
Jess Frazelle ddd0c47412 Merge pull request #16497 from MHBauer/attach-refactor-new
refactor attach to not use internal data structures
2015-09-23 16:52:39 -07:00
Doug Davis 0a734182eb Move more 'daemon' errors to the new error package
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-23 09:51:45 -07:00
Morgan Bauer 8aef1a33eb
refactor attach to not use internal data structures
- refactor to make it easier to split the api in the future

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-23 08:55:21 -07:00
Phil Estes 9dc0973655 Merge pull request #16331 from calavera/server_middlewares
Organize server pre-func logic in middlewares.
2015-09-22 16:10:42 -05:00
Alexandre Beslic f97e6f2a37 Merge pull request #16419 from HuKeping/comments
Comment: add description for some code which hard to understand
2015-09-21 20:44:54 -07:00
Hu Keping 3b443abec2 Comment: add description for some code which hard to understand
Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-22 09:34:49 +08:00
David Calavera 0fea04d27e Organize server pre-func logic in middlewares.
It defines global middlewares for every request.
This makes the server slightly more composable.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-21 14:27:06 -04:00
Antonio Murdaca 76b3b54314 api: server: container: fix wrong comment
This comment was wrongly referring to the old job mechanism and it
wasn't clear what it was trying to document.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-09-21 00:46:16 +02:00
Doug Davis a283a30fb0 Move api/errors/ to errors/
Per @calavera's suggestion: https://github.com/docker/docker/pull/16355#issuecomment-141139220

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-17 11:54:14 -07:00
Jess Frazelle 828e4ac45a Merge pull request #16355 from duglin/DaemonErrors
Convert some "daemon" static error strings to the new errocode package format
2015-09-17 11:48:37 -07:00
David Calavera cafee2eb45 Merge pull request #16361 from MHBauer/stats-refactor
refactor stats to not use internal data structures
2015-09-17 08:28:07 -07:00
Morgan Bauer 2d5d606fd3
refactor stats to not use internal data structures
- refactor to make it easier to split the api in the future
 - addition to check the existing test case and make sure it contains
   some expected output

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-16 17:28:52 -07:00
Doug Davis f7d4b4fe2b Convert some "daemon" static error strings to the new errocode package format
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-16 16:16:42 -07:00
Sebastiaan van Stijn 259a0fb16c Merge pull request #15786 from HuKeping/stats-network
Refactor the statistics of network in docker stats
2015-09-16 20:29:16 +02:00
Madhav Puri 54240f8da9 Support for passing build-time variables in build context
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.

- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.

- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.

Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
2015-09-16 03:31:15 -07:00
Brian Goff e91f2c26ce Merge pull request #15069 from duglin/UseErrorPackage
Use the new error package
2015-09-15 09:28:53 -04:00
Hu Keping d3379946ec Refactor the statistics of network in docker stats
For now docker stats will sum the rxbytes, txbytes, etc. of all
the interfaces.

It is OK for the output of CLI `docker stats` but not good for
the API response, especially when the container is in sereval
subnets.

It's better to leave these origianl data to user.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-15 15:40:34 +08:00
Doug Davis 628b9a41b0 Use the new error package
This is the first step in converting out static strings into well-defined
error types.  This shows just a few examples of it to get a feel for how things
will look. Once we agree on the basic outline we can then work on converting
the rest of the code over.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-14 13:11:01 -07:00
Nalin Dahyabhai e611a189cb Add log reading to the journald log driver
If a logdriver doesn't register a callback function to validate log
options, it won't be usable.  Fix the journald driver by adding a dummy
validator.

Teach the client and the daemon's "logs" logic that the server can also
supply "logs" data via the "journald" driver.  Update documentation and
tests that depend on error messages.

Add support for reading log data from the systemd journal to the
journald log driver.  The internal logic uses a goroutine to scan the
journal for matching entries after any specified cutoff time, formats
the messages from those entries as JSONLog messages, and stuffs the
results down a pipe whose reading end we hand back to the caller.

If we are missing any of the 'linux', 'cgo', or 'journald' build tags,
however, we don't implement a reader, so the 'logs' endpoint will still
return an error.

Make the necessary changes to the build setup to ensure that support for
reading container logs from the systemd journal is built.

Rename the Jmap member of the journald logdriver's struct to "vars" to
make it non-public, and to make it easier to tell that it's just there
to hold additional variable values that we want journald to record along
with log data that we're sending to it.

In the client, don't assume that we know which logdrivers the server
implements, and remove the check that looks at the server.  It's
redundant because the server already knows, and the check also makes
using older clients with newer servers (which may have new logdrivers in
them) unnecessarily hard.

When we try to "logs" and have to report that the container's logdriver
doesn't support reading, send the error message through the
might-be-a-multiplexer so that clients which are expecting multiplexed
data will be able to properly display the error, instead of tripping
over the data and printing a less helpful "Unrecognized input header"
error.

Signed-off-by: Nalin Dahyabhai <nalin@redhat.com> (github: nalind)
2015-09-11 16:50:03 -04:00
David Calavera 0e50d946a2 Signal to stop a container.
Allow to set the signal to stop a container in `docker run`:
- Use `--stop-signal` with docker-run to set the default signal the container will use to exit.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-10 19:56:05 -04:00
Doug Davis f790496d8b Move context stuff to its own package
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-09 14:23:25 -07:00
Doug Davis 8b454dd79e Add a "context" to the api/server/* code
This defines a 'context' object that is passed to each API handler.
Right now the context just has a unique 'requestID' for each API call.
The next steps would be:
- use this 'requestID' in our logging.
- determine the best way to format the logging to include this info.

In particular for log events that generate multiple entries in the log
we can use the requestID to help correlate the log entries.

Adding the requestID to the logging will be a challenge since it could mean
changing every single logrus.XXX() call to pass in the 'context' object.

But first step is to agree on a format, which we can discus in a subsequent
PR, but my initial thoughts are to add it right after the timestamp:

current format:
INFO[0039] POST /v1.21/build?buildargs=%7B%22foo%22%3A%22xxx%22%7D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null

proposed format:
INFO[0039-83dea1222191] POST /v1.21/build?buildargs=%7B%22foo%22%3A%22xxx%22%7D&cgroupparent=&cpuperiod=0&cpuquota=0&cpusetcpus=&cpusetmems=&cpushares=0&dockerfile=Dockerfile&memory=0&memswap=0&rm=1&t=&ulimits=null

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-09 12:51:20 -07:00
Brian Goff 9ca4aa4797 Merge pull request #15798 from calavera/volume_driver_host_config
Move VolumeDriver to HostConfig to make containers portable.
2015-09-08 22:05:40 -04:00
Madhu Venugopal eb5c81e799 Merge pull request #16130 from aboch/vnd
Vendoring libnetwork 3e31cead05cba8ec20241630d051e6d73765b3a2
2015-09-08 11:29:34 -07:00
Alessandro Boch 4d648f924a Vendoring libnetwork 3e31cead05cba8ec20241630d051e6d73765b3a2
+ Fix a couple of bugs introduced by previous vendoring:
  - in bitseq which prevents to use experimental overlay networking
  - in docker service ls cli o/p
+ Add missing http subrouter for newly introduced sandboxes
+ Fix fragmentation issue on vxlan header addition for overlay network driver
+ Remove libnetwork test code utilities from vendoring

Signed-off-by: Alessandro Boch <aboch@docker.com>
2015-09-08 07:36:35 -07:00
Brian Goff dc8a39036b Don't lookup container 4 times for stats
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-09-08 10:12:46 -04:00
Hu Keping d9bf8163ad Fix connection block when using docker stats API
For now CLI `docker stats` will not block even if the container was
not running is because there is a 2s timeout setting when waiting for
the response.

I think why we hang there waiting for the container to run is because we
want to get the stats of container immediately when it starts running.
But it will block when use the API directly, for example
- curl
- Google Chrome plugin, Postman
- Firefox plugin, RESTClient

This patch keeps the feature that getting info immediately when container
starts running and in the meantime, it will not block when using the API
directrly.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-06 17:21:41 +08:00
David Calavera 6549d6517b Move VolumeDriver to HostConfig to make containers portable.
Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-04 12:42:44 -04:00
David Calavera b5cc077864 Remove /containers/ps from the api router.
This route was deprecated more than two years ago in the linked
commit[1]. It's not referenced anywhere in the documentation and it's
time to stop maintaning it.

[1]: 4f9443927e

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-01 05:17:35 -04:00
Josh Hawn 111d2f3487 [daemon] Refactor image_delete.go
This file was not well documented and had very high cyclomatic complexity.
This patch completely rearranges this file and the ImageDelete method to
be easier to follow and more maintainable in the future.

Docker-DCO-1.1-Signed-off-by: Josh Hawn <josh.hawn@docker.com> (github: jlhawn)
2015-08-28 11:01:24 -07:00
Morgan Bauer abd72d4008
golint fixes for daemon/ package
- some method names were changed to have a 'Locking' suffix, as the
 downcased versions already existed, and the existing functions simply
 had locks around the already downcased version.
 - deleting unused functions
 - package comment
 - magic numbers replaced by golang constants
 - comments all over

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-08-27 22:07:42 -07:00
Brian Goff b3b7eb2723 Add volume API/CLI
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-08-26 13:37:52 -04:00
David Calavera b857dadb33 Merge pull request #15711 from monitorjbl/import-commit-message
adding message option to the import subcommand
2015-08-26 11:42:10 +02:00
Brian Goff e134ccada7 Merge pull request #14978 from gesellix/websocket-disable-check-origin
skip checkOrigin during websocket handshake
2015-08-25 12:56:30 -04:00
David Calavera 6f8c4480e4 Merge pull request #14665 from coolljt0725/fix_build_with_resource_limit
Fix build with resource limit which system not support.
2015-08-25 16:42:13 +02:00
Taylor Jones 936b2c6afe adding message option to the import subcommand
Signed-off-by: Taylor Jones <monitorjbl@gmail.com>
2015-08-25 02:55:04 +00:00
David Calavera 354a2e105d Allow HEAD requests to container archives via CORS.
Add HEAD to Access-Control-Allow-Methods.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-08-24 05:57:22 -04:00
Shijiang Wei fcf9daad91 fix a panic when the exec fails to start
Signed-off-by: Shijiang Wei <mountkin@gmail.com>
2015-08-22 00:01:44 +08:00
Lei Jitang 770daa15f3 Fix build with resource limit which system is not support.
Signed-off-by: Lei Jitang <leijitang@huawei.com>
2015-08-18 11:54:00 +08:00
Brian Goff 5ffcecf130 Fix issue with exec TTY caused by 15446
The bool logic around setting up the TTY ended up getting flipped
accidentally.
Also added a test for exec with TTY.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-08-14 14:55:26 -07:00
Jessie Frazelle d3198fa8c4 Merge pull request #15446 from cpuguy83/better_err_on_exec_err
Return better errors from exec
2015-08-12 11:13:29 -07:00
Mrunal Patel d626f77e36 Fix incorrect comment
Signed-off-by: Mrunal Patel <mrunalp@gmail.com>
2015-08-11 19:45:23 -04:00
Tobias Gesellchen 9bde19e72a skip checkOrigin during websocket handshake
Signed-off-by: Tobias Gesellchen <tobias@gesellix.de>
2015-08-12 00:05:59 +02:00
Tonis Tiigi 6e3740450d Mark network ui endpoints unix-only
Signed-off-by: Tonis Tiigi <tonistiigi@gmail.com>
2015-08-10 21:49:27 -07:00
Brian Goff f078f75bf2 Return better errors from exec
Also cleans up some of the API side of exec.
Was writing the header twice (two different headers).

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-08-10 09:53:19 -04:00