Commit Graph

462 Commits

Author SHA1 Message Date
Phil Estes 442b45628e Add user namespace (mapping) support to the Docker engine
Adds support for the daemon to handle user namespace maps as a
per-daemon setting.

Support for handling uid/gid mapping is added to the builder,
archive/unarchive packages and functions, all graphdrivers (except
Windows), and the test suite is updated to handle user namespace daemon
rootgraph changes.

Docker-DCO-1.1-Signed-off-by: Phil Estes <estesp@linux.vnet.ibm.com> (github: estesp)
2015-10-09 17:47:37 -04:00
Morgan Bauer ee36b57c8f
continue daemon refactor of Repositories()
- use daemon member directly rather than through a function call
 - create GetImage method for use external to deamon

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-10-08 15:06:55 -07:00
Jess Frazelle 67e38cc54c Merge pull request #16638 from MHBauer/daemon-derepo
refactor daemon image tagging
2015-10-08 14:14:20 -07:00
Jess Frazelle 01cd67e752 Merge pull request #16366 from MHBauer/logs-refactor
refactor logs to not use internal data structures
2015-10-08 14:05:02 -07:00
Vincent Demeester caf7d83c67 Merge pull request #16826 from LK4D4/refactor_listenfd
Refactor listenFD function
2015-10-08 22:08:52 +02:00
Jess Frazelle cc411c054f Merge pull request #16785 from cpuguy83/exec_cleanup
Cleanup some issues with exec
2015-10-08 12:02:03 -07:00
Morgan Bauer 1eecc1e7e5
refactor logs to not use internal data structures
- refactor to make it easier to split the api in the future
 - additional tests for non existent container case

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-10-07 15:44:16 -07:00
Morgan Bauer 77c592d26a
refactor daemon
- create pass through in daemon for access to functions on daemon
   member
 - import image
push image
export image and corrections
lookup image & comments
load image
list images
image history & comments

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-10-07 15:29:02 -07:00
Alexander Morozov fb04043ca5 Refactor listenFD function
* fixed weird logic with "*"
* return error if fdNum is failed to parse
* check if listener at offset is nil
* close unused listeners

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-10-07 14:26:48 -07:00
Alexander Morozov 281a48d092 Return listenbuffer behavior
Now we're start to serve early, but all Accept calls are intercepted by
listenbuffer or systemd socket.

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-10-07 14:18:42 -07:00
Arnaud Porterie 8e31036816 Merge pull request #16645 from mavenugo/ux
Docker Network UX & remote API changes
2015-10-07 10:44:58 -07:00
Madhu Venugopal 2ab94e11a2 Network remote APIs using new router, --net=<user-defined-network> changes
* Moving Network Remote APIs out of experimental
* --net can now accept user created networks using network drivers/plugins
* Removed the experimental services concept and --default-network option
* Neccessary backend changes to accomodate multiple networks per container
* Integration Tests

Signed-off-by: David Calavera <david.calavera@gmail.com>
Signed-off-by: Madhu Venugopal <madhu@docker.com>
2015-10-07 03:54:19 -07:00
Brian Goff 2606a2e4d3 Merge pull request #16147 from tiborvass/refactor-builder
Refactor builder with new Go interfaces
2015-10-06 20:36:07 -04:00
Tibor Vass e0ef11a4c2 Abstract builder and implement server-side dockerfile builder
This patch creates interfaces in builder/ for building Docker images.
It is a first step in a series of patches to remove the daemon
dependency on builder and later allow a client-side Dockerfile builder
as well as potential builder plugins.

It is needed because we cannot remove the /build API endpoint, so we
need to keep the server-side Dockerfile builder, but we also want to
reuse the same Dockerfile parser and evaluator for both server-side and
client-side.

builder/dockerfile/ and api/server/builder.go contain implementations
of those interfaces as a refactoring of the current code.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-06 19:10:19 -04:00
Jess Frazelle c1c3475c6a Merge pull request #16786 from LK4D4/fix_systemd_serve
Remove wait on start channel for butts socket
2015-10-06 14:04:15 -07:00
Brian Goff 561005e5ca Cleanup some issues with exec
Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-10-06 15:22:20 -04:00
Alexander Morozov 806e4c4a02 Remove wait on start channel for systemd socket
Because Serve will be called after daemon creation

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-10-05 17:41:25 -07:00
Doug Davis cc833c5e2b Fix daemon logs so that "API listen on" is INFO not ERROR
W/o this PR the logs would show:
```
ERRO[0000] API listen on [::]:2375
ERRO[0000] API listen on /var/run/docker.sock
```
which is misleading since these aren't "errors" they're just informational.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-10-05 16:17:25 -07:00
Tibor Vass f41230b93a Move builder files to builder/dockerfile
Signed-off-by: Tibor Vass <tibor@docker.com>
2015-10-05 18:26:47 -04:00
Vincent Demeester 80f2e3f0c6 Merge pull request #16769 from LK4D4/fix_volume_socket
Allocate resources for server API before daemon creation
2015-10-05 20:48:41 +02:00
Alexander Morozov 5eda566f93 Allocate resources for server API before daemon creation
It prevents occupying of those resources (ports, unix-sockets) by
containers.
Also fixed false-positive test for that case.

Fix #15912

Signed-off-by: Alexander Morozov <lk4d4@docker.com>
2015-10-05 09:32:08 -07:00
Daniel Nephin 08b117517d Resolves #16458 - filter events by labels.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-10-02 16:33:20 -04:00
Brian Goff 2d43d93410 Make exec start return proper error codes
Exec start was sending HTTP 500 for every error.

Fixed an error where pausing a container and then calling exec start
caused the daemon to freeze.

Updated API docs which incorrectly showed that a successful exec start
was an HTTP 201, in reality it is HTTP 200.

Signed-off-by: Brian Goff <cpuguy83@gmail.com>
2015-10-02 14:40:22 -04:00
David Calavera da982cf551 Separate API router from server.
Implement basic interfaces to write custom routers that can be plugged
to the server. Remove server coupling with the daemon.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-29 19:43:03 -04:00
Tibor Vass 94e3b0f428 Use golang.org/x/net/context in api/server/
This patch removes the internal context package and uses golang's
package instead.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-09-29 17:40:55 -04:00
Tibor Vass b08f071e18 Revert "Merge pull request #16228 from duglin/ContextualizeEvents"
Although having a request ID available throughout the codebase is very
valuable, the impact of requiring a Context as an argument to every
function in the codepath of an API request, is too significant and was
not properly understood at the time of the review.

Furthermore, mixing API-layer code with non-API-layer code makes the
latter usable only by API-layer code (one that has a notion of Context).

This reverts commit de41640435, reversing
changes made to 7daeecd42d.

Signed-off-by: Tibor Vass <tibor@docker.com>

Conflicts:
	api/server/container.go
	builder/internals.go
	daemon/container_unix.go
	daemon/create.go
2015-09-29 14:26:51 -04:00
Tibor Vass 79c31f4b13 Revert "Merge pull request #16567 from calavera/context_per_request"
This reverts commit ff92f45be4, reversing
changes made to 80e31df3b6.

Reverting to make the next revert easier.

Signed-off-by: Tibor Vass <tibor@docker.com>
2015-09-29 13:40:46 -04:00
Jess Frazelle ff92f45be4 Merge pull request #16567 from calavera/context_per_request
Define a context per request.
2015-09-25 13:01:51 -07:00
Morgan Bauer 93bd57b0b2
refactor create to not expose internal data structures
- use existing exposed type

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-25 09:35:18 -07:00
David Calavera 27c76522de Define a context per request.
Avoid creating a global context object that will be used while the daemon is running.

Not only this object won't ever be garbage collected, but it won't ever be used for anything else than creating other contexts in each request. I think it's a bad practive to have something like this sprawling aroud the code.

This change removes that global object and initializes a context in the cases we don't have already one, like shutting down the server.
This also removes a bunch of context arguments from functions that did nothing with it.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-24 18:44:53 -04:00
Doug Davis 26b1064967 Add context.RequestID to event stream
This PR adds a "request ID" to each event generated, the 'docker events'
stream now looks like this:

```
2015-09-10T15:02:50.000000000-07:00 [reqid: c01e3534ddca] de7c5d4ca927253cf4e978ee9c4545161e406e9b5a14617efb52c658b249174a: (from ubuntu) create
```
Note the `[reqID: c01e3534ddca]` part, that's new.

Each HTTP request will generate its own unique ID. So, if you do a
`docker build` you'll see a series of events all with the same reqID.
This allow for log processing tools to determine which events are all related
to the same http request.

I didn't propigate the context to all possible funcs in the daemon,
I decided to just do the ones that needed it in order to get the reqID
into the events. I'd like to have people review this direction first, and
if we're ok with it then I'll make sure we're consistent about when
we pass around the context - IOW, make sure that all funcs at the same level
have a context passed in even if they don't call the log funcs - this will
ensure we're consistent w/o passing it around for all calls unnecessarily.

ping @icecrime @calavera @crosbymichael

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-24 11:56:37 -07:00
Brian Goff ebe7ef9fc2 Merge pull request #16411 from duglin/DaemonErrors
Move more 'daemon' errors to the new error package
2015-09-23 21:20:56 -04:00
Jess Frazelle ddd0c47412 Merge pull request #16497 from MHBauer/attach-refactor-new
refactor attach to not use internal data structures
2015-09-23 16:52:39 -07:00
Doug Davis 0a734182eb Move more 'daemon' errors to the new error package
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-23 09:51:45 -07:00
Morgan Bauer 8aef1a33eb
refactor attach to not use internal data structures
- refactor to make it easier to split the api in the future

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-23 08:55:21 -07:00
Phil Estes 9dc0973655 Merge pull request #16331 from calavera/server_middlewares
Organize server pre-func logic in middlewares.
2015-09-22 16:10:42 -05:00
Alexandre Beslic f97e6f2a37 Merge pull request #16419 from HuKeping/comments
Comment: add description for some code which hard to understand
2015-09-21 20:44:54 -07:00
Hu Keping 3b443abec2 Comment: add description for some code which hard to understand
Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-22 09:34:49 +08:00
David Calavera 0fea04d27e Organize server pre-func logic in middlewares.
It defines global middlewares for every request.
This makes the server slightly more composable.

Signed-off-by: David Calavera <david.calavera@gmail.com>
2015-09-21 14:27:06 -04:00
Antonio Murdaca 76b3b54314 api: server: container: fix wrong comment
This comment was wrongly referring to the old job mechanism and it
wasn't clear what it was trying to document.

Signed-off-by: Antonio Murdaca <runcom@linux.com>
2015-09-21 00:46:16 +02:00
Doug Davis a283a30fb0 Move api/errors/ to errors/
Per @calavera's suggestion: https://github.com/docker/docker/pull/16355#issuecomment-141139220

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-17 11:54:14 -07:00
Jess Frazelle 828e4ac45a Merge pull request #16355 from duglin/DaemonErrors
Convert some "daemon" static error strings to the new errocode package format
2015-09-17 11:48:37 -07:00
David Calavera cafee2eb45 Merge pull request #16361 from MHBauer/stats-refactor
refactor stats to not use internal data structures
2015-09-17 08:28:07 -07:00
Morgan Bauer 2d5d606fd3
refactor stats to not use internal data structures
- refactor to make it easier to split the api in the future
 - addition to check the existing test case and make sure it contains
   some expected output

Signed-off-by: Morgan Bauer <mbauer@us.ibm.com>
2015-09-16 17:28:52 -07:00
Doug Davis f7d4b4fe2b Convert some "daemon" static error strings to the new errocode package format
Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-16 16:16:42 -07:00
Sebastiaan van Stijn 259a0fb16c Merge pull request #15786 from HuKeping/stats-network
Refactor the statistics of network in docker stats
2015-09-16 20:29:16 +02:00
Madhav Puri 54240f8da9 Support for passing build-time variables in build context
- The build-time variables are passed as environment-context for command(s)
run as part of the RUN primitve. These variables are not persisted in environment of
intermediate and final images when passed as context for RUN. The build environment
is prepended to the intermediate continer's command string for aiding cache lookups.
It also helps with build traceability. But this also makes the feature less secure from
point of view of passing build time secrets.

- The build-time variables also get used to expand the symbols used in certain
Dockerfile primitves like ADD, COPY, USER etc, without an explicit prior definiton using a
ENV primitive. These variables get persisted in the intermediate and final images
whenever they are expanded.

- The build-time variables are only expanded or passed to the RUN primtive if they
are defined in Dockerfile using the ARG primitive or belong to list of built-in variables.
HTTP_PROXY, HTTPS_PROXY, http_proxy, https_proxy, FTP_PROXY and NO_PROXY are built-in
variables that needn't be explicitly defined in Dockerfile to use this feature.

Signed-off-by: Madhav Puri <madhav.puri@gmail.com>
2015-09-16 03:31:15 -07:00
Brian Goff e91f2c26ce Merge pull request #15069 from duglin/UseErrorPackage
Use the new error package
2015-09-15 09:28:53 -04:00
Hu Keping d3379946ec Refactor the statistics of network in docker stats
For now docker stats will sum the rxbytes, txbytes, etc. of all
the interfaces.

It is OK for the output of CLI `docker stats` but not good for
the API response, especially when the container is in sereval
subnets.

It's better to leave these origianl data to user.

Signed-off-by: Hu Keping <hukeping@huawei.com>
2015-09-15 15:40:34 +08:00
Doug Davis 628b9a41b0 Use the new error package
This is the first step in converting out static strings into well-defined
error types.  This shows just a few examples of it to get a feel for how things
will look. Once we agree on the basic outline we can then work on converting
the rest of the code over.

Signed-off-by: Doug Davis <dug@us.ibm.com>
2015-09-14 13:11:01 -07:00