Commit Graph

148 Commits

Author SHA1 Message Date
mnowster 6b68a84b9b Merge pull request #2019 from mrfuxi/less-verbose-extends
Allow to extend service using shorthand notation. Closes #1989
2015-10-14 17:13:09 +01:00
Daniel Nephin b9c76dfabd Merge pull request #2181 from mnowster/2128-split-drive-not-handling-relative-paths
2128 split drive not handling relative paths
2015-10-14 11:04:42 -04:00
Mohit Soni f4efa29377 Added support for cgroup_parent
This change adds cgroup-parent support to compose project. It allows
each service to specify a 'cgroup_parent' option.

Signed-off-by: Mohit Soni <mosoni@paypal.com>
2015-10-14 15:02:42 +01:00
Mazz Mosley c1d5ecaafe Workaround splitdrive limitations
splitdrive doesn't handle relative paths, so if volume_path contains
a relative path, we handle that differently and manually set drive to ''.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-13 17:27:25 +01:00
Mazz Mosley bc6b3f970b container paths don't need to be expanded
They should not ever be relative.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-13 17:03:09 +01:00
Mazz Mosley f4cd5b1d45 Handle windows volume paths
When a relative path is expanded and we're on a windows platform,
it expands to include the drive, eg C:\ , which was causing a ConfigError
as we split on ":" in parse_volume_spec and that was giving too many parts.

Use os.path.splitdrive instead of manually calculating the drive.

This should help us deal with windows drives as part of the volume
path better than us doing it manually.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-10-01 16:06:57 +01:00
Aanand Prasad e38334efbd Don't expand volume names
Only expand volume host paths if they begin with a dot.

This is a breaking change. The deprecation warning preparing users for
this change has been removed.

Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-10-01 14:48:38 +01:00
Karol Duleba 91b227d133 Allow to extend service using shorthand notation. Closes #1989
Signed-off-by: Karol Duleba <mr.fuxi@gmail.com>
2015-09-24 18:01:47 +01:00
Daniel Nephin 22bc174650 Refactor config.load() to remove reduce() and document some types.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-18 17:11:16 -04:00
Daniel Nephin 577439ea7f Add a debug log message for config filenames.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:28:16 -04:00
Daniel Nephin 39ae85db8a Support a default docker-compose.override.yml for overrides
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin fe5daf860d Move find_candidates_in_parent_dirs() into a config module so that config doesn't import from cli.
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin 89be7f1fa7 Unit tests for multiple files
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
Daniel Nephin 10b3188214 Support multiple config files
Signed-off-by: Daniel Nephin <dnephin@docker.com>
2015-09-17 14:13:50 -04:00
mnowster 1c823597bb Merge pull request #1995 from predakanga/master
Added support for IPC namespaces, fixes GH-1689
2015-09-17 17:14:44 +01:00
Aanand Prasad 87a50317ad Merge pull request #1954 from nhumrich/master
Allow for user relative paths on EXTEND
2015-09-10 16:35:40 -07:00
Nick H a372275c6e Allow for user relative paths
'~/' in a path currently doesnt work, you get the following error:

[Errno 2] No such file or directory: u'/home/USER/folder/~/some/path/.yml'

Signed-off-by: Nick H <nick.humrich@gmail.com>
2015-09-09 14:54:08 -06:00
Lachlan Pease 2468235472 Added support for IPC namespaces, fixes GH-1689
Signed-off-by: Lachlan Pease <predatory.kangaroo@gmail.com>
2015-09-06 12:09:24 +10:00
Daniel Nephin e801981fed Sort config keys
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2015-09-03 12:04:38 -04:00
Mazz Mosley 9b8e404d13 Pass service_name to process_errors
Previously on Buffy... The process_errors was parsing a load of
ValidationErrors that we get back from jsonschema which included
assumptions about the state of the instance we're validating.

Now it's split in two and we're doing field separate to service,
those assumptions don't hold and we can't logically retrieve the
service_name from the error parsing when we're doing service schema
validation, have to explicitly pass this in.

process_errors is high on my list for some future re-factoring to help
make it a bit clearer, smaller state of doing things.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 4b487e3957 Refactor extends back out of __init__
If make_service_dict is our factory function then we'll give it the
responsibility of validation/construction and resolving.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 950577d60f Split validation into fields and service
We want to give feedback to the user as soon as possible about the
validity of the config supplied for the services.

When extending a service, we can validate that the fields are
correct against our schema but we must wait until the *end* of
the extends cycle once all of the extended dicts have been merged
into the service dict, to perform the final validation check on the
config to ensure it is a complete valid service.

Doing this before that had happened resulted in false reports of
invalid config, as common config when split out, by itself, is not
a valid service but *is* valid config to be included.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 4a8b2947ca Interpolate extended config
This refactoring is now really coming together. Construction is
happening in the __init__, which is a constructor and helps
clean up the design and clarity of intent of the code. We can now
see (nearly) everything that is being constructed when a ServiceLoader
is created. It needs all of these data constructs to perform the
domain logic and actions. Which are now clearer to see and moving
more towards the principle of functions doing (mostly)one thing and
function names being more descriptive.

resolve_extends is now concerned with the resolving of extends, rather
than the construction, validation, pre processing and *then* resolving
of extends.

Happy days :)

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 36757cde1c Validate extended service against our schema
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 37bf8235b7 Get extended config path
Refactored out into it's own function.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 538a501eec Refactor validating extends file path
Separating out the steps we need to resolve extends, so that it
will be clear to insert pre-processing of interpolation and
validation.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 02c52ae673 Move resolve_environment within __init__
resolve_environment is specific to ServiceLoader, the function
does not need to be on the global scope, it is a part of the
ServiceLoader object. The environment needs to be resolved
before we can make any service dicts, it belongs in the
constructor.

This is cleaning up the design a little and being clearer
about intent and scope of functions.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 8a6061bfb9 __init__ takes service name and dict
Moving service name and dict out of the function make_service_dict
and into __init__. We always call make_service_dict with those so
let's put them in the initialiser. Slightly cleaner design intent.

The whole purpose of the ServiceLoader is to take a
service name&service dictionary then validate, process and return
service dictionaries ready to be created.

This is also another step towards cleaning the code up so we can
interpolate and validate an extended dictionary.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley 1344533b24 filename is not optional
While it can be set to ultimately a value of None, when a
config file is read in from stdin, it is not optional.

We kinda make use of it's ability to be set to None in our
tests but functionally and design wise, it is required.

If filename is not set, extends does not work.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Mazz Mosley c907f35e74 Raise if working_dir is None
Check for this in the init so we can remove the duplication of
raising in further functions.

A ServiceLoader isn't valid without one.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-09-02 15:42:38 +01:00
Daniel Nephin d264c2e33a Resolves #1804
Fix mutation of service.options when a label or environment variable is specified in the config.

Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2015-08-27 18:05:21 -04:00
Daniel Nephin 59d4f304ee Run pre-commit on all files
Signed-off-by: Daniel Nephin <dnephin@gmail.com>
2015-08-24 17:04:50 -04:00
Aanand Prasad a806d9e77b Merge pull request #1855 from mnowster/number-is-a-valid-service-name
A number is a valid service name
2015-08-20 12:02:58 +01:00
mnowster 30b64e65cd Merge pull request #1859 from mrfuxi/extend-unexisting-service
Raise configuration error when trying to extend service that does not exist
2015-08-19 14:23:28 +01:00
Aanand Prasad 530d20db6d Fix volume path warning
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-08-14 11:15:22 +01:00
Mazz Mosley 67995ab9e3 Pre-process validation steps
In order to validate a service name that has been specified as an
integer we need to run that as a pre-process validation step
*before* we pass the config to be validated against the schema.

It is not possible to validate it *in* the schema, it causes a
type error. Even though a number is a valid service name, it
must be a cast as a string within the yaml to avoid type error.

Taken this opportunity to move the code design in a direction
towards:

1. pre-process
2. validate
3. construct

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-08-13 16:32:20 +01:00
Karol Duleba 2e7f08c2ef Raise configuration error when trying to extend service that does not exist. Fixes #1826
Signed-off-by: Karol Duleba <mr.fuxi@gmail.com>
2015-08-12 22:34:25 +01:00
Aanand Prasad fb4c9fbb8b Merge pull request #1808 from mnowster/129-validate-compose-yml
129 validate compose yml
2015-08-12 11:15:47 +01:00
Aanand Prasad 46e8e4322a Show a warning when a relative path is specified without "./"
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-08-10 19:56:36 +01:00
Mazz Mosley 2e428f94ca Refactor validation out
Move validation out into its own file without causing circular
import errors.

Fix some of the tests to import from the right place.

Also fix tests that were not using valid test data, as the validation
schema is now firing telling you that you couldn't "just" have this
dict without a build/image config key.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-08-07 12:06:34 +01:00
Mazz Mosley 0557b5dce6 Remove dead code
These functions weren't being called by anything.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-08-07 12:06:34 +01:00
Mazz Mosley d8aee782c8 Error handling
jsonschema provides a rich error tree of info, by parsing each error
we can pull out relevant info and re-write the error messages.

This covers current error handling behaviour.

This includes new error handling behaviour for types and formatting of
the ports field.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-08-07 12:06:34 +01:00
Mazz Mosley 8d6694085d Include remaining valid config properties
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-08-07 12:06:34 +01:00
Mazz Mosley 6c7c598546 Format validation of ports
Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-08-07 12:06:34 +01:00
Mazz Mosley da36ee7cbc Perform schema validation
Define a schema that we can pass to jsonschema to validate against the
config a user has supplied. This will help catch a wide variety of common
errors that occur.

If the config does not pass schema validation then it raises an exception
and prints out human readable reasons.

Signed-off-by: Mazz Mosley <mazz@houseofmnowster.com>
2015-08-07 12:06:34 +01:00
Aanand Prasad 8b5bd945d0 Interpolate environment variables
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-08-06 11:18:57 +01:00
Aanand Prasad 31ac3ce22a Split out compose.config.errors
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-08-05 15:50:22 +01:00
Aanand Prasad 92ef1f5702 Make compose.config a proper module
Signed-off-by: Aanand Prasad <aanand.prasad@gmail.com>
2015-08-05 15:50:22 +01:00