From 29bd2b0d3e844fde284ee6efe823f2e52f476342 Mon Sep 17 00:00:00 2001 From: Guillaume Tardif Date: Tue, 17 Nov 2020 17:07:03 +0100 Subject: [PATCH] Adding more basic examples of compose files, and small updates on ECS services dependencies, service name resolution Signed-off-by: Guillaume Tardif --- engine/context/aci-integration.md | 34 ++++++++++++++++++++++----- engine/context/ecs-integration.md | 38 +++++++++++++++++++++++++++---- 2 files changed, 61 insertions(+), 11 deletions(-) diff --git a/engine/context/aci-integration.md b/engine/context/aci-integration.md index f3cb5b06b5..9ea36ac37a 100644 --- a/engine/context/aci-integration.md +++ b/engine/context/aci-integration.md @@ -177,14 +177,25 @@ Total CPUs reclaimed: 2.01, total memory reclaimed: 2.30 GB ## Exposing ports -Single containers and Compose applications can optionally expose ports. For single containers, this is done using the `--publish` (`-p`) flag of the `docker run` command and for Compose applications, you must specify exposed ports in the Compose file service definition. +Single containers and Compose applications can optionally expose ports. +For single containers, this is done using the `--publish` (`-p`) flag of the `docker run` command : `docker run -p 80:80 nginx`. + +For Compose applications, you must specify exposed ports in the Compose file service definition: + +```yaml +services: + nginx: + image: nginx + ports: + - "80:80" +``` + > **Note** > > ACI does not allow port mapping (that is, changing port number while exposing port). Therefore, the source and target ports must be the same when deploying to ACI. > -> -> All containers in the same Compose application are deployed in the same ACI container group. Containers in the same Compose application cannot expose the same port when deployed to ACI. +> All containers in the same Compose application are deployed in the same ACI container group. Different containers in the same Compose application cannot expose the same port when deployed to ACI. By default, when exposing ports for your application, a random public IP address is associated with the container group supporting the deployed application (single container or Compose application). This IP address can be obtained when listing containers with `docker ps` or using `docker inspect`. @@ -192,12 +203,25 @@ This IP address can be obtained when listing containers with `docker ps` or usin ### DNS label name In addition to exposing ports on a random IP address, you can specify a DNS label name to expose your application on an FQDN of the form: `.region.azurecontainer.io`. -You can set this name with the `--domainname` flag when performing a `docker run`, or by using the `domainname` field in the Compose file when performing a `docker compose up`. + +You can set this name with the `--domainname` flag when performing a `docker run`, or by using the `domainname` field in the Compose file when performing a `docker compose up`: + +```yaml +services: + nginx: + image: nginx + domainname: "myapp" + ports: + - "80:80" +``` + > **Note** > > The domain of a Compose application can only be set once, if you specify the > `domainname` for several services, the value must be identical. +> +> The FQDN `.region.azurecontainer.io` must be available. ## Using Azure file share as volumes in ACI containers @@ -328,8 +352,6 @@ services: healthcheck: test: ["CMD", "curl", "-f", "http://localhost:80"] interval: 10s - timeout: 2s - start_period: 40s ``` ## Private Docker Hub images and using the Azure Container Registry diff --git a/engine/context/ecs-integration.md b/engine/context/ecs-integration.md index fa6678af2d..16dbc7ca67 100644 --- a/engine/context/ecs-integration.md +++ b/engine/context/ecs-integration.md @@ -149,22 +149,29 @@ services: > **Note** > -> If you set the Compose file version to 3.8 or later, you can use the same Compose file for local deployment using `docker-compose`. Custom extensions will be ignored in this case. +> If you set the Compose file version to 3.8 or later, you can use the same Compose file for local deployment using `docker-compose`. Custom ECS extensions will be ignored in this case. ## Service discovery -Service-to-service communication is implemented by the [Security Groups](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html){: target="_blank" rel="noopener" class="_"} rules, allowing services sharing a common Compose file “network” to communicate together. This allows individual services to run with distinct constraints (memory, cpu) and replication rules. However, it comes with a constraint that Docker images have to handle service discovery and wait for dependent services to be available. +Service-to-service communication is implemented transparently by default, so you can deploy your Compose applications with multiple interconnected services without changing the compose file between local and ECS deployment. Individual services can run with distinct constraints (memory, cpu) and replication rules. ### Service names -Services are registered by the Docker Compose CLI on [AWS Cloud Map](https://docs.aws.amazon.com/cloud-map/latest/dg/what-is-cloud-map.html){: target="_blank" rel="noopener" class="_"} during application deployment. They are declared as fully qualified domain names of the form: `..local`. Services can retrieve their dependencies using this fully qualified name, or can just use a short service name (as they do with docker-compose). +Services are registered automatically by the Docker Compose CLI on [AWS Cloud Map](https://docs.aws.amazon.com/cloud-map/latest/dg/what-is-cloud-map.html){: target="_blank" rel="noopener" class="_"} during application deployment. They are declared as fully qualified domain names of the form: `..local`. + +Services can retrieve their dependencies using Compose service names (as they do when deploying locally with docker-compose), or optionally use the fully qualified names. ### Dependent service startup time and DNS resolution -Services get concurrently scheduled on ECS when a Compose file is deployed. AWS Cloud Map introduces an initial delay for DNS service to be able to resolve your services domain names. As a result, your code needs to be adjusted to support this delay by waiting for dependent services to be ready, or by adding a wait-script as the entrypoint to your Docker image, as documented in [Control startup order](https://docs.docker.com/compose/startup-order/). +Services get concurrently scheduled on ECS when a Compose file is deployed. AWS Cloud Map introduces an initial delay for DNS service to be able to resolve your services domain names. Your code needs to support this delay by waiting for dependent services to be ready, or by adding a wait-script as the entrypoint to your Docker image, as documented in [Control startup order](https://docs.docker.com/compose/startup-order/). +Note this need to wait for dependent services in your Compose application also exists when deploying locally with docker-compose, but the delay is typically shorter. Issues might become more visible when deploying to ECS if services do not wait for their dependencies to be available. Alternatively, you can use the [depends_on](https://github.com/compose-spec/compose-spec/blob/master/spec.md#depends_on){: target="_blank" rel="noopener" class="_"} feature of the Compose file format. By doing this, dependent service will be created first, and application deployment will wait for it to be up and running before starting the creation of the dependent services. +### Service isolation + +Service isolation is implemented by the [Security Groups](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-security-groups.html){: target="_blank" rel="noopener" class="_"} rules, allowing services sharing a common Compose file “network” to communicate together using their Compose service names. + ## Volumes ECS integration supports volume management based on Amazon Elastic File System (Amazon EFS). @@ -279,6 +286,15 @@ value `*` to get all keys bound in your container. ## Auto scaling +Scaling service static information (non auto-scaling) can be specified using the normal Compose syntax: + +```yaml +services: + foo: + deploy: + replicas: 3 +``` + The Compose file model does not define any attributes to declare auto-scaling conditions. Therefore, we rely on `x-aws-autoscaling` custom extension to define the auto-scaling range, as well as cpu _or_ memory to define target metric, expressed as resource usage percent. @@ -336,7 +352,19 @@ The Docker Compose CLI relies on [Amazon CloudFormation](https://docs.aws.amazon ## Using existing AWS network resources -By default, the Docker Compose CLI creates an ECS cluster for your Compose application, a Security Group per network in your Compose file on your AWS account’s default VPC, and a LoadBalancer to route traffic to your services. If your AWS account does not have [permissions](https://github.com/docker/ecs-plugin/blob/master/docs/requirements.md#permissions){: target="_blank" rel="noopener" class="_"} to create such resources, or if you want to manage these yourself, you can use the following custom Compose extensions: +By default, the Docker Compose CLI creates an ECS cluster for your Compose application, a Security Group per network in your Compose file on your AWS account’s default VPC, and a LoadBalancer to route traffic to your services. + +With the following basic compose file, the Docker Compose CLI will automatically create these ECS constructs including the load balancer to route traffic to the exposed port 80. + +```yaml +services: + nginx: + image: nginx + ports: + - "80:80" +``` + +If your AWS account does not have [permissions](https://github.com/docker/ecs-plugin/blob/master/docs/requirements.md#permissions){: target="_blank" rel="noopener" class="_"} to create such resources, or if you want to manage these yourself, you can use the following custom Compose extensions: - Use `x-aws-cluster` as a top-level element in your Compose file to set the ARN of an ECS cluster when deploying a Compose application. Otherwise, a