Review DTR storage backend

This commit is contained in:
Joao Fernandes 2017-03-20 15:10:31 -07:00 committed by Joao Fernandes
parent ab84302012
commit 7dfb08830a
4 changed files with 256 additions and 200 deletions

View File

@ -5,151 +5,70 @@ keywords: docker, documentation, about, technology, understanding, configuration
title: Configure DTR image storage
---
After installing Docker Trusted Registry, one of your first tasks is to
designate and configure the Trusted Registry storage backend. This document
provides the following:
By default DTR uses the local filesystem of the node where it is running to
store your Docker images. You can configure DTR to use an external storage
backend, for improved performance or high availability.
* Information describing your storage backend options.
* Configuration steps using either the Trusted Registry UI or a YAML file.
![architecture diagram](../../../images/configure-external-storage-1.svg)
The default storage backend, `filesystem`, stores and serves images from the
*local* filesystem. In a HA setup this fails, as each node can only access its
own files.
If your DTR deployment only has a single replica, you can continue using the
local filesystem to store your Docker images. If your DTR deployment has
multiple replicas, for high availability, you need to ensure all replicas are
using the same storage backend. When a user pulls an image, the node serving
the request needs to have access to that image.
DTR allows you to confiugure your image storage via distributed stores, such as
Amazon S3, NFS, or Google Cloud Storage. This flexibility to configure to a
different storage backend allows you to:
DTR supports these storage systems:
* Scale your Trusted Registry
* Leverage storage redundancy
* Store your images anywhere in the cloud
* Take advantage of other features that are critical to your organization
* Local filesystem
* NFS
* Amazon S3 or compatible
* Google Cloud Storage
* Microsoft Azure Blob storage
* OpenStack Swift
At first, you might have explored Docker Trusted Registry and Docker Engine by
installing them on your system in order to familiarize yourself with them.
However, for various reasons such as deployment purposes or continuous
integration, it makes sense to think about your long term organizations needs
when selecting a storage backend. The Trusted Registry natively supports TLS and
basic authentication.
To configure the storage backend, you can log into the **DTR web UI**
as an administrator user, navigate to the **Settings** page, and choose
**Storage**.
## Understand the Trusted Registry storage backend
![dtr settings](../../../images/configure-external-storage-2.png){: .with-border}
Your Trusted Registry data (images etc.) are stored using the configured
**storage driver** within DTR's settings. This defaults to the local
filesystem which uses your OS' posix operations to store and serve images.
The storage configuration page in the DTR web UI has options for the most
common configuration options, but you can also upload a yaml configuration file.
Additionally, the Trusted Registry supports these cloud-based storage drivers:
The format of this configuration file is similar to the one used by
[Docker Registry](/registry/configuration.md).
* Amazon Simple Storage Solution **S3** (and S3-compatible servers)
* OpenStack **Swift**
* Microsoft **Azure** Blob Storage
* **Google Cloud** Storage
## Local filesystem
### Filesystem
By default, DTR creates a volume names `dtr-registry-<replica-id>` to store
your images using the local filesystem. You can customize the name and path of
the volume used by DTR, using the `docker/dtr reconfigure --dtr-storage-volume`
option.
The `filesystem` driver operates on the host's local filesystem. In HA
environments this needs to be shared via NFS, otherwise each node in your setup
will only be able to see their own local data. For more information on
configuring NFS [see the NFS docs](/datacenter/dtr/2.2/guides/admin/
configure/external-storage/nfs/).
If you're deploying DTR with high-availability, you need to use NFS or any other
centralized storage backend so that all your DTR replicas have access to the
same images.
By default, docker creates a volume named `dtr-registry-${replica-id}` which is
used to host your data. You can supply a different volume name or directory
when installing or reconfiguring docker to change where DTR stores your data
locally.
To check how much space your images are taking in the local filesystem, you
can ssh into the node where DTR is deployed and run:
When using your local filesystem (or NFS) to serve images ensure there is enough
available space, otherwise pushes will begin to fail.
```
# Find the path to the volume
docker volume inspect dtr-registry-<replica-id>
You can see the total space used locally by running `du -hs "path-to-volume"`.
The path to the docker volume can be found by running `docker volume ls` to list
volumes and `docker volume inspect dtr-registry-$replicaId` to show the path.
# Check the disk usage
du -hs <path-to-volume>
```
### Amazon S3
## NFS
DTR supports AWS S3 plus other file servers that are S3 compatible, such as
Minio. For more information on configuring S3 or a compatible backend see the
[S3 configuration guide](
/datacenter/dtr/2.2/guides/admin/configure/external-storage/s3/).
You can configure your DTR replicas to store images on a NFS partition, so that
all replicas can share the same storage backend.
[Learn how to configure DTR with NFS](nfs.md).
### OpenStack Swift
## Amazon S3
OpenStack Swift, also known as OpenStack Object Storage, is an open source
object storage system that is licensed under the Apache 2.0 license. Refer to [Swift documentation](http://docs.openstack.org/developer/swift/) to get started.
### Microsoft Azure
This storage backend uses Microsofts Azure Blob storage. Data is stored within
a paid Windows Azure storage account. Refer to Microsoft's Azure
[documentation](https://azure.microsoft.com/en-us/services/storage/) which
explains how to set up your Storage account.
## Configure your Trusted Registry storage backend
Once you select your driver, you need to configure it through the UI or use a
YAML file (which is discussed further in this document.)
1. From the main Trusted Registry screen, navigate to Settings > Storage.
2. Under Storage Backend, use the drop down menu to select your storage. The screen refreshes to reflect your option.
3. Enter your configuration settings. If you're not sure what a particular parameter does, then find your driver from the following headings so that you can see a detailed explanation.
4. Click Save. The Trusted Registry restarts so that your changes take effect.
>**Note**: Changing your storage backend requires you to restart the Trusted Registry.
See the [Registry configuration](/registry/configuration.md)
documentation for the full options specific to each driver. Storage drivers can
be customized through the [Docker Registry storage driver
API](/registry/storage-drivers/index.md#storage-driver-api).
### Filesystem settings
The [filesystem storage backend](/registry/configuration.md#filesystem)
has only one setting, the "Storage directory".
### S3 settings
If you select the [S3 storage backend](/registry/configuration.md#s3), then you
need to set "AWS region", "Bucket name", "Access Key", and "Secret Key".
### Azure settings
Set the "Account name", "Account key", "Container", and "Realm" on the [Azure storage backend](/registry/configuration.md#azure) page.
### Openstack Swift settings
View the [Openstack Swift settings](/registry/configuration.md#openstack-swift)
documentation so that you can set up your storage settings: authurl, username,
password, container, tenant, tenantid, domain, domainid, insecureskipverify,
region, chunksize, and prefix.
## Configure using a YAML file
If the previous quick setup options are not sufficient to configure your
Registry options, you can upload a YAML file. The schema of this file is
identical to that used by the [Registry](/registry/configuration.md).
There are several benefits to using a YAML file as it can provide an
additional level of granularity in defining your storage backend. Advantages
include:
* Overriding specific configuration options.
* Overriding the entire configuration file.
* Selecting from the entire list of configuration options.
**To configure**:
1. Navigate to the Trusted Registry UI > Settings > Storage.
2. Select Download to get the text based file. It contains a minimum amount
of information and you're going to need additional data based on your driver and
business requirements.
3. Go [here](/registry/configuration.md#list-of-configuration-options") to see the open source YAML file. Copy the sections you need and paste into your `storage.yml` file. Note that some settings may contradict others, so
ensure your choices make sense.
4. Save the YAML file and return to the UI.
5. On the Storage screen, upload the file, review your changes, and click Save.
## Where to go next
* [Set up high availability](../set-up-high-availability.md)
DTR supports AWS3 or other storage systems that are S3-compatible like Minio.
[Learn how to configure DTR with Amazon S3](s3.md).

View File

@ -1,55 +1,35 @@
---
description: S3 storage configuration for Docker Trusted Registry
keywords: docker, dtr, storage driver, s3 storage, s3 compatible
title: Configuring S3 storage within DTR
title: Store images on Amazon S3
description: Learn how to configure Docker Trusted Registry to store Docker images on
Amazon S3
keywords: docker, dtr, storage driver, s3
---
You can configure DTR to store Docker images on Amazon S3, or other file servers
with an S3-compatible API like Minio.
DTR supports AWS S3 to store your images, plus other file servers that have an
S3 compatible API such as Minio. Other blobstores that are S3 compatible
generally use the same terminology, though setup may be slightly different.
Amazon S3 and compatible services store files in "buckets", and users have
permissions to read, write, and delete files from those buckets. When you
integrate DTR with Amazon S3, DTR sends all read and write operations to the
S3 bucket so that the images are persisted there.
### About S3
## Create a bucket on Amazon S3
S3 stores data as objects within “buckets” where you read, write, and delete
objects in that bucket. All read and write operations will be sent to S3 (or
your S3-compatible server), ensuring availability and durability of your images.
Before configuring DTR you need to create a bucket on Amazon S3.
To get faster pulls and pushes, you should create the S3 bucket on a region
that's physically close to the servers where DTR is running.
### Configuring S3 itself
Start by
[creating a bucket](http://docs.aws.amazon.com/AmazonS3/latest/gsg/CreatingABucket.html).
Then, as a best practice you should
[create a new IAM user](http://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html)
just for the DTR
integration and apply a IAM policy that ensures the user has limited permissions.
This section deals with creating and configuring bucket policies within AWS; if
you're using an S3-compatible server you may skip this section.
This user only needs permissions to access the bucket that you'll use to store
images, and be able to read, write, and delete files.
Prior to configuring DTR you need to make a "bucket" within S3. Buckets are
uniquely named containers in which S3 stores files.
You must:
1. Create a bucket within S3, choosing a region which is closest to your DTR
2. Note the bucket and region name for configuring DTR
You then need to configure authorization for your bucket. You can choose to use
an [access and secret key combination](
http://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html) for
the entirety of DTR (simple, but potentially less secure) or configure an [IAM
policy for the bucket and DTR](
http://docs.aws.amazon.com/AmazonS3/latest/dev/example-policies-s3.html) (more
complex to configure but also more secure, as access is restricted to only the
bucket).
If using an access key and secret key you should copy these and begin
configuring your storage settings within DTR
**Creating an IAM policy for your bucket**
You can set a policy through your AWS console to manage permissions for DTR.
For more information about setting IAM policies using the command line or the
console, review the AWS [Overview of IAM Policies](
http://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html) article
or visit the console Policies page.
The following example describes the minimum permissions set which allows
Trusted Registry users to access, push, pull, and delete images.
Here's an example of a policy like that:
```json
{
@ -66,7 +46,7 @@ Trusted Registry users to access, push, pull, and delete images.
"s3:ListBucket",
"s3:GetBucketLocation"
],
"Resource": "arn:aws:s3:::<INSERT YOUR BUCKET HERE>"
"Resource": "arn:aws:s3:::<bucket-name>"
},
{
"Effect": "Allow",
@ -75,42 +55,39 @@ Trusted Registry users to access, push, pull, and delete images.
"s3:GetObject",
"s3:DeleteObject"
],
"Resource": "arn:aws:s3:::<INSERT YOUR BUCKET HERE>/*"
"Resource": "arn:aws:s3:::<bucket-name>/*"
}
]
}
```
To set a policy through the AWS command line, save the policy into a file,
for example `TrustedRegistryUserPerms.json`, and pass it to the
put-user-policy AWS command:
```
$ aws iam put-user-policy --user-name MyUser --policy-name TrustedRegistryUserPerms --policy-document file://C:\Temp\TrustedRegistryUserPerms.json
```
You can also save this policy using the AWS console online.
### Configuring your storage settings
## Configure DTR
To configure your storage settings you must be a DTR administrator, and you must
have a bucket created within S3 with the
Once you've created a bucket and user, you can configure DTR to use it.
Navigate to the **DTR web UI**, go to **Settings**, and choose **Storage**.
You first need to create a bucket within Amazon S3
![](../../../images/s3-1.png){: .with-border}
1. First navigate to the storage settings tab, within "settings"
2. Choose "S3" from the storage list (even when using an S3-compatible backend):
![](../../../images/s3-1.png){: .with-border}
3. Fill out the form using your bucket name, region, and optionally access keys.
If you're using an IAM policy the secret key and access key can be
left blank. Also, when using an S3-compatible server these are likely your
username and password.
4. If using an S3 compatible server, change the region endpoint to the URL of
your server
5. Within "Advanced Settings" you can choose to use V4 auth and HTTPS. By
default HTTPS is on and V4 auth is off.
Select the **S3** option, and fill-in the information about the bucket and
user.
When hitting "Save" DTR validates that it can read and write to your new
settings and saves them once validated.
| Field | Description |
|:----------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Root directory | The path in the bucket where images are stored |
| AWS Region name | The region where the bucket is. [Learn more](http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) |
| S3 bucket name | The name of the bucket to store the images |
| AWS access key | The access key to use to access the S3 bucket. This can be left empty if you're using an AIM policy. [Learn more](http://docs.aws.amazon.com/general/latest/gr/managing-aws-access-keys.html) |
| AWS secret key | The secret key to use to access the S3 bucket. This can be left empty if you're using an AIM policy |
| Region endpoint | The endpoint name for the region you're using. [Learn more](http://docs.aws.amazon.com/general/latest/gr/rande.html#s3_region) |
There are also some advanced settings.
| Field | Description |
|:---------------|:----------------------------------------------------------------------------------------------------------------------------------------------------------|
| Version 4 auth | Authenticate the requests using AWS signature version 4. [Learn more](http://docs.aws.amazon.com/AmazonS3/latest/API/sig-v4-authenticating-requests.html) |
| Use HTTPS | Secure all requests with HTTPS, or make requests in an insecure way |
Once you click **Save**, DTR validates the configurations and saves the changes.

View File

@ -0,0 +1,160 @@
<?xml version="1.0" encoding="UTF-8"?>
<svg width="740px" height="250px" viewBox="0 0 740 250" version="1.1" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<!-- Generator: Sketch 42 (36781) - http://www.bohemiancoding.com/sketch -->
<title>architecture-3</title>
<desc>Created with Sketch.</desc>
<defs>
<circle id="path-1" cx="4" cy="4" r="4"></circle>
<mask id="mask-2" maskContentUnits="userSpaceOnUse" maskUnits="objectBoundingBox" x="-2" y="-2" width="12" height="12">
<rect x="-2" y="-2" width="12" height="12" fill="white"></rect>
<use xlink:href="#path-1" fill="black"></use>
</mask>
<circle id="path-3" cx="4" cy="4" r="4"></circle>
<mask id="mask-4" maskContentUnits="userSpaceOnUse" maskUnits="objectBoundingBox" x="-2" y="-2" width="12" height="12">
<rect x="-2" y="-2" width="12" height="12" fill="white"></rect>
<use xlink:href="#path-3" fill="black"></use>
</mask>
<circle id="path-5" cx="4" cy="4" r="4"></circle>
<mask id="mask-6" maskContentUnits="userSpaceOnUse" maskUnits="objectBoundingBox" x="-2" y="-2" width="12" height="12">
<rect x="-2" y="-2" width="12" height="12" fill="white"></rect>
<use xlink:href="#path-5" fill="black"></use>
</mask>
</defs>
<g id="dtr-diagrams" stroke="none" stroke-width="1" fill="none" fill-rule="evenodd">
<g id="architecture-3">
<g id="group" transform="translate(93.000000, 3.000000)">
<g id="dtr">
<text id="DTR-cluster" font-family="OpenSans-Semibold, Open Sans" font-size="10" font-weight="500" fill="#82949E">
<tspan x="7.025" y="236.009524">DTR cluster</tspan>
</text>
<g id="network" transform="translate(33.000000, 178.000000)">
<rect id="Rectangle-138" fill="#FFB463" x="0" y="0" width="366" height="22" rx="2"></rect>
<text id="overlay-network" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="146.05127" y="15">overlay network</tspan>
</text>
</g>
<g id="arrows" transform="translate(104.000000, 147.000000)">
<g id="arrow-copy-2" transform="translate(218.500000, 17.000000) rotate(-90.000000) translate(-218.500000, -17.000000) translate(202.000000, 13.000000)">
<path d="M2,4 L33,4" id="Line" stroke="#FFB463" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"></path>
<g id="Oval">
<use fill="#FFB463" fill-rule="evenodd" xlink:href="#path-1"></use>
<use stroke="#F7F8F9" mask="url(#mask-2)" stroke-width="4" xlink:href="#path-1"></use>
</g>
</g>
<g id="arrow-copy-3" transform="translate(111.500000, 17.000000) rotate(-90.000000) translate(-111.500000, -17.000000) translate(95.000000, 13.000000)">
<path d="M2,4 L33,4" id="Line" stroke="#FFB463" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"></path>
<g id="Oval">
<use fill="#FFB463" fill-rule="evenodd" xlink:href="#path-3"></use>
<use stroke="#F7F8F9" mask="url(#mask-4)" stroke-width="4" xlink:href="#path-3"></use>
</g>
</g>
<g id="arrow-copy-4" transform="translate(4.500000, 17.000000) rotate(-90.000000) translate(-4.500000, -17.000000) translate(-12.000000, 13.000000)">
<path d="M2,4 L33,4" id="Line" stroke="#FFB463" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"></path>
<g id="Oval">
<use fill="#FFB463" fill-rule="evenodd" xlink:href="#path-5"></use>
<use stroke="#F7F8F9" mask="url(#mask-6)" stroke-width="4" xlink:href="#path-5"></use>
</g>
</g>
</g>
<g id="nodes" transform="translate(60.000000, 45.000000)">
<g id="node-3" transform="translate(214.000000, 0.000000)">
<g id="node">
<g id="node-label">
<path d="M0,2.00295631 C0,0.896754086 0.897702336,0 1.99174577,0 L71,0 L71,10.6452381 C71,16.5244408 66.2312425,21.2904762 60.3513837,21.2904762 L0,21.2904762 L0,2.00295631 Z" id="Rectangle-127" fill="#445D6E"></path>
<text id="worker-node" font-family="OpenSans, Open Sans" font-size="8" font-weight="normal" fill="#FFFFFF">
<tspan x="6" y="14">worker node</tspan>
</text>
</g>
</g>
<g id="engine" transform="translate(1.000000, 79.000000)">
<rect id="Rectangle-138" fill="#1488C6" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="CS-Docker-Engine" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="7.26025391" y="15">CS Docker Engine</tspan>
</text>
</g>
<g id="ucp" transform="translate(1.000000, 56.000000)">
<rect id="Rectangle-138" fill="#1488C6" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="UCP-worker" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="20.4755859" y="15">UCP worker</tspan>
</text>
</g>
<g id="dtr" transform="translate(1.000000, 33.000000)">
<rect id="Rectangle-138" fill="#FFB463" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="DTR" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="38.4980469" y="15">DTR</tspan>
</text>
</g>
<rect id="node-border" stroke="#445D6E" stroke-width="2" x="0" y="0" width="97" height="102" rx="2"></rect>
</g>
<g id="node-2" transform="translate(107.000000, 0.000000)">
<g id="node">
<g id="node-label">
<path d="M0,2.00295631 C0,0.896754086 0.897702336,0 1.99174577,0 L71,0 L71,10.6452381 C71,16.5244408 66.2312425,21.2904762 60.3513837,21.2904762 L0,21.2904762 L0,2.00295631 Z" id="Rectangle-127" fill="#445D6E"></path>
<text id="worker-node" font-family="OpenSans, Open Sans" font-size="8" font-weight="normal" fill="#FFFFFF">
<tspan x="6" y="14">worker node</tspan>
</text>
</g>
</g>
<g id="engine" transform="translate(1.000000, 79.000000)">
<rect id="Rectangle-138" fill="#1488C6" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="CS-Docker-Engine" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="7.26025391" y="15">CS Docker Engine</tspan>
</text>
</g>
<g id="ucp" transform="translate(1.000000, 56.000000)">
<rect id="Rectangle-138" fill="#1488C6" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="UCP-worker" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="20.4755859" y="15">UCP worker</tspan>
</text>
</g>
<g id="dtr" transform="translate(1.000000, 33.000000)">
<rect id="Rectangle-138" fill="#FFB463" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="DTR" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="38.4980469" y="15">DTR</tspan>
</text>
</g>
<rect id="node-border" stroke="#445D6E" stroke-width="2" x="0" y="0" width="97" height="102" rx="2"></rect>
</g>
<g id="node-1">
<g id="node">
<g id="node-label">
<path d="M0,2.00295631 C0,0.896754086 0.897702336,0 1.99174577,0 L71,0 L71,10.6452381 C71,16.5244408 66.2312425,21.2904762 60.3513837,21.2904762 L0,21.2904762 L0,2.00295631 Z" id="Rectangle-127" fill="#445D6E"></path>
<text id="worker-node" font-family="OpenSans, Open Sans" font-size="8" font-weight="normal" fill="#FFFFFF">
<tspan x="6" y="14">worker node</tspan>
</text>
</g>
</g>
<g id="engine" transform="translate(1.000000, 79.000000)">
<rect id="Rectangle-138" fill="#1488C6" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="CS-Docker-Engine" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="7.26025391" y="15">CS Docker Engine</tspan>
</text>
</g>
<g id="ucp" transform="translate(1.000000, 56.000000)">
<rect id="Rectangle-138" fill="#1488C6" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="UCP-worker" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="20.4755859" y="15">UCP worker</tspan>
</text>
</g>
<g id="dtr" transform="translate(1.000000, 33.000000)">
<rect id="Rectangle-138" fill="#FFB463" x="0" y="0" width="95" height="22" rx="2"></rect>
<text id="DTR" font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#FFFFFF">
<tspan x="38.4980469" y="15">DTR</tspan>
</text>
</g>
<rect id="node-border" stroke="#445D6E" stroke-width="2" x="0" y="0" width="97" height="102" rx="2"></rect>
</g>
</g>
<rect id="group" stroke="#82949E" stroke-width="2" stroke-dasharray="5,5,5,5" x="0" y="0" width="433" height="245" rx="2"></rect>
</g>
<g id="object-storage" transform="translate(457.000000, 45.000000)">
<rect id="node-border" fill="#445D6E" x="0" y="0" width="97" height="102" rx="2"></rect>
<text font-family="OpenSans, Open Sans" font-size="10" font-weight="normal" fill="#F7F8F9">
<tspan x="34.7006836" y="48">object</tspan>
<tspan x="31.4633789" y="62">storage</tspan>
</text>
</g>
</g>
</g>
</g>
</svg>

After

Width:  |  Height:  |  Size: 12 KiB

Binary file not shown.

After

Width:  |  Height:  |  Size: 268 KiB