diff --git a/docs/en/README.md b/docs/en/README.md index 07346f06b..1b19ffd95 100644 --- a/docs/en/README.md +++ b/docs/en/README.md @@ -20,7 +20,7 @@ Organization of document is as following: * [scheduler](cli-reference/scheduler.md) * [manager](cli-reference/manager.md) * [Runtime Integration](runtime-integration/README.md) - * [containerd](runtime-integration/containerd.md) + * [containerd](runtime-integration/containerd/README.md) * [cri-o](runtime-integration/cri-o.md) * [docker](runtime-integration/docker.md) * [Preheat](preheat/README.md) diff --git a/docs/en/design/architecture.md b/docs/en/design/architecture.md index 9aacaf514..ed9ab5e33 100644 --- a/docs/en/design/architecture.md +++ b/docs/en/design/architecture.md @@ -37,11 +37,11 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented) ### Entity relationship -![alt][TODO association] +![alt][association] ### Image file download process -![alt][TODO download-process] +![alt][download-process] ### Sub-system architecture @@ -100,4 +100,6 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented) - Client connection management -[arch]: ../images/arch.png \ No newline at end of file +[arch]: ../images/arch.png +[association]: ../images/association.png +[download-process]: ../images/download-process.png \ No newline at end of file diff --git a/docs/en/images/association.png b/docs/en/images/association.png new file mode 100644 index 000000000..c293abfd0 Binary files /dev/null and b/docs/en/images/association.png differ diff --git a/docs/en/images/download-process.png b/docs/en/images/download-process.png new file mode 100644 index 000000000..9d1e5007f Binary files /dev/null and b/docs/en/images/download-process.png differ diff --git a/docs/en/quick-start.md b/docs/en/quick-start.md index df52ca5bd..98993a9dd 100644 --- a/docs/en/quick-start.md +++ b/docs/en/quick-start.md @@ -9,10 +9,11 @@ This table describes some container runtimes version and documents. | Runtime | Version | Document | CRI Support | Pull Command | | --- | --- | --- | --- | --- | -| Containerd without CRI | All | [Link](runtime-integration/containerd.md) | No | ctr image pull docker.io/library/alpine | -| Containerd with CRI | v1.1.0+ | [Link](runtime-integration/containerd.md) | Yes | crictl pull docker.io/library/alpine:latest | +| Containerd* | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest | +| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine | | CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest | +**:`containerd` is recommended* ## Runtime Configuration Guide for Dragonfly Helm Chart Dragonfly helm supports config docker automatically. diff --git a/docs/en/runtime-integration/README.md b/docs/en/runtime-integration/README.md index 00fc328bd..8e02a2cda 100644 --- a/docs/en/runtime-integration/README.md +++ b/docs/en/runtime-integration/README.md @@ -2,6 +2,6 @@ Table of contents: -* [containerd](containerd.md) +* [containerd](containerd/README.md) * [cri-o](cri-o.md) * [docker](docker.md) diff --git a/docs/en/runtime-integration/containerd.md b/docs/en/runtime-integration/containerd.md deleted file mode 100644 index 30ed56f4a..000000000 --- a/docs/en/runtime-integration/containerd.md +++ /dev/null @@ -1,276 +0,0 @@ -# Use dfget daemon as HTTP proxy for containerd - -Currently, `ctr` command of containerd doesn't support private registries with `registry-mirrors`, -in order to do so, we need to use HTTP proxy for containerd. - -## Quick Start - -### Step 1: Generate CA certificate for HTTP proxy - -Generate a CA certificate private key. - -```bash -openssl genrsa -out ca.key 2048 -``` - -Open openssl config file `openssl.conf`. Note set `basicConstraints` to true, that you can modify the values. - -```text -[ req ] -#default_bits = 2048 -#default_md = sha256 -#default_keyfile = privkey.pem -distinguished_name = req_distinguished_name -attributes = req_attributes -extensions = v3_ca -req_extensions = v3_ca - -[ req_distinguished_name ] -countryName = Country Name (2 letter code) -countryName_min = 2 -countryName_max = 2 -stateOrProvinceName = State or Province Name (full name) -localityName = Locality Name (eg, city) -0.organizationName = Organization Name (eg, company) -organizationalUnitName = Organizational Unit Name (eg, section) -commonName = Common Name (eg, fully qualified host name) -commonName_max = 64 -emailAddress = Email Address -emailAddress_max = 64 - -[ req_attributes ] -challengePassword = A challenge password -challengePassword_min = 4 -challengePassword_max = 20 - -[ v3_ca ] -basicConstraints = CA:TRUE -``` - -Generate the CA certificate. - -```bash -openssl req -new -key ca.key -nodes -out ca.csr -config openssl.conf -openssl x509 -req -days 36500 -extfile openssl.conf -extensions v3_ca -in ca.csr -signkey ca.key -out ca.crt -``` - -### Step 2: Configure dfget daemon - -To use dfget daemon as HTTP proxy, first you need to append a proxy rule in -`/etc/dragonfly/dfget.yaml`, This will proxy `your.private.registry`'s requests for image layers: - -```yaml -proxy: - security: - insecure: true - tcpListen: - listen: 0.0.0.0 - port: 65001 - proxies: - - regx: blobs/sha256.* - hijackHTTPS: - # CA certificate's path used to hijack https requests - cert: ca.crt - key: ca.key - hosts: - - regx: your.private.registry -``` - -### Step 3: Configure containerd - -Set dfget damone as `HTTP_PROXY` and `HTTPS_PROXY` for containerd in -`/etc/systemd/system/containerd.service.d/http-proxy.conf`: - -``` -[Service] -Environment="HTTP_PROXY=http://127.0.0.1:65001" -Environment="HTTPS_PROXY=http://127.0.0.1:65001" -``` - -### Step 4: Pull images with proxy - -Through the above steps, we can start to validate if Dragonfly works as expected. - -And you can pull the image as usual, for example: - -```bash -ctr image pull your.private.registry/namespace/image:latest -``` - -## Custom assets - -### Registry uses a self-signed certificate - -If your registry uses a self-signed certificate, you can either choose to -ignore the certificate error with: - -```yaml -proxy: - security: - insecure: true - tcpListen: - listen: 0.0.0.0 - port: 65001 - proxies: - - regx: blobs/sha256.* - hijackHTTPS: - # CA certificate's path used to hijack https requests - cert: ca.crt - key: ca.key - hosts: - - regx: your.private.registry - insecure: true -``` - -Or provide a certificate with: - -```yaml -proxy: - security: - insecure: true - tcpListen: - listen: 0.0.0.0 - port: 65001 - proxies: - - regx: blobs/sha256.* - hijackHTTPS: - # CA certificate's path used to hijack https requests - cert: ca.crt - key: ca.key - hosts: - - regx: your.private.registry - certs: ["server.crt"] -``` - -You can get the certificate of your server with: - -``` -openssl x509 -in <(openssl s_client -showcerts -servername xxx -connect xxx:443 -prexit 2>/dev/null) -``` - -# Use dfget daemon as registry mirror for Containerd with CRI support - -From v1.1.0, Containerd supports registry mirrors, we can configure Containerd via this feature for HA. - -## Quick Start - -### Step 1: Configure dfget daemon - -To use dfget daemon as registry mirror, first you need to ensure configuration in `/etc/dragonfly/dfget.yaml`: - -```yaml -proxy: - security: - insecure: true - tcpListen: - listen: 0.0.0.0 - port: 65001 - registryMirror: - # multiple registries support, if only mirror single registry, disable this - dynamic: true - url: https://index.docker.io - proxies: - - regx: blobs/sha256.* -``` - -Run dfget daemon - -```shell -dfget daemon -``` - -## Step 2: Configure Containerd - -### Option 1: Single Registry - -Enable mirrors in Containerd registries configuration in -`/etc/containerd/config.toml`: - -```toml -# explicitly use v2 config format, if already v2, skip the "version = 2" -version = 2 - -[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"] - endpoint = ["http://127.0.0.1:65001","https://registry-1.docker.io"] -``` - -In this config, there is two mirror endpoints for "docker.io", Containerd will pull images with `http://127.0.0.1:65001` first. -If `http://127.0.0.1:65001` is not available, the default `https://registry-1.docker.io` will be used for HA. - -> More details about Containerd configuration: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/registry.md#configure-registry-endpoint - -> Containerd has deprecated the above config from v1.4.0, new format for reference: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/config.md#registry-configuration - -### Option 2: Multiple Registries - -This option only supports Containerd 1.5.0+. - -#### 1. Enable Containerd Registries Config Path - -Enable mirrors in Containerd registries config path in -`/etc/containerd/config.toml`: - -```toml -# explicitly use v2 config format, if already v2, skip the "version = 2" -version = 2 - -[plugins."io.containerd.grpc.v1.cri".registry] - config_path = "/etc/containerd/certs.d" -``` - -#### 2. Generate Per Registry hosts.toml - -##### Option 1: Generate hosts.toml manually - -Path: `/etc/containerd/certs.d/example.com/hosts.toml` - -Replace `example.com` according the different registry domains. - -Content: - -```toml -server = "https://example.com" - -[host."http://127.0.0.1:65001"] - capabilities = ["pull", "resolve"] - [host."http://127.0.0.1:65001".header] - X-Dragonfly-Registry = ["https://example.com"] -``` - -##### Option 2: Generate hosts.toml automatically - -You can also generate hosts.toml with https://github.com/dragonflyoss/Dragonfly2/blob/main/hack/gen-containerd-hosts.sh - -```shell -bash gen-containerd-hosts.sh example.com -``` - -> More details about registry configuration: https://github.com/containerd/containerd/blob/main/docs/hosts.md#registry-configuration---examples - -## Step 3: Restart Containerd Daemon - -``` -systemctl restart containerd -``` - -## Step 4: Pull Image - -You can pull image like this: - -``` -crictl pull docker.io/library/busybox -``` - -## Step 5: Validate Dragonfly - -You can execute the following command to check if the busybox image is distributed via Dragonfly. - -```bash -grep 'register peer task result' /var/log/dragonfly/daemon/*.log -``` - -If the output of command above has content like - -``` -{"level":"info","ts":"2021-02-23 20:03:20.306","caller":"client/client.go:83","msg":"register peer task result:true[200] for taskId:adf62a86f001e17037eedeaaba3393f3519b80ce,peerIp:10.15.233.91,securityDomain:,idc:,scheduler:127.0.0.1:8002","peerId":"10.15.233.91-65000-43096-1614081800301788000","errMsg":null} -``` diff --git a/docs/en/runtime-integration/containerd/README.md b/docs/en/runtime-integration/containerd/README.md new file mode 100644 index 000000000..4405fcb6b --- /dev/null +++ b/docs/en/runtime-integration/containerd/README.md @@ -0,0 +1,8 @@ +# Containerd + +This documentation will help you to integrate Dragonfly2 into Containerd. We recommend to use `mirror`. + +Table of contents: + +* [mirror](mirror.md) +* [proxy](proxy.md) diff --git a/docs/en/runtime-integration/containerd/mirror.md b/docs/en/runtime-integration/containerd/mirror.md new file mode 100644 index 000000000..3588a7759 --- /dev/null +++ b/docs/en/runtime-integration/containerd/mirror.md @@ -0,0 +1,126 @@ +# Use dfget daemon for containerd + +From v1.1.0, Containerd supports registry mirrors, we can configure Containerd via this feature for HA. + +## Quick Start + +### Step 1: Configure dfget daemon + +To use dfget daemon as registry mirror, first you need to ensure configuration in `/etc/dragonfly/dfget.yaml`: + +```yaml +proxy: + security: + insecure: true + tcpListen: + listen: 0.0.0.0 + port: 65001 + registryMirror: + # multiple registries support, if only mirror single registry, disable this + dynamic: true + url: https://index.docker.io + proxies: + - regx: blobs/sha256.* +``` + +Run dfget daemon + +```shell +dfget daemon +``` + +## Step 2: Configure Containerd + +### Option 1: Single Registry + +Enable mirrors in Containerd registries configuration in +`/etc/containerd/config.toml`: + +```toml +# explicitly use v2 config format, if already v2, skip the "version = 2" +version = 2 + +[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"] + endpoint = ["http://127.0.0.1:65001","https://registry-1.docker.io"] +``` + +In this config, there is two mirror endpoints for "docker.io", Containerd will pull images with `http://127.0.0.1:65001` first. +If `http://127.0.0.1:65001` is not available, the default `https://registry-1.docker.io` will be used for HA. + +> More details about Containerd configuration: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/registry.md#configure-registry-endpoint + +> Containerd has deprecated the above config from v1.4.0, new format for reference: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/config.md#registry-configuration + +### Option 2: Multiple Registries + +This option only supports Containerd 1.5.0+. + +#### 1. Enable Containerd Registries Config Path + +Enable mirrors in Containerd registries config path in +`/etc/containerd/config.toml`: + +```toml +# explicitly use v2 config format, if already v2, skip the "version = 2" +version = 2 + +[plugins."io.containerd.grpc.v1.cri".registry] + config_path = "/etc/containerd/certs.d" +``` + +#### 2. Generate Per Registry hosts.toml + +##### Option 1: Generate hosts.toml manually + +Path: `/etc/containerd/certs.d/example.com/hosts.toml` + +Replace `example.com` according the different registry domains. + +Content: + +```toml +server = "https://example.com" + +[host."http://127.0.0.1:65001"] + capabilities = ["pull", "resolve"] + [host."http://127.0.0.1:65001".header] + X-Dragonfly-Registry = ["https://example.com"] +``` + +##### Option 2: Generate hosts.toml automatically + +You can also generate hosts.toml with https://github.com/dragonflyoss/Dragonfly2/blob/main/hack/gen-containerd-hosts.sh + +```shell +bash gen-containerd-hosts.sh example.com +``` + +> More details about registry configuration: https://github.com/containerd/containerd/blob/main/docs/hosts.md#registry-configuration---examples + +## Step 3: Restart Containerd Daemon + +``` +systemctl restart containerd +``` + +## Step 4: Pull Image + +You can pull image like this: + +``` +crictl pull docker.io/library/busybox +``` + +## Step 5: Validate Dragonfly + +You can execute the following command to check if the busybox image is distributed via Dragonfly. + +```bash +grep 'register peer task result' /var/log/dragonfly/daemon/*.log +``` + +If the output of command above has content like + +``` +{"level":"info","ts":"2021-02-23 20:03:20.306","caller":"client/client.go:83","msg":"register peer task result:true[200] for taskId:adf62a86f001e17037eedeaaba3393f3519b80ce,peerIp:10.15.233.91,securityDomain:,idc:,scheduler:127.0.0.1:8002","peerId":"10.15.233.91-65000-43096-1614081800301788000","errMsg":null} +``` \ No newline at end of file diff --git a/docs/en/runtime-integration/containerd/proxy.md b/docs/en/runtime-integration/containerd/proxy.md new file mode 100644 index 000000000..8e09ad28c --- /dev/null +++ b/docs/en/runtime-integration/containerd/proxy.md @@ -0,0 +1,149 @@ +# Use dfget daemon as HTTP proxy for containerd + +Currently, `ctr` command of containerd doesn't support private registries with `registry-mirrors`, +in order to do so, we need to use HTTP proxy for containerd. + +## Quick Start + +### Step 1: Generate CA certificate for HTTP proxy + +Generate a CA certificate private key. + +```bash +openssl genrsa -out ca.key 2048 +``` + +Open openssl config file `openssl.conf`. Note set `basicConstraints` to true, that you can modify the values. + +```text +[ req ] +#default_bits = 2048 +#default_md = sha256 +#default_keyfile = privkey.pem +distinguished_name = req_distinguished_name +attributes = req_attributes +extensions = v3_ca +req_extensions = v3_ca + +[ req_distinguished_name ] +countryName = Country Name (2 letter code) +countryName_min = 2 +countryName_max = 2 +stateOrProvinceName = State or Province Name (full name) +localityName = Locality Name (eg, city) +0.organizationName = Organization Name (eg, company) +organizationalUnitName = Organizational Unit Name (eg, section) +commonName = Common Name (eg, fully qualified host name) +commonName_max = 64 +emailAddress = Email Address +emailAddress_max = 64 + +[ req_attributes ] +challengePassword = A challenge password +challengePassword_min = 4 +challengePassword_max = 20 + +[ v3_ca ] +basicConstraints = CA:TRUE +``` + +Generate the CA certificate. + +```bash +openssl req -new -key ca.key -nodes -out ca.csr -config openssl.conf +openssl x509 -req -days 36500 -extfile openssl.conf -extensions v3_ca -in ca.csr -signkey ca.key -out ca.crt +``` + +### Step 2: Configure dfget daemon + +To use dfget daemon as HTTP proxy, first you need to append a proxy rule in +`/etc/dragonfly/dfget.yaml`, This will proxy `your.private.registry`'s requests for image layers: + +```yaml +proxy: + security: + insecure: true + tcpListen: + listen: 0.0.0.0 + port: 65001 + proxies: + - regx: blobs/sha256.* + hijackHTTPS: + # CA certificate's path used to hijack https requests + cert: ca.crt + key: ca.key + hosts: + - regx: your.private.registry +``` + +### Step 3: Configure containerd + +Set dfget damone as `HTTP_PROXY` and `HTTPS_PROXY` for containerd in +`/etc/systemd/system/containerd.service.d/http-proxy.conf`: + +``` +[Service] +Environment="HTTP_PROXY=http://127.0.0.1:65001" +Environment="HTTPS_PROXY=http://127.0.0.1:65001" +``` + +### Step 4: Pull images with proxy + +Through the above steps, we can start to validate if Dragonfly works as expected. + +And you can pull the image as usual, for example: + +```bash +ctr image pull your.private.registry/namespace/image:latest +``` + +## Custom assets + +### Registry uses a self-signed certificate + +If your registry uses a self-signed certificate, you can either choose to +ignore the certificate error with: + +```yaml +proxy: + security: + insecure: true + tcpListen: + listen: 0.0.0.0 + port: 65001 + proxies: + - regx: blobs/sha256.* + hijackHTTPS: + # CA certificate's path used to hijack https requests + cert: ca.crt + key: ca.key + hosts: + - regx: your.private.registry + insecure: true +``` + +Or provide a certificate with: + +```yaml +proxy: + security: + insecure: true + tcpListen: + listen: 0.0.0.0 + port: 65001 + proxies: + - regx: blobs/sha256.* + hijackHTTPS: + # CA certificate's path used to hijack https requests + cert: ca.crt + key: ca.key + hosts: + - regx: your.private.registry + certs: ["server.crt"] +``` + +You can get the certificate of your server with: + +``` +openssl x509 -in <(openssl s_client -showcerts -servername xxx -connect xxx:443 -prexit 2>/dev/null) +``` \ No newline at end of file