docs(runtime): upgrade containerd runtime (#748)

* docs(runtime): upgrade containerd runtime

Signed-off-by: cndoit18 <cndoit18@outlook.com>

* add images

Signed-off-by: cndoit18 <cndoit18@outlook.com>

docs/en/README.md

@@ -20,7 +20,7 @@ Organization of document is as following:
   * [scheduler](cli-reference/scheduler.md)
   * [manager](cli-reference/manager.md)
 * [Runtime Integration](runtime-integration/README.md)
-  * [containerd](runtime-integration/containerd.md)
+  * [containerd](runtime-integration/containerd/README.md)
   * [cri-o](runtime-integration/cri-o.md)
   * [docker](runtime-integration/docker.md)
 * [Preheat](preheat/README.md)

docs/en/design/architecture.md

@@ -37,11 +37,11 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented)
 
 ### Entity relationship
 
-![alt][TODO association]
+![alt][association]
 
 ### Image file download process
 
-![alt][TODO download-process]
+![alt][download-process]
 
 ### Sub-system architecture
 
@@ -100,4 +100,6 @@ Provide enterprise-level (efficient, stable, secure, low-cost, product-oriented)
 - Client connection management
 
 
-[arch]: ../images/arch.png
+[arch]: ../images/arch.png
+[association]: ../images/association.png
+[download-process]: ../images/download-process.png

docs/en/quick-start.md

@@ -9,10 +9,11 @@ This table describes some container runtimes version and documents.
 
 | Runtime | Version | Document | CRI Support | Pull Command |
 | --- | --- | --- | --- | --- | 
-| Containerd without CRI | All | [Link](runtime-integration/containerd.md) | No | ctr image pull docker.io/library/alpine |
+| Containerd<sup>*</sup> | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest |
-| Containerd with CRI | v1.1.0+ | [Link](runtime-integration/containerd.md) | Yes | crictl pull docker.io/library/alpine:latest |
+| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine |
 | CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest |
 
+**:`containerd` is recommended*
 ## Runtime Configuration Guide for Dragonfly Helm Chart
 
 Dragonfly helm supports config docker automatically.

docs/en/runtime-integration/README.md

@@ -2,6 +2,6 @@
 
 Table of contents:
 
-* [containerd](containerd.md)
+* [containerd](containerd/README.md)
 * [cri-o](cri-o.md)
 * [docker](docker.md)

docs/en/runtime-integration/containerd.md

@@ -1,276 +0,0 @@
-# Use dfget daemon as HTTP proxy for containerd
-
-Currently, `ctr` command of containerd doesn't support private registries with `registry-mirrors`,
-in order to do so, we need to use HTTP proxy for containerd.
-
-## Quick Start
-
-### Step 1: Generate CA certificate for HTTP proxy
-
-Generate a CA certificate private key.
-
-```bash
-openssl genrsa -out ca.key 2048
-```
-
-Open openssl config file `openssl.conf`. Note set `basicConstraints` to true, that you can modify the values.
-
-```text
-[ req ]
-#default_bits		= 2048
-#default_md		= sha256
-#default_keyfile 	= privkey.pem
-distinguished_name	= req_distinguished_name
-attributes		= req_attributes
-extensions               = v3_ca
-req_extensions           = v3_ca
-
-[ req_distinguished_name ]
-countryName			= Country Name (2 letter code)
-countryName_min			= 2
-countryName_max			= 2
-stateOrProvinceName		= State or Province Name (full name)
-localityName			= Locality Name (eg, city)
-0.organizationName		= Organization Name (eg, company)
-organizationalUnitName		= Organizational Unit Name (eg, section)
-commonName			= Common Name (eg, fully qualified host name)
-commonName_max			= 64
-emailAddress			= Email Address
-emailAddress_max		= 64
-
-[ req_attributes ]
-challengePassword		= A challenge password
-challengePassword_min		= 4
-challengePassword_max		= 20
-
-[ v3_ca ]
-basicConstraints         = CA:TRUE
-```
-
-Generate the CA certificate.
-
-```bash
-openssl req -new -key ca.key -nodes -out ca.csr -config openssl.conf
-openssl x509 -req -days 36500 -extfile openssl.conf -extensions v3_ca -in ca.csr -signkey ca.key -out ca.crt
-```
-
-### Step 2: Configure dfget daemon
-
-To use dfget daemon as HTTP proxy, first you need to append a proxy rule in
-`/etc/dragonfly/dfget.yaml`, This will proxy `your.private.registry`'s requests for image layers:
-
-```yaml
-proxy:
-  security:
-    insecure: true
-  tcpListen:
-    listen: 0.0.0.0
-    port: 65001
-  proxies:
-    - regx: blobs/sha256.*
-  hijackHTTPS:
-    # CA certificate's path used to hijack https requests
-    cert: ca.crt
-    key: ca.key
-    hosts:
-      - regx: your.private.registry
-```
-
-### Step 3: Configure containerd
-
-Set dfget damone as `HTTP_PROXY` and `HTTPS_PROXY` for containerd in
-`/etc/systemd/system/containerd.service.d/http-proxy.conf`:
-
-```
-[Service]
-Environment="HTTP_PROXY=http://127.0.0.1:65001"
-Environment="HTTPS_PROXY=http://127.0.0.1:65001"
-```
-
-### Step 4: Pull images with proxy
-
-Through the above steps, we can start to validate if Dragonfly works as expected.
-
-And you can pull the image as usual, for example:
-
-```bash
-ctr image pull your.private.registry/namespace/image:latest
-```
-
-## Custom assets
-
-### Registry uses a self-signed certificate
-
-If your registry uses a self-signed certificate, you can either choose to
-ignore the certificate error with:
-
-```yaml
-proxy:
-  security:
-    insecure: true
-  tcpListen:
-    listen: 0.0.0.0
-    port: 65001
-  proxies:
-    - regx: blobs/sha256.*
-  hijackHTTPS:
-    # CA certificate's path used to hijack https requests
-    cert: ca.crt
-    key: ca.key
-    hosts:
-      - regx: your.private.registry
-        insecure: true
-```
-
-Or provide a certificate with:
-
-```yaml
-proxy:
-  security:
-    insecure: true
-  tcpListen:
-    listen: 0.0.0.0
-    port: 65001
-  proxies:
-    - regx: blobs/sha256.*
-  hijackHTTPS:
-    # CA certificate's path used to hijack https requests
-    cert: ca.crt
-    key: ca.key
-    hosts:
-      - regx: your.private.registry
-        certs: ["server.crt"]
-```
-
-You can get the certificate of your server with:
-
-```
-openssl x509 -in <(openssl s_client -showcerts -servername xxx -connect xxx:443 -prexit 2>/dev/null)
-```
-
-# Use dfget daemon as registry mirror for Containerd with CRI support
-
-From v1.1.0, Containerd supports registry mirrors, we can configure Containerd via this feature for HA.
-
-## Quick Start
-
-### Step 1: Configure dfget daemon
-
-To use dfget daemon as registry mirror, first you need to ensure configuration in `/etc/dragonfly/dfget.yaml`:
-
-```yaml
-proxy:
-  security:
-    insecure: true
-  tcpListen:
-    listen: 0.0.0.0
-    port: 65001
-  registryMirror:
-    # multiple registries support, if only mirror single registry, disable this
-    dynamic: true
-    url: https://index.docker.io
-  proxies:
-    - regx: blobs/sha256.*
-```
-
-Run dfget daemon
-
-```shell
-dfget daemon
-```
-
-## Step 2: Configure Containerd
-
-### Option 1: Single Registry
-
-Enable mirrors in Containerd registries configuration in
-`/etc/containerd/config.toml`:
-
-```toml
-# explicitly use v2 config format, if already v2, skip the "version = 2"
-version = 2
-
-[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
-  endpoint = ["http://127.0.0.1:65001","https://registry-1.docker.io"]
-```
-
-In this config, there is two mirror endpoints for "docker.io", Containerd will pull images with `http://127.0.0.1:65001` first.
-If `http://127.0.0.1:65001` is not available, the default `https://registry-1.docker.io` will be used for HA.
-
-> More details about Containerd configuration: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/registry.md#configure-registry-endpoint
-
-> Containerd has deprecated the above config from v1.4.0, new format for reference: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/config.md#registry-configuration
-
-### Option 2: Multiple Registries
-
-This option only supports Containerd 1.5.0+.
-
-#### 1. Enable Containerd Registries Config Path
-
-Enable mirrors in Containerd registries config path in
-`/etc/containerd/config.toml`:
-
-```toml
-# explicitly use v2 config format, if already v2, skip the "version = 2"
-version = 2
-
-[plugins."io.containerd.grpc.v1.cri".registry]
-  config_path = "/etc/containerd/certs.d"
-```
-
-#### 2. Generate Per Registry hosts.toml
-
-##### Option 1: Generate hosts.toml manually
-
-Path: `/etc/containerd/certs.d/example.com/hosts.toml`
-
-Replace `example.com` according the different registry domains.
-
-Content:
-
-```toml
-server = "https://example.com"
-
-[host."http://127.0.0.1:65001"]
-  capabilities = ["pull", "resolve"]
-  [host."http://127.0.0.1:65001".header]
-    X-Dragonfly-Registry = ["https://example.com"]
-```
-
-##### Option 2: Generate hosts.toml automatically
-
-You can also generate hosts.toml with https://github.com/dragonflyoss/Dragonfly2/blob/main/hack/gen-containerd-hosts.sh
-
-```shell
-bash gen-containerd-hosts.sh example.com
-```
-
-> More details about registry configuration: https://github.com/containerd/containerd/blob/main/docs/hosts.md#registry-configuration---examples
-
-## Step 3: Restart Containerd Daemon
-
-```
-systemctl restart containerd
-```
-
-## Step 4: Pull Image
-
-You can pull image like this:
-
-```
-crictl pull docker.io/library/busybox
-```
-
-## Step 5: Validate Dragonfly
-
-You can execute the following command to check if the busybox image is distributed via Dragonfly.
-
-```bash
-grep 'register peer task result' /var/log/dragonfly/daemon/*.log
-```
-
-If the output of command above has content like
-
-```
-{"level":"info","ts":"2021-02-23 20:03:20.306","caller":"client/client.go:83","msg":"register peer task result:true[200] for taskId:adf62a86f001e17037eedeaaba3393f3519b80ce,peerIp:10.15.233.91,securityDomain:,idc:,scheduler:127.0.0.1:8002","peerId":"10.15.233.91-65000-43096-1614081800301788000","errMsg":null}
-```

docs/en/runtime-integration/containerd/README.md

@@ -0,0 +1,8 @@
+# Containerd
+
+This documentation will help you to integrate Dragonfly2 into Containerd. We recommend to use `mirror`.
+
+Table of contents:
+
+* [mirror](mirror.md)
+* [proxy](proxy.md)

docs/en/runtime-integration/containerd/mirror.md

@@ -0,0 +1,126 @@
+# Use dfget daemon for containerd
+
+From v1.1.0, Containerd supports registry mirrors, we can configure Containerd via this feature for HA.
+
+## Quick Start
+
+### Step 1: Configure dfget daemon
+
+To use dfget daemon as registry mirror, first you need to ensure configuration in `/etc/dragonfly/dfget.yaml`:
+
+```yaml
+proxy:
+  security:
+    insecure: true
+  tcpListen:
+    listen: 0.0.0.0
+    port: 65001
+  registryMirror:
+    # multiple registries support, if only mirror single registry, disable this
+    dynamic: true
+    url: https://index.docker.io
+  proxies:
+    - regx: blobs/sha256.*
+```
+
+Run dfget daemon
+
+```shell
+dfget daemon
+```
+
+## Step 2: Configure Containerd
+
+### Option 1: Single Registry
+
+Enable mirrors in Containerd registries configuration in
+`/etc/containerd/config.toml`:
+
+```toml
+# explicitly use v2 config format, if already v2, skip the "version = 2"
+version = 2
+
+[plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
+  endpoint = ["http://127.0.0.1:65001","https://registry-1.docker.io"]
+```
+
+In this config, there is two mirror endpoints for "docker.io", Containerd will pull images with `http://127.0.0.1:65001` first.
+If `http://127.0.0.1:65001` is not available, the default `https://registry-1.docker.io` will be used for HA.
+
+> More details about Containerd configuration: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/registry.md#configure-registry-endpoint
+
+> Containerd has deprecated the above config from v1.4.0, new format for reference: https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/config.md#registry-configuration
+
+### Option 2: Multiple Registries
+
+This option only supports Containerd 1.5.0+.
+
+#### 1. Enable Containerd Registries Config Path
+
+Enable mirrors in Containerd registries config path in
+`/etc/containerd/config.toml`:
+
+```toml
+# explicitly use v2 config format, if already v2, skip the "version = 2"
+version = 2
+
+[plugins."io.containerd.grpc.v1.cri".registry]
+  config_path = "/etc/containerd/certs.d"
+```
+
+#### 2. Generate Per Registry hosts.toml
+
+##### Option 1: Generate hosts.toml manually
+
+Path: `/etc/containerd/certs.d/example.com/hosts.toml`
+
+Replace `example.com` according the different registry domains.
+
+Content:
+
+```toml
+server = "https://example.com"
+
+[host."http://127.0.0.1:65001"]
+  capabilities = ["pull", "resolve"]
+  [host."http://127.0.0.1:65001".header]
+    X-Dragonfly-Registry = ["https://example.com"]
+```
+
+##### Option 2: Generate hosts.toml automatically
+
+You can also generate hosts.toml with https://github.com/dragonflyoss/Dragonfly2/blob/main/hack/gen-containerd-hosts.sh
+
+```shell
+bash gen-containerd-hosts.sh example.com
+```
+
+> More details about registry configuration: https://github.com/containerd/containerd/blob/main/docs/hosts.md#registry-configuration---examples
+
+## Step 3: Restart Containerd Daemon
+
+```
+systemctl restart containerd
+```
+
+## Step 4: Pull Image
+
+You can pull image like this:
+
+```
+crictl pull docker.io/library/busybox
+```
+
+## Step 5: Validate Dragonfly
+
+You can execute the following command to check if the busybox image is distributed via Dragonfly.
+
+```bash
+grep 'register peer task result' /var/log/dragonfly/daemon/*.log
+```
+
+If the output of command above has content like
+
+```
+{"level":"info","ts":"2021-02-23 20:03:20.306","caller":"client/client.go:83","msg":"register peer task result:true[200] for taskId:adf62a86f001e17037eedeaaba3393f3519b80ce,peerIp:10.15.233.91,securityDomain:,idc:,scheduler:127.0.0.1:8002","peerId":"10.15.233.91-65000-43096-1614081800301788000","errMsg":null}
+```

docs/en/runtime-integration/containerd/proxy.md

@@ -0,0 +1,149 @@
+# Use dfget daemon as HTTP proxy for containerd
+
+Currently, `ctr` command of containerd doesn't support private registries with `registry-mirrors`,
+in order to do so, we need to use HTTP proxy for containerd.
+
+## Quick Start
+
+### Step 1: Generate CA certificate for HTTP proxy
+
+Generate a CA certificate private key.
+
+```bash
+openssl genrsa -out ca.key 2048
+```
+
+Open openssl config file `openssl.conf`. Note set `basicConstraints` to true, that you can modify the values.
+
+```text
+[ req ]
+#default_bits		= 2048
+#default_md		= sha256
+#default_keyfile 	= privkey.pem
+distinguished_name	= req_distinguished_name
+attributes		= req_attributes
+extensions               = v3_ca
+req_extensions           = v3_ca
+
+[ req_distinguished_name ]
+countryName			= Country Name (2 letter code)
+countryName_min			= 2
+countryName_max			= 2
+stateOrProvinceName		= State or Province Name (full name)
+localityName			= Locality Name (eg, city)
+0.organizationName		= Organization Name (eg, company)
+organizationalUnitName		= Organizational Unit Name (eg, section)
+commonName			= Common Name (eg, fully qualified host name)
+commonName_max			= 64
+emailAddress			= Email Address
+emailAddress_max		= 64
+
+[ req_attributes ]
+challengePassword		= A challenge password
+challengePassword_min		= 4
+challengePassword_max		= 20
+
+[ v3_ca ]
+basicConstraints         = CA:TRUE
+```
+
+Generate the CA certificate.
+
+```bash
+openssl req -new -key ca.key -nodes -out ca.csr -config openssl.conf
+openssl x509 -req -days 36500 -extfile openssl.conf -extensions v3_ca -in ca.csr -signkey ca.key -out ca.crt
+```
+
+### Step 2: Configure dfget daemon
+
+To use dfget daemon as HTTP proxy, first you need to append a proxy rule in
+`/etc/dragonfly/dfget.yaml`, This will proxy `your.private.registry`'s requests for image layers:
+
+```yaml
+proxy:
+  security:
+    insecure: true
+  tcpListen:
+    listen: 0.0.0.0
+    port: 65001
+  proxies:
+    - regx: blobs/sha256.*
+  hijackHTTPS:
+    # CA certificate's path used to hijack https requests
+    cert: ca.crt
+    key: ca.key
+    hosts:
+      - regx: your.private.registry
+```
+
+### Step 3: Configure containerd
+
+Set dfget damone as `HTTP_PROXY` and `HTTPS_PROXY` for containerd in
+`/etc/systemd/system/containerd.service.d/http-proxy.conf`:
+
+```
+[Service]
+Environment="HTTP_PROXY=http://127.0.0.1:65001"
+Environment="HTTPS_PROXY=http://127.0.0.1:65001"
+```
+
+### Step 4: Pull images with proxy
+
+Through the above steps, we can start to validate if Dragonfly works as expected.
+
+And you can pull the image as usual, for example:
+
+```bash
+ctr image pull your.private.registry/namespace/image:latest
+```
+
+## Custom assets
+
+### Registry uses a self-signed certificate
+
+If your registry uses a self-signed certificate, you can either choose to
+ignore the certificate error with:
+
+```yaml
+proxy:
+  security:
+    insecure: true
+  tcpListen:
+    listen: 0.0.0.0
+    port: 65001
+  proxies:
+    - regx: blobs/sha256.*
+  hijackHTTPS:
+    # CA certificate's path used to hijack https requests
+    cert: ca.crt
+    key: ca.key
+    hosts:
+      - regx: your.private.registry
+        insecure: true
+```
+
+Or provide a certificate with:
+
+```yaml
+proxy:
+  security:
+    insecure: true
+  tcpListen:
+    listen: 0.0.0.0
+    port: 65001
+  proxies:
+    - regx: blobs/sha256.*
+  hijackHTTPS:
+    # CA certificate's path used to hijack https requests
+    cert: ca.crt
+    key: ca.key
+    hosts:
+      - regx: your.private.registry
+        certs: ["server.crt"]
+```
+
+You can get the certificate of your server with:
+
+```
+openssl x509 -in <(openssl s_client -showcerts -servername xxx -connect xxx:443 -prexit 2>/dev/null)
+```