From c7d02ef8be36029d53ea437b3047841664298c7c Mon Sep 17 00:00:00 2001 From: Gaius Date: Wed, 16 Feb 2022 17:18:01 +0800 Subject: [PATCH] docs: add metrics document (#1075) * docs: add metrics document Signed-off-by: Gaius * docs: format docs Signed-off-by: Gaius * chore: change markdownlint rules Signed-off-by: Gaius --- .markdownlint.yml | 2 + docs/en/README.md | 56 +++++++------ docs/en/cli-reference/README.md | 8 +- docs/en/cli-reference/cdn.md | 6 +- docs/en/cli-reference/dfget.md | 4 + docs/en/cli-reference/manager.md | 4 +- docs/en/cli-reference/scheduler.md | 6 +- docs/en/deployment/README.md | 4 +- docs/en/deployment/configuration/cdn.yaml | 6 +- docs/en/deployment/configuration/dfget.yaml | 52 ++++++------ docs/en/deployment/configuration/manager.yaml | 2 +- .../deployment/configuration/scheduler.yaml | 18 ++-- docs/en/deployment/installation/README.md | 6 +- .../deployment/installation/docker/README.md | 4 +- .../installation/docker/standalone/README.md | 8 +- .../installation/docker/standalone/cdn.md | 42 +++++----- .../docker/standalone/dfdaemon.md | 40 ++++----- .../installation/docker/standalone/manager.md | 40 ++++----- .../docker/standalone/scheduler.md | 40 ++++----- .../installation/executable-files/README.md | 8 +- .../installation/executable-files/cdn.md | 82 +++++++++--------- .../installation/executable-files/dfdaemon.md | 48 +++++------ .../installation/executable-files/manager.md | 50 +++++------ .../executable-files/scheduler.md | 50 +++++------ .../installation/kubernetes/README.md | 6 +- .../installation/kubernetes/helm.md | 74 ++++++++-------- docs/en/design/README.md | 10 +-- docs/en/design/architecture.md | 2 +- docs/en/developer-guide/README.md | 8 ++ .../{developer-guide.md => development.md} | 2 + docs/en/developer-guide/plugin.md | 19 ++++- docs/en/preheat/README.md | 4 +- docs/en/quick-start.md | 20 +++-- docs/en/runtime-integration/README.md | 6 +- .../runtime-integration/containerd/README.md | 4 +- .../runtime-integration/containerd/mirror.md | 2 +- .../runtime-integration/containerd/proxy.md | 2 +- docs/en/runtime-integration/docker.md | 2 +- docs/en/troubleshooting/README.md | 22 ++--- docs/en/user-guide/README.md | 7 ++ docs/en/user-guide/metrics.md | 67 +++++++++++++++ docs/zh-CN/README.md | 56 +++++++------ docs/zh-CN/cli-reference/README.md | 8 +- docs/zh-CN/cli-reference/cdn.md | 4 +- docs/zh-CN/cli-reference/dfget.md | 10 ++- docs/zh-CN/cli-reference/manager.md | 6 +- docs/zh-CN/cli-reference/scheduler.md | 4 +- docs/zh-CN/deployment/README.md | 4 +- docs/zh-CN/deployment/configuration/cdn.yaml | 10 +-- .../zh-CN/deployment/configuration/dfget.yaml | 52 ++++++------ .../deployment/configuration/manager.yaml | 2 +- .../deployment/configuration/scheduler.yaml | 20 ++--- docs/zh-CN/deployment/installation/README.md | 6 +- .../deployment/installation/docker/README.md | 4 +- .../installation/docker/standalone/README.md | 8 +- .../installation/docker/standalone/cdn.md | 40 ++++----- .../docker/standalone/dfdaemon.md | 40 ++++----- .../installation/docker/standalone/manager.md | 40 ++++----- .../docker/standalone/scheduler.md | 40 ++++----- .../installation/executable-files/README.md | 8 +- .../installation/executable-files/cdn.md | 84 +++++++++---------- .../installation/executable-files/dfdaemon.md | 46 +++++----- .../installation/executable-files/manager.md | 48 +++++------ .../executable-files/scheduler.md | 48 +++++------ .../installation/kubernetes/README.md | 6 +- .../installation/kubernetes/helm.md | 32 +++---- docs/zh-CN/design/README.md | 10 +-- docs/zh-CN/design/architecture.md | 8 +- docs/zh-CN/developer-guide/README.md | 8 ++ docs/zh-CN/developer-guide/developer-guide.md | 64 -------------- docs/zh-CN/developer-guide/development.md | 3 + docs/zh-CN/preheat/README.md | 4 +- docs/zh-CN/quick-start.md | 22 ++--- docs/zh-CN/runtime-integration/README.md | 6 +- .../runtime-integration/containerd/README.md | 4 +- .../runtime-integration/containerd/mirror.md | 2 +- .../runtime-integration/containerd/proxy.md | 2 +- docs/zh-CN/runtime-integration/docker.md | 2 +- docs/zh-CN/troubleshooting/README.md | 30 +++---- docs/zh-CN/user-guide/README.md | 7 ++ docs/zh-CN/user-guide/metrics.md | 67 +++++++++++++++ 81 files changed, 929 insertions(+), 779 deletions(-) create mode 100644 docs/en/developer-guide/README.md rename docs/en/developer-guide/{developer-guide.md => development.md} (99%) create mode 100644 docs/en/user-guide/README.md create mode 100644 docs/en/user-guide/metrics.md create mode 100644 docs/zh-CN/developer-guide/README.md delete mode 100644 docs/zh-CN/developer-guide/developer-guide.md create mode 100644 docs/zh-CN/developer-guide/development.md create mode 100644 docs/zh-CN/user-guide/README.md create mode 100644 docs/zh-CN/user-guide/metrics.md diff --git a/.markdownlint.yml b/.markdownlint.yml index f65b87cea..d4a796083 100644 --- a/.markdownlint.yml +++ b/.markdownlint.yml @@ -1,3 +1,5 @@ MD010: false MD013: line_length: 120 +MD046: + style: "fenced" diff --git a/docs/en/README.md b/docs/en/README.md index ef6a3cc3c..544d36920 100644 --- a/docs/en/README.md +++ b/docs/en/README.md @@ -9,29 +9,33 @@ part is that all need. Organization of document is as following: -* [Quick Start](quick-start.md) -* [Deployment](deployment/README.md) - * [Installation](deployment/installation) - * [Configuration](deployment/configuration) -* [Troubleshooting](troubleshooting/README.md) -* [CLI Reference](cli-reference/README.md) - * [dfget](cli-reference/dfget.md) - * [cdn](cli-reference/cdn.md) - * [scheduler](cli-reference/scheduler.md) - * [manager](cli-reference/manager.md) -* [Preheat](preheat/README.md) - * [Console](preheat/console.md) - * [Api](preheat/api.md) -* [Runtime Integration](runtime-integration/README.md) - * [containerd](runtime-integration/containerd/README.md) - * [cri-o](runtime-integration/cri-o.md) - * [docker](runtime-integration/docker.md) -* [Design](design/README.md) - * [architecture](design/architecture.md) - * [manager](design/manager.md) - * [TODO scheduler](design/scheduler.md) - * [TODO cdn](design/cdn.md) - * [TODO dfdaemon](design/dfdaemon.md) -* [Developer Guide](developer-guide/developer-guide.md) -* [Test Guide](test-guide/test-guide.md) -* [API Reference](api-reference/api-reference.md) +- [Quick Start](quick-start.md) +- [Deployment](deployment/README.md) + - [Installation](deployment/installation) + - [Configuration](deployment/configuration) +- [Troubleshooting](troubleshooting/README.md) +- [CLI Reference](cli-reference/README.md) + - [dfget](cli-reference/dfget.md) + - [cdn](cli-reference/cdn.md) + - [scheduler](cli-reference/scheduler.md) + - [manager](cli-reference/manager.md) +- [Preheat](preheat/README.md) + - [Console](preheat/console.md) + - [Api](preheat/api.md) +- [Runtime Integration](runtime-integration/README.md) + - [containerd](runtime-integration/containerd/README.md) + - [cri-o](runtime-integration/cri-o.md) + - [docker](runtime-integration/docker.md) +- [Design](design/README.md) + - [architecture](design/architecture.md) + - [manager](design/manager.md) + - [TODO scheduler](design/scheduler.md) + - [TODO cdn](design/cdn.md) + - [TODO dfdaemon](design/dfdaemon.md) +- [Developer Guide](developer-guide/README.md) + - [Development](developer-guide/development.md) + - [Plugin](developer-guide/plugin.md) +- [User Guide](user-guide/README.md) + - [Prometheus Metrics](user-guide/metrics.md) +- [Test Guide](test-guide/test-guide.md) +- [API Reference](api-reference/api-reference.md) diff --git a/docs/en/cli-reference/README.md b/docs/en/cli-reference/README.md index be2d7e7e9..71d1db287 100644 --- a/docs/en/cli-reference/README.md +++ b/docs/en/cli-reference/README.md @@ -10,7 +10,7 @@ all commandline docs are auto generated via source code. Table of contents: -* [dfget](dfget.md) -* [cdn](cdn.md) -* [scheduler](scheduler.md) -* [manager](manager.md) +- [dfget](dfget.md) +- [cdn](cdn.md) +- [scheduler](scheduler.md) +- [manager](manager.md) diff --git a/docs/en/cli-reference/cdn.md b/docs/en/cli-reference/cdn.md index de0877415..991b0e2b4 100644 --- a/docs/en/cli-reference/cdn.md +++ b/docs/en/cli-reference/cdn.md @@ -13,7 +13,7 @@ cdn [command] ## Available Commands ```text -doc generate documents +doc generate documents help Help about any command version show version ``` @@ -21,8 +21,9 @@ version show version ## Flags + ```text - --config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/cdn.yaml, it can + --config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/cdn.yaml, it can also be set by environment variable cdn_config --console whether logger output records to the stdout -h, --help help for scheduler @@ -31,4 +32,5 @@ version show version --service-name string name of the service for tracer (default "dragonfly-cdn") --verbose whether logger use debug level ``` + diff --git a/docs/en/cli-reference/dfget.md b/docs/en/cli-reference/dfget.md index a861b195c..9ef91ef61 100644 --- a/docs/en/cli-reference/dfget.md +++ b/docs/en/cli-reference/dfget.md @@ -32,6 +32,7 @@ set environment variable console=true if you want to print logs to Terminal ### Options + ``` --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) --cacerts strings the cacert file which is used to verify remote server when supernode interact with the source. @@ -65,6 +66,7 @@ set environment variable console=true if you want to print logs to Terminal -u, --url string URL of user requested downloading file(only HTTP/HTTPs supported) --verbose enable verbose mode, all debug log will be display ``` + ## dfget daemon @@ -76,6 +78,7 @@ set environment variable console=true if you want to print logs to Terminal ### Daemon Options + ``` --advertise-ip string the ip report to scheduler, normal same with listen ip (default "10.15.232.63") --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) @@ -104,4 +107,5 @@ set environment variable console=true if you want to print logs to Terminal --upload-rate ratelimit upload rate limit for other peers (default 104857600.000000) --verbose print verbose log and enable golang debug info ``` + diff --git a/docs/en/cli-reference/manager.md b/docs/en/cli-reference/manager.md index 811e196f1..c76a9378d 100644 --- a/docs/en/cli-reference/manager.md +++ b/docs/en/cli-reference/manager.md @@ -31,7 +31,7 @@ endpoint: /swagger/doc.json ## Prometheus metrics monitoring -endpoint: /metrics +endpoint: /metrics ## HealthCheck @@ -46,6 +46,7 @@ manager --jaeger http://localhost:14250/api/traces ## Options + ```text --config string the path of configuration file with yaml extension name, default is /etc/dragonfly/manager.yaml, it can also be set by env var: MANAGER_CONFIG --console whether logger output records to the stdout @@ -55,6 +56,7 @@ manager --jaeger http://localhost:14250/api/traces --service-name string name of the service for tracer (default "dragonfly-manager") --verbose whether logger use debug level ``` + ## Manager Config diff --git a/docs/en/cli-reference/scheduler.md b/docs/en/cli-reference/scheduler.md index 53cc02d0b..a2716c2bf 100644 --- a/docs/en/cli-reference/scheduler.md +++ b/docs/en/cli-reference/scheduler.md @@ -16,7 +16,7 @@ scheduler [command] ## Available Commands ```text -doc generate documents +doc generate documents help Help about any command version show version ``` @@ -24,8 +24,9 @@ version show version ## Flags + ```text - --config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/scheduler.yaml, it can + --config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/scheduler.yaml, it can also be set by environment variable scheduler_config --console whether logger output records to the stdout -h, --help help for scheduler @@ -34,4 +35,5 @@ version show version --service-name string name of the service for tracer (default "dragonfly-scheduler") --verbose whether logger use debug level ``` + diff --git a/docs/en/deployment/README.md b/docs/en/deployment/README.md index 6432fa274..0708ed41f 100644 --- a/docs/en/deployment/README.md +++ b/docs/en/deployment/README.md @@ -2,8 +2,8 @@ Table of contents: -* [installation](installation/README.md) -* [configuration](configuration) +- [installation](installation/README.md) +- [configuration](configuration) ## Installation diff --git a/docs/en/deployment/configuration/cdn.yaml b/docs/en/deployment/configuration/cdn.yaml index 5f2f8d353..081815327 100644 --- a/docs/en/deployment/configuration/cdn.yaml +++ b/docs/en/deployment/configuration/cdn.yaml @@ -44,11 +44,11 @@ base: # logDir is the log storage directory # in linux, default value is /var/log/dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly/logs - logDir: "" + logDir: '' # manager configuration manager: - addr: "" + addr: '' cdnClusterID: 0 keepAlive: interval: 5s @@ -116,7 +116,7 @@ verbose: false pprof-port: -1 # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' # service name used in tracer # default: dragonfly-cdn diff --git a/docs/en/deployment/configuration/dfget.yaml b/docs/en/deployment/configuration/dfget.yaml index 620e4f240..b7c786db0 100644 --- a/docs/en/deployment/configuration/dfget.yaml +++ b/docs/en/deployment/configuration/dfget.yaml @@ -8,22 +8,22 @@ gcInterval: 1m0s # daemon work directory, daemon will change current working directory to this # in linux, default value is /usr/local/dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly -workHome: "" +workHome: '' # cacheDir is dynconfig cache storage directory # in linux, default value is /var/cache/dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly/cache -cacheDir: "" +cacheDir: '' # logDir is the log storage directory # in linux, default value is /var/log/dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly/logs -logDir: "" +logDir: '' # dataDir is the download data storage directory # in linux, default value is /var/lib/dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly/data -dataDir: "" +dataDir: '' # when daemon exit, keep peer task data or not # it is usefully when upgrade daemon service, all local cache will be saved @@ -41,7 +41,7 @@ verbose: false pprof-port: -1 # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' # all addresses of all schedulers # the schedulers of all daemons should be same in one region or zone. @@ -75,13 +75,13 @@ host: # when local ip is different with access ip, advertiseIP should be set advertiseIP: 0.0.0.0 # geographical location, separated by "|" characters - location: "" + location: '' # idc deployed by daemon - idc: "" + idc: '' # security domain deployed by daemon, network isolation between different security domains - securityDomain: "" + securityDomain: '' # network topology, separated by "|" characters - netTopology: "" + netTopology: '' # daemon hostname # hostname: "" @@ -116,9 +116,9 @@ download: # security option security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tlsConfig: null # download service listen address # current, only support unix domain socket @@ -131,9 +131,9 @@ download: peerGRPC: security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tcpListen: # listen address listen: 0.0.0.0 @@ -151,9 +151,9 @@ upload: rateLimit: 100Mi security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tcpListen: # listen address listen: 0.0.0.0 @@ -196,17 +196,17 @@ proxy: # when defaultFilter: "Expires&Signature", for example: # http://localhost/xyz?Expires=111&Signature=222 and http://localhost/xyz?Expires=333&Signature=999 # is same task - defaultFilter: "Expires&Signature" + defaultFilter: 'Expires&Signature' security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tcpListen: # namespace stands the linux net namespace, like /proc/1/ns/net # it's useful for running daemon in pod with ip allocated and listening the special port in host net namespace # Linux only - namespace: "" + namespace: '' # listen address listen: 0.0.0.0 # listen port, daemon will try to listen @@ -248,8 +248,8 @@ proxy: hijackHTTPS: # key pair used to hijack https requests - cert: "" - key: "" + cert: '' + key: '' hosts: - regx: mirror.aliyuncs.com:443 # regexp to match request hosts # whether to ignore https certificate errors @@ -260,7 +260,7 @@ proxy: maxConcurrency: 0 whiteList: # the host of the whitelist - - host: "" + - host: '' # match whitelist hosts regx: # port that need to be added to the whitelist diff --git a/docs/en/deployment/configuration/manager.yaml b/docs/en/deployment/configuration/manager.yaml index d1b61d0b4..85466f259 100644 --- a/docs/en/deployment/configuration/manager.yaml +++ b/docs/en/deployment/configuration/manager.yaml @@ -70,4 +70,4 @@ verbose: false pprof-port: -1 # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' diff --git a/docs/en/deployment/configuration/scheduler.yaml b/docs/en/deployment/configuration/scheduler.yaml index 73d3d2872..138acebea 100644 --- a/docs/en/deployment/configuration/scheduler.yaml +++ b/docs/en/deployment/configuration/scheduler.yaml @@ -11,11 +11,11 @@ server: # cacheDir is dynconfig cache storage directory # in linux, default value is /var/cache/dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly/cache - cacheDir: "" + cacheDir: '' # logDir is the log storage directory # in linux, default value is /var/log/dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly/logs - logDir: "" + logDir: '' # scheduler policy configuration scheduler: @@ -57,11 +57,11 @@ dynConfig: # scheduler host configuration host: # idc is the idc of scheduler instance - idc: "" + idc: '' # netTopology is the net topology of scheduler instance - netTopology: "" + netTopology: '' # location is the location of scheduler instance - location: "" + location: '' # manager configuration manager: @@ -89,11 +89,11 @@ job: # redis configuration redis: # host - host: "" + host: '' # port port: 6379 # password - password: "" + password: '' # brokerDB brokerDB: 1 # backendDB @@ -104,7 +104,7 @@ metrics: # scheduler enable metrics service enable: false # metrics service address - addr: ":8000" + addr: ':8000' # enable peer host metrics enablePeerHost: false @@ -119,4 +119,4 @@ verbose: false pprof-port: -1 # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' diff --git a/docs/en/deployment/installation/README.md b/docs/en/deployment/installation/README.md index 3a987bb88..39c41c13a 100644 --- a/docs/en/deployment/installation/README.md +++ b/docs/en/deployment/installation/README.md @@ -2,6 +2,6 @@ Table of contents: -* [docker](docker/README.md) -* [executable-files](executable-files/README.md) -* [kubernetes](kubernetes/README.md) +- [docker](docker/README.md) +- [executable-files](executable-files/README.md) +- [kubernetes](kubernetes/README.md) diff --git a/docs/en/deployment/installation/docker/README.md b/docs/en/deployment/installation/docker/README.md index 4ae0d666c..5e65b2f90 100644 --- a/docs/en/deployment/installation/docker/README.md +++ b/docs/en/deployment/installation/docker/README.md @@ -6,5 +6,5 @@ or install separately according to modules. Table of contents: -* [Docker Compose](docker-compose.md) -* [Standalone](standalone/README.md) +- [Docker Compose](docker-compose.md) +- [Standalone](standalone/README.md) diff --git a/docs/en/deployment/installation/docker/standalone/README.md b/docs/en/deployment/installation/docker/standalone/README.md index 19d5e4736..fe7db3192 100644 --- a/docs/en/deployment/installation/docker/standalone/README.md +++ b/docs/en/deployment/installation/docker/standalone/README.md @@ -14,10 +14,10 @@ is the installation instructions of dragonfly. ## Install separately -* [install-client](./dfdaemon.md) is the installation instructions of dfdaemon +- [install-client](./dfdaemon.md) is the installation instructions of dfdaemon -* [install-manager](./manager.md) is the installation instructions of manager +- [install-manager](./manager.md) is the installation instructions of manager -* [install-scheduler](./scheduler.md) is the installation instructions of scheduler +- [install-scheduler](./scheduler.md) is the installation instructions of scheduler -* [install-cdn](./cdn.md) is the installation instructions of CDN +- [install-cdn](./cdn.md) is the installation instructions of CDN diff --git a/docs/en/deployment/installation/docker/standalone/cdn.md b/docs/en/deployment/installation/docker/standalone/cdn.md index a75b8b8c1..af21b68a8 100644 --- a/docs/en/deployment/installation/docker/standalone/cdn.md +++ b/docs/en/deployment/installation/docker/standalone/cdn.md @@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly CDN server. When deploying with Docker, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## Procedure - When Deploying with Docker @@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly. 1. Obtain the latest Docker image of the cdn. - ```sh - docker pull dragonflyoss/cdn - ``` + ```sh + docker pull dragonflyoss/cdn + ``` Or you can build your own cdn image. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Build the Docker image. - ```sh - TAG="2.0.0" - make docker-build-cdn D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-cdn D7Y_VERSION=$TAG + ``` 4. Obtain the latest Docker image ID of the cdn. - ```sh - docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 + ``` ### Start cdn @@ -57,7 +57,7 @@ Or you can build your own cdn image. ```sh docker run -d --name cdn --restart=always \ -p 8001:8001 -p 8003:8003 \ - -v /home/admin/ftp:/home/admin/ftp ${cdnDockerImageId} \ + -v /home/admin/ftp:/home/admin/ftp ${cdnDockerImageId} \ --download-port=8001 ``` diff --git a/docs/en/deployment/installation/docker/standalone/dfdaemon.md b/docs/en/deployment/installation/docker/standalone/dfdaemon.md index e76f37a5b..95cbaac07 100644 --- a/docs/en/deployment/installation/docker/standalone/dfdaemon.md +++ b/docs/en/deployment/installation/docker/standalone/dfdaemon.md @@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly dfdaemon server. When deploying with Docker, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## Procedure - When Deploying with Docker @@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly. 1. Obtain the latest Docker image of the dfdaemon. - ```sh - docker pull dragonflyoss/dfdaemon - ``` + ```sh + docker pull dragonflyoss/dfdaemon + ``` Or you can build your own dfdaemon image. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Build the Docker image. - ```sh - TAG="2.0.0" - make docker-build-dfdaemon D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-dfdaemon D7Y_VERSION=$TAG + ``` 4. Obtain the latest Docker image ID of the dfdaemon. - ```sh - docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1 + ``` ### Start dfdaemon diff --git a/docs/en/deployment/installation/docker/standalone/manager.md b/docs/en/deployment/installation/docker/standalone/manager.md index 00fa1b8c6..8494590fa 100644 --- a/docs/en/deployment/installation/docker/standalone/manager.md +++ b/docs/en/deployment/installation/docker/standalone/manager.md @@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly manager server. When deploying with Docker, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## Procedure - When Deploying with Docker @@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly. 1. Obtain the latest Docker image of the manager. - ```sh - docker pull dragonflyoss/manager - ``` + ```sh + docker pull dragonflyoss/manager + ``` Or you can build your own manager image. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Build the Docker image. - ```sh - TAG="2.0.0" - make docker-build-manager D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-manager D7Y_VERSION=$TAG + ``` 4. Obtain the latest Docker image ID of the manager. - ```sh - docker image ls | grep 'manager' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'manager' | awk '{print $3}' | head -n1 + ``` ### Start manager diff --git a/docs/en/deployment/installation/docker/standalone/scheduler.md b/docs/en/deployment/installation/docker/standalone/scheduler.md index b0b9eb6a2..3a696be84 100644 --- a/docs/en/deployment/installation/docker/standalone/scheduler.md +++ b/docs/en/deployment/installation/docker/standalone/scheduler.md @@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly scheduler server. When deploying with Docker, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## Procedure - When Deploying with Docker @@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly. 1. Obtain the latest Docker image of the scheduler. - ```sh - docker pull dragonflyoss/scheduler - ``` + ```sh + docker pull dragonflyoss/scheduler + ``` Or you can build your own scheduler image. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Build the Docker image. - ```sh - TAG="2.0.0" - make docker-build-scheduler D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-scheduler D7Y_VERSION=$TAG + ``` 4. Obtain the latest Docker image ID of the scheduler. - ```sh - docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1 + ``` ### Start scheduler diff --git a/docs/en/deployment/installation/executable-files/README.md b/docs/en/deployment/installation/executable-files/README.md index eb9a1a10a..78fb12b86 100644 --- a/docs/en/deployment/installation/executable-files/README.md +++ b/docs/en/deployment/installation/executable-files/README.md @@ -5,10 +5,10 @@ the executable files separately according to the modules. ## Install separately -* [install-client](dfdaemon.md) is the installation instructions of dfdaemon +- [install-client](dfdaemon.md) is the installation instructions of dfdaemon -* [install-manager](manager.md) is the installation instructions of manager +- [install-manager](manager.md) is the installation instructions of manager -* [install-scheduler](scheduler.md) is the installation instructions of scheduler +- [install-scheduler](scheduler.md) is the installation instructions of scheduler -* [install-cdn](cdn.md) is the installation instructions of CDN +- [install-cdn](cdn.md) is the installation instructions of CDN diff --git a/docs/en/deployment/installation/executable-files/cdn.md b/docs/en/deployment/installation/executable-files/cdn.md index 03acac5ef..b8917c1bb 100644 --- a/docs/en/deployment/installation/executable-files/cdn.md +++ b/docs/en/deployment/installation/executable-files/cdn.md @@ -6,54 +6,54 @@ This topic explains how to install the Dragonfly CDN server. When deploying with physical machines, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Golang|1.12.x -Nginx|0.8+ +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | +| Nginx | 0.8+ | ## Procedure - When Deploying with Physical Machines ### Get cdn executable file 1. Download a binary package of the cdn. You can download one of -the latest builds for Dragonfly on the -[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). + the latest builds for Dragonfly on the + [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. Unzip the package. - ```bash - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. Move the `cdn` to your `PATH` environment variable to -make sure you can directly use `cdn` command. + make sure you can directly use `cdn` command. Or you can build your own cdn executable file. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Compile the source code. - ```sh - make build-cdn && make install-cdn - ``` + ```sh + make build-cdn && make install-cdn + ``` ### Start cdn @@ -69,31 +69,31 @@ You can start a file server in any way. However, the following conditions must be met: - It must be rooted at `cdnHomeDir/ftp` which is -defined in the previous step. + defined in the previous step. - It must listen on the port `cdnDownloadPort` which is -defined in the previous step. + defined in the previous step. Let's take nginx as an example. 1. Add the following configuration items to -the Nginx configuration file. + the Nginx configuration file. - ```conf - server { - # Must be ${cdnDownloadPort} - listen 8001; - location / { - # Must be ${cdnHomeDir}/ftp - root /home/admin/ftp; - } - } - ``` + ```conf + server { + # Must be ${cdnDownloadPort} + listen 8001; + location / { + # Must be ${cdnHomeDir}/ftp + root /home/admin/ftp; + } + } + ``` 2. Start Nginx. - ```sh - sudo nginx - ``` + ```sh + sudo nginx + ``` After cdn is installed, run the following commands to verify if Nginx and **cdn** are started, diff --git a/docs/en/deployment/installation/executable-files/dfdaemon.md b/docs/en/deployment/installation/executable-files/dfdaemon.md index 5fcb2cb96..7104d15c0 100644 --- a/docs/en/deployment/installation/executable-files/dfdaemon.md +++ b/docs/en/deployment/installation/executable-files/dfdaemon.md @@ -6,53 +6,53 @@ This topic explains how to install the Dragonfly dfdaemon server. When deploying with physical machines, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Golang|1.12.x +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | ## Procedure - When Deploying with Physical Machines ### Get dfget executable file 1. Download a binary package of the dfget. You can download one of -the latest builds for Dragonfly on the -[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). + the latest builds for Dragonfly on the + [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. Unzip the package. - ```bash - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. Move the `dfget` to your `PATH` environment variable to -make sure you can directly use `dfget` command. + make sure you can directly use `dfget` command. Or you can build your own dfget executable file. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Compile the source code. - ```sh - make build-dfget && make install-dfget - ``` + ```sh + make build-dfget && make install-dfget + ``` ### Start dfdaemon diff --git a/docs/en/deployment/installation/executable-files/manager.md b/docs/en/deployment/installation/executable-files/manager.md index b2d8dc78a..79131b7f4 100644 --- a/docs/en/deployment/installation/executable-files/manager.md +++ b/docs/en/deployment/installation/executable-files/manager.md @@ -6,54 +6,54 @@ This topic explains how to install the Dragonfly manager server. When deploying with physical machines, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Golang|1.12.x -Nginx|0.8+ +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | +| Nginx | 0.8+ | ## Procedure - When Deploying with Physical Machines ### Get manager executable file 1. Download a binary package of the manager. You can download -one of the latest builds for Dragonfly on -the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). + one of the latest builds for Dragonfly on + the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. Unzip the package. - ```sh - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```sh + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. Move the `manager` to your `PATH` environment variable -to make sure you can directly use `manager` command. + to make sure you can directly use `manager` command. Or you can build your own manager executable file. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Compile the source code. - ```sh - make build-manager && make install-manager - ``` + ```sh + make build-manager && make install-manager + ``` ### Start manager diff --git a/docs/en/deployment/installation/executable-files/scheduler.md b/docs/en/deployment/installation/executable-files/scheduler.md index 0c6d6be96..79ce267e6 100644 --- a/docs/en/deployment/installation/executable-files/scheduler.md +++ b/docs/en/deployment/installation/executable-files/scheduler.md @@ -6,54 +6,54 @@ This topic explains how to install the Dragonfly scheduler server. When deploying with physical machines, the following conditions must be met. -Required Software | Version Limit ----|--- -Git|1.9.1+ -Golang|1.12.x -Nginx|0.8+ +| Required Software | Version Limit | +| ----------------- | ------------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | +| Nginx | 0.8+ | ## Procedure - When Deploying with Physical Machines ### Get scheduler executable file 1. Download a binary package of the scheduler. -You can download one of the latest builds for -Dragonfly on the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). + You can download one of the latest builds for + Dragonfly on the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. Unzip the package. - ```bash - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. Move the `scheduler` to your `PATH` environment -variable to make sure you can directly use `scheduler` command. + variable to make sure you can directly use `scheduler` command. Or you can build your own scheduler executable file. 1. Obtain the source code of Dragonfly. - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. Enter the project directory. - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. Compile the source code. - ```sh - make build-scheduler && make install-scheduler - ``` + ```sh + make build-scheduler && make install-scheduler + ``` ### Start scheduler diff --git a/docs/en/deployment/installation/kubernetes/README.md b/docs/en/deployment/installation/kubernetes/README.md index a91c0658b..5c264bb5d 100644 --- a/docs/en/deployment/installation/kubernetes/README.md +++ b/docs/en/deployment/installation/kubernetes/README.md @@ -6,6 +6,6 @@ daemon as `DaemonSets`, manager as `Deployments`. Table of contents: -* [Helm](helm.md) -* [Kustomize](kustomize.md) -* [TODO Upgrade Guide](upgrade-guide.md) +- [Helm](helm.md) +- [Kustomize](kustomize.md) +- [TODO Upgrade Guide](upgrade-guide.md) diff --git a/docs/en/deployment/installation/kubernetes/helm.md b/docs/en/deployment/installation/kubernetes/helm.md index 4fb3bda86..a20c2e463 100644 --- a/docs/en/deployment/installation/kubernetes/helm.md +++ b/docs/en/deployment/installation/kubernetes/helm.md @@ -8,21 +8,21 @@ you can skip [Configure Runtime](#configure-runtime-manually) manually. ### 1. Docker > **We did not recommend to using dragonfly -with docker in Kubernetes** -due to many reasons: -no fallback image pulling policy. -deprecated in Kubernetes. +> with docker in Kubernetes** +> due to many reasons: +> no fallback image pulling policy. +> deprecated in Kubernetes. > Because the original `daemonset` in -Kubernetes did not support `Surging Rolling Update` policy. +> Kubernetes did not support `Surging Rolling Update` policy. > When kill current dfdaemon pod, -the new pod image can not be pulled anymore. +> the new pod image can not be pulled anymore. > If you can not change runtime from docker to others, -remind to choose a plan when upgrade dfdaemon: +> remind to choose a plan when upgrade dfdaemon: > pull newly dfdaemon image manually before upgrade dragonfly, -or use [ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob) to -pull image automate. +> or use [ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob) to +> pull image automate. > keep the image registry of dragonfly is -different from common registries and add host in `containerRuntime.docker.skipHosts`. +> different from common registries and add host in `containerRuntime.docker.skipHosts`. Dragonfly helm supports config docker automatically. @@ -33,6 +33,7 @@ Config cases: Chart customize values.yaml: + ```yaml containerRuntime: docker: @@ -42,9 +43,10 @@ containerRuntime: # When use certs and inject hosts in docker, no necessary to restart docker daemon. injectHosts: true registryDomains: - - "harbor.example.com" - - "harbor.example.net" + - "harbor.example.com" + - "harbor.example.net" ``` + This config enables docker pulling images from @@ -57,7 +59,7 @@ Advantages: - Support upgrade dfdaemon smoothness > In this mode, when dfdaemon pod deleted, - the `preStop` hook will remove all injected hosts info in /etc/hosts, +> the `preStop` hook will remove all injected hosts info in /etc/hosts, > all images traffic fallbacks to original registries. Limitations: @@ -69,6 +71,7 @@ Limitations: Chart customize values.yaml: + ```yaml containerRuntime: docker: @@ -78,9 +81,10 @@ containerRuntime: # If did not want restart docker daemon, keep containerRuntime.docker.restart=false and containerRuntime.docker.injectHosts=true. restart: true skipHosts: - - "127.0.0.1" - - "docker.io" # Dragonfly use this image registry to upgrade itself, so we need skip it. Change it in real environment. + - "127.0.0.1" + - "docker.io" # Dragonfly use this image registry to upgrade itself, so we need skip it. Change it in real environment. ``` + This config enables docker pulling images from arbitrary registries via Dragonfly. @@ -124,30 +128,28 @@ containerRuntime: - Option 1 - Allow charts to inject config_path and restart containerd. - This option also enable multiple registry mirrors support. + This option also enable multiple registry mirrors support. - > Caution: if there are already many other mirror config in config.toml, - should not use this option, or migrate your config with `config_path`. + > Caution: if there are already many other mirror config in config.toml, + > should not use this option, or migrate your config with `config_path`. - Chart customize values.yaml: + Chart customize values.yaml: - ```yaml - containerRuntime: - containerd: - enable: true - injectConfigPath: true - ``` + ```yaml + containerRuntime: + containerd: + enable: true + injectConfigPath: true + ``` - Option 2 - Just mirror only one registry -which `dfdaemon.config.proxy.registryMirror.url` is + which `dfdaemon.config.proxy.registryMirror.url` is Chart customize values.yaml: - Chart customize values.yaml: - - ```yaml - containerRuntime: - containerd: - enable: true - ``` + ```yaml + containerRuntime: + containerd: + enable: true + ``` #### Case 3: Version 1 @@ -178,9 +180,9 @@ containerRuntime: enable: true # Registries full urls registries: - - "https://ghcr.io" - - "https://quay.io" - - "https://harbor.example.com:8443" + - "https://ghcr.io" + - "https://quay.io" + - "https://harbor.example.com:8443" ``` ## Prepare Kubernetes Cluster diff --git a/docs/en/design/README.md b/docs/en/design/README.md index f3eb7e71c..8ec0cc5b7 100644 --- a/docs/en/design/README.md +++ b/docs/en/design/README.md @@ -8,8 +8,8 @@ all APIs in Dragonfly and some technical things else. Table of contents: -* [architecture](architecture.md) -* [manager](manager.md) -* [TODO scheduler](scheduler.md) -* [TODO cdn](cdn.md) -* [TODO dfdaemon](dfdaemon.md) +- [architecture](architecture.md) +- [manager](manager.md) +- [TODO scheduler](scheduler.md) +- [TODO cdn](cdn.md) +- [TODO dfdaemon](dfdaemon.md) diff --git a/docs/en/design/architecture.md b/docs/en/design/architecture.md index 0e8dcdaca..764161e48 100644 --- a/docs/en/design/architecture.md +++ b/docs/en/design/architecture.md @@ -72,7 +72,7 @@ data market, multi-tenancy, and permission control, etc. #### CDN system -![cdn][TODO cdn] +![cdn][todo cdn] ### Sub-system function diff --git a/docs/en/developer-guide/README.md b/docs/en/developer-guide/README.md new file mode 100644 index 000000000..3071256f4 --- /dev/null +++ b/docs/en/developer-guide/README.md @@ -0,0 +1,8 @@ +# Developer Guide + +Understand how to develop dragonfly project and plugins. + +Table of contents: + +- [Development](development.md) +- [Plugin](plugin.md) diff --git a/docs/en/developer-guide/developer-guide.md b/docs/en/developer-guide/development.md similarity index 99% rename from docs/en/developer-guide/developer-guide.md rename to docs/en/developer-guide/development.md index ba08c74bd..7ac889bb8 100644 --- a/docs/en/developer-guide/developer-guide.md +++ b/docs/en/developer-guide/development.md @@ -24,6 +24,7 @@ Attaching to cdn, scheduler, dfdaemon Show dragonfly logs. + ```bash $ tail -f log/**/*.log ==> log/dragonfly/cdn/core.log <== @@ -49,6 +50,7 @@ $ tail -f log/**/*.log {"level":"info","ts":"2021-02-26 05:43:37.342","caller":"worker/sender.go:49","msg":"start sender worker : 50"} {"level":"info","ts":"2021-02-26 05:43:37.343","caller":"worker/worker_group.go:64","msg":"start scheduler worker number:6"} ``` + ## Step 4: Stop dragonfly diff --git a/docs/en/developer-guide/plugin.md b/docs/en/developer-guide/plugin.md index 5120a2f98..69dbd4920 100644 --- a/docs/en/developer-guide/plugin.md +++ b/docs/en/developer-guide/plugin.md @@ -12,22 +12,26 @@ The resource plugin is used to download custom resource like `dfget -u d7yfs://h All resource plugins need to implement `d7y.io/dragonfly/v2/pkg/source.ResourceClient` and a function + + `func DragonflyPluginInit(option map[string]string) (interface{}, map[string]string, error)`. + + ```golang // ResourceClient defines the API interface to interact with source. type ResourceClient interface { // GetContentLength get length of resource content // return source.UnknownSourceFileLen if response status is not StatusOK and StatusPartialContent GetContentLength(request *Request) (int64, error) - + // IsSupportRange checks if resource supports breakpoint continuation // return false if response status is not StatusPartialContent IsSupportRange(request *Request) (bool, error) - + // IsExpired checks if a resource received or stored is the same. // return false and non-nil err to prevent the source from exploding if // fails to get the result, it is considered that the source has not expired @@ -35,11 +39,12 @@ type ResourceClient interface { // Download downloads from source Download(request *Request) (*Response, error) - + // GetLastModified gets last modified timestamp milliseconds of resource GetLastModified(request *Request) (int64, error) } ``` + ### Example Code @@ -47,6 +52,7 @@ type ResourceClient interface { #### 1. main.go + ```golang package main @@ -107,11 +113,13 @@ func DragonflyPluginInit(option map[string]string) (interface{}, map[string]stri ``` + #### 2. go.mod + ``` module example.com/d7yfs @@ -132,6 +140,7 @@ require ( // fix golang build error: `plugin was built with a different version of package d7y.io/dragonfly/v2/internal/dflog` replace d7y.io/dragonfly/v2 => /Dragonfly2 ``` + ### Build @@ -141,6 +150,7 @@ replace d7y.io/dragonfly/v2 => /Dragonfly2 > Update `D7Y_COMMIT` in the following script. + ```shell # golang plugin need cgo # original Dragonfly2 image is built with CGO_ENABLED=0 for alpine linux @@ -159,6 +169,7 @@ go mod tidy -compat=1.17 go build -ldflags="-X main.buildTime=${BUILD_TIME} -X main.buildCommit=${BUILD_COMMIT}" \ -buildmode=plugin -o=/usr/local/dragonfly/plugins/d7y-resource-plugin-d7yfs.so ./main.go ``` + #### 2. Validate plugin @@ -170,10 +181,12 @@ go build -ldflags="-X main.buildTime=${BUILD_TIME} -X main.buildCommit=${BUILD_C Example output: + ```text search plugin in /usr/local/dragonfly/plugins resource plugin d7yfs, location: d7y-resource-plugin-d7yfs.so, attribute: {"buildCommit":"bb65f13","buildTime":"2021-12-13T08:53:04Z","name":"d7yfs","schema":"d7yfs","type":"resource","vendor":"d7y"} ``` + ## Searcher plugin diff --git a/docs/en/preheat/README.md b/docs/en/preheat/README.md index 2803e7331..cac2e7cc6 100644 --- a/docs/en/preheat/README.md +++ b/docs/en/preheat/README.md @@ -5,5 +5,5 @@ Users can be preheat in the `console` or directly call `preheat api` for preheat Table of contents: -* [Console](console.md) -* [Api](api.md) +- [Console](console.md) +- [Api](api.md) diff --git a/docs/en/quick-start.md b/docs/en/quick-start.md index 1613462a2..b15910a8f 100644 --- a/docs/en/quick-start.md +++ b/docs/en/quick-start.md @@ -11,14 +11,16 @@ We recommend to use `Containerd with CRI` and `CRI-O` client. This table describes some container runtimes version and documents. -| Runtime | Version | Document | CRI Support | Pull Command | -| --- | --- | --- | --- | --- | -| Containerd* | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest | -| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine | -| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest | + +| Runtime | Version | Document | CRI Support | Pull Command | +| ----------------------- | -------- | ------------------------------------------------ | ----------- | ------------------------------------------- | +| Containerd\* | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest | +| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine | +| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest | + -**: `containerd` is recommended* +`containerd` is recommended. ## Runtime Configuration Guide for Dragonfly Helm Chart @@ -40,8 +42,8 @@ containerRuntime: # When use certs and inject hosts in docker, no necessary to restart docker daemon. injectHosts: true registryDomains: - - "harbor.example.com" - - "harbor.example.net" + - 'harbor.example.com' + - 'harbor.example.net' ``` This config enables docker pulling images from registries @@ -50,7 +52,7 @@ When deploying Dragonfly with above config, it's unnecessary to restart docker d Limitations: -* Only support implicit registries +- Only support implicit registries ## Prepare Kubernetes Cluster diff --git a/docs/en/runtime-integration/README.md b/docs/en/runtime-integration/README.md index 8e02a2cda..f66edbc14 100644 --- a/docs/en/runtime-integration/README.md +++ b/docs/en/runtime-integration/README.md @@ -2,6 +2,6 @@ Table of contents: -* [containerd](containerd/README.md) -* [cri-o](cri-o.md) -* [docker](docker.md) +- [containerd](containerd/README.md) +- [cri-o](cri-o.md) +- [docker](docker.md) diff --git a/docs/en/runtime-integration/containerd/README.md b/docs/en/runtime-integration/containerd/README.md index e1ea07e0a..ed5dab568 100644 --- a/docs/en/runtime-integration/containerd/README.md +++ b/docs/en/runtime-integration/containerd/README.md @@ -5,5 +5,5 @@ Containerd. We recommend to use `mirror`. Table of contents: -* [mirror](mirror.md) -* [proxy](proxy.md) +- [mirror](mirror.md) +- [proxy](proxy.md) diff --git a/docs/en/runtime-integration/containerd/mirror.md b/docs/en/runtime-integration/containerd/mirror.md index 7960c6a8a..d4fc4ffd1 100644 --- a/docs/en/runtime-integration/containerd/mirror.md +++ b/docs/en/runtime-integration/containerd/mirror.md @@ -71,7 +71,7 @@ In this config, registry auth configuration needs to be based on `127.0.0.1:6500 > More details about Containerd configuration: > Containerd has deprecated the above config from v1.4.0, - new format for reference: +> new format for reference: ### Option 2: Multiple Registries diff --git a/docs/en/runtime-integration/containerd/proxy.md b/docs/en/runtime-integration/containerd/proxy.md index 4cfe047db..d2913d625 100644 --- a/docs/en/runtime-integration/containerd/proxy.md +++ b/docs/en/runtime-integration/containerd/proxy.md @@ -142,7 +142,7 @@ proxy: key: ca.key hosts: - regx: your.private.registry - certs: ["server.crt"] + certs: ['server.crt'] ``` You can get the certificate of your server with: diff --git a/docs/en/runtime-integration/docker.md b/docs/en/runtime-integration/docker.md index 5cf6916da..80586a997 100644 --- a/docs/en/runtime-integration/docker.md +++ b/docs/en/runtime-integration/docker.md @@ -150,7 +150,7 @@ proxy: key: ca.key hosts: - regx: your.private.registry - certs: ["server.crt"] + certs: ['server.crt'] ``` You can get the certificate of your server with: diff --git a/docs/en/troubleshooting/README.md b/docs/en/troubleshooting/README.md index f05da7698..4f7a03be9 100644 --- a/docs/en/troubleshooting/README.md +++ b/docs/en/troubleshooting/README.md @@ -26,17 +26,17 @@ change log level to info 1. Confirm limit rate in [dfget.yaml](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/en/deployment/configuration/dfget.yaml#L65) - ```yaml - download: - # total download limit per second - totalRateLimit: 200Mi - # per peer task download limit per second - # default is 20Mi, this default is in consideration of extreme environments - perPeerRateLimit: 100Mi - upload: - # upload limit per second - rateLimit: 100Mi - ``` + ```yaml + download: + # total download limit per second + totalRateLimit: 200Mi + # per peer task download limit per second + # default is 20Mi, this default is in consideration of extreme environments + perPeerRateLimit: 100Mi + upload: + # upload limit per second + rateLimit: 100Mi + ``` 2. Confirm source connection speed in CDN and dfdaemon diff --git a/docs/en/user-guide/README.md b/docs/en/user-guide/README.md new file mode 100644 index 000000000..e63f23bcc --- /dev/null +++ b/docs/en/user-guide/README.md @@ -0,0 +1,7 @@ +# User Guide + +Understand how to use Dragonfly from installing the server and client to downloading files. + +Table of contents: + +- [Prometheus Metrics](metrics.md) diff --git a/docs/en/user-guide/metrics.md b/docs/en/user-guide/metrics.md new file mode 100644 index 000000000..7eac09404 --- /dev/null +++ b/docs/en/user-guide/metrics.md @@ -0,0 +1,67 @@ +# Prometheus Metrics + +This doc contains all the metrics that Dragonfly components currently support. +Now we support metrics for Dfdaemon, Manager, Scheduler and CDN. +The metrics path is fixed to /metrics. The following metrics are exported. + +## Dfdaemon + +GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + + + +| Name | Labels | Type | Description | +| :------------------------------------------------------- | :----- | :------ | :---------------------------------------------------- | +| dragonfly_dfdaemon_proxy_request_total | method | counter | Counter of the total proxy request. | +| dragonfly_dfdaemon_proxy_request_via_dragonfly_total | | counter | Counter of the total proxy request via Dragonfly. | +| dragonfly_dfdaemon_proxy_request_not_via_dragonfly_total | | counter | Counter of the total proxy request not via Dragonfly. | +| dragonfly_dfdaemon_proxy_request_running_total | method | counter | Current running count of proxy request. | +| dragonfly_dfdaemon_proxy_request_bytes_total | method | counter | Counter of the total byte of all proxy request. | +| dragonfly_dfdaemon_peer_task_total | | counter | Counter of the total peer tasks. | +| dragonfly_dfdaemon_peer_task_failed_total | | counter | Counter of the total failed peer tasks. | +| dragonfly_dfdaemon_piece_task_total | | counter | Counter of the total failed piece tasks. | +| dragonfly_dfdaemon_piece_task_failed_total | | counter | Dragonfly dfget tasks. | +| dragonfly_dfdaemon_file_task_total | | counter | Counter of the total file tasks. | +| dragonfly_dfdaemon_stream_task_total | | counter | Counter of the total stream tasks. | +| dragonfly_dfdaemon_peer_task_cache_hit_total | | counter | Counter of the total cache hit peer tasks. | + + + +## Manager + +GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + +## Scheduler + +GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + + + +| Name | Labels | Type | Description | +| :----------------------------------------------------------- | :----------------------------------------- | :-------- | :--------------------------------------------------------- | +| dragonfly_scheduler_register_peer_task_total | | counter | Counter of the number of the register peer task. | +| dragonfly_scheduler_register_peer_task_failure_total | | counter | Counter of the number of failed of the register peer task. | +| dragonfly_scheduler_download_total | | counter | Counter of the number of the downloading. | +| dragonfly_scheduler_download_failure_total | | counter | Counter of the number of failed of the downloading. | +| dragonfly_scheduler_p2p_traffic | | counter | Counter of the number of p2p traffic. | +| dragonfly_scheduler_peer_host_traffic | traffic_type, peer_host_uuid, peer_host_ip | counter | Counter of the number of per peer host traffic. | +| dragonfly_scheduler_peer_task_total | type | counter | Counter of the number of peer task. | +| dragonfly_scheduler_peer_task_download_duration_milliseconds | | histogram | Histogram of the time each peer task downloading. | +| dragonfly_scheduler_concurrent_schedule_total | | gauge | Gauge of the number of concurrent of the scheduling. | + + + +## CDN + +GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + + + +| Name | Labels | Type | Description | +| :-------------------------------------- | :----- | :------ | :----------------------------------------------------- | +| dragonfly_cdn_download_total | | counter | Counter of the number of the downloading. | +| dragonfly_cdn_download_failure_total | | counter | Counter of the number of failed of the downloading. | +| dragonfly_cdn_download_traffic | | counter | Counter of the number of download traffic. | +| dragonfly_cdn_concurrent_download_total | | gauge | Gauger of the number of concurrent of the downloading. | + + diff --git a/docs/zh-CN/README.md b/docs/zh-CN/README.md index ff7ceaf3b..33f638523 100644 --- a/docs/zh-CN/README.md +++ b/docs/zh-CN/README.md @@ -6,29 +6,33 @@ Dragonfly 文档介绍了 Dragonfly 各方面的信息。如果您对 Dragonfly 文档文件组织如下： -* [快速开始](quick-start.md) -* [部署](deployment/README.md) - * [安装](deployment/installation) - * [配置](deployment/configuration) -* [故障排查](troubleshooting/README.md) -* [CLI参考](cli-reference/README.md) - * [dfget](cli-reference/dfget.md) - * [cdn](cli-reference/cdn.md) - * [scheduler](cli-reference/scheduler.md) - * [manager](cli-reference/manager.md) -* [预热](preheat/README.md) - * [Console](preheat/console.md) - * [Api](preheat/api.md) -* [运行时集成](runtime-integration/README.md) - * [containerd](runtime-integration/containerd/README.md) - * [cri-o](runtime-integration/cri-o.md) - * [docker](runtime-integration/docker.md) -* [架构设计](design/README.md) - * [整体架构](design/architecture.md) - * [manager](design/manager.md) - * [TODO scheduler](design/scheduler.md) - * [TODO cdn](design/cdn.md) - * [TODO dfdaemon](design/dfdaemon.md) -* [开发者指南](developer-guide/developer-guide.md) -* [测试指南](test-guide/test-guide.md) -* [API 参考](api-reference/api-reference.md) +- [快速开始](quick-start.md) +- [部署](deployment/README.md) + - [安装](deployment/installation) + - [配置](deployment/configuration) +- [故障排查](troubleshooting/README.md) +- [CLI 参考](cli-reference/README.md) + - [dfget](cli-reference/dfget.md) + - [cdn](cli-reference/cdn.md) + - [scheduler](cli-reference/scheduler.md) + - [manager](cli-reference/manager.md) +- [预热](preheat/README.md) + - [Console](preheat/console.md) + - [Api](preheat/api.md) +- [运行时集成](runtime-integration/README.md) + - [containerd](runtime-integration/containerd/README.md) + - [cri-o](runtime-integration/cri-o.md) + - [docker](runtime-integration/docker.md) +- [架构设计](design/README.md) + - [整体架构](design/architecture.md) + - [manager](design/manager.md) + - [TODO scheduler](design/scheduler.md) + - [TODO cdn](design/cdn.md) + - [TODO dfdaemon](design/dfdaemon.md) +- [开发者指南](developer-guide/README.md) + - [开发](developer-guide/developer.md) + - [插件](developer-guide/plugin.md) +- [用户指南](user-guide/README.md) + - [Prometheus 数据指标](user-guide/metrics.md) +- [测试指南](test-guide/test-guide.md) +- [API 参考](api-reference/api-reference.md) diff --git a/docs/zh-CN/cli-reference/README.md b/docs/zh-CN/cli-reference/README.md index d00ba7efe..a1a42bdf2 100644 --- a/docs/zh-CN/cli-reference/README.md +++ b/docs/zh-CN/cli-reference/README.md @@ -8,7 +8,7 @@ 内容列表: -* [dfget](dfget.md) -* [cdn](cdn.md) -* [scheduler](scheduler.md) -* [manager](manager.md) +- [dfget](dfget.md) +- [cdn](cdn.md) +- [scheduler](scheduler.md) +- [manager](manager.md) diff --git a/docs/zh-CN/cli-reference/cdn.md b/docs/zh-CN/cli-reference/cdn.md index 7aec15f89..3009fb1f3 100644 --- a/docs/zh-CN/cli-reference/cdn.md +++ b/docs/zh-CN/cli-reference/cdn.md @@ -12,7 +12,7 @@ cdn [command] ## 可用子命令 ```text -doc 生成帮助文档 +doc 生成帮助文档 help 命令的帮助文档 version 查看 cdn 当前版本 ``` @@ -20,6 +20,7 @@ version 查看 cdn 当前版本 ## 可选参数 + ```text --config string 配置文件地址 (默认值为/Users/${USER_HOME}/.dragonfly/config/cdn.yaml)，也可以使用环境变量 cdn_config 进行配置 --console logger 日志是否输出记录到标准输出 @@ -29,4 +30,5 @@ version 查看 cdn 当前版本 --service-name string tracer分布式追踪日志中的服务名称 (默认 "dragonfly-scheduler") --verbose 是否使用debug级别的日志输出 ``` + diff --git a/docs/zh-CN/cli-reference/dfget.md b/docs/zh-CN/cli-reference/dfget.md index a463df731..25df256c8 100644 --- a/docs/zh-CN/cli-reference/dfget.md +++ b/docs/zh-CN/cli-reference/dfget.md @@ -2,7 +2,7 @@ ## dfget -`dfget` 是Dragonfly中用来下载和上传文件的客户端。 +`dfget` 是 Dragonfly 中用来下载和上传文件的客户端。 ### dfget 简介 @@ -24,11 +24,12 @@ dfget --schedulers 127.0.0.1:8002 -o /path/to/output -u "http://example.com/obje ### 查看 dfget 日志 -如果您想要在Terminal中查看日志，请使用 --console参数，或者将环境变量 console 设置为 true。 +如果您想要在 Terminal 中查看日志，请使用 --console 参数，或者将环境变量 console 设置为 true。 ### dfget 的可选参数 + ```text --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) --cacerts strings the cacert file which is used to verify remote server when supernode interact with the source. @@ -62,17 +63,19 @@ dfget --schedulers 127.0.0.1:8002 -o /path/to/output -u "http://example.com/obje -u, --url string URL of user requested downloading file(only HTTP/HTTPs supported) --verbose enable verbose mode, all debug log will be display ``` + ## dfget daemon ### 查看 daemon 日志 -如果您想要在Terminal中查看日志，请使用 --console参数，或者将环境变量 console 设置为 true。 +如果您想要在 Terminal 中查看日志，请使用 --console 参数，或者将环境变量 console 设置为 true。 ### daemon 的可选参数 + ```text --advertise-ip string the ip report to scheduler, normal same with listen ip (default "10.15.232.63") --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) @@ -101,4 +104,5 @@ dfget --schedulers 127.0.0.1:8002 -o /path/to/output -u "http://example.com/obje --upload-rate ratelimit upload rate limit for other peers (default 104857600.000000) --verbose print verbose log and enable golang debug info ``` + diff --git a/docs/zh-CN/cli-reference/manager.md b/docs/zh-CN/cli-reference/manager.md index 49b1945cb..13a47d190 100644 --- a/docs/zh-CN/cli-reference/manager.md +++ b/docs/zh-CN/cli-reference/manager.md @@ -21,11 +21,11 @@ manager [Option] 启动时配置参数: --pprof-port port ``` -## Swagger支持 +## Swagger 支持 端点: /swagger/doc.json -## Prometheus集成 +## Prometheus 集成 端点: /metrics @@ -33,7 +33,7 @@ manager [Option] 端点: /healthy/ -## jaeger集成 +## jaeger 集成 ```text 启动时配置 --jaeger http://localhost:14250/api/traces diff --git a/docs/zh-CN/cli-reference/scheduler.md b/docs/zh-CN/cli-reference/scheduler.md index f934989e0..df032d1f3 100644 --- a/docs/zh-CN/cli-reference/scheduler.md +++ b/docs/zh-CN/cli-reference/scheduler.md @@ -1,7 +1,7 @@ # Scheduler Scheduler 是一个常驻后台运行的进程，用于接收和管理客户端的下载任务， -通知CDN进行回源， 在下载过程中生成维护P2P网络，给客户端推送适合的下载节点 +通知 CDN 进行回源， 在下载过程中生成维护 P2P 网络，给客户端推送适合的下载节点 ## 用法 @@ -13,7 +13,7 @@ scheduler [command] ## 可用子命令 ```text -doc 生成帮助文档 +doc 生成帮助文档 help 命令的帮助文档 version 查看 scheduler 当前版本 ``` diff --git a/docs/zh-CN/deployment/README.md b/docs/zh-CN/deployment/README.md index 03b6a43c0..8c63bdb2f 100644 --- a/docs/zh-CN/deployment/README.md +++ b/docs/zh-CN/deployment/README.md @@ -2,8 +2,8 @@ 内容: -* [安装](installation/README.md) -* [配置](configuration) +- [安装](installation/README.md) +- [配置](configuration) ## 安装 diff --git a/docs/zh-CN/deployment/configuration/cdn.yaml b/docs/zh-CN/deployment/configuration/cdn.yaml index be87fe999..75050b443 100644 --- a/docs/zh-CN/deployment/configuration/cdn.yaml +++ b/docs/zh-CN/deployment/configuration/cdn.yaml @@ -48,12 +48,12 @@ base: # cdn 日志目录 # linux 上默认目录 /var/log/dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs - logDir: "" + logDir: '' # CDN 连接的 manager，可以不指定。 # 各项配置默认值如下。如果 addr 为空字符串，CDN将不会连接manager。 manager: - addr: "" + addr: '' cdnClusterID: 0 keepAlive: interval: 5s @@ -61,9 +61,9 @@ base: # 主机信息 host: # 地理位置 - location: "" + location: '' # IDC(Internet Data Center)，互联网数据中心 - idc: "" + idc: '' # 开启数据收集服务 # metrics: @@ -123,7 +123,7 @@ pprof-port: -1 # jaeger 地址 # 默认使用空字符串（不配置 jaeger）, 例如: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' # tracer 中使用的 service-name # 默认值：dragonfly-cdn diff --git a/docs/zh-CN/deployment/configuration/dfget.yaml b/docs/zh-CN/deployment/configuration/dfget.yaml index ccc395f5e..6cdd275c9 100644 --- a/docs/zh-CN/deployment/configuration/dfget.yaml +++ b/docs/zh-CN/deployment/configuration/dfget.yaml @@ -8,22 +8,22 @@ gcInterval: 1m0s # daemon 工作目录 # linux 上默认目录 /usr/local/dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly -workHome: "" +workHome: '' # daemon 动态配置缓存目录 # linux 上默认目录 /var/cache/dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/cache -cacheDir: "" +cacheDir: '' # daemon 日志目录 # linux 上默认目录 /var/log/dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs -logDir: "" +logDir: '' # daemon 数据目录 # linux 上默认目录为 /var/lib/dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/data/ -dataDir: "" +dataDir: '' # 当 daemon 退出是, 是否保存缓存数据 # 保留缓存数据在升级 daemon 的时候比较有用 @@ -41,7 +41,7 @@ pprof-port: -1 # jaeger 地址 # 默认使用空字符串（不配置 jaeger）, 例如: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' # 调度器地址 # 尽量使用同一个地区的调度器. @@ -73,13 +73,13 @@ host: # 其他 daemon 可以通过这个 IP 地址连接过来 advertiseIP: 0.0.0.0 # 地理信息, 通过 "|" 符号分隔 - location: "" + location: '' # 机房信息 - idc: "" + idc: '' # 安全域信息，不同安全域之间网络隔离 - securityDomain: "" + securityDomain: '' # 网络拓扑结构，通过 "|" 符号分隔 - netTopology: "" + netTopology: '' # 主机名称 # hostname: "" @@ -96,9 +96,9 @@ download: # 安全选项 security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tlsConfig: null # 下载服务监听地址，dfget 下载文件将通过该地址连接到 daemon # 目前是支持 unix domain socket @@ -111,9 +111,9 @@ download: peerGRPC: security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tcpListen: # 监听地址 listen: 0.0.0.0 @@ -130,9 +130,9 @@ upload: rateLimit: 100Mi security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tcpListen: # 监听地址 listen: 0.0.0.0 @@ -174,17 +174,17 @@ proxy: # 例如：defaultFilter: "Expires&Signature": # http://localhost/xyz?Expires=111&Signature=222 and http://localhost/xyz?Expires=333&Signature=999 # 是相同的 task - defaultFilter: "Expires&Signature" + defaultFilter: 'Expires&Signature' security: insecure: true - cacert: "" - cert: "" - key: "" + cacert: '' + cert: '' + key: '' tcpListen: # 监听的网络命名空间, 例如：/proc/1/ns/net # 主要用在部署 kubernetes 中的时候，daemon 不使用 host network 时，监听宿主机的端口 # 仅支持 Linux - namespace: "" + namespace: '' # 监听地址 listen: 0.0.0.0 # 监听端口 @@ -222,8 +222,8 @@ proxy: hijackHTTPS: # https 劫持的证书和密钥 # 建议自签 CA 并更新主机证书链 - cert: "" - key: "" + cert: '' + key: '' # 需要走蜻蜓 p2p 的流量 hosts: - regx: mirror.aliyuncs.com:443 # 正则匹配 @@ -236,7 +236,7 @@ proxy: # 白名单，如果设置了，仅白名单内可以走代理，其他的都拒绝 whiteList: # 主机信息 - - host: "" + - host: '' # 正则匹配 regx: # 端口白名单 diff --git a/docs/zh-CN/deployment/configuration/manager.yaml b/docs/zh-CN/deployment/configuration/manager.yaml index dbefdffa1..7b7d3ff67 100644 --- a/docs/zh-CN/deployment/configuration/manager.yaml +++ b/docs/zh-CN/deployment/configuration/manager.yaml @@ -69,4 +69,4 @@ pprof-port: -1 # jaeger 地址 # 默认使用空字符串（不配置 jaeger）, 例如: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' diff --git a/docs/zh-CN/deployment/configuration/scheduler.yaml b/docs/zh-CN/deployment/configuration/scheduler.yaml index f7278bf49..270b2b0f1 100644 --- a/docs/zh-CN/deployment/configuration/scheduler.yaml +++ b/docs/zh-CN/deployment/configuration/scheduler.yaml @@ -11,11 +11,11 @@ server: # daemon 动态配置缓存目录 # linux 上默认目录 /var/cache/dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/cache - cacheDir: "" + cacheDir: '' # daemon 日志目录 # linux 上默认目录 /var/log/dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs - logDir: "" + logDir: '' # scheduler 调度策略配置 scheduler: @@ -55,18 +55,18 @@ dynConfig: # 实例主机信息 host: # 实例所在机房 - idc: "" + idc: '' # 实例网络拓扑信息 - netTopology: "" + netTopology: '' # 实例所在的地理位置信息 - location: "" + location: '' # manager 配置 manager: # 启动与 manager 的连接 enable: true # manager 访问地址 - addr: "" + addr: '' # 注册的 scheduler 集群 ID schedulerClusterID: # manager 心跳配置 @@ -87,11 +87,11 @@ job: # redis 配置 redis: # 服务地址 - host: "" + host: '' # 服务端口 port: 6379 # 密码 - password: "" + password: '' # broker 数据库 brokerDB: 1 # backend 数据库 @@ -102,7 +102,7 @@ metrics: # 启动数据收集服务 enable: false # 数据服务地址 - addr: ":8000" + addr: ':8000' # 开机收集 peer host 数据 enablePeerHost: false @@ -117,4 +117,4 @@ pprof-port: -1 # jaeger 地址 # 默认使用空字符串（不配置 jaeger）, 例如: http://jaeger.dragonfly.svc:14268/api/traces -jaeger: "" +jaeger: '' diff --git a/docs/zh-CN/deployment/installation/README.md b/docs/zh-CN/deployment/installation/README.md index d8da82b5d..70d03e578 100644 --- a/docs/zh-CN/deployment/installation/README.md +++ b/docs/zh-CN/deployment/installation/README.md @@ -2,6 +2,6 @@ 内容: -* [docker](docker/README.md) -* [executable-files](executable-files/README.md) -* [kubernetes](kubernetes/README.md) +- [docker](docker/README.md) +- [executable-files](executable-files/README.md) +- [kubernetes](kubernetes/README.md) diff --git a/docs/zh-CN/deployment/installation/docker/README.md b/docs/zh-CN/deployment/installation/docker/README.md index d5b415a85..d3332c42a 100644 --- a/docs/zh-CN/deployment/installation/docker/README.md +++ b/docs/zh-CN/deployment/installation/docker/README.md @@ -4,5 +4,5 @@ 安装选项: -* [Docker Compose](docker-compose.md) -* [Standalone](standalone/README.md) +- [Docker Compose](docker-compose.md) +- [Standalone](standalone/README.md) diff --git a/docs/zh-CN/deployment/installation/docker/standalone/README.md b/docs/zh-CN/deployment/installation/docker/standalone/README.md index f7a24dcd9..88ba6ee1a 100644 --- a/docs/zh-CN/deployment/installation/docker/standalone/README.md +++ b/docs/zh-CN/deployment/installation/docker/standalone/README.md @@ -10,10 +10,10 @@ ## 按模块单独安装 -* [install-dfdaemon](./dfdaemon.md) 是 dfdaemon 的安装指南 +- [install-dfdaemon](./dfdaemon.md) 是 dfdaemon 的安装指南 -* [install-manager](./manager.md) 是 manager 的安装指南 +- [install-manager](./manager.md) 是 manager 的安装指南 -* [install-scheduler](./scheduler.md) 是 scheduler 的安装指南 +- [install-scheduler](./scheduler.md) 是 scheduler 的安装指南 -* [install-cdn](./cdn.md) 是 CDN 的安装指南 +- [install-cdn](./cdn.md) 是 CDN 的安装指南 diff --git a/docs/zh-CN/deployment/installation/docker/standalone/cdn.md b/docs/zh-CN/deployment/installation/docker/standalone/cdn.md index 1a4e07c87..180128d8e 100644 --- a/docs/zh-CN/deployment/installation/docker/standalone/cdn.md +++ b/docs/zh-CN/deployment/installation/docker/standalone/cdn.md @@ -6,10 +6,10 @@ 使用 Docker 部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## 使用 Docker 部署 @@ -19,36 +19,36 @@ Docker|1.12.0+ 1. 获取最新的 CDN 镜像 - ```sh - docker pull dragonflyoss/cdn - ``` + ```sh + docker pull dragonflyoss/cdn + ``` 或者您可以构建自己的 CDN 镜像 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 构建 CDN 的 Docker 镜像 - ```sh - TAG="2.0.0" - make docker-build-cdn D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-cdn D7Y_VERSION=$TAG + ``` 4. 获取最新的 CDN 镜像 ID - ```sh - docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 + ``` ### 启动 cdn diff --git a/docs/zh-CN/deployment/installation/docker/standalone/dfdaemon.md b/docs/zh-CN/deployment/installation/docker/standalone/dfdaemon.md index 908f93bd2..38dd094ed 100644 --- a/docs/zh-CN/deployment/installation/docker/standalone/dfdaemon.md +++ b/docs/zh-CN/deployment/installation/docker/standalone/dfdaemon.md @@ -6,10 +6,10 @@ 使用 Docker 部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## 使用 Docker 部署 @@ -19,36 +19,36 @@ Docker|1.12.0+ 1. 获取最新的 dfdaemon 镜像 - ```sh - docker pull dragonflyoss/dfdaemon - ``` + ```sh + docker pull dragonflyoss/dfdaemon + ``` 或者您可以构建自己的 dfdaemon 镜像 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 构建 dfdaemon 的 Docker 镜像 - ```sh - TAG="2.0.0" - make docker-build-dfdaemon D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-dfdaemon D7Y_VERSION=$TAG + ``` 4. 获取最新的 dfdaemon 镜像 ID - ```sh - docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1 + ``` ### 启动 dfdaemon diff --git a/docs/zh-CN/deployment/installation/docker/standalone/manager.md b/docs/zh-CN/deployment/installation/docker/standalone/manager.md index 83f585480..de6d99b15 100644 --- a/docs/zh-CN/deployment/installation/docker/standalone/manager.md +++ b/docs/zh-CN/deployment/installation/docker/standalone/manager.md @@ -6,10 +6,10 @@ 使用 Docker 部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## 使用 Docker 部署 @@ -19,36 +19,36 @@ Docker|1.12.0+ 1. 获取最新的 Manager 镜像 - ```sh - docker pull dragonflyoss/manager - ``` + ```sh + docker pull dragonflyoss/manager + ``` 或者您可以构建自己的 manager 镜像 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 构建 manager 的 Docker 镜像 - ```sh - TAG="2.0.0" - make docker-build-manager D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-manager D7Y_VERSION=$TAG + ``` 4. 获取最新的 manager 镜像 ID - ```sh - docker image ls | grep 'manager' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'manager' | awk '{print $3}' | head -n1 + ``` ### 启动 manager diff --git a/docs/zh-CN/deployment/installation/docker/standalone/scheduler.md b/docs/zh-CN/deployment/installation/docker/standalone/scheduler.md index 9675526ac..9ef331807 100644 --- a/docs/zh-CN/deployment/installation/docker/standalone/scheduler.md +++ b/docs/zh-CN/deployment/installation/docker/standalone/scheduler.md @@ -6,10 +6,10 @@ 使用 Docker 部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Docker|1.12.0+ +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Docker | 1.12.0+ | ## 使用 Docker 部署 @@ -19,36 +19,36 @@ Docker|1.12.0+ 1. 获取最新的 scheduler 镜像 - ```sh - docker pull dragonflyoss/scheduler - ``` + ```sh + docker pull dragonflyoss/scheduler + ``` 或者您可以构建自己的 scheduler 镜像 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 构建 scheduler 的 Docker 镜像 - ```sh - TAG="2.0.0" - make docker-build-scheduler D7Y_VERSION=$TAG - ``` + ```sh + TAG="2.0.0" + make docker-build-scheduler D7Y_VERSION=$TAG + ``` 4. 获取最新的 scheduler 镜像 ID - ```sh - docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1 - ``` + ```sh + docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1 + ``` ### 启动 scheduler diff --git a/docs/zh-CN/deployment/installation/executable-files/README.md b/docs/zh-CN/deployment/installation/executable-files/README.md index 8653facf9..969124d00 100644 --- a/docs/zh-CN/deployment/installation/executable-files/README.md +++ b/docs/zh-CN/deployment/installation/executable-files/README.md @@ -4,10 +4,10 @@ ## 按模块单独安装 -* [install-dfdaemon](dfdaemon.md) 是 dfdaemon 的安装指南 +- [install-dfdaemon](dfdaemon.md) 是 dfdaemon 的安装指南 -* [install-manager](manager.md) 是 manager 的安装指南 +- [install-manager](manager.md) 是 manager 的安装指南 -* [install-scheduler](scheduler.md) 是 scheduler 的安装指南 +- [install-scheduler](scheduler.md) 是 scheduler 的安装指南 -* [install-cdn](cdn.md) 是 CDN 的安装指南 +- [install-cdn](cdn.md) 是 CDN 的安装指南 diff --git a/docs/zh-CN/deployment/installation/executable-files/cdn.md b/docs/zh-CN/deployment/installation/executable-files/cdn.md index 5f4714f3a..56b227656 100644 --- a/docs/zh-CN/deployment/installation/executable-files/cdn.md +++ b/docs/zh-CN/deployment/installation/executable-files/cdn.md @@ -6,31 +6,31 @@ 直接在物理机上部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Golang|1.12.x -Nginx|0.8+ +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | +| Nginx | 0.8+ | ## 在物理机上部署 ### 获取 CDN 可执行文件 1. 下载 Dragonfly 项目的压缩包。您可以从 -[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) -下载一个已发布的最近版本 + [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) + 下载一个已发布的最近版本 - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. 解压压缩包 - ```bash - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. 把 `cdn` 移动到环境变量 `PATH` 下以确保您可以直接使用 `cdn` 命令 @@ -38,21 +38,21 @@ Nginx|0.8+ 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 编译源码 - ```sh - make build-cdn && make install-cdn - ``` + ```sh + make build-cdn && make install-cdn + ``` ### 启动 cdn @@ -73,26 +73,26 @@ cdn --port=8003 --download-port=$cdnDownloadPort 1. 将下面的配置添加到 Nginx 配置文件中 - ```conf - server { - # Must be ${cdnDownloadPort} - listen 8001; - location / { - # Must be ${cdnHomeDir}/ftp - root /home/admin/ftp; - } - } - ``` + ```conf + server { + # Must be ${cdnDownloadPort} + listen 8001; + location / { + # Must be ${cdnHomeDir}/ftp + root /home/admin/ftp; + } + } + ``` 2. 启动 Nginx. - ```sh - sudo nginx - ``` + ```sh + sudo nginx + ``` - CDN 部署完成之后，运行以下命令以检查 Nginx 和 **cdn** 是否正在运行，以及 `8001` 和 `8003` 端口是否可用。 + CDN 部署完成之后，运行以下命令以检查 Nginx 和 **cdn** 是否正在运行，以及 `8001` 和 `8003` 端口是否可用。 - ```sh - telnet 127.0.0.1 8001 - telnet 127.0.0.1 8003 - ``` + ```sh + telnet 127.0.0.1 8001 + telnet 127.0.0.1 8003 + ``` diff --git a/docs/zh-CN/deployment/installation/executable-files/dfdaemon.md b/docs/zh-CN/deployment/installation/executable-files/dfdaemon.md index 7bcd9d1a6..9ab547096 100644 --- a/docs/zh-CN/deployment/installation/executable-files/dfdaemon.md +++ b/docs/zh-CN/deployment/installation/executable-files/dfdaemon.md @@ -6,30 +6,30 @@ 直接在物理机上部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Golang|1.12.x +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | ## 在物理机上部署 ### 获取 dfget 可执行文件 1. 下载 Dragonfly 项目的压缩包。您可以从 -[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) -下载一个已发布的最近版本 + [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) + 下载一个已发布的最近版本 - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. 解压压缩包 - ```bash - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. 把 `dfget` 移动到环境变量 `PATH` 下以确保您可以直接使用 `dfget` 命令 @@ -37,21 +37,21 @@ Golang|1.12.x 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 编译源码 - ```sh - make build-dfget && make install-dfget - ``` + ```sh + make build-dfget && make install-dfget + ``` ### 启动 dfdaemon diff --git a/docs/zh-CN/deployment/installation/executable-files/manager.md b/docs/zh-CN/deployment/installation/executable-files/manager.md index f64fc1564..e33836d50 100644 --- a/docs/zh-CN/deployment/installation/executable-files/manager.md +++ b/docs/zh-CN/deployment/installation/executable-files/manager.md @@ -6,31 +6,31 @@ 直接在物理机上部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Golang|1.12.x -Nginx|0.8+ +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | +| Nginx | 0.8+ | ## 在物理机上部署 ### 获取 manager 可执行文件 1. 下载 Dragonfly 项目的压缩包。您可以从 -[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) -下载一个已发布的最近版本 + [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) + 下载一个已发布的最近版本 - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. 解压压缩包 - ```bash - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. 把 `manager` 移动到环境变量 `PATH` 下以确保您可以直接使用 `manager` 命令 @@ -38,21 +38,21 @@ Nginx|0.8+ 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 编译源码 - ```sh - make build-manager && make install-manager - ``` + ```sh + make build-manager && make install-manager + ``` ### 启动 manager diff --git a/docs/zh-CN/deployment/installation/executable-files/scheduler.md b/docs/zh-CN/deployment/installation/executable-files/scheduler.md index 070be89ed..2f5328baa 100644 --- a/docs/zh-CN/deployment/installation/executable-files/scheduler.md +++ b/docs/zh-CN/deployment/installation/executable-files/scheduler.md @@ -6,31 +6,31 @@ 直接在物理机上部署时，以下条件必须满足： -所需软件 | 版本要求 ----|--- -Git|1.9.1+ -Golang|1.12.x -Nginx|0.8+ +| 所需软件 | 版本要求 | +| -------- | -------- | +| Git | 1.9.1+ | +| Golang | 1.12.x | +| Nginx | 0.8+ | ## 在物理机上部署 ### 获取 scheduler 可执行文件 1. 下载 Dragonfly 项目的压缩包。您可以从 -[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) -下载一个已发布的最近版本 + [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) + 下载一个已发布的最近版本 - ```sh - version=2.0.0 - wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz - ``` + ```sh + version=2.0.0 + wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz + ``` 2. 解压压缩包 - ```bash - # Replace `xxx` with the installation directory. - tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx - ``` + ```bash + # Replace `xxx` with the installation directory. + tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx + ``` 3. 把 `scheduler` 移动到环境变量 `PATH` 下以确保您可以直接使用 `scheduler` 命令 @@ -38,21 +38,21 @@ Nginx|0.8+ 1. 获取 Dragonfly 的源码 - ```sh - git clone https://github.com/dragonflyoss/Dragonfly2.git - ``` + ```sh + git clone https://github.com/dragonflyoss/Dragonfly2.git + ``` 2. 打开项目文件夹 - ```sh - cd Dragonfly2 - ``` + ```sh + cd Dragonfly2 + ``` 3. 编译源码 - ```sh - make build-scheduler && make install-scheduler - ``` + ```sh + make build-scheduler && make install-scheduler + ``` ### 启动 scheduler diff --git a/docs/zh-CN/deployment/installation/kubernetes/README.md b/docs/zh-CN/deployment/installation/kubernetes/README.md index 5e1d2d680..bab0a89f0 100644 --- a/docs/zh-CN/deployment/installation/kubernetes/README.md +++ b/docs/zh-CN/deployment/installation/kubernetes/README.md @@ -6,6 +6,6 @@ dfdaemon 会作为 `DaemonSets` 部署, manager 会作为 `Deployments` 部署 部署方式: -* [Helm](helm.md) -* [Kustomize](kustomize.md) -* [TODO Upgrade Guide](upgrade-guide.md) +- [Helm](helm.md) +- [Kustomize](kustomize.md) +- [TODO Upgrade Guide](upgrade-guide.md) diff --git a/docs/zh-CN/deployment/installation/kubernetes/helm.md b/docs/zh-CN/deployment/installation/kubernetes/helm.md index 616a6b345..4ff7a6f8a 100644 --- a/docs/zh-CN/deployment/installation/kubernetes/helm.md +++ b/docs/zh-CN/deployment/installation/kubernetes/helm.md @@ -11,10 +11,10 @@ > 因为当前 Kubernetes 里的 `daemonset` 并不支持 `Surging Rolling Update` 策略, > 一旦旧的 dfdaemon pod 被删除后，新的 dfdaemon 就再也拉取不了了。 > 如果无法更换容器运行时的话，那在升级蜻蜓的时候，请从下面两种方案选择比较适合的： -> 选项1：先手动拉取新的 dfdaemon 镜像，或者使用 -[ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob) -去自动拉取。 -> 选项2：保持蜻蜓的镜像中心和通用的镜像中心不一样，同时将蜻蜓镜像中心相关的 host 加入 `containerRuntime.docker.skipHosts`。 +> 选项 1：先手动拉取新的 dfdaemon 镜像，或者使用 +> [ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob) +> 去自动拉取。 +> 选项 2：保持蜻蜓的镜像中心和通用的镜像中心不一样，同时将蜻蜓镜像中心相关的 host 加入 `containerRuntime.docker.skipHosts`。 Dragonfly Helm 支持自动更改 docker 配置。 @@ -32,8 +32,8 @@ containerRuntime: # When use certs and inject hosts in docker, no necessary to restart docker daemon. injectHosts: true registryDomains: - - "harbor.example.com" - - "harbor.example.net" + - 'harbor.example.com' + - 'harbor.example.net' ``` 此配置允许 docker 通过 Dragonfly 拉取 `harbor.example.com` 和 `harbor.example.net` 域名镜像。 @@ -41,13 +41,13 @@ containerRuntime: 优点： -* 支持 dfdaemon 自身平滑升级 +- 支持 dfdaemon 自身平滑升级 > 这种模式下，当删除 dfdaemon pod 的时候，`preStop` 钩子将会清理已经注入到 `/etc/hosts` 下的所有主机信息，所有流量将会走原来的镜像中心。 限制: -* 只支持指定域名。 +- 只支持指定域名。 #### 情况 2: 支持任意仓库 @@ -72,8 +72,8 @@ containerRuntime: 限制: -* 必须开启 docker 的 `live-restore` 功能 -* 需要重启 docker daemon +- 必须开启 docker 的 `live-restore` 功能 +- 需要重启 docker daemon ### 2. Containerd @@ -100,7 +100,7 @@ containerRuntime: #### 情况 2: V2 版本不使用配置文件 -* 选项 1 - 允许 Charts 注入 `config_path` 并重新启动 containerd。 +- 选项 1 - 允许 Charts 注入 `config_path` 并重新启动 containerd。 此选项启用多个镜像仓库支持。 @@ -115,7 +115,7 @@ containerRuntime: injectConfigPath: true ``` -* 选项 2 - 只使用一个镜像仓库 `dfdaemon.config.proxy.registryMirror.url` +- 选项 2 - 只使用一个镜像仓库 `dfdaemon.config.proxy.registryMirror.url` 定制 values.yaml 文件: @@ -153,9 +153,9 @@ containerRuntime: enable: true # Registries full urls registries: - - "https://ghcr.io" - - "https://quay.io" - - "https://harbor.example.com:8443" + - 'https://ghcr.io' + - 'https://quay.io' + - 'https://harbor.example.com:8443' ``` ## 准备 Kubernetes 集群 @@ -230,7 +230,7 @@ manager: externalManager: enable: true - host: "dragonfly-manager.dragonfly-system.svc.cluster.local" + host: 'dragonfly-manager.dragonfly-system.svc.cluster.local' restPort: 8080 grpcPort: 65003 diff --git a/docs/zh-CN/design/README.md b/docs/zh-CN/design/README.md index 094fdb0df..1f5c3b979 100644 --- a/docs/zh-CN/design/README.md +++ b/docs/zh-CN/design/README.md @@ -5,8 +5,8 @@ Dragonfly 的所有组件设计的架构，组件之间的交互工作流程，D 具体设计: -* [architecture](architecture.md) -* [manager](manager.md) -* [TODO scheduler](scheduler.md) -* [TODO cdn](cdn.md) -* [TODO dfdaemon](dfdaemon.md) +- [architecture](architecture.md) +- [manager](manager.md) +- [TODO scheduler](scheduler.md) +- [TODO cdn](cdn.md) +- [TODO dfdaemon](dfdaemon.md) diff --git a/docs/zh-CN/design/architecture.md b/docs/zh-CN/design/architecture.md index ee33e9790..a6dc221c7 100644 --- a/docs/zh-CN/design/architecture.md +++ b/docs/zh-CN/design/architecture.md @@ -16,12 +16,12 @@ ## 核心能力 -- 通过统一的回源适配层及实现支持各种不同类型存储（HDFS、各云厂商的存储服务、Maven、YUM等等）的P2P文件分发能力 +- 通过统一的回源适配层及实现支持各种不同类型存储（HDFS、各云厂商的存储服务、Maven、YUM 等等）的 P2P 文件分发能力 - 支持更多的分发模式: 主动拉取、主动推送、实时同步、远程复制、自动预热、跨云传输等等 -- 系统之间分离解耦，调度与CDN插件化，且支持按需部署，可轻可重，可内可外，灵活满足不同场景的实际需要 +- 系统之间分离解耦，调度与 CDN 插件化，且支持按需部署，可轻可重，可内可外，灵活满足不同场景的实际需要 - 基于 grpc 全新设计的 P2P 协议框架，效率及稳定性更佳 - 支持加密传输、基于账号的传输鉴权与限流、多租户隔离机制 -- 支持更高效的IO方式: 多线程IO、内存映射、DMA等 +- 支持更高效的 IO 方式: 多线程 IO、内存映射、DMA 等 - 支持动态压缩、内存文件系统以及更高效的调度算法，提升分发效率 - 客户端通过 C/S 模式支持第三方软件原生集成蜻蜓的 P2P 能力 - 产品化能力: 支持文件上传、各种分发模式的任务管理、数据视图以及全局管控等功能 @@ -44,7 +44,7 @@ ### 子模块架构 -#### CDN模块 +#### CDN 模块 ![cdn][cdn] diff --git a/docs/zh-CN/developer-guide/README.md b/docs/zh-CN/developer-guide/README.md new file mode 100644 index 000000000..66b9b18a5 --- /dev/null +++ b/docs/zh-CN/developer-guide/README.md @@ -0,0 +1,8 @@ +# 开发者指南 + +理解如何开发 Dragonfly 项目以及相关插件。 + +内容: + +- [开发](development.md) +- [插件](plugin.md) diff --git a/docs/zh-CN/developer-guide/developer-guide.md b/docs/zh-CN/developer-guide/developer-guide.md deleted file mode 100644 index d8e2dd485..000000000 --- a/docs/zh-CN/developer-guide/developer-guide.md +++ /dev/null @@ -1,64 +0,0 @@ -# 开发 - -快速搭建本地开发环境 - -## 第一步：安装 docker 和 docker compose - -详见文档 [docs.docker.com] - -## 第二步：启动 dragonfly - -打开 dragonfly 文档、启动 docker-compose. - -```bash -$ docker-compose up -Creating network "dragonfly2_default" with the default driver -Creating cdn ... done -Creating scheduler ... done -Creating dfdaemon ... done -Attaching to cdn, scheduler, dfdaemon -``` - -## 第三步：日志分析 - -查看 dragonfly 日志 - - -```bash -$ tail -f log/**/*.log -==> log/dragonfly/cdn/core.log <== -{"level":"info","ts":"2021-02-26 05:43:36.896","caller":"cmd/root.go:90","msg":"success to init local ip of cdn, start to run cdn system, use ip: 172.21.0.2"} -{"level":"info","ts":"2021-02-26 05:43:36.901","caller":"plugins/plugins.go:51","msg":"add plugin[storage][local]"} -{"level":"info","ts":"2021-02-26 05:43:36.902","caller":"plugins/plugins.go:37","msg":"plugin[sourceclient][http] is disabled"} - -==> log/dragonfly/dfdaemon/core.log <== -{"level":"info","ts":"2021-02-26 05:43:37.797","caller":"proxy/proxy_manager.go:63","msg":"registry mirror: https://index.docker.io"} -{"level":"info","ts":"2021-02-26 05:43:37.798","caller":"proxy/proxy_manager.go:68","msg":"load 1 proxy rules"} -{"level":"info","ts":"2021-02-26 05:43:37.799","caller":"proxy/proxy_manager.go:78","msg":"[1] proxy blobs/sha256.* with dragonfly "} -{"level":"debug","ts":"2021-02-26 05:43:37.799","caller":"rpc/server_listen.go:31","msg":"start to listen port: 0.0.0.0:65000"} -{"level":"debug","ts":"2021-02-26 05:43:37.800","caller":"rpc/server_listen.go:31","msg":"start to listen port: 0.0.0.0:65002"} -{"level":"debug","ts":"2021-02-26 05:43:37.800","caller":"rpc/server_listen.go:31","msg":"start to listen port: 0.0.0.0:65001"} -{"level":"info","ts":"2021-02-26 05:43:37.800","caller":"daemon/peerhost.go:274","msg":"serve download grpc at unix:///tmp/dfdamon.sock"} -{"level":"info","ts":"2021-02-26 05:43:37.801","caller":"daemon/peerhost.go:285","msg":"serve peer grpc at tcp://[::]:65000"} -{"level":"info","ts":"2021-02-26 05:43:37.801","caller":"daemon/peerhost.go:320","msg":"serve upload service at tcp://[::]:65002"} -{"level":"info","ts":"2021-02-26 05:43:37.801","caller":"daemon/peerhost.go:306","msg":"serve proxy at tcp://0.0.0.0:65001"} - -==> log/dragonfly/scheduler/core.log <== -{"level":"info","ts":"2021-02-26 05:43:37.332","caller":"cmd/root.go:57","msg":"start to run scheduler"} -{"level":"info","ts":"2021-02-26 05:43:37.338","caller":"server/server.go:35","msg":"start server at port %!s(int=8002)"} -{"level":"info","ts":"2021-02-26 05:43:37.342","caller":"worker/sender.go:49","msg":"start sender worker : 50"} -{"level":"info","ts":"2021-02-26 05:43:37.343","caller":"worker/worker_group.go:64","msg":"start scheduler worker number:6"} -``` - - -## 第四步：关闭 dragonfly - -```bash -$ docker-compose down -Removing dfdaemon ... done -Removing scheduler ... done -Removing cdn ... done -Removing network dragonfly2_default -``` - -[docs.docker.com]: https://docs.docker.com diff --git a/docs/zh-CN/developer-guide/development.md b/docs/zh-CN/developer-guide/development.md new file mode 100644 index 000000000..1a9b2ed0e --- /dev/null +++ b/docs/zh-CN/developer-guide/development.md @@ -0,0 +1,3 @@ +# 开发 + +中文文档编写中，请先参考正在编写的[英文文档](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/en/developer-guide/development.md) diff --git a/docs/zh-CN/preheat/README.md b/docs/zh-CN/preheat/README.md index 3f05da650..162c714a5 100644 --- a/docs/zh-CN/preheat/README.md +++ b/docs/zh-CN/preheat/README.md @@ -4,5 +4,5 @@ P2P 加速可预热两种类型数据 `image` 和 `file`, 用户可以在控制 预热方式: -* [Console](console.md) -* [Api](api.md) +- [Console](console.md) +- [Api](api.md) diff --git a/docs/zh-CN/quick-start.md b/docs/zh-CN/quick-start.md index defc4bb8c..4723e1243 100644 --- a/docs/zh-CN/quick-start.md +++ b/docs/zh-CN/quick-start.md @@ -9,14 +9,16 @@ 下表列出了一些容器的运行时、版本和文档。 -| Runtime | Version | Document | CRI Support | Pull Command | -| --- | --- | --- | --- | --- | -| Containerd* | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest | -| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine | -| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest | + +| Runtime | Version | Document | CRI Support | Pull Command | +| ----------------------- | -------- | ------------------------------------------------ | ----------- | ------------------------------------------- | +| Containerd\* | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest | +| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine | +| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest | + -**: 推荐使用`containerd`* +推荐使用 `containerd`. ## Helm Chart 运行时配置 @@ -34,8 +36,8 @@ containerRuntime: # When use certs and inject hosts in docker, no necessary to restart docker daemon. injectHosts: true registryDomains: - - "harbor.example.com" - - "harbor.example.net" + - 'harbor.example.com' + - 'harbor.example.net' ``` 此配置允许 docker 通过 Dragonfly 拉取 `harbor.example.com` 和 `harbor.example.net` 域名镜像。 @@ -43,13 +45,13 @@ containerRuntime: 优点： -* 支持 dfdaemon 自身平滑升级 +- 支持 dfdaemon 自身平滑升级 > 这种模式下，当删除 dfdaemon pod 的时候，`preStop` 钩子将会清理已经注入到 `/etc/hosts` 下的所有主机信息，所有流量将会走原来的镜像中心。 限制: -* 只支持指定域名。 +- 只支持指定域名。 ## 准备 Kubernetes 集群 diff --git a/docs/zh-CN/runtime-integration/README.md b/docs/zh-CN/runtime-integration/README.md index 69aad590a..9d730a2fa 100644 --- a/docs/zh-CN/runtime-integration/README.md +++ b/docs/zh-CN/runtime-integration/README.md @@ -2,6 +2,6 @@ 运行时: -* [containerd](containerd/README.md) -* [cri-o](cri-o.md) -* [docker](docker.md) +- [containerd](containerd/README.md) +- [cri-o](cri-o.md) +- [docker](docker.md) diff --git a/docs/zh-CN/runtime-integration/containerd/README.md b/docs/zh-CN/runtime-integration/containerd/README.md index 2f3bb5860..3bab28dda 100644 --- a/docs/zh-CN/runtime-integration/containerd/README.md +++ b/docs/zh-CN/runtime-integration/containerd/README.md @@ -4,5 +4,5 @@ 集成方式: -* [mirror](mirror.md) -* [proxy](proxy.md) +- [mirror](mirror.md) +- [proxy](proxy.md) diff --git a/docs/zh-CN/runtime-integration/containerd/mirror.md b/docs/zh-CN/runtime-integration/containerd/mirror.md index f4c14e706..9159cbc07 100644 --- a/docs/zh-CN/runtime-integration/containerd/mirror.md +++ b/docs/zh-CN/runtime-integration/containerd/mirror.md @@ -82,7 +82,7 @@ version = 2 #### 2. 生成每个仓库的 hosts.toml -##### 选项 1: 手动生成 hosts.toml +##### 选项 1: 手动生成 hosts.toml 路径: `/etc/containerd/certs.d/example.com/hosts.toml` diff --git a/docs/zh-CN/runtime-integration/containerd/proxy.md b/docs/zh-CN/runtime-integration/containerd/proxy.md index efedc257d..5d84b5145 100644 --- a/docs/zh-CN/runtime-integration/containerd/proxy.md +++ b/docs/zh-CN/runtime-integration/containerd/proxy.md @@ -139,7 +139,7 @@ proxy: key: ca.key hosts: - regx: your.private.registry - certs: ["server.crt"] + certs: ['server.crt'] ``` 您能使用以下命令获取您服务器的证书： diff --git a/docs/zh-CN/runtime-integration/docker.md b/docs/zh-CN/runtime-integration/docker.md index bb4f9b5e5..b2acf5a64 100644 --- a/docs/zh-CN/runtime-integration/docker.md +++ b/docs/zh-CN/runtime-integration/docker.md @@ -146,7 +146,7 @@ proxy: key: ca.key hosts: - regx: your.private.registry - certs: ["server.crt"] + certs: ['server.crt'] ``` 您能使用以下命令获取您服务器的证书： diff --git a/docs/zh-CN/troubleshooting/README.md b/docs/zh-CN/troubleshooting/README.md index bc75f36ab..efbc7dfb1 100644 --- a/docs/zh-CN/troubleshooting/README.md +++ b/docs/zh-CN/troubleshooting/README.md @@ -26,16 +26,16 @@ change log level to info 1. 确认限速值是否合适 [dfget.yaml](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/zh-CN/config/dfget.yaml#L61) - ```yaml - download: - # 总下载限速 - totalRateLimit: 200Mi - # 单个任务下载限速 - perPeerRateLimit: 100Mi # 为了兼容极限环境下，默认值为 20Mi，可以按需调整 - upload: - # 上传限速 - rateLimit: 100Mi - ``` + ```yaml + download: + # 总下载限速 + totalRateLimit: 200Mi + # 单个任务下载限速 + perPeerRateLimit: 100Mi # 为了兼容极限环境下，默认值为 20Mi，可以按需调整 + upload: + # 上传限速 + rateLimit: 100Mi + ``` 2. 确认回源速度是否正常 @@ -45,10 +45,10 @@ change log level to info 2. 检查源站可连接行(DNS 错误 or 证书) - 示例: + 示例: - ```shell - curl https://example.harbor.local/ - ``` + ```shell + curl https://example.harbor.local/ + ``` - 如果`curl`有报错，请查看具体错误 + 如果`curl`有报错，请查看具体错误 diff --git a/docs/zh-CN/user-guide/README.md b/docs/zh-CN/user-guide/README.md new file mode 100644 index 000000000..3a4204656 --- /dev/null +++ b/docs/zh-CN/user-guide/README.md @@ -0,0 +1,7 @@ +# User Guide + +让用户快速了解 Dragonfly 如何进行 P2P 下载以及其他使用相关内容。 + +内容: + +- [Prometheus 数据指标](metrics.md) diff --git a/docs/zh-CN/user-guide/metrics.md b/docs/zh-CN/user-guide/metrics.md new file mode 100644 index 000000000..9a9a6414c --- /dev/null +++ b/docs/zh-CN/user-guide/metrics.md @@ -0,0 +1,67 @@ +# Prometheus 数据指标 + +文档包含 Dragonfly 组件当前支持的所有指标。 +现在支持 Dfdaemon、Manager、Scheduler 和 CDN 服务的指标。 +获取数据指标的路径固定为 /metrics。 + +## Dfdaemon + +GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + + + +| 名字 | 标签 | 类型 | 描述 | +| :------------------------------------------------------- | :----- | :------ | :---------------------------------- | +| dragonfly_dfdaemon_proxy_request_total | method | counter | 代理请求总次数。 | +| dragonfly_dfdaemon_proxy_request_via_dragonfly_total | | counter | 代理通过 drgonfly 的请求次数。 | +| dragonfly_dfdaemon_proxy_request_not_via_dragonfly_total | | counter | 代理没有通过 dragonfly 的请求次数。 | +| dragonfly_dfdaemon_proxy_request_running_total | method | counter | 当前代理请求总次数。 | +| dragonfly_dfdaemon_proxy_request_bytes_total | method | counter | 所有代理请求的总字节数。 | +| dragonfly_dfdaemon_peer_task_total | | counter | 任务的总个数。 | +| dragonfly_dfdaemon_peer_task_failed_total | | counter | 失败任务的总个数。 | +| dragonfly_dfdaemon_piece_task_total | | counter | 分片的总个数。 | +| dragonfly_dfdaemon_piece_task_failed_total | | counter | 失败的分片总个数。 | +| dragonfly_dfdaemon_file_task_total | | counter | 文件类型任务总个数。 | +| dragonfly_dfdaemon_stream_task_total | | counter | 流式类型任务总个数。 | +| dragonfly_dfdaemon_peer_task_cache_hit_total | | counter | 命中缓存任务个数。 | + + + +## Manager + +GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + +## Scheduler + +GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + + + +| 名字 | 标签 | 类型 | 描述 | +| :----------------------------------------------------------- | :----------------------------------------- | :-------- | :-------------------- | +| dragonfly_scheduler_register_peer_task_total | | counter | 注册任务总次数。 | +| dragonfly_scheduler_register_peer_task_failure_total | | counter | 注册任务失败次数。 | +| dragonfly_scheduler_download_total | | counter | 下载任务总次数。 | +| dragonfly_scheduler_download_failure_total | | counter | 下载任务失败次数。 | +| dragonfly_scheduler_p2p_traffic | | counter | P2P 流量。 | +| dragonfly_scheduler_peer_host_traffic | traffic_type, peer_host_uuid, peer_host_ip | counter | 每个主机的 P2P 流量。 | +| dragonfly_scheduler_peer_task_total | type | counter | 下载任务总个数。 | +| dragonfly_scheduler_peer_task_download_duration_milliseconds | | histogram | 任务下载耗时。 | +| dragonfly_scheduler_concurrent_schedule_total | | gauge | 并行调度任务个数。 | + + + +## CDN + +GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus). + + + +| 名字 | 标签 | 类型 | 描述 | +| :-------------------------------------- | :--- | :------ | :----------------- | +| dragonfly_cdn_download_total | | counter | 下载任务总次数。 | +| dragonfly_cdn_download_failure_total | | counter | 下载任务失败次数。 | +| dragonfly_cdn_download_traffic | | counter | 经过 CDN 流量。 | +| dragonfly_cdn_concurrent_download_total | | gauge | 并行下载任务个数。 | + +