docs: add metrics document (#1075)

* docs: add metrics document

Signed-off-by: Gaius <gaius.qi@gmail.com>

* docs: format docs

Signed-off-by: Gaius <gaius.qi@gmail.com>

* chore: change markdownlint rules

Signed-off-by: Gaius <gaius.qi@gmail.com>
This commit is contained in:
Gaius 2022-02-16 17:18:01 +08:00
parent 4a171ac5cb
commit c7d02ef8be
No known key found for this signature in database
GPG Key ID: 8B4E5D1290FA2FFB
81 changed files with 929 additions and 779 deletions

View File

@ -1,3 +1,5 @@
MD010: false MD010: false
MD013: MD013:
line_length: 120 line_length: 120
MD046:
style: "fenced"

View File

@ -9,29 +9,33 @@ part is that all need.
Organization of document is as following: Organization of document is as following:
* [Quick Start](quick-start.md) - [Quick Start](quick-start.md)
* [Deployment](deployment/README.md) - [Deployment](deployment/README.md)
* [Installation](deployment/installation) - [Installation](deployment/installation)
* [Configuration](deployment/configuration) - [Configuration](deployment/configuration)
* [Troubleshooting](troubleshooting/README.md) - [Troubleshooting](troubleshooting/README.md)
* [CLI Reference](cli-reference/README.md) - [CLI Reference](cli-reference/README.md)
* [dfget](cli-reference/dfget.md) - [dfget](cli-reference/dfget.md)
* [cdn](cli-reference/cdn.md) - [cdn](cli-reference/cdn.md)
* [scheduler](cli-reference/scheduler.md) - [scheduler](cli-reference/scheduler.md)
* [manager](cli-reference/manager.md) - [manager](cli-reference/manager.md)
* [Preheat](preheat/README.md) - [Preheat](preheat/README.md)
* [Console](preheat/console.md) - [Console](preheat/console.md)
* [Api](preheat/api.md) - [Api](preheat/api.md)
* [Runtime Integration](runtime-integration/README.md) - [Runtime Integration](runtime-integration/README.md)
* [containerd](runtime-integration/containerd/README.md) - [containerd](runtime-integration/containerd/README.md)
* [cri-o](runtime-integration/cri-o.md) - [cri-o](runtime-integration/cri-o.md)
* [docker](runtime-integration/docker.md) - [docker](runtime-integration/docker.md)
* [Design](design/README.md) - [Design](design/README.md)
* [architecture](design/architecture.md) - [architecture](design/architecture.md)
* [manager](design/manager.md) - [manager](design/manager.md)
* [TODO scheduler](design/scheduler.md) - [TODO scheduler](design/scheduler.md)
* [TODO cdn](design/cdn.md) - [TODO cdn](design/cdn.md)
* [TODO dfdaemon](design/dfdaemon.md) - [TODO dfdaemon](design/dfdaemon.md)
* [Developer Guide](developer-guide/developer-guide.md) - [Developer Guide](developer-guide/README.md)
* [Test Guide](test-guide/test-guide.md) - [Development](developer-guide/development.md)
* [API Reference](api-reference/api-reference.md) - [Plugin](developer-guide/plugin.md)
- [User Guide](user-guide/README.md)
- [Prometheus Metrics](user-guide/metrics.md)
- [Test Guide](test-guide/test-guide.md)
- [API Reference](api-reference/api-reference.md)

View File

@ -10,7 +10,7 @@ all commandline docs are auto generated via source code.
Table of contents: Table of contents:
* [dfget](dfget.md) - [dfget](dfget.md)
* [cdn](cdn.md) - [cdn](cdn.md)
* [scheduler](scheduler.md) - [scheduler](scheduler.md)
* [manager](manager.md) - [manager](manager.md)

View File

@ -13,7 +13,7 @@ cdn [command]
## Available Commands ## Available Commands
```text ```text
doc generate documents doc generate documents
help Help about any command help Help about any command
version show version version show version
``` ```
@ -21,8 +21,9 @@ version show version
## Flags ## Flags
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```text ```text
--config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/cdn.yaml, it can --config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/cdn.yaml, it can
also be set by environment variable cdn_config also be set by environment variable cdn_config
--console whether logger output records to the stdout --console whether logger output records to the stdout
-h, --help help for scheduler -h, --help help for scheduler
@ -31,4 +32,5 @@ version show version
--service-name string name of the service for tracer (default "dragonfly-cdn") --service-name string name of the service for tracer (default "dragonfly-cdn")
--verbose whether logger use debug level --verbose whether logger use debug level
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->

View File

@ -32,6 +32,7 @@ set environment variable console=true if you want to print logs to Terminal
### Options ### Options
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
``` ```
--alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s)
--cacerts strings the cacert file which is used to verify remote server when supernode interact with the source. --cacerts strings the cacert file which is used to verify remote server when supernode interact with the source.
@ -65,6 +66,7 @@ set environment variable console=true if you want to print logs to Terminal
-u, --url string URL of user requested downloading file(only HTTP/HTTPs supported) -u, --url string URL of user requested downloading file(only HTTP/HTTPs supported)
--verbose enable verbose mode, all debug log will be display --verbose enable verbose mode, all debug log will be display
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
## dfget daemon ## dfget daemon
@ -76,6 +78,7 @@ set environment variable console=true if you want to print logs to Terminal
### Daemon Options ### Daemon Options
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
``` ```
--advertise-ip string the ip report to scheduler, normal same with listen ip (default "10.15.232.63") --advertise-ip string the ip report to scheduler, normal same with listen ip (default "10.15.232.63")
--alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s)
@ -104,4 +107,5 @@ set environment variable console=true if you want to print logs to Terminal
--upload-rate ratelimit upload rate limit for other peers (default 104857600.000000) --upload-rate ratelimit upload rate limit for other peers (default 104857600.000000)
--verbose print verbose log and enable golang debug info --verbose print verbose log and enable golang debug info
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->

View File

@ -31,7 +31,7 @@ endpoint: /swagger/doc.json
## Prometheus metrics monitoring ## Prometheus metrics monitoring
endpoint: /metrics endpoint: /metrics
## HealthCheck ## HealthCheck
@ -46,6 +46,7 @@ manager --jaeger http://localhost:14250/api/traces
## Options ## Options
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```text ```text
--config string the path of configuration file with yaml extension name, default is /etc/dragonfly/manager.yaml, it can also be set by env var: MANAGER_CONFIG --config string the path of configuration file with yaml extension name, default is /etc/dragonfly/manager.yaml, it can also be set by env var: MANAGER_CONFIG
--console whether logger output records to the stdout --console whether logger output records to the stdout
@ -55,6 +56,7 @@ manager --jaeger http://localhost:14250/api/traces
--service-name string name of the service for tracer (default "dragonfly-manager") --service-name string name of the service for tracer (default "dragonfly-manager")
--verbose whether logger use debug level --verbose whether logger use debug level
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
## Manager Config ## Manager Config

View File

@ -16,7 +16,7 @@ scheduler [command]
## Available Commands ## Available Commands
```text ```text
doc generate documents doc generate documents
help Help about any command help Help about any command
version show version version show version
``` ```
@ -24,8 +24,9 @@ version show version
## Flags ## Flags
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```text ```text
--config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/scheduler.yaml, it can --config string the path of configuration file with yaml extension name, default is /Users/${USER_HOME}/.dragonfly/config/scheduler.yaml, it can
also be set by environment variable scheduler_config also be set by environment variable scheduler_config
--console whether logger output records to the stdout --console whether logger output records to the stdout
-h, --help help for scheduler -h, --help help for scheduler
@ -34,4 +35,5 @@ version show version
--service-name string name of the service for tracer (default "dragonfly-scheduler") --service-name string name of the service for tracer (default "dragonfly-scheduler")
--verbose whether logger use debug level --verbose whether logger use debug level
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->

View File

@ -2,8 +2,8 @@
Table of contents: Table of contents:
* [installation](installation/README.md) - [installation](installation/README.md)
* [configuration](configuration) - [configuration](configuration)
## Installation ## Installation

View File

@ -44,11 +44,11 @@ base:
# logDir is the log storage directory # logDir is the log storage directory
# in linux, default value is /var/log/dragonfly # in linux, default value is /var/log/dragonfly
# in macos(just for testing), default value is /Users/$USER/.dragonfly/logs # in macos(just for testing), default value is /Users/$USER/.dragonfly/logs
logDir: "" logDir: ''
# manager configuration # manager configuration
manager: manager:
addr: "" addr: ''
cdnClusterID: 0 cdnClusterID: 0
keepAlive: keepAlive:
interval: 5s interval: 5s
@ -116,7 +116,7 @@ verbose: false
pprof-port: -1 pprof-port: -1
# jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''
# service name used in tracer # service name used in tracer
# default: dragonfly-cdn # default: dragonfly-cdn

View File

@ -8,22 +8,22 @@ gcInterval: 1m0s
# daemon work directory, daemon will change current working directory to this # daemon work directory, daemon will change current working directory to this
# in linux, default value is /usr/local/dragonfly # in linux, default value is /usr/local/dragonfly
# in macos(just for testing), default value is /Users/$USER/.dragonfly # in macos(just for testing), default value is /Users/$USER/.dragonfly
workHome: "" workHome: ''
# cacheDir is dynconfig cache storage directory # cacheDir is dynconfig cache storage directory
# in linux, default value is /var/cache/dragonfly # in linux, default value is /var/cache/dragonfly
# in macos(just for testing), default value is /Users/$USER/.dragonfly/cache # in macos(just for testing), default value is /Users/$USER/.dragonfly/cache
cacheDir: "" cacheDir: ''
# logDir is the log storage directory # logDir is the log storage directory
# in linux, default value is /var/log/dragonfly # in linux, default value is /var/log/dragonfly
# in macos(just for testing), default value is /Users/$USER/.dragonfly/logs # in macos(just for testing), default value is /Users/$USER/.dragonfly/logs
logDir: "" logDir: ''
# dataDir is the download data storage directory # dataDir is the download data storage directory
# in linux, default value is /var/lib/dragonfly # in linux, default value is /var/lib/dragonfly
# in macos(just for testing), default value is /Users/$USER/.dragonfly/data # in macos(just for testing), default value is /Users/$USER/.dragonfly/data
dataDir: "" dataDir: ''
# when daemon exit, keep peer task data or not # when daemon exit, keep peer task data or not
# it is usefully when upgrade daemon service, all local cache will be saved # it is usefully when upgrade daemon service, all local cache will be saved
@ -41,7 +41,7 @@ verbose: false
pprof-port: -1 pprof-port: -1
# jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''
# all addresses of all schedulers # all addresses of all schedulers
# the schedulers of all daemons should be same in one region or zone. # the schedulers of all daemons should be same in one region or zone.
@ -75,13 +75,13 @@ host:
# when local ip is different with access ip, advertiseIP should be set # when local ip is different with access ip, advertiseIP should be set
advertiseIP: 0.0.0.0 advertiseIP: 0.0.0.0
# geographical location, separated by "|" characters # geographical location, separated by "|" characters
location: "" location: ''
# idc deployed by daemon # idc deployed by daemon
idc: "" idc: ''
# security domain deployed by daemon, network isolation between different security domains # security domain deployed by daemon, network isolation between different security domains
securityDomain: "" securityDomain: ''
# network topology, separated by "|" characters # network topology, separated by "|" characters
netTopology: "" netTopology: ''
# daemon hostname # daemon hostname
# hostname: "" # hostname: ""
@ -116,9 +116,9 @@ download:
# security option # security option
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tlsConfig: null tlsConfig: null
# download service listen address # download service listen address
# current, only support unix domain socket # current, only support unix domain socket
@ -131,9 +131,9 @@ download:
peerGRPC: peerGRPC:
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tcpListen: tcpListen:
# listen address # listen address
listen: 0.0.0.0 listen: 0.0.0.0
@ -151,9 +151,9 @@ upload:
rateLimit: 100Mi rateLimit: 100Mi
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tcpListen: tcpListen:
# listen address # listen address
listen: 0.0.0.0 listen: 0.0.0.0
@ -196,17 +196,17 @@ proxy:
# when defaultFilter: "Expires&Signature", for example: # when defaultFilter: "Expires&Signature", for example:
# http://localhost/xyz?Expires=111&Signature=222 and http://localhost/xyz?Expires=333&Signature=999 # http://localhost/xyz?Expires=111&Signature=222 and http://localhost/xyz?Expires=333&Signature=999
# is same task # is same task
defaultFilter: "Expires&Signature" defaultFilter: 'Expires&Signature'
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tcpListen: tcpListen:
# namespace stands the linux net namespace, like /proc/1/ns/net # namespace stands the linux net namespace, like /proc/1/ns/net
# it's useful for running daemon in pod with ip allocated and listening the special port in host net namespace # it's useful for running daemon in pod with ip allocated and listening the special port in host net namespace
# Linux only # Linux only
namespace: "" namespace: ''
# listen address # listen address
listen: 0.0.0.0 listen: 0.0.0.0
# listen port, daemon will try to listen # listen port, daemon will try to listen
@ -248,8 +248,8 @@ proxy:
hijackHTTPS: hijackHTTPS:
# key pair used to hijack https requests # key pair used to hijack https requests
cert: "" cert: ''
key: "" key: ''
hosts: hosts:
- regx: mirror.aliyuncs.com:443 # regexp to match request hosts - regx: mirror.aliyuncs.com:443 # regexp to match request hosts
# whether to ignore https certificate errors # whether to ignore https certificate errors
@ -260,7 +260,7 @@ proxy:
maxConcurrency: 0 maxConcurrency: 0
whiteList: whiteList:
# the host of the whitelist # the host of the whitelist
- host: "" - host: ''
# match whitelist hosts # match whitelist hosts
regx: regx:
# port that need to be added to the whitelist # port that need to be added to the whitelist

View File

@ -70,4 +70,4 @@ verbose: false
pprof-port: -1 pprof-port: -1
# jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''

View File

@ -11,11 +11,11 @@ server:
# cacheDir is dynconfig cache storage directory # cacheDir is dynconfig cache storage directory
# in linux, default value is /var/cache/dragonfly # in linux, default value is /var/cache/dragonfly
# in macos(just for testing), default value is /Users/$USER/.dragonfly/cache # in macos(just for testing), default value is /Users/$USER/.dragonfly/cache
cacheDir: "" cacheDir: ''
# logDir is the log storage directory # logDir is the log storage directory
# in linux, default value is /var/log/dragonfly # in linux, default value is /var/log/dragonfly
# in macos(just for testing), default value is /Users/$USER/.dragonfly/logs # in macos(just for testing), default value is /Users/$USER/.dragonfly/logs
logDir: "" logDir: ''
# scheduler policy configuration # scheduler policy configuration
scheduler: scheduler:
@ -57,11 +57,11 @@ dynConfig:
# scheduler host configuration # scheduler host configuration
host: host:
# idc is the idc of scheduler instance # idc is the idc of scheduler instance
idc: "" idc: ''
# netTopology is the net topology of scheduler instance # netTopology is the net topology of scheduler instance
netTopology: "" netTopology: ''
# location is the location of scheduler instance # location is the location of scheduler instance
location: "" location: ''
# manager configuration # manager configuration
manager: manager:
@ -89,11 +89,11 @@ job:
# redis configuration # redis configuration
redis: redis:
# host # host
host: "" host: ''
# port # port
port: 6379 port: 6379
# password # password
password: "" password: ''
# brokerDB # brokerDB
brokerDB: 1 brokerDB: 1
# backendDB # backendDB
@ -104,7 +104,7 @@ metrics:
# scheduler enable metrics service # scheduler enable metrics service
enable: false enable: false
# metrics service address # metrics service address
addr: ":8000" addr: ':8000'
# enable peer host metrics # enable peer host metrics
enablePeerHost: false enablePeerHost: false
@ -119,4 +119,4 @@ verbose: false
pprof-port: -1 pprof-port: -1
# jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces # jaeger endpoint url, like: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''

View File

@ -2,6 +2,6 @@
Table of contents: Table of contents:
* [docker](docker/README.md) - [docker](docker/README.md)
* [executable-files](executable-files/README.md) - [executable-files](executable-files/README.md)
* [kubernetes](kubernetes/README.md) - [kubernetes](kubernetes/README.md)

View File

@ -6,5 +6,5 @@ or install separately according to modules.
Table of contents: Table of contents:
* [Docker Compose](docker-compose.md) - [Docker Compose](docker-compose.md)
* [Standalone](standalone/README.md) - [Standalone](standalone/README.md)

View File

@ -14,10 +14,10 @@ is the installation instructions of dragonfly.
## Install separately ## Install separately
* [install-client](./dfdaemon.md) is the installation instructions of dfdaemon - [install-client](./dfdaemon.md) is the installation instructions of dfdaemon
* [install-manager](./manager.md) is the installation instructions of manager - [install-manager](./manager.md) is the installation instructions of manager
* [install-scheduler](./scheduler.md) is the installation instructions of scheduler - [install-scheduler](./scheduler.md) is the installation instructions of scheduler
* [install-cdn](./cdn.md) is the installation instructions of CDN - [install-cdn](./cdn.md) is the installation instructions of CDN

View File

@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly CDN server.
When deploying with Docker, the following conditions must be met. When deploying with Docker, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## Procedure - When Deploying with Docker ## Procedure - When Deploying with Docker
@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly.
1. Obtain the latest Docker image of the cdn. 1. Obtain the latest Docker image of the cdn.
```sh ```sh
docker pull dragonflyoss/cdn docker pull dragonflyoss/cdn
``` ```
Or you can build your own cdn image. Or you can build your own cdn image.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Build the Docker image. 3. Build the Docker image.
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-cdn D7Y_VERSION=$TAG make docker-build-cdn D7Y_VERSION=$TAG
``` ```
4. Obtain the latest Docker image ID of the cdn. 4. Obtain the latest Docker image ID of the cdn.
```sh ```sh
docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 docker image ls | grep 'cdn' | awk '{print $3}' | head -n1
``` ```
### Start cdn ### Start cdn
@ -57,7 +57,7 @@ Or you can build your own cdn image.
```sh ```sh
docker run -d --name cdn --restart=always \ docker run -d --name cdn --restart=always \
-p 8001:8001 -p 8003:8003 \ -p 8001:8001 -p 8003:8003 \
-v /home/admin/ftp:/home/admin/ftp ${cdnDockerImageId} \ -v /home/admin/ftp:/home/admin/ftp ${cdnDockerImageId} \
--download-port=8001 --download-port=8001
``` ```

View File

@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly dfdaemon server.
When deploying with Docker, the following conditions must be met. When deploying with Docker, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## Procedure - When Deploying with Docker ## Procedure - When Deploying with Docker
@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly.
1. Obtain the latest Docker image of the dfdaemon. 1. Obtain the latest Docker image of the dfdaemon.
```sh ```sh
docker pull dragonflyoss/dfdaemon docker pull dragonflyoss/dfdaemon
``` ```
Or you can build your own dfdaemon image. Or you can build your own dfdaemon image.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Build the Docker image. 3. Build the Docker image.
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-dfdaemon D7Y_VERSION=$TAG make docker-build-dfdaemon D7Y_VERSION=$TAG
``` ```
4. Obtain the latest Docker image ID of the dfdaemon. 4. Obtain the latest Docker image ID of the dfdaemon.
```sh ```sh
docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1 docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1
``` ```
### Start dfdaemon ### Start dfdaemon

View File

@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly manager server.
When deploying with Docker, the following conditions must be met. When deploying with Docker, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## Procedure - When Deploying with Docker ## Procedure - When Deploying with Docker
@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly.
1. Obtain the latest Docker image of the manager. 1. Obtain the latest Docker image of the manager.
```sh ```sh
docker pull dragonflyoss/manager docker pull dragonflyoss/manager
``` ```
Or you can build your own manager image. Or you can build your own manager image.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Build the Docker image. 3. Build the Docker image.
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-manager D7Y_VERSION=$TAG make docker-build-manager D7Y_VERSION=$TAG
``` ```
4. Obtain the latest Docker image ID of the manager. 4. Obtain the latest Docker image ID of the manager.
```sh ```sh
docker image ls | grep 'manager' | awk '{print $3}' | head -n1 docker image ls | grep 'manager' | awk '{print $3}' | head -n1
``` ```
### Start manager ### Start manager

View File

@ -6,10 +6,10 @@ This topic explains how to install the Dragonfly scheduler server.
When deploying with Docker, the following conditions must be met. When deploying with Docker, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## Procedure - When Deploying with Docker ## Procedure - When Deploying with Docker
@ -19,36 +19,36 @@ You can get it from [DockerHub](https://hub.docker.com/) directly.
1. Obtain the latest Docker image of the scheduler. 1. Obtain the latest Docker image of the scheduler.
```sh ```sh
docker pull dragonflyoss/scheduler docker pull dragonflyoss/scheduler
``` ```
Or you can build your own scheduler image. Or you can build your own scheduler image.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Build the Docker image. 3. Build the Docker image.
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-scheduler D7Y_VERSION=$TAG make docker-build-scheduler D7Y_VERSION=$TAG
``` ```
4. Obtain the latest Docker image ID of the scheduler. 4. Obtain the latest Docker image ID of the scheduler.
```sh ```sh
docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1 docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1
``` ```
### Start scheduler ### Start scheduler

View File

@ -5,10 +5,10 @@ the executable files separately according to the modules.
## Install separately ## Install separately
* [install-client](dfdaemon.md) is the installation instructions of dfdaemon - [install-client](dfdaemon.md) is the installation instructions of dfdaemon
* [install-manager](manager.md) is the installation instructions of manager - [install-manager](manager.md) is the installation instructions of manager
* [install-scheduler](scheduler.md) is the installation instructions of scheduler - [install-scheduler](scheduler.md) is the installation instructions of scheduler
* [install-cdn](cdn.md) is the installation instructions of CDN - [install-cdn](cdn.md) is the installation instructions of CDN

View File

@ -6,54 +6,54 @@ This topic explains how to install the Dragonfly CDN server.
When deploying with physical machines, the following conditions must be met. When deploying with physical machines, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
Nginx|0.8+ | Nginx | 0.8+ |
## Procedure - When Deploying with Physical Machines ## Procedure - When Deploying with Physical Machines
### Get cdn executable file ### Get cdn executable file
1. Download a binary package of the cdn. You can download one of 1. Download a binary package of the cdn. You can download one of
the latest builds for Dragonfly on the the latest builds for Dragonfly on the
[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases).
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. Unzip the package. 2. Unzip the package.
```bash ```bash
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. Move the `cdn` to your `PATH` environment variable to 3. Move the `cdn` to your `PATH` environment variable to
make sure you can directly use `cdn` command. make sure you can directly use `cdn` command.
Or you can build your own cdn executable file. Or you can build your own cdn executable file.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Compile the source code. 3. Compile the source code.
```sh ```sh
make build-cdn && make install-cdn make build-cdn && make install-cdn
``` ```
### Start cdn ### Start cdn
@ -69,31 +69,31 @@ You can start a file server in any way.
However, the following conditions must be met: However, the following conditions must be met:
- It must be rooted at `cdnHomeDir/ftp` which is - It must be rooted at `cdnHomeDir/ftp` which is
defined in the previous step. defined in the previous step.
- It must listen on the port `cdnDownloadPort` which is - It must listen on the port `cdnDownloadPort` which is
defined in the previous step. defined in the previous step.
Let's take nginx as an example. Let's take nginx as an example.
1. Add the following configuration items to 1. Add the following configuration items to
the Nginx configuration file. the Nginx configuration file.
```conf ```conf
server { server {
# Must be ${cdnDownloadPort} # Must be ${cdnDownloadPort}
listen 8001; listen 8001;
location / { location / {
# Must be ${cdnHomeDir}/ftp # Must be ${cdnHomeDir}/ftp
root /home/admin/ftp; root /home/admin/ftp;
} }
} }
``` ```
2. Start Nginx. 2. Start Nginx.
```sh ```sh
sudo nginx sudo nginx
``` ```
After cdn is installed, run the following commands to After cdn is installed, run the following commands to
verify if Nginx and **cdn** are started, verify if Nginx and **cdn** are started,

View File

@ -6,53 +6,53 @@ This topic explains how to install the Dragonfly dfdaemon server.
When deploying with physical machines, the following conditions must be met. When deploying with physical machines, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
## Procedure - When Deploying with Physical Machines ## Procedure - When Deploying with Physical Machines
### Get dfget executable file ### Get dfget executable file
1. Download a binary package of the dfget. You can download one of 1. Download a binary package of the dfget. You can download one of
the latest builds for Dragonfly on the the latest builds for Dragonfly on the
[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases).
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. Unzip the package. 2. Unzip the package.
```bash ```bash
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. Move the `dfget` to your `PATH` environment variable to 3. Move the `dfget` to your `PATH` environment variable to
make sure you can directly use `dfget` command. make sure you can directly use `dfget` command.
Or you can build your own dfget executable file. Or you can build your own dfget executable file.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Compile the source code. 3. Compile the source code.
```sh ```sh
make build-dfget && make install-dfget make build-dfget && make install-dfget
``` ```
### Start dfdaemon ### Start dfdaemon

View File

@ -6,54 +6,54 @@ This topic explains how to install the Dragonfly manager server.
When deploying with physical machines, the following conditions must be met. When deploying with physical machines, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
Nginx|0.8+ | Nginx | 0.8+ |
## Procedure - When Deploying with Physical Machines ## Procedure - When Deploying with Physical Machines
### Get manager executable file ### Get manager executable file
1. Download a binary package of the manager. You can download 1. Download a binary package of the manager. You can download
one of the latest builds for Dragonfly on one of the latest builds for Dragonfly on
the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases).
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. Unzip the package. 2. Unzip the package.
```sh ```sh
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. Move the `manager` to your `PATH` environment variable 3. Move the `manager` to your `PATH` environment variable
to make sure you can directly use `manager` command. to make sure you can directly use `manager` command.
Or you can build your own manager executable file. Or you can build your own manager executable file.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Compile the source code. 3. Compile the source code.
```sh ```sh
make build-manager && make install-manager make build-manager && make install-manager
``` ```
### Start manager ### Start manager

View File

@ -6,54 +6,54 @@ This topic explains how to install the Dragonfly scheduler server.
When deploying with physical machines, the following conditions must be met. When deploying with physical machines, the following conditions must be met.
Required Software | Version Limit | Required Software | Version Limit |
---|--- | ----------------- | ------------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
Nginx|0.8+ | Nginx | 0.8+ |
## Procedure - When Deploying with Physical Machines ## Procedure - When Deploying with Physical Machines
### Get scheduler executable file ### Get scheduler executable file
1. Download a binary package of the scheduler. 1. Download a binary package of the scheduler.
You can download one of the latest builds for You can download one of the latest builds for
Dragonfly on the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases). Dragonfly on the [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases).
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. Unzip the package. 2. Unzip the package.
```bash ```bash
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. Move the `scheduler` to your `PATH` environment 3. Move the `scheduler` to your `PATH` environment
variable to make sure you can directly use `scheduler` command. variable to make sure you can directly use `scheduler` command.
Or you can build your own scheduler executable file. Or you can build your own scheduler executable file.
1. Obtain the source code of Dragonfly. 1. Obtain the source code of Dragonfly.
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. Enter the project directory. 2. Enter the project directory.
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. Compile the source code. 3. Compile the source code.
```sh ```sh
make build-scheduler && make install-scheduler make build-scheduler && make install-scheduler
``` ```
### Start scheduler ### Start scheduler

View File

@ -6,6 +6,6 @@ daemon as `DaemonSets`, manager as `Deployments`.
Table of contents: Table of contents:
* [Helm](helm.md) - [Helm](helm.md)
* [Kustomize](kustomize.md) - [Kustomize](kustomize.md)
* [TODO Upgrade Guide](upgrade-guide.md) - [TODO Upgrade Guide](upgrade-guide.md)

View File

@ -8,21 +8,21 @@ you can skip [Configure Runtime](#configure-runtime-manually) manually.
### 1. Docker ### 1. Docker
> **We did not recommend to using dragonfly > **We did not recommend to using dragonfly
with docker in Kubernetes** > with docker in Kubernetes**
due to many reasons: > due to many reasons:
no fallback image pulling policy. > no fallback image pulling policy.
deprecated in Kubernetes. > deprecated in Kubernetes.
> Because the original `daemonset` in > Because the original `daemonset` in
Kubernetes did not support `Surging Rolling Update` policy. > Kubernetes did not support `Surging Rolling Update` policy.
> When kill current dfdaemon pod, > When kill current dfdaemon pod,
the new pod image can not be pulled anymore. > the new pod image can not be pulled anymore.
> If you can not change runtime from docker to others, > If you can not change runtime from docker to others,
remind to choose a plan when upgrade dfdaemon: > remind to choose a plan when upgrade dfdaemon:
> pull newly dfdaemon image manually before upgrade dragonfly, > pull newly dfdaemon image manually before upgrade dragonfly,
or use [ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob) to > or use [ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob) to
pull image automate. > pull image automate.
> keep the image registry of dragonfly is > keep the image registry of dragonfly is
different from common registries and add host in `containerRuntime.docker.skipHosts`. > different from common registries and add host in `containerRuntime.docker.skipHosts`.
Dragonfly helm supports config docker automatically. Dragonfly helm supports config docker automatically.
@ -33,6 +33,7 @@ Config cases:
Chart customize values.yaml: Chart customize values.yaml:
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```yaml ```yaml
containerRuntime: containerRuntime:
docker: docker:
@ -42,9 +43,10 @@ containerRuntime:
# When use certs and inject hosts in docker, no necessary to restart docker daemon. # When use certs and inject hosts in docker, no necessary to restart docker daemon.
injectHosts: true injectHosts: true
registryDomains: registryDomains:
- "harbor.example.com" - "harbor.example.com"
- "harbor.example.net" - "harbor.example.net"
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
This config enables docker pulling images from This config enables docker pulling images from
@ -57,7 +59,7 @@ Advantages:
- Support upgrade dfdaemon smoothness - Support upgrade dfdaemon smoothness
> In this mode, when dfdaemon pod deleted, > In this mode, when dfdaemon pod deleted,
the `preStop` hook will remove all injected hosts info in /etc/hosts, > the `preStop` hook will remove all injected hosts info in /etc/hosts,
> all images traffic fallbacks to original registries. > all images traffic fallbacks to original registries.
Limitations: Limitations:
@ -69,6 +71,7 @@ Limitations:
Chart customize values.yaml: Chart customize values.yaml:
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```yaml ```yaml
containerRuntime: containerRuntime:
docker: docker:
@ -78,9 +81,10 @@ containerRuntime:
# If did not want restart docker daemon, keep containerRuntime.docker.restart=false and containerRuntime.docker.injectHosts=true. # If did not want restart docker daemon, keep containerRuntime.docker.restart=false and containerRuntime.docker.injectHosts=true.
restart: true restart: true
skipHosts: skipHosts:
- "127.0.0.1" - "127.0.0.1"
- "docker.io" # Dragonfly use this image registry to upgrade itself, so we need skip it. Change it in real environment. - "docker.io" # Dragonfly use this image registry to upgrade itself, so we need skip it. Change it in real environment.
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
This config enables docker pulling images from arbitrary registries via Dragonfly. This config enables docker pulling images from arbitrary registries via Dragonfly.
@ -124,30 +128,28 @@ containerRuntime:
- Option 1 - Allow charts to inject config_path and restart containerd. - Option 1 - Allow charts to inject config_path and restart containerd.
This option also enable multiple registry mirrors support. This option also enable multiple registry mirrors support.
> Caution: if there are already many other mirror config in config.toml, > Caution: if there are already many other mirror config in config.toml,
should not use this option, or migrate your config with `config_path`. > should not use this option, or migrate your config with `config_path`.
Chart customize values.yaml: Chart customize values.yaml:
```yaml ```yaml
containerRuntime: containerRuntime:
containerd: containerd:
enable: true enable: true
injectConfigPath: true injectConfigPath: true
``` ```
- Option 2 - Just mirror only one registry - Option 2 - Just mirror only one registry
which `dfdaemon.config.proxy.registryMirror.url` is which `dfdaemon.config.proxy.registryMirror.url` is Chart customize values.yaml:
Chart customize values.yaml: ```yaml
containerRuntime:
```yaml containerd:
containerRuntime: enable: true
containerd: ```
enable: true
```
#### Case 3: Version 1 #### Case 3: Version 1
@ -178,9 +180,9 @@ containerRuntime:
enable: true enable: true
# Registries full urls # Registries full urls
registries: registries:
- "https://ghcr.io" - "https://ghcr.io"
- "https://quay.io" - "https://quay.io"
- "https://harbor.example.com:8443" - "https://harbor.example.com:8443"
``` ```
## Prepare Kubernetes Cluster ## Prepare Kubernetes Cluster

View File

@ -8,8 +8,8 @@ all APIs in Dragonfly and some technical things else.
Table of contents: Table of contents:
* [architecture](architecture.md) - [architecture](architecture.md)
* [manager](manager.md) - [manager](manager.md)
* [TODO scheduler](scheduler.md) - [TODO scheduler](scheduler.md)
* [TODO cdn](cdn.md) - [TODO cdn](cdn.md)
* [TODO dfdaemon](dfdaemon.md) - [TODO dfdaemon](dfdaemon.md)

View File

@ -72,7 +72,7 @@ data market, multi-tenancy, and permission control, etc.
#### CDN system #### CDN system
![cdn][TODO cdn] ![cdn][todo cdn]
### Sub-system function ### Sub-system function

View File

@ -0,0 +1,8 @@
# Developer Guide
Understand how to develop dragonfly project and plugins.
Table of contents:
- [Development](development.md)
- [Plugin](plugin.md)

View File

@ -24,6 +24,7 @@ Attaching to cdn, scheduler, dfdaemon
Show dragonfly logs. Show dragonfly logs.
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```bash ```bash
$ tail -f log/**/*.log $ tail -f log/**/*.log
==> log/dragonfly/cdn/core.log <== ==> log/dragonfly/cdn/core.log <==
@ -49,6 +50,7 @@ $ tail -f log/**/*.log
{"level":"info","ts":"2021-02-26 05:43:37.342","caller":"worker/sender.go:49","msg":"start sender worker : 50"} {"level":"info","ts":"2021-02-26 05:43:37.342","caller":"worker/sender.go:49","msg":"start sender worker : 50"}
{"level":"info","ts":"2021-02-26 05:43:37.343","caller":"worker/worker_group.go:64","msg":"start scheduler worker number:6"} {"level":"info","ts":"2021-02-26 05:43:37.343","caller":"worker/worker_group.go:64","msg":"start scheduler worker number:6"}
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
## Step 4: Stop dragonfly ## Step 4: Stop dragonfly

View File

@ -12,22 +12,26 @@ The resource plugin is used to download custom resource like `dfget -u d7yfs://h
All resource plugins need to implement `d7y.io/dragonfly/v2/pkg/source.ResourceClient` All resource plugins need to implement `d7y.io/dragonfly/v2/pkg/source.ResourceClient`
and a function and a function
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
`func DragonflyPluginInit(option map[string]string) (interface{}, map[string]string, error)`. `func DragonflyPluginInit(option map[string]string) (interface{}, map[string]string, error)`.
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```golang ```golang
// ResourceClient defines the API interface to interact with source. // ResourceClient defines the API interface to interact with source.
type ResourceClient interface { type ResourceClient interface {
// GetContentLength get length of resource content // GetContentLength get length of resource content
// return source.UnknownSourceFileLen if response status is not StatusOK and StatusPartialContent // return source.UnknownSourceFileLen if response status is not StatusOK and StatusPartialContent
GetContentLength(request *Request) (int64, error) GetContentLength(request *Request) (int64, error)
// IsSupportRange checks if resource supports breakpoint continuation // IsSupportRange checks if resource supports breakpoint continuation
// return false if response status is not StatusPartialContent // return false if response status is not StatusPartialContent
IsSupportRange(request *Request) (bool, error) IsSupportRange(request *Request) (bool, error)
// IsExpired checks if a resource received or stored is the same. // IsExpired checks if a resource received or stored is the same.
// return false and non-nil err to prevent the source from exploding if // return false and non-nil err to prevent the source from exploding if
// fails to get the result, it is considered that the source has not expired // fails to get the result, it is considered that the source has not expired
@ -35,11 +39,12 @@ type ResourceClient interface {
// Download downloads from source // Download downloads from source
Download(request *Request) (*Response, error) Download(request *Request) (*Response, error)
// GetLastModified gets last modified timestamp milliseconds of resource // GetLastModified gets last modified timestamp milliseconds of resource
GetLastModified(request *Request) (int64, error) GetLastModified(request *Request) (int64, error)
} }
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
### Example Code ### Example Code
@ -47,6 +52,7 @@ type ResourceClient interface {
#### 1. main.go #### 1. main.go
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```golang ```golang
package main package main
@ -107,11 +113,13 @@ func DragonflyPluginInit(option map[string]string) (interface{}, map[string]stri
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
#### 2. go.mod #### 2. go.mod
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
``` ```
module example.com/d7yfs module example.com/d7yfs
@ -132,6 +140,7 @@ require (
// fix golang build error: `plugin was built with a different version of package d7y.io/dragonfly/v2/internal/dflog` // fix golang build error: `plugin was built with a different version of package d7y.io/dragonfly/v2/internal/dflog`
replace d7y.io/dragonfly/v2 => /Dragonfly2 replace d7y.io/dragonfly/v2 => /Dragonfly2
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
### Build ### Build
@ -141,6 +150,7 @@ replace d7y.io/dragonfly/v2 => /Dragonfly2
> Update `D7Y_COMMIT` in the following script. > Update `D7Y_COMMIT` in the following script.
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```shell ```shell
# golang plugin need cgo # golang plugin need cgo
# original Dragonfly2 image is built with CGO_ENABLED=0 for alpine linux # original Dragonfly2 image is built with CGO_ENABLED=0 for alpine linux
@ -159,6 +169,7 @@ go mod tidy -compat=1.17
go build -ldflags="-X main.buildTime=${BUILD_TIME} -X main.buildCommit=${BUILD_COMMIT}" \ go build -ldflags="-X main.buildTime=${BUILD_TIME} -X main.buildCommit=${BUILD_COMMIT}" \
-buildmode=plugin -o=/usr/local/dragonfly/plugins/d7y-resource-plugin-d7yfs.so ./main.go -buildmode=plugin -o=/usr/local/dragonfly/plugins/d7y-resource-plugin-d7yfs.so ./main.go
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
#### 2. Validate plugin #### 2. Validate plugin
@ -170,10 +181,12 @@ go build -ldflags="-X main.buildTime=${BUILD_TIME} -X main.buildCommit=${BUILD_C
Example output: Example output:
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```text ```text
search plugin in /usr/local/dragonfly/plugins search plugin in /usr/local/dragonfly/plugins
resource plugin d7yfs, location: d7y-resource-plugin-d7yfs.so, attribute: {"buildCommit":"bb65f13","buildTime":"2021-12-13T08:53:04Z","name":"d7yfs","schema":"d7yfs","type":"resource","vendor":"d7y"} resource plugin d7yfs, location: d7y-resource-plugin-d7yfs.so, attribute: {"buildCommit":"bb65f13","buildTime":"2021-12-13T08:53:04Z","name":"d7yfs","schema":"d7yfs","type":"resource","vendor":"d7y"}
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
## Searcher plugin ## Searcher plugin

View File

@ -5,5 +5,5 @@ Users can be preheat in the `console` or directly call `preheat api` for preheat
Table of contents: Table of contents:
* [Console](console.md) - [Console](console.md)
* [Api](api.md) - [Api](api.md)

View File

@ -11,14 +11,16 @@ We recommend to use `Containerd with CRI` and `CRI-O` client.
This table describes some container runtimes version and documents. This table describes some container runtimes version and documents.
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
| Runtime | Version | Document | CRI Support | Pull Command |
| --- | --- | --- | --- | --- | | Runtime | Version | Document | CRI Support | Pull Command |
| Containerd<sup>*</sup> | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest | | ----------------------- | -------- | ------------------------------------------------ | ----------- | ------------------------------------------- |
| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine | | Containerd<sup>\*</sup> | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest |
| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest | | Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine |
| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest |
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
**: `containerd` is recommended* `containerd` is recommended.
## Runtime Configuration Guide for Dragonfly Helm Chart ## Runtime Configuration Guide for Dragonfly Helm Chart
@ -40,8 +42,8 @@ containerRuntime:
# When use certs and inject hosts in docker, no necessary to restart docker daemon. # When use certs and inject hosts in docker, no necessary to restart docker daemon.
injectHosts: true injectHosts: true
registryDomains: registryDomains:
- "harbor.example.com" - 'harbor.example.com'
- "harbor.example.net" - 'harbor.example.net'
``` ```
This config enables docker pulling images from registries This config enables docker pulling images from registries
@ -50,7 +52,7 @@ When deploying Dragonfly with above config, it's unnecessary to restart docker d
Limitations: Limitations:
* Only support implicit registries - Only support implicit registries
## Prepare Kubernetes Cluster ## Prepare Kubernetes Cluster

View File

@ -2,6 +2,6 @@
Table of contents: Table of contents:
* [containerd](containerd/README.md) - [containerd](containerd/README.md)
* [cri-o](cri-o.md) - [cri-o](cri-o.md)
* [docker](docker.md) - [docker](docker.md)

View File

@ -5,5 +5,5 @@ Containerd. We recommend to use `mirror`.
Table of contents: Table of contents:
* [mirror](mirror.md) - [mirror](mirror.md)
* [proxy](proxy.md) - [proxy](proxy.md)

View File

@ -71,7 +71,7 @@ In this config, registry auth configuration needs to be based on `127.0.0.1:6500
> More details about Containerd configuration: <https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/registry.md#configure-registry-endpoint> > More details about Containerd configuration: <https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/registry.md#configure-registry-endpoint>
> Containerd has deprecated the above config from v1.4.0, > Containerd has deprecated the above config from v1.4.0,
new format for reference: <https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/config.md#registry-configuration> > new format for reference: <https://github.com/containerd/containerd/blob/v1.5.2/docs/cri/config.md#registry-configuration>
### Option 2: Multiple Registries ### Option 2: Multiple Registries

View File

@ -142,7 +142,7 @@ proxy:
key: ca.key key: ca.key
hosts: hosts:
- regx: your.private.registry - regx: your.private.registry
certs: ["server.crt"] certs: ['server.crt']
``` ```
You can get the certificate of your server with: You can get the certificate of your server with:

View File

@ -150,7 +150,7 @@ proxy:
key: ca.key key: ca.key
hosts: hosts:
- regx: your.private.registry - regx: your.private.registry
certs: ["server.crt"] certs: ['server.crt']
``` ```
You can get the certificate of your server with: You can get the certificate of your server with:

View File

@ -26,17 +26,17 @@ change log level to info
1. Confirm limit rate in [dfget.yaml](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/en/deployment/configuration/dfget.yaml#L65) 1. Confirm limit rate in [dfget.yaml](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/en/deployment/configuration/dfget.yaml#L65)
```yaml ```yaml
download: download:
# total download limit per second # total download limit per second
totalRateLimit: 200Mi totalRateLimit: 200Mi
# per peer task download limit per second # per peer task download limit per second
# default is 20Mi, this default is in consideration of extreme environments # default is 20Mi, this default is in consideration of extreme environments
perPeerRateLimit: 100Mi perPeerRateLimit: 100Mi
upload: upload:
# upload limit per second # upload limit per second
rateLimit: 100Mi rateLimit: 100Mi
``` ```
2. Confirm source connection speed in CDN and dfdaemon 2. Confirm source connection speed in CDN and dfdaemon

View File

@ -0,0 +1,7 @@
# User Guide
Understand how to use Dragonfly from installing the server and client to downloading files.
Table of contents:
- [Prometheus Metrics](metrics.md)

View File

@ -0,0 +1,67 @@
# Prometheus Metrics
This doc contains all the metrics that Dragonfly components currently support.
Now we support metrics for Dfdaemon, Manager, Scheduler and CDN.
The metrics path is fixed to /metrics. The following metrics are exported.
## Dfdaemon
GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
<!-- markdownlint-disable -->
| Name | Labels | Type | Description |
| :------------------------------------------------------- | :----- | :------ | :---------------------------------------------------- |
| dragonfly_dfdaemon_proxy_request_total | method | counter | Counter of the total proxy request. |
| dragonfly_dfdaemon_proxy_request_via_dragonfly_total | | counter | Counter of the total proxy request via Dragonfly. |
| dragonfly_dfdaemon_proxy_request_not_via_dragonfly_total | | counter | Counter of the total proxy request not via Dragonfly. |
| dragonfly_dfdaemon_proxy_request_running_total | method | counter | Current running count of proxy request. |
| dragonfly_dfdaemon_proxy_request_bytes_total | method | counter | Counter of the total byte of all proxy request. |
| dragonfly_dfdaemon_peer_task_total | | counter | Counter of the total peer tasks. |
| dragonfly_dfdaemon_peer_task_failed_total | | counter | Counter of the total failed peer tasks. |
| dragonfly_dfdaemon_piece_task_total | | counter | Counter of the total failed piece tasks. |
| dragonfly_dfdaemon_piece_task_failed_total | | counter | Dragonfly dfget tasks. |
| dragonfly_dfdaemon_file_task_total | | counter | Counter of the total file tasks. |
| dragonfly_dfdaemon_stream_task_total | | counter | Counter of the total stream tasks. |
| dragonfly_dfdaemon_peer_task_cache_hit_total | | counter | Counter of the total cache hit peer tasks. |
<!-- markdownlint-restore -->
## Manager
GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
## Scheduler
GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
<!-- markdownlint-disable -->
| Name | Labels | Type | Description |
| :----------------------------------------------------------- | :----------------------------------------- | :-------- | :--------------------------------------------------------- |
| dragonfly_scheduler_register_peer_task_total | | counter | Counter of the number of the register peer task. |
| dragonfly_scheduler_register_peer_task_failure_total | | counter | Counter of the number of failed of the register peer task. |
| dragonfly_scheduler_download_total | | counter | Counter of the number of the downloading. |
| dragonfly_scheduler_download_failure_total | | counter | Counter of the number of failed of the downloading. |
| dragonfly_scheduler_p2p_traffic | | counter | Counter of the number of p2p traffic. |
| dragonfly_scheduler_peer_host_traffic | traffic_type, peer_host_uuid, peer_host_ip | counter | Counter of the number of per peer host traffic. |
| dragonfly_scheduler_peer_task_total | type | counter | Counter of the number of peer task. |
| dragonfly_scheduler_peer_task_download_duration_milliseconds | | histogram | Histogram of the time each peer task downloading. |
| dragonfly_scheduler_concurrent_schedule_total | | gauge | Gauge of the number of concurrent of the scheduling. |
<!-- markdownlint-restore -->
## CDN
GRPC metrics are exposed via [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
<!-- markdownlint-disable -->
| Name | Labels | Type | Description |
| :-------------------------------------- | :----- | :------ | :----------------------------------------------------- |
| dragonfly_cdn_download_total | | counter | Counter of the number of the downloading. |
| dragonfly_cdn_download_failure_total | | counter | Counter of the number of failed of the downloading. |
| dragonfly_cdn_download_traffic | | counter | Counter of the number of download traffic. |
| dragonfly_cdn_concurrent_download_total | | gauge | Gauger of the number of concurrent of the downloading. |
<!-- markdownlint-restore -->

View File

@ -6,29 +6,33 @@ Dragonfly 文档介绍了 Dragonfly 各方面的信息。如果您对 Dragonfly
文档文件组织如下: 文档文件组织如下:
* [快速开始](quick-start.md) - [快速开始](quick-start.md)
* [部署](deployment/README.md) - [部署](deployment/README.md)
* [安装](deployment/installation) - [安装](deployment/installation)
* [配置](deployment/configuration) - [配置](deployment/configuration)
* [故障排查](troubleshooting/README.md) - [故障排查](troubleshooting/README.md)
* [CLI参考](cli-reference/README.md) - [CLI 参考](cli-reference/README.md)
* [dfget](cli-reference/dfget.md) - [dfget](cli-reference/dfget.md)
* [cdn](cli-reference/cdn.md) - [cdn](cli-reference/cdn.md)
* [scheduler](cli-reference/scheduler.md) - [scheduler](cli-reference/scheduler.md)
* [manager](cli-reference/manager.md) - [manager](cli-reference/manager.md)
* [预热](preheat/README.md) - [预热](preheat/README.md)
* [Console](preheat/console.md) - [Console](preheat/console.md)
* [Api](preheat/api.md) - [Api](preheat/api.md)
* [运行时集成](runtime-integration/README.md) - [运行时集成](runtime-integration/README.md)
* [containerd](runtime-integration/containerd/README.md) - [containerd](runtime-integration/containerd/README.md)
* [cri-o](runtime-integration/cri-o.md) - [cri-o](runtime-integration/cri-o.md)
* [docker](runtime-integration/docker.md) - [docker](runtime-integration/docker.md)
* [架构设计](design/README.md) - [架构设计](design/README.md)
* [整体架构](design/architecture.md) - [整体架构](design/architecture.md)
* [manager](design/manager.md) - [manager](design/manager.md)
* [TODO scheduler](design/scheduler.md) - [TODO scheduler](design/scheduler.md)
* [TODO cdn](design/cdn.md) - [TODO cdn](design/cdn.md)
* [TODO dfdaemon](design/dfdaemon.md) - [TODO dfdaemon](design/dfdaemon.md)
* [开发者指南](developer-guide/developer-guide.md) - [开发者指南](developer-guide/README.md)
* [测试指南](test-guide/test-guide.md) - [开发](developer-guide/developer.md)
* [API 参考](api-reference/api-reference.md) - [插件](developer-guide/plugin.md)
- [用户指南](user-guide/README.md)
- [Prometheus 数据指标](user-guide/metrics.md)
- [测试指南](test-guide/test-guide.md)
- [API 参考](api-reference/api-reference.md)

View File

@ -8,7 +8,7 @@
内容列表: 内容列表:
* [dfget](dfget.md) - [dfget](dfget.md)
* [cdn](cdn.md) - [cdn](cdn.md)
* [scheduler](scheduler.md) - [scheduler](scheduler.md)
* [manager](manager.md) - [manager](manager.md)

View File

@ -12,7 +12,7 @@ cdn [command]
## 可用子命令 ## 可用子命令
```text ```text
doc 生成帮助文档 doc 生成帮助文档
help 命令的帮助文档 help 命令的帮助文档
version 查看 cdn 当前版本 version 查看 cdn 当前版本
``` ```
@ -20,6 +20,7 @@ version 查看 cdn 当前版本
## 可选参数 ## 可选参数
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```text ```text
--config string 配置文件地址 (默认值为/Users/${USER_HOME}/.dragonfly/config/cdn.yaml),也可以使用环境变量 cdn_config 进行配置 --config string 配置文件地址 (默认值为/Users/${USER_HOME}/.dragonfly/config/cdn.yaml),也可以使用环境变量 cdn_config 进行配置
--console logger 日志是否输出记录到标准输出 --console logger 日志是否输出记录到标准输出
@ -29,4 +30,5 @@ version 查看 cdn 当前版本
--service-name string tracer分布式追踪日志中的服务名称 (默认 "dragonfly-scheduler") --service-name string tracer分布式追踪日志中的服务名称 (默认 "dragonfly-scheduler")
--verbose 是否使用debug级别的日志输出 --verbose 是否使用debug级别的日志输出
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->

View File

@ -2,7 +2,7 @@
## dfget ## dfget
`dfget` 是Dragonfly中用来下载和上传文件的客户端。 `dfget` Dragonfly 中用来下载和上传文件的客户端。
### dfget 简介 ### dfget 简介
@ -24,11 +24,12 @@ dfget --schedulers 127.0.0.1:8002 -o /path/to/output -u "http://example.com/obje
### 查看 dfget 日志 ### 查看 dfget 日志
如果您想要在Terminal中查看日志请使用 --console参数或者将环境变量 console 设置为 true。 如果您想要在 Terminal 中查看日志,请使用 --console 参数,或者将环境变量 console 设置为 true。
### dfget 的可选参数 ### dfget 的可选参数
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```text ```text
--alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s)
--cacerts strings the cacert file which is used to verify remote server when supernode interact with the source. --cacerts strings the cacert file which is used to verify remote server when supernode interact with the source.
@ -62,17 +63,19 @@ dfget --schedulers 127.0.0.1:8002 -o /path/to/output -u "http://example.com/obje
-u, --url string URL of user requested downloading file(only HTTP/HTTPs supported) -u, --url string URL of user requested downloading file(only HTTP/HTTPs supported)
--verbose enable verbose mode, all debug log will be display --verbose enable verbose mode, all debug log will be display
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
## dfget daemon ## dfget daemon
### 查看 daemon 日志 ### 查看 daemon 日志
如果您想要在Terminal中查看日志请使用 --console参数或者将环境变量 console 设置为 true。 如果您想要在 Terminal 中查看日志,请使用 --console 参数,或者将环境变量 console 设置为 true。
### daemon 的可选参数 ### daemon 的可选参数
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
```text ```text
--advertise-ip string the ip report to scheduler, normal same with listen ip (default "10.15.232.63") --advertise-ip string the ip report to scheduler, normal same with listen ip (default "10.15.232.63")
--alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s) --alivetime duration alive duration for which uploader keeps no accessing by any uploading requests, after this period uploader will automatically exit (default 5m0s)
@ -101,4 +104,5 @@ dfget --schedulers 127.0.0.1:8002 -o /path/to/output -u "http://example.com/obje
--upload-rate ratelimit upload rate limit for other peers (default 104857600.000000) --upload-rate ratelimit upload rate limit for other peers (default 104857600.000000)
--verbose print verbose log and enable golang debug info --verbose print verbose log and enable golang debug info
``` ```
<!-- markdownlint-restore --> <!-- markdownlint-restore -->

View File

@ -21,11 +21,11 @@ manager [Option]
启动时配置参数: --pprof-port port 启动时配置参数: --pprof-port port
``` ```
## Swagger支持 ## Swagger 支持
端点: /swagger/doc.json 端点: /swagger/doc.json
## Prometheus集成 ## Prometheus 集成
端点: /metrics 端点: /metrics
@ -33,7 +33,7 @@ manager [Option]
端点: /healthy/ 端点: /healthy/
## jaeger集成 ## jaeger 集成
```text ```text
启动时配置 --jaeger http://localhost:14250/api/traces 启动时配置 --jaeger http://localhost:14250/api/traces

View File

@ -1,7 +1,7 @@
# Scheduler # Scheduler
Scheduler 是一个常驻后台运行的进程,用于接收和管理客户端的下载任务, Scheduler 是一个常驻后台运行的进程,用于接收和管理客户端的下载任务,
通知CDN进行回源 在下载过程中生成维护P2P网络给客户端推送适合的下载节点 通知 CDN 进行回源, 在下载过程中生成维护 P2P 网络,给客户端推送适合的下载节点
## 用法 ## 用法
@ -13,7 +13,7 @@ scheduler [command]
## 可用子命令 ## 可用子命令
```text ```text
doc 生成帮助文档 doc 生成帮助文档
help 命令的帮助文档 help 命令的帮助文档
version 查看 scheduler 当前版本 version 查看 scheduler 当前版本
``` ```

View File

@ -2,8 +2,8 @@
内容: 内容:
* [安装](installation/README.md) - [安装](installation/README.md)
* [配置](configuration) - [配置](configuration)
## 安装 ## 安装

View File

@ -48,12 +48,12 @@ base:
# cdn 日志目录 # cdn 日志目录
# linux 上默认目录 /var/log/dragonfly # linux 上默认目录 /var/log/dragonfly
# macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs
logDir: "" logDir: ''
# CDN 连接的 manager可以不指定。 # CDN 连接的 manager可以不指定。
# 各项配置默认值如下。如果 addr 为空字符串CDN将不会连接manager。 # 各项配置默认值如下。如果 addr 为空字符串CDN将不会连接manager。
manager: manager:
addr: "" addr: ''
cdnClusterID: 0 cdnClusterID: 0
keepAlive: keepAlive:
interval: 5s interval: 5s
@ -61,9 +61,9 @@ base:
# 主机信息 # 主机信息
host: host:
# 地理位置 # 地理位置
location: "" location: ''
# IDC(Internet Data Center),互联网数据中心 # IDC(Internet Data Center),互联网数据中心
idc: "" idc: ''
# 开启数据收集服务 # 开启数据收集服务
# metrics: # metrics:
@ -123,7 +123,7 @@ pprof-port: -1
# jaeger 地址 # jaeger 地址
# 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces # 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''
# tracer 中使用的 service-name # tracer 中使用的 service-name
# 默认值dragonfly-cdn # 默认值dragonfly-cdn

View File

@ -8,22 +8,22 @@ gcInterval: 1m0s
# daemon 工作目录 # daemon 工作目录
# linux 上默认目录 /usr/local/dragonfly # linux 上默认目录 /usr/local/dragonfly
# macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly
workHome: "" workHome: ''
# daemon 动态配置缓存目录 # daemon 动态配置缓存目录
# linux 上默认目录 /var/cache/dragonfly # linux 上默认目录 /var/cache/dragonfly
# macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/cache # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/cache
cacheDir: "" cacheDir: ''
# daemon 日志目录 # daemon 日志目录
# linux 上默认目录 /var/log/dragonfly # linux 上默认目录 /var/log/dragonfly
# macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs
logDir: "" logDir: ''
# daemon 数据目录 # daemon 数据目录
# linux 上默认目录为 /var/lib/dragonfly # linux 上默认目录为 /var/lib/dragonfly
# macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/data/ # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/data/
dataDir: "" dataDir: ''
# 当 daemon 退出是, 是否保存缓存数据 # 当 daemon 退出是, 是否保存缓存数据
# 保留缓存数据在升级 daemon 的时候比较有用 # 保留缓存数据在升级 daemon 的时候比较有用
@ -41,7 +41,7 @@ pprof-port: -1
# jaeger 地址 # jaeger 地址
# 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces # 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''
# 调度器地址 # 调度器地址
# 尽量使用同一个地区的调度器. # 尽量使用同一个地区的调度器.
@ -73,13 +73,13 @@ host:
# 其他 daemon 可以通过这个 IP 地址连接过来 # 其他 daemon 可以通过这个 IP 地址连接过来
advertiseIP: 0.0.0.0 advertiseIP: 0.0.0.0
# 地理信息, 通过 "|" 符号分隔 # 地理信息, 通过 "|" 符号分隔
location: "" location: ''
# 机房信息 # 机房信息
idc: "" idc: ''
# 安全域信息,不同安全域之间网络隔离 # 安全域信息,不同安全域之间网络隔离
securityDomain: "" securityDomain: ''
# 网络拓扑结构,通过 "|" 符号分隔 # 网络拓扑结构,通过 "|" 符号分隔
netTopology: "" netTopology: ''
# 主机名称 # 主机名称
# hostname: "" # hostname: ""
@ -96,9 +96,9 @@ download:
# 安全选项 # 安全选项
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tlsConfig: null tlsConfig: null
# 下载服务监听地址dfget 下载文件将通过该地址连接到 daemon # 下载服务监听地址dfget 下载文件将通过该地址连接到 daemon
# 目前是支持 unix domain socket # 目前是支持 unix domain socket
@ -111,9 +111,9 @@ download:
peerGRPC: peerGRPC:
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tcpListen: tcpListen:
# 监听地址 # 监听地址
listen: 0.0.0.0 listen: 0.0.0.0
@ -130,9 +130,9 @@ upload:
rateLimit: 100Mi rateLimit: 100Mi
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tcpListen: tcpListen:
# 监听地址 # 监听地址
listen: 0.0.0.0 listen: 0.0.0.0
@ -174,17 +174,17 @@ proxy:
# 例如defaultFilter: "Expires&Signature": # 例如defaultFilter: "Expires&Signature":
# http://localhost/xyz?Expires=111&Signature=222 and http://localhost/xyz?Expires=333&Signature=999 # http://localhost/xyz?Expires=111&Signature=222 and http://localhost/xyz?Expires=333&Signature=999
# 是相同的 task # 是相同的 task
defaultFilter: "Expires&Signature" defaultFilter: 'Expires&Signature'
security: security:
insecure: true insecure: true
cacert: "" cacert: ''
cert: "" cert: ''
key: "" key: ''
tcpListen: tcpListen:
# 监听的网络命名空间, 例如:/proc/1/ns/net # 监听的网络命名空间, 例如:/proc/1/ns/net
# 主要用在部署 kubernetes 中的时候daemon 不使用 host network 时,监听宿主机的端口 # 主要用在部署 kubernetes 中的时候daemon 不使用 host network 时,监听宿主机的端口
# 仅支持 Linux # 仅支持 Linux
namespace: "" namespace: ''
# 监听地址 # 监听地址
listen: 0.0.0.0 listen: 0.0.0.0
# 监听端口 # 监听端口
@ -222,8 +222,8 @@ proxy:
hijackHTTPS: hijackHTTPS:
# https 劫持的证书和密钥 # https 劫持的证书和密钥
# 建议自签 CA 并更新主机证书链 # 建议自签 CA 并更新主机证书链
cert: "" cert: ''
key: "" key: ''
# 需要走蜻蜓 p2p 的流量 # 需要走蜻蜓 p2p 的流量
hosts: hosts:
- regx: mirror.aliyuncs.com:443 # 正则匹配 - regx: mirror.aliyuncs.com:443 # 正则匹配
@ -236,7 +236,7 @@ proxy:
# 白名单,如果设置了,仅白名单内可以走代理,其他的都拒绝 # 白名单,如果设置了,仅白名单内可以走代理,其他的都拒绝
whiteList: whiteList:
# 主机信息 # 主机信息
- host: "" - host: ''
# 正则匹配 # 正则匹配
regx: regx:
# 端口白名单 # 端口白名单

View File

@ -69,4 +69,4 @@ pprof-port: -1
# jaeger 地址 # jaeger 地址
# 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces # 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''

View File

@ -11,11 +11,11 @@ server:
# daemon 动态配置缓存目录 # daemon 动态配置缓存目录
# linux 上默认目录 /var/cache/dragonfly # linux 上默认目录 /var/cache/dragonfly
# macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/cache # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/cache
cacheDir: "" cacheDir: ''
# daemon 日志目录 # daemon 日志目录
# linux 上默认目录 /var/log/dragonfly # linux 上默认目录 /var/log/dragonfly
# macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs # macos(仅开发、测试), 默认目录是 /Users/$USER/.dragonfly/logs
logDir: "" logDir: ''
# scheduler 调度策略配置 # scheduler 调度策略配置
scheduler: scheduler:
@ -55,18 +55,18 @@ dynConfig:
# 实例主机信息 # 实例主机信息
host: host:
# 实例所在机房 # 实例所在机房
idc: "" idc: ''
# 实例网络拓扑信息 # 实例网络拓扑信息
netTopology: "" netTopology: ''
# 实例所在的地理位置信息 # 实例所在的地理位置信息
location: "" location: ''
# manager 配置 # manager 配置
manager: manager:
# 启动与 manager 的连接 # 启动与 manager 的连接
enable: true enable: true
# manager 访问地址 # manager 访问地址
addr: "" addr: ''
# 注册的 scheduler 集群 ID # 注册的 scheduler 集群 ID
schedulerClusterID: schedulerClusterID:
# manager 心跳配置 # manager 心跳配置
@ -87,11 +87,11 @@ job:
# redis 配置 # redis 配置
redis: redis:
# 服务地址 # 服务地址
host: "" host: ''
# 服务端口 # 服务端口
port: 6379 port: 6379
# 密码 # 密码
password: "" password: ''
# broker 数据库 # broker 数据库
brokerDB: 1 brokerDB: 1
# backend 数据库 # backend 数据库
@ -102,7 +102,7 @@ metrics:
# 启动数据收集服务 # 启动数据收集服务
enable: false enable: false
# 数据服务地址 # 数据服务地址
addr: ":8000" addr: ':8000'
# 开机收集 peer host 数据 # 开机收集 peer host 数据
enablePeerHost: false enablePeerHost: false
@ -117,4 +117,4 @@ pprof-port: -1
# jaeger 地址 # jaeger 地址
# 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces # 默认使用空字符串(不配置 jaeger, 例如: http://jaeger.dragonfly.svc:14268/api/traces
jaeger: "" jaeger: ''

View File

@ -2,6 +2,6 @@
内容: 内容:
* [docker](docker/README.md) - [docker](docker/README.md)
* [executable-files](executable-files/README.md) - [executable-files](executable-files/README.md)
* [kubernetes](kubernetes/README.md) - [kubernetes](kubernetes/README.md)

View File

@ -4,5 +4,5 @@
安装选项: 安装选项:
* [Docker Compose](docker-compose.md) - [Docker Compose](docker-compose.md)
* [Standalone](standalone/README.md) - [Standalone](standalone/README.md)

View File

@ -10,10 +10,10 @@
## 按模块单独安装 ## 按模块单独安装
* [install-dfdaemon](./dfdaemon.md) 是 dfdaemon 的安装指南 - [install-dfdaemon](./dfdaemon.md) 是 dfdaemon 的安装指南
* [install-manager](./manager.md) 是 manager 的安装指南 - [install-manager](./manager.md) 是 manager 的安装指南
* [install-scheduler](./scheduler.md) 是 scheduler 的安装指南 - [install-scheduler](./scheduler.md) 是 scheduler 的安装指南
* [install-cdn](./cdn.md) 是 CDN 的安装指南 - [install-cdn](./cdn.md) 是 CDN 的安装指南

View File

@ -6,10 +6,10 @@
使用 Docker 部署时,以下条件必须满足: 使用 Docker 部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## 使用 Docker 部署 ## 使用 Docker 部署
@ -19,36 +19,36 @@ Docker|1.12.0+
1. 获取最新的 CDN 镜像 1. 获取最新的 CDN 镜像
```sh ```sh
docker pull dragonflyoss/cdn docker pull dragonflyoss/cdn
``` ```
或者您可以构建自己的 CDN 镜像 或者您可以构建自己的 CDN 镜像
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 构建 CDN 的 Docker 镜像 3. 构建 CDN 的 Docker 镜像
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-cdn D7Y_VERSION=$TAG make docker-build-cdn D7Y_VERSION=$TAG
``` ```
4. 获取最新的 CDN 镜像 ID 4. 获取最新的 CDN 镜像 ID
```sh ```sh
docker image ls | grep 'cdn' | awk '{print $3}' | head -n1 docker image ls | grep 'cdn' | awk '{print $3}' | head -n1
``` ```
### 启动 cdn ### 启动 cdn

View File

@ -6,10 +6,10 @@
使用 Docker 部署时,以下条件必须满足: 使用 Docker 部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## 使用 Docker 部署 ## 使用 Docker 部署
@ -19,36 +19,36 @@ Docker|1.12.0+
1. 获取最新的 dfdaemon 镜像 1. 获取最新的 dfdaemon 镜像
```sh ```sh
docker pull dragonflyoss/dfdaemon docker pull dragonflyoss/dfdaemon
``` ```
或者您可以构建自己的 dfdaemon 镜像 或者您可以构建自己的 dfdaemon 镜像
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 构建 dfdaemon 的 Docker 镜像 3. 构建 dfdaemon 的 Docker 镜像
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-dfdaemon D7Y_VERSION=$TAG make docker-build-dfdaemon D7Y_VERSION=$TAG
``` ```
4. 获取最新的 dfdaemon 镜像 ID 4. 获取最新的 dfdaemon 镜像 ID
```sh ```sh
docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1 docker image ls | grep 'dfdaemon' | awk '{print $3}' | head -n1
``` ```
### 启动 dfdaemon ### 启动 dfdaemon

View File

@ -6,10 +6,10 @@
使用 Docker 部署时,以下条件必须满足: 使用 Docker 部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## 使用 Docker 部署 ## 使用 Docker 部署
@ -19,36 +19,36 @@ Docker|1.12.0+
1. 获取最新的 Manager 镜像 1. 获取最新的 Manager 镜像
```sh ```sh
docker pull dragonflyoss/manager docker pull dragonflyoss/manager
``` ```
或者您可以构建自己的 manager 镜像 或者您可以构建自己的 manager 镜像
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 构建 manager 的 Docker 镜像 3. 构建 manager 的 Docker 镜像
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-manager D7Y_VERSION=$TAG make docker-build-manager D7Y_VERSION=$TAG
``` ```
4. 获取最新的 manager 镜像 ID 4. 获取最新的 manager 镜像 ID
```sh ```sh
docker image ls | grep 'manager' | awk '{print $3}' | head -n1 docker image ls | grep 'manager' | awk '{print $3}' | head -n1
``` ```
### 启动 manager ### 启动 manager

View File

@ -6,10 +6,10 @@
使用 Docker 部署时,以下条件必须满足: 使用 Docker 部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Docker|1.12.0+ | Docker | 1.12.0+ |
## 使用 Docker 部署 ## 使用 Docker 部署
@ -19,36 +19,36 @@ Docker|1.12.0+
1. 获取最新的 scheduler 镜像 1. 获取最新的 scheduler 镜像
```sh ```sh
docker pull dragonflyoss/scheduler docker pull dragonflyoss/scheduler
``` ```
或者您可以构建自己的 scheduler 镜像 或者您可以构建自己的 scheduler 镜像
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 构建 scheduler 的 Docker 镜像 3. 构建 scheduler 的 Docker 镜像
```sh ```sh
TAG="2.0.0" TAG="2.0.0"
make docker-build-scheduler D7Y_VERSION=$TAG make docker-build-scheduler D7Y_VERSION=$TAG
``` ```
4. 获取最新的 scheduler 镜像 ID 4. 获取最新的 scheduler 镜像 ID
```sh ```sh
docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1 docker image ls | grep 'scheduler' | awk '{print $3}' | head -n1
``` ```
### 启动 scheduler ### 启动 scheduler

View File

@ -4,10 +4,10 @@
## 按模块单独安装 ## 按模块单独安装
* [install-dfdaemon](dfdaemon.md) 是 dfdaemon 的安装指南 - [install-dfdaemon](dfdaemon.md) 是 dfdaemon 的安装指南
* [install-manager](manager.md) 是 manager 的安装指南 - [install-manager](manager.md) 是 manager 的安装指南
* [install-scheduler](scheduler.md) 是 scheduler 的安装指南 - [install-scheduler](scheduler.md) 是 scheduler 的安装指南
* [install-cdn](cdn.md) 是 CDN 的安装指南 - [install-cdn](cdn.md) 是 CDN 的安装指南

View File

@ -6,31 +6,31 @@
直接在物理机上部署时,以下条件必须满足: 直接在物理机上部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
Nginx|0.8+ | Nginx | 0.8+ |
## 在物理机上部署 ## 在物理机上部署
### 获取 CDN 可执行文件 ### 获取 CDN 可执行文件
1. 下载 Dragonfly 项目的压缩包。您可以从 1. 下载 Dragonfly 项目的压缩包。您可以从
[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases)
下载一个已发布的最近版本 下载一个已发布的最近版本
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. 解压压缩包 2. 解压压缩包
```bash ```bash
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. 把 `cdn` 移动到环境变量 `PATH` 下以确保您可以直接使用 `cdn` 命令 3. 把 `cdn` 移动到环境变量 `PATH` 下以确保您可以直接使用 `cdn` 命令
@ -38,21 +38,21 @@ Nginx|0.8+
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 编译源码 3. 编译源码
```sh ```sh
make build-cdn && make install-cdn make build-cdn && make install-cdn
``` ```
### 启动 cdn ### 启动 cdn
@ -73,26 +73,26 @@ cdn --port=8003 --download-port=$cdnDownloadPort
1. 将下面的配置添加到 Nginx 配置文件中 1. 将下面的配置添加到 Nginx 配置文件中
```conf ```conf
server { server {
# Must be ${cdnDownloadPort} # Must be ${cdnDownloadPort}
listen 8001; listen 8001;
location / { location / {
# Must be ${cdnHomeDir}/ftp # Must be ${cdnHomeDir}/ftp
root /home/admin/ftp; root /home/admin/ftp;
} }
} }
``` ```
2. 启动 Nginx. 2. 启动 Nginx.
```sh ```sh
sudo nginx sudo nginx
``` ```
CDN 部署完成之后,运行以下命令以检查 Nginx 和 **cdn** 是否正在运行,以及 `8001``8003` 端口是否可用。 CDN 部署完成之后,运行以下命令以检查 Nginx 和 **cdn** 是否正在运行,以及 `8001``8003` 端口是否可用。
```sh ```sh
telnet 127.0.0.1 8001 telnet 127.0.0.1 8001
telnet 127.0.0.1 8003 telnet 127.0.0.1 8003
``` ```

View File

@ -6,30 +6,30 @@
直接在物理机上部署时,以下条件必须满足: 直接在物理机上部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
## 在物理机上部署 ## 在物理机上部署
### 获取 dfget 可执行文件 ### 获取 dfget 可执行文件
1. 下载 Dragonfly 项目的压缩包。您可以从 1. 下载 Dragonfly 项目的压缩包。您可以从
[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases)
下载一个已发布的最近版本 下载一个已发布的最近版本
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. 解压压缩包 2. 解压压缩包
```bash ```bash
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. 把 `dfget` 移动到环境变量 `PATH` 下以确保您可以直接使用 `dfget` 命令 3. 把 `dfget` 移动到环境变量 `PATH` 下以确保您可以直接使用 `dfget` 命令
@ -37,21 +37,21 @@ Golang|1.12.x
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 编译源码 3. 编译源码
```sh ```sh
make build-dfget && make install-dfget make build-dfget && make install-dfget
``` ```
### 启动 dfdaemon ### 启动 dfdaemon

View File

@ -6,31 +6,31 @@
直接在物理机上部署时,以下条件必须满足: 直接在物理机上部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
Nginx|0.8+ | Nginx | 0.8+ |
## 在物理机上部署 ## 在物理机上部署
### 获取 manager 可执行文件 ### 获取 manager 可执行文件
1. 下载 Dragonfly 项目的压缩包。您可以从 1. 下载 Dragonfly 项目的压缩包。您可以从
[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases)
下载一个已发布的最近版本 下载一个已发布的最近版本
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. 解压压缩包 2. 解压压缩包
```bash ```bash
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. 把 `manager` 移动到环境变量 `PATH` 下以确保您可以直接使用 `manager` 命令 3. 把 `manager` 移动到环境变量 `PATH` 下以确保您可以直接使用 `manager` 命令
@ -38,21 +38,21 @@ Nginx|0.8+
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 编译源码 3. 编译源码
```sh ```sh
make build-manager && make install-manager make build-manager && make install-manager
``` ```
### 启动 manager ### 启动 manager

View File

@ -6,31 +6,31 @@
直接在物理机上部署时,以下条件必须满足: 直接在物理机上部署时,以下条件必须满足:
所需软件 | 版本要求 | 所需软件 | 版本要求 |
---|--- | -------- | -------- |
Git|1.9.1+ | Git | 1.9.1+ |
Golang|1.12.x | Golang | 1.12.x |
Nginx|0.8+ | Nginx | 0.8+ |
## 在物理机上部署 ## 在物理机上部署
### 获取 scheduler 可执行文件 ### 获取 scheduler 可执行文件
1. 下载 Dragonfly 项目的压缩包。您可以从 1. 下载 Dragonfly 项目的压缩包。您可以从
[github releases page](https://github.com/dragonflyoss/Dragonfly2/releases) [github releases page](https://github.com/dragonflyoss/Dragonfly2/releases)
下载一个已发布的最近版本 下载一个已发布的最近版本
```sh ```sh
version=2.0.0 version=2.0.0
wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz wget https://github.com/dragonflyoss/Dragonfly2/releases/download/v$version/Dragonfly2_$version_linux_amd64.tar.gz
``` ```
2. 解压压缩包 2. 解压压缩包
```bash ```bash
# Replace `xxx` with the installation directory. # Replace `xxx` with the installation directory.
tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx tar -zxf Dragonfly2_2.0.0_linux_amd64.tar.gz -C xxx
``` ```
3. 把 `scheduler` 移动到环境变量 `PATH` 下以确保您可以直接使用 `scheduler` 命令 3. 把 `scheduler` 移动到环境变量 `PATH` 下以确保您可以直接使用 `scheduler` 命令
@ -38,21 +38,21 @@ Nginx|0.8+
1. 获取 Dragonfly 的源码 1. 获取 Dragonfly 的源码
```sh ```sh
git clone https://github.com/dragonflyoss/Dragonfly2.git git clone https://github.com/dragonflyoss/Dragonfly2.git
``` ```
2. 打开项目文件夹 2. 打开项目文件夹
```sh ```sh
cd Dragonfly2 cd Dragonfly2
``` ```
3. 编译源码 3. 编译源码
```sh ```sh
make build-scheduler && make install-scheduler make build-scheduler && make install-scheduler
``` ```
### 启动 scheduler ### 启动 scheduler

View File

@ -6,6 +6,6 @@ dfdaemon 会作为 `DaemonSets` 部署, manager 会作为 `Deployments` 部署
部署方式: 部署方式:
* [Helm](helm.md) - [Helm](helm.md)
* [Kustomize](kustomize.md) - [Kustomize](kustomize.md)
* [TODO Upgrade Guide](upgrade-guide.md) - [TODO Upgrade Guide](upgrade-guide.md)

View File

@ -11,10 +11,10 @@
> 因为当前 Kubernetes 里的 `daemonset` 并不支持 `Surging Rolling Update` 策略, > 因为当前 Kubernetes 里的 `daemonset` 并不支持 `Surging Rolling Update` 策略,
> 一旦旧的 dfdaemon pod 被删除后,新的 dfdaemon 就再也拉取不了了。 > 一旦旧的 dfdaemon pod 被删除后,新的 dfdaemon 就再也拉取不了了。
> 如果无法更换容器运行时的话,那在升级蜻蜓的时候,请从下面两种方案选择比较适合的: > 如果无法更换容器运行时的话,那在升级蜻蜓的时候,请从下面两种方案选择比较适合的:
> 选项1先手动拉取新的 dfdaemon 镜像,或者使用 > 选项 1先手动拉取新的 dfdaemon 镜像,或者使用
[ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob) > [ImagePullJob](https://openkruise.io/docs/user-manuals/imagepulljob)
去自动拉取。 > 去自动拉取。
> 选项2保持蜻蜓的镜像中心和通用的镜像中心不一样同时将蜻蜓镜像中心相关的 host 加入 `containerRuntime.docker.skipHosts` > 选项 2保持蜻蜓的镜像中心和通用的镜像中心不一样同时将蜻蜓镜像中心相关的 host 加入 `containerRuntime.docker.skipHosts`
Dragonfly Helm 支持自动更改 docker 配置。 Dragonfly Helm 支持自动更改 docker 配置。
@ -32,8 +32,8 @@ containerRuntime:
# When use certs and inject hosts in docker, no necessary to restart docker daemon. # When use certs and inject hosts in docker, no necessary to restart docker daemon.
injectHosts: true injectHosts: true
registryDomains: registryDomains:
- "harbor.example.com" - 'harbor.example.com'
- "harbor.example.net" - 'harbor.example.net'
``` ```
此配置允许 docker 通过 Dragonfly 拉取 `harbor.example.com``harbor.example.net` 域名镜像。 此配置允许 docker 通过 Dragonfly 拉取 `harbor.example.com``harbor.example.net` 域名镜像。
@ -41,13 +41,13 @@ containerRuntime:
优点: 优点:
* 支持 dfdaemon 自身平滑升级 - 支持 dfdaemon 自身平滑升级
> 这种模式下,当删除 dfdaemon pod 的时候,`preStop` 钩子将会清理已经注入到 `/etc/hosts` 下的所有主机信息,所有流量将会走原来的镜像中心。 > 这种模式下,当删除 dfdaemon pod 的时候,`preStop` 钩子将会清理已经注入到 `/etc/hosts` 下的所有主机信息,所有流量将会走原来的镜像中心。
限制: 限制:
* 只支持指定域名。 - 只支持指定域名。
#### 情况 2: 支持任意仓库 #### 情况 2: 支持任意仓库
@ -72,8 +72,8 @@ containerRuntime:
限制: 限制:
* 必须开启 docker 的 `live-restore` 功能 - 必须开启 docker 的 `live-restore` 功能
* 需要重启 docker daemon - 需要重启 docker daemon
### 2. Containerd ### 2. Containerd
@ -100,7 +100,7 @@ containerRuntime:
#### 情况 2: V2 版本不使用配置文件 #### 情况 2: V2 版本不使用配置文件
* 选项 1 - 允许 Charts 注入 `config_path` 并重新启动 containerd。 - 选项 1 - 允许 Charts 注入 `config_path` 并重新启动 containerd。
此选项启用多个镜像仓库支持。 此选项启用多个镜像仓库支持。
@ -115,7 +115,7 @@ containerRuntime:
injectConfigPath: true injectConfigPath: true
``` ```
* 选项 2 - 只使用一个镜像仓库 `dfdaemon.config.proxy.registryMirror.url` - 选项 2 - 只使用一个镜像仓库 `dfdaemon.config.proxy.registryMirror.url`
定制 values.yaml 文件: 定制 values.yaml 文件:
@ -153,9 +153,9 @@ containerRuntime:
enable: true enable: true
# Registries full urls # Registries full urls
registries: registries:
- "https://ghcr.io" - 'https://ghcr.io'
- "https://quay.io" - 'https://quay.io'
- "https://harbor.example.com:8443" - 'https://harbor.example.com:8443'
``` ```
## 准备 Kubernetes 集群 ## 准备 Kubernetes 集群
@ -230,7 +230,7 @@ manager:
externalManager: externalManager:
enable: true enable: true
host: "dragonfly-manager.dragonfly-system.svc.cluster.local" host: 'dragonfly-manager.dragonfly-system.svc.cluster.local'
restPort: 8080 restPort: 8080
grpcPort: 65003 grpcPort: 65003

View File

@ -5,8 +5,8 @@ Dragonfly 的所有组件设计的架构组件之间的交互工作流程D
具体设计: 具体设计:
* [architecture](architecture.md) - [architecture](architecture.md)
* [manager](manager.md) - [manager](manager.md)
* [TODO scheduler](scheduler.md) - [TODO scheduler](scheduler.md)
* [TODO cdn](cdn.md) - [TODO cdn](cdn.md)
* [TODO dfdaemon](dfdaemon.md) - [TODO dfdaemon](dfdaemon.md)

View File

@ -16,12 +16,12 @@
## 核心能力 ## 核心能力
- 通过统一的回源适配层及实现支持各种不同类型存储HDFS、各云厂商的存储服务、Maven、YUM等等的P2P文件分发能力 - 通过统一的回源适配层及实现支持各种不同类型存储HDFS、各云厂商的存储服务、Maven、YUM 等等)的 P2P 文件分发能力
- 支持更多的分发模式: 主动拉取、主动推送、实时同步、远程复制、自动预热、跨云传输等等 - 支持更多的分发模式: 主动拉取、主动推送、实时同步、远程复制、自动预热、跨云传输等等
- 系统之间分离解耦调度与CDN插件化且支持按需部署可轻可重可内可外灵活满足不同场景的实际需要 - 系统之间分离解耦,调度与 CDN 插件化,且支持按需部署,可轻可重,可内可外,灵活满足不同场景的实际需要
- 基于 grpc 全新设计的 P2P 协议框架,效率及稳定性更佳 - 基于 grpc 全新设计的 P2P 协议框架,效率及稳定性更佳
- 支持加密传输、基于账号的传输鉴权与限流、多租户隔离机制 - 支持加密传输、基于账号的传输鉴权与限流、多租户隔离机制
- 支持更高效的IO方式: 多线程IO、内存映射、DMA等 - 支持更高效的 IO 方式: 多线程 IO、内存映射、DMA
- 支持动态压缩、内存文件系统以及更高效的调度算法,提升分发效率 - 支持动态压缩、内存文件系统以及更高效的调度算法,提升分发效率
- 客户端通过 C/S 模式支持第三方软件原生集成蜻蜓的 P2P 能力 - 客户端通过 C/S 模式支持第三方软件原生集成蜻蜓的 P2P 能力
- 产品化能力: 支持文件上传、各种分发模式的任务管理、数据视图以及全局管控等功能 - 产品化能力: 支持文件上传、各种分发模式的任务管理、数据视图以及全局管控等功能
@ -44,7 +44,7 @@
### 子模块架构 ### 子模块架构
#### CDN模块 #### CDN 模块
![cdn][cdn] ![cdn][cdn]

View File

@ -0,0 +1,8 @@
# 开发者指南
理解如何开发 Dragonfly 项目以及相关插件。
内容:
- [开发](development.md)
- [插件](plugin.md)

View File

@ -1,64 +0,0 @@
# 开发
快速搭建本地开发环境
## 第一步:安装 docker 和 docker compose
详见文档 [docs.docker.com]
## 第二步:启动 dragonfly
打开 dragonfly 文档、启动 docker-compose.
```bash
$ docker-compose up
Creating network "dragonfly2_default" with the default driver
Creating cdn ... done
Creating scheduler ... done
Creating dfdaemon ... done
Attaching to cdn, scheduler, dfdaemon
```
## 第三步:日志分析
查看 dragonfly 日志
<!-- markdownlint-disable -->
```bash
$ tail -f log/**/*.log
==> log/dragonfly/cdn/core.log <==
{"level":"info","ts":"2021-02-26 05:43:36.896","caller":"cmd/root.go:90","msg":"success to init local ip of cdn, start to run cdn system, use ip: 172.21.0.2"}
{"level":"info","ts":"2021-02-26 05:43:36.901","caller":"plugins/plugins.go:51","msg":"add plugin[storage][local]"}
{"level":"info","ts":"2021-02-26 05:43:36.902","caller":"plugins/plugins.go:37","msg":"plugin[sourceclient][http] is disabled"}
==> log/dragonfly/dfdaemon/core.log <==
{"level":"info","ts":"2021-02-26 05:43:37.797","caller":"proxy/proxy_manager.go:63","msg":"registry mirror: https://index.docker.io"}
{"level":"info","ts":"2021-02-26 05:43:37.798","caller":"proxy/proxy_manager.go:68","msg":"load 1 proxy rules"}
{"level":"info","ts":"2021-02-26 05:43:37.799","caller":"proxy/proxy_manager.go:78","msg":"[1] proxy blobs/sha256.* with dragonfly "}
{"level":"debug","ts":"2021-02-26 05:43:37.799","caller":"rpc/server_listen.go:31","msg":"start to listen port: 0.0.0.0:65000"}
{"level":"debug","ts":"2021-02-26 05:43:37.800","caller":"rpc/server_listen.go:31","msg":"start to listen port: 0.0.0.0:65002"}
{"level":"debug","ts":"2021-02-26 05:43:37.800","caller":"rpc/server_listen.go:31","msg":"start to listen port: 0.0.0.0:65001"}
{"level":"info","ts":"2021-02-26 05:43:37.800","caller":"daemon/peerhost.go:274","msg":"serve download grpc at unix:///tmp/dfdamon.sock"}
{"level":"info","ts":"2021-02-26 05:43:37.801","caller":"daemon/peerhost.go:285","msg":"serve peer grpc at tcp://[::]:65000"}
{"level":"info","ts":"2021-02-26 05:43:37.801","caller":"daemon/peerhost.go:320","msg":"serve upload service at tcp://[::]:65002"}
{"level":"info","ts":"2021-02-26 05:43:37.801","caller":"daemon/peerhost.go:306","msg":"serve proxy at tcp://0.0.0.0:65001"}
==> log/dragonfly/scheduler/core.log <==
{"level":"info","ts":"2021-02-26 05:43:37.332","caller":"cmd/root.go:57","msg":"start to run scheduler"}
{"level":"info","ts":"2021-02-26 05:43:37.338","caller":"server/server.go:35","msg":"start server at port %!s(int=8002)"}
{"level":"info","ts":"2021-02-26 05:43:37.342","caller":"worker/sender.go:49","msg":"start sender worker : 50"}
{"level":"info","ts":"2021-02-26 05:43:37.343","caller":"worker/worker_group.go:64","msg":"start scheduler worker number:6"}
```
<!-- markdownlint-restore -->
## 第四步:关闭 dragonfly
```bash
$ docker-compose down
Removing dfdaemon ... done
Removing scheduler ... done
Removing cdn ... done
Removing network dragonfly2_default
```
[docs.docker.com]: https://docs.docker.com

View File

@ -0,0 +1,3 @@
# 开发
中文文档编写中,请先参考正在编写的[英文文档](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/en/developer-guide/development.md)

View File

@ -4,5 +4,5 @@ P2P 加速可预热两种类型数据 `image` 和 `file`, 用户可以在控制
预热方式: 预热方式:
* [Console](console.md) - [Console](console.md)
* [Api](api.md) - [Api](api.md)

View File

@ -9,14 +9,16 @@
下表列出了一些容器的运行时、版本和文档。 下表列出了一些容器的运行时、版本和文档。
<!-- markdownlint-disable --> <!-- markdownlint-disable -->
| Runtime | Version | Document | CRI Support | Pull Command |
| --- | --- | --- | --- | --- | | Runtime | Version | Document | CRI Support | Pull Command |
| Containerd<sup>*</sup> | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest | | ----------------------- | -------- | ------------------------------------------------ | ----------- | ------------------------------------------- |
| Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine | | Containerd<sup>\*</sup> | v1.1.0+ | [Link](runtime-integration/containerd/mirror.md) | Yes | crictl pull docker.io/library/alpine:latest |
| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest | | Containerd without CRI | < v1.1.0 | [Link](runtime-integration/containerd/proxy.md) | No | ctr image pull docker.io/library/alpine |
| CRI-O | All | [Link](runtime-integration/cri-o.md) | Yes | crictl pull docker.io/library/alpine:latest |
<!-- markdownlint-restore --> <!-- markdownlint-restore -->
**: 推荐使用`containerd`* 推荐使用 `containerd`.
## Helm Chart 运行时配置 ## Helm Chart 运行时配置
@ -34,8 +36,8 @@ containerRuntime:
# When use certs and inject hosts in docker, no necessary to restart docker daemon. # When use certs and inject hosts in docker, no necessary to restart docker daemon.
injectHosts: true injectHosts: true
registryDomains: registryDomains:
- "harbor.example.com" - 'harbor.example.com'
- "harbor.example.net" - 'harbor.example.net'
``` ```
此配置允许 docker 通过 Dragonfly 拉取 `harbor.example.com``harbor.example.net` 域名镜像。 此配置允许 docker 通过 Dragonfly 拉取 `harbor.example.com``harbor.example.net` 域名镜像。
@ -43,13 +45,13 @@ containerRuntime:
优点: 优点:
* 支持 dfdaemon 自身平滑升级 - 支持 dfdaemon 自身平滑升级
> 这种模式下,当删除 dfdaemon pod 的时候,`preStop` 钩子将会清理已经注入到 `/etc/hosts` 下的所有主机信息,所有流量将会走原来的镜像中心。 > 这种模式下,当删除 dfdaemon pod 的时候,`preStop` 钩子将会清理已经注入到 `/etc/hosts` 下的所有主机信息,所有流量将会走原来的镜像中心。
限制: 限制:
* 只支持指定域名。 - 只支持指定域名。
## 准备 Kubernetes 集群 ## 准备 Kubernetes 集群

View File

@ -2,6 +2,6 @@
运行时: 运行时:
* [containerd](containerd/README.md) - [containerd](containerd/README.md)
* [cri-o](cri-o.md) - [cri-o](cri-o.md)
* [docker](docker.md) - [docker](docker.md)

View File

@ -4,5 +4,5 @@
集成方式: 集成方式:
* [mirror](mirror.md) - [mirror](mirror.md)
* [proxy](proxy.md) - [proxy](proxy.md)

View File

@ -82,7 +82,7 @@ version = 2
#### 2. 生成每个仓库的 hosts.toml #### 2. 生成每个仓库的 hosts.toml
##### 选项 1: 手动生成 hosts.toml ##### 选项 1: 手动生成 hosts.toml
路径: `/etc/containerd/certs.d/example.com/hosts.toml` 路径: `/etc/containerd/certs.d/example.com/hosts.toml`

View File

@ -139,7 +139,7 @@ proxy:
key: ca.key key: ca.key
hosts: hosts:
- regx: your.private.registry - regx: your.private.registry
certs: ["server.crt"] certs: ['server.crt']
``` ```
您能使用以下命令获取您服务器的证书: 您能使用以下命令获取您服务器的证书:

View File

@ -146,7 +146,7 @@ proxy:
key: ca.key key: ca.key
hosts: hosts:
- regx: your.private.registry - regx: your.private.registry
certs: ["server.crt"] certs: ['server.crt']
``` ```
您能使用以下命令获取您服务器的证书: 您能使用以下命令获取您服务器的证书:

View File

@ -26,16 +26,16 @@ change log level to info
1. 确认限速值是否合适 [dfget.yaml](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/zh-CN/config/dfget.yaml#L61) 1. 确认限速值是否合适 [dfget.yaml](https://github.com/dragonflyoss/Dragonfly2/blob/main/docs/zh-CN/config/dfget.yaml#L61)
```yaml ```yaml
download: download:
# 总下载限速 # 总下载限速
totalRateLimit: 200Mi totalRateLimit: 200Mi
# 单个任务下载限速 # 单个任务下载限速
perPeerRateLimit: 100Mi # 为了兼容极限环境下,默认值为 20Mi可以按需调整 perPeerRateLimit: 100Mi # 为了兼容极限环境下,默认值为 20Mi可以按需调整
upload: upload:
# 上传限速 # 上传限速
rateLimit: 100Mi rateLimit: 100Mi
``` ```
2. 确认回源速度是否正常 2. 确认回源速度是否正常
@ -45,10 +45,10 @@ change log level to info
2. 检查源站可连接行(DNS 错误 or 证书) 2. 检查源站可连接行(DNS 错误 or 证书)
示例: 示例:
```shell ```shell
curl https://example.harbor.local/ curl https://example.harbor.local/
``` ```
如果`curl`有报错,请查看具体错误 如果`curl`有报错,请查看具体错误

View File

@ -0,0 +1,7 @@
# User Guide
让用户快速了解 Dragonfly 如何进行 P2P 下载以及其他使用相关内容。
内容:
- [Prometheus 数据指标](metrics.md)

View File

@ -0,0 +1,67 @@
# Prometheus 数据指标
文档包含 Dragonfly 组件当前支持的所有指标。
现在支持 Dfdaemon、Manager、Scheduler 和 CDN 服务的指标。
获取数据指标的路径固定为 /metrics。
## Dfdaemon
GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
<!-- markdownlint-disable -->
| 名字 | 标签 | 类型 | 描述 |
| :------------------------------------------------------- | :----- | :------ | :---------------------------------- |
| dragonfly_dfdaemon_proxy_request_total | method | counter | 代理请求总次数。 |
| dragonfly_dfdaemon_proxy_request_via_dragonfly_total | | counter | 代理通过 drgonfly 的请求次数。 |
| dragonfly_dfdaemon_proxy_request_not_via_dragonfly_total | | counter | 代理没有通过 dragonfly 的请求次数。 |
| dragonfly_dfdaemon_proxy_request_running_total | method | counter | 当前代理请求总次数。 |
| dragonfly_dfdaemon_proxy_request_bytes_total | method | counter | 所有代理请求的总字节数。 |
| dragonfly_dfdaemon_peer_task_total | | counter | 任务的总个数。 |
| dragonfly_dfdaemon_peer_task_failed_total | | counter | 失败任务的总个数。 |
| dragonfly_dfdaemon_piece_task_total | | counter | 分片的总个数。 |
| dragonfly_dfdaemon_piece_task_failed_total | | counter | 失败的分片总个数。 |
| dragonfly_dfdaemon_file_task_total | | counter | 文件类型任务总个数。 |
| dragonfly_dfdaemon_stream_task_total | | counter | 流式类型任务总个数。 |
| dragonfly_dfdaemon_peer_task_cache_hit_total | | counter | 命中缓存任务个数。 |
<!-- markdownlint-restore -->
## Manager
GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
## Scheduler
GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
<!-- markdownlint-disable -->
| 名字 | 标签 | 类型 | 描述 |
| :----------------------------------------------------------- | :----------------------------------------- | :-------- | :-------------------- |
| dragonfly_scheduler_register_peer_task_total | | counter | 注册任务总次数。 |
| dragonfly_scheduler_register_peer_task_failure_total | | counter | 注册任务失败次数。 |
| dragonfly_scheduler_download_total | | counter | 下载任务总次数。 |
| dragonfly_scheduler_download_failure_total | | counter | 下载任务失败次数。 |
| dragonfly_scheduler_p2p_traffic | | counter | P2P 流量。 |
| dragonfly_scheduler_peer_host_traffic | traffic_type, peer_host_uuid, peer_host_ip | counter | 每个主机的 P2P 流量。 |
| dragonfly_scheduler_peer_task_total | type | counter | 下载任务总个数。 |
| dragonfly_scheduler_peer_task_download_duration_milliseconds | | histogram | 任务下载耗时。 |
| dragonfly_scheduler_concurrent_schedule_total | | gauge | 并行调度任务个数。 |
<!-- markdownlint-restore -->
## CDN
GRPC 数据指标基于 [go-grpc-prometheus](https://github.com/grpc-ecosystem/go-grpc-prometheus).
<!-- markdownlint-disable -->
| 名字 | 标签 | 类型 | 描述 |
| :-------------------------------------- | :--- | :------ | :----------------- |
| dragonfly_cdn_download_total | | counter | 下载任务总次数。 |
| dragonfly_cdn_download_failure_total | | counter | 下载任务失败次数。 |
| dragonfly_cdn_download_traffic | | counter | 经过 CDN 流量。 |
| dragonfly_cdn_concurrent_download_total | | gauge | 并行下载任务个数。 |
<!-- markdownlint-restore -->