Commit Graph

2448 Commits

Author SHA1 Message Date
Gaius 94fde2608c
refactor: scheduler register task (#924)
* refactor: scheduler register task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* refactor: scheduler register task

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:46 +08:00
sunwp 0babd6e519
CDN config (#920)
* CDN config

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
Jim Ma 705c5ed342
fix: plugin dir is empty (#922)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:46 +08:00
Jim Ma e24b1621e4
docs: update plugin docs (#921)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:46 +08:00
Gaius 475d86bca1
fix: peer gc (#918)
* fix: peer gc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* Check whether the peer is in zombie state

Signed-off-by: sunwp <244372610@qq.com>

Co-authored-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
sunwp e252ef5880
Adjust CDN module package (#893)
* adjust CDN code package structure

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
Gaius 40a281a752
feat: dfdaemon list scheduler cluster with multi idc (#917)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius 0751333215
feat: update submodule (#916)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
sunwp afdf91976c
feat: update task access time (#909)
* update task access time when task is downloading

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:45 +08:00
Jim Ma 717b997e6e
feat: optmize dfget package upgrade support (#804)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:45 +08:00
Jim Ma 6b2c4eceee
feat: support create container without docker-compose (#915)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:45 +08:00
Eng Zer Jun f579d3c9ce
refactor: move from io/ioutil to io and os packages (#906)
* chore: run `go fmt ./...`

This commit synchronizes `//go:build` lines with `// +build` lines.

Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-28 17:32:45 +08:00
Jim Ma 10ec7d2bbc
fix: go plugin test build error (#912)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius a3659820f0
test: idgen peer id (#913)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius b5799ff97b
fix: typo (#911)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius 9800a211da
feat: add data directory (#910)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
Jim Ma 029c4eea36
fix: total pieces not set when back source (#908)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:44 +08:00
Gaius 75c7be0769
feat: add data storage directory (#907)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
Gaius 8f903ccf24
docs: dir path (#904)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
sunwp eaca95d24e
Load source plugins (#894)
* unit test

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:44 +08:00
Jim Ma ca2f196aff
fix: mismatch digest peer task did not mark invalid (#903)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:44 +08:00
sunwp 0096c613b0
Fix hdfs test (#902)
* unit test

Signed-off-by: sunwp <244372610@qq.com>

* fix hdfs range test

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:44 +08:00
Jim Ma 6134277c5c
chore: support multi daemons e2e test (#896)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:44 +08:00
Gaius e006aaece0
fix: dfget dfpath (#901)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
sunwp 23747be9d3
unit test (#898)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:44 +08:00
Gaius bcae898ef4
feat: dfdaemon update content length (#895)
* feat: dfdaemon update content length

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: error check

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
Gaius c2c721bc78
refactor: dfpath pkg (#879)
* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: path

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: plugin

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
sunwp 929fe7db2d
Fix load source plugins (#892)
* load source plugins

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:43 +08:00
Gaius 04a3550925
fix: scheduler success event (#891)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
sunwp dc50bd31d5
reslove scheduler obtain seeds cancal result in CDN task download cancel (#890)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:43 +08:00
Jim Ma ff3a542fde
chore: update UnknownSourceFileLen (#888)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:43 +08:00
Gaius 18e7822b51
fix: add cdn cluster to scheduler cluster (#887)
* feat: add cdn cluster to scheduler cluster

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update console submodule

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
Jim Ma 5b016d92b4
fix: small size task failed due to digest error (#886)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:43 +08:00
sunwp 6866ecc3fe
task manager (#885)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:43 +08:00
sunwp 456e776bfd
correct metadata spell (#884)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
sunwp 46105b0986
unify back source interface (#877)
* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* metaData Spelling correction

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs unit test

Signed-off-by: sunwp <244372610@qq.com>

* reset fqdn

Signed-off-by: sunwp <244372610@qq.com>

* new seed task with header

Signed-off-by: sunwp <244372610@qq.com>

* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* add unit tests & rename CheckRespCode -> CheckResponseCode & rename GetLastModifiedMillis -> GetLastModified

Signed-off-by: sunwp <244372610@qq.com>

* add request test

Signed-off-by: sunwp <244372610@qq.com>

* golang lint

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
sunwp b0078162ca
Fixed range GRPC check (#872)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
Gaius afe5bdb63a
fix: searcher log (#878)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00
Gaius 4b511f5d87
feat: lint sh (#876)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00
Jim Ma a376f3171f
docs: add plugin guide (#875)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:42 +08:00
Gaius c6c693179f
chore: update changelog
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00
Gaius 6465ec37e8
feat: update helm charts (#870)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00
Gaius 80eb7f284f
feat: update version to v2.0.1 (#869)
* feat: update version v2.0.0

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00
Gaius 3086147df6
docs: keep alive (#868)
* docs: keep alive

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add transport log

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
Gaius 5b5c4ed8fd
feat: add oauth timeout (#867)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
Jim Ma 540cd233a0
feat: support customize transport in daemon (#866)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:41 +08:00
Gaius 3ee7bd791f
feat: console (#865)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
Gaius f6ce817b72
feat: move dfnet to internal (#862)
* feat: move dfnet to internal

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename dfutils to util

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
Gaius 8b72144e1e
fix: error log (#863)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
Jim Ma b82e0fa944
chore: upgrade to golang 1.17 and alpine 3.14 (#861)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:41 +08:00