Commit Graph

1907 Commits

Author SHA1 Message Date
sunwp a1ab42790f
Optimaze GC log (#923)
* optimaze gc log

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:47 +08:00
Jim Ma ce2503f6c5
fix: not handle base.Code_SchedTaskStatusError in client (#938)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:47 +08:00
yojay11717 b2848a982c
fix typos for test/readme (#933)
Signed-off-by: yojay11717 <lanyujie@inspur.com>
2023-06-28 17:32:47 +08:00
Gaius e8556d8fc4
feat: add kmutex and krwmutex (#934)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:47 +08:00
sunwp 7965ed70fe
Set totalPieceCount when the file length is available (#930)
* set totalPieceCount when the file length is available

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:47 +08:00
Jim Ma 7ba341fc5c
fix: infinitely get pieces when piece num is invalid (#926)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:47 +08:00
Gaius cb4202319e
feat: make idgen package public (#931)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:47 +08:00
Gaius 1cde964259
feat: make dfpath public (#929)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:46 +08:00
sunwp 39787c5bf1
fix Multiple test downloads failed (#928)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
Gaius 94fde2608c
refactor: scheduler register task (#924)
* refactor: scheduler register task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* refactor: scheduler register task

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:46 +08:00
sunwp 0babd6e519
CDN config (#920)
* CDN config

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
Jim Ma 705c5ed342
fix: plugin dir is empty (#922)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:46 +08:00
Jim Ma e24b1621e4
docs: update plugin docs (#921)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:46 +08:00
Gaius 475d86bca1
fix: peer gc (#918)
* fix: peer gc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* Check whether the peer is in zombie state

Signed-off-by: sunwp <244372610@qq.com>

Co-authored-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
sunwp e252ef5880
Adjust CDN module package (#893)
* adjust CDN code package structure

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
Gaius 40a281a752
feat: dfdaemon list scheduler cluster with multi idc (#917)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius 0751333215
feat: update submodule (#916)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
sunwp afdf91976c
feat: update task access time (#909)
* update task access time when task is downloading

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:45 +08:00
Jim Ma 717b997e6e
feat: optmize dfget package upgrade support (#804)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:45 +08:00
Jim Ma 6b2c4eceee
feat: support create container without docker-compose (#915)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:45 +08:00
Eng Zer Jun f579d3c9ce
refactor: move from io/ioutil to io and os packages (#906)
* chore: run `go fmt ./...`

This commit synchronizes `//go:build` lines with `// +build` lines.

Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-28 17:32:45 +08:00
Jim Ma 10ec7d2bbc
fix: go plugin test build error (#912)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius a3659820f0
test: idgen peer id (#913)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius b5799ff97b
fix: typo (#911)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius 9800a211da
feat: add data directory (#910)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
Jim Ma 029c4eea36
fix: total pieces not set when back source (#908)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:44 +08:00
Gaius 75c7be0769
feat: add data storage directory (#907)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
Gaius 8f903ccf24
docs: dir path (#904)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
sunwp eaca95d24e
Load source plugins (#894)
* unit test

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:44 +08:00
Jim Ma ca2f196aff
fix: mismatch digest peer task did not mark invalid (#903)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:44 +08:00
sunwp 0096c613b0
Fix hdfs test (#902)
* unit test

Signed-off-by: sunwp <244372610@qq.com>

* fix hdfs range test

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:44 +08:00
Jim Ma 6134277c5c
chore: support multi daemons e2e test (#896)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:44 +08:00
Gaius e006aaece0
fix: dfget dfpath (#901)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
sunwp 23747be9d3
unit test (#898)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:44 +08:00
Gaius bcae898ef4
feat: dfdaemon update content length (#895)
* feat: dfdaemon update content length

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: error check

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
Gaius c2c721bc78
refactor: dfpath pkg (#879)
* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: path

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: plugin

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
sunwp 929fe7db2d
Fix load source plugins (#892)
* load source plugins

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:43 +08:00
Gaius 04a3550925
fix: scheduler success event (#891)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
sunwp dc50bd31d5
reslove scheduler obtain seeds cancal result in CDN task download cancel (#890)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:43 +08:00
Jim Ma ff3a542fde
chore: update UnknownSourceFileLen (#888)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:43 +08:00
Gaius 18e7822b51
fix: add cdn cluster to scheduler cluster (#887)
* feat: add cdn cluster to scheduler cluster

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update console submodule

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
Jim Ma 5b016d92b4
fix: small size task failed due to digest error (#886)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:43 +08:00
sunwp 6866ecc3fe
task manager (#885)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:43 +08:00
sunwp 456e776bfd
correct metadata spell (#884)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
sunwp 46105b0986
unify back source interface (#877)
* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* metaData Spelling correction

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs unit test

Signed-off-by: sunwp <244372610@qq.com>

* reset fqdn

Signed-off-by: sunwp <244372610@qq.com>

* new seed task with header

Signed-off-by: sunwp <244372610@qq.com>

* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* add unit tests & rename CheckRespCode -> CheckResponseCode & rename GetLastModifiedMillis -> GetLastModified

Signed-off-by: sunwp <244372610@qq.com>

* add request test

Signed-off-by: sunwp <244372610@qq.com>

* golang lint

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
sunwp b0078162ca
Fixed range GRPC check (#872)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
Gaius afe5bdb63a
fix: searcher log (#878)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00
Gaius 4b511f5d87
feat: lint sh (#876)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00
Jim Ma a376f3171f
docs: add plugin guide (#875)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:42 +08:00
Gaius c6c693179f
chore: update changelog
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:42 +08:00