Commit Graph

71 Commits

Author SHA1 Message Date
Jim Ma 6f76c74838
chore: clarify daemon interface (#991)
1. Rename FilePeerTask to FileTask.
2. Rename StreamPeerTask to StreamTask.
3. Extract pulling piece task logic to pieceTaskPoller.

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:51 +08:00
Jim Ma c38bba2612
feat: merge same tasks in daemon (#977)
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:49 +08:00
Eng Zer Jun f579d3c9ce
refactor: move from io/ioutil to io and os packages (#906)
* chore: run `go fmt ./...`

This commit synchronizes `//go:build` lines with `// +build` lines.

Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-28 17:32:45 +08:00
sunwp 46105b0986
unify back source interface (#877)
* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* metaData Spelling correction

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs unit test

Signed-off-by: sunwp <244372610@qq.com>

* reset fqdn

Signed-off-by: sunwp <244372610@qq.com>

* new seed task with header

Signed-off-by: sunwp <244372610@qq.com>

* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* add unit tests & rename CheckRespCode -> CheckResponseCode & rename GetLastModifiedMillis -> GetLastModified

Signed-off-by: sunwp <244372610@qq.com>

* add request test

Signed-off-by: sunwp <244372610@qq.com>

* golang lint

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
Gaius f6ce817b72
feat: move dfnet to internal (#862)
* feat: move dfnet to internal

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename dfutils to util

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
sunwp 44730a2e46
unify piece size (#854)
* unify piece size

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:40 +08:00
sunwp c1d84fb56d
validate grpc model (#825)
* base validate

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:40 +08:00
sunwp d79a2a6877
move RPC code definition to proto file (#829)
* move rpc code to proto
* replace Cdn to CDN

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:38 +08:00
Gaius e837ae9bdf
chore: add lint errcheck and fix errcheck(#766)
* feat: add lint errcheck and fix errcheck

Signed-off-by: Gaius <gaius.qi@gmail.com>

* replace assert to require

Signed-off-by: 孙伟鹏 <weipeng.swp@alibaba-inc.com>

Co-authored-by: 孙伟鹏 <weipeng.swp@alibaba-inc.com>
2023-06-28 17:32:35 +08:00
Jim Ma 806f2ef11d
feat: calculate piece metadata digest (#787)
* feat: calculate piece metadata digest
* fix: back source not work in midway
* chore: add partial back source test
* chore: optimize digest log

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:35 +08:00
Gaius 466912aa48
docs: scheduler config (#698)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:25 +08:00
sunwp 359d11367e
chore: rename cdnsystem to cdn (#626)
Signed-off-by: santong <244372610@qq.com>
2023-06-28 17:31:46 +08:00
Jim Ma 94e35fa9df
chore: optimize compute piece size function (#528)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:34 +08:00
Jim Ma fdf2b688fa
feat: back source when no available peers or scheduler error (#521)
* feature: back source when no available peers

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:34 +08:00
Gaius 27103e4b4e
feat: rename manager grpc (#510)
* feat: rename manager grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:26 +08:00
Jim Ma 777e98784e
feature: export peer.TaskManager for embedding dragonfly in custom binary (#434)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:30:19 +08:00
sunwp 6cce5a9872
Feature/code spec (#336)
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:24:44 +08:00
Jim Ma 5b9c3033f8
Support peer task data multiplex (#347)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:23:35 +08:00
Jim Ma a777c8882c
Fix: use atomic to avoid data race in client (#254)
* fix: use atomic to avoid data race

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feature: temporary disable cdnsystem&manager test for CI

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feature: update .github/workflows/ci.yaml and codecov.yml

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feature: update ratelimiter test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feature: update dynconfig test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: ratelimiter and file utils test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: ratelimiter and queue test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: httputil test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feat: remove httputils

Signed-off-by: Gaius <gaius.qi@gmail.com>

Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:38 +08:00
Jim Ma 3955a7d716
daemon: update default timeout and add context for downloading piece (#190)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:21:33 +08:00
Jim Ma 0af8817c1f
Merge branch main-rc to branch main
Signed-off-by: Jim Ma <majinjing3@gmail.com>

Co-authored-by: 李玉海 <yuhai.lyh@alibaba-inc.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
Co-authored-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
Co-authored-by: 加菲 <garfieldhu@aliyun.com>
2023-06-28 17:20:20 +08:00