98e95e693c
feat: implement bidirectional fetch pieces ( #1165 )
...
* feat: implement grpc server side sync pieces
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:06 +08:00
ac4c4bb912
feat: reuse partial completed task ( #1107 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:00 +08:00
4597f74283
feat: merge ranged request storage into parent ( #1078 )
...
* feat: merge ranged request storage into parent
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:59 +08:00
6fc2bb4de2
chore: add content length for fast stream peer task ( #1061 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:56 +08:00
f80c75efdb
Feature: prefetch ranged requests ( #1053 )
...
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:56 +08:00
e9f824e0a7
chore: optimize defer and test ( #1010 )
...
* chore: optimize defer and test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: random test failed
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:53 +08:00
c00f07c2de
Extract storage driver into peer tasks ( #998 )
...
* chore: extract storage instead load every time
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: gofmt
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:52 +08:00
6f76c74838
chore: clarify daemon interface ( #991 )
...
1. Rename FilePeerTask to FileTask.
2. Rename StreamPeerTask to StreamTask.
3. Extract pulling piece task logic to pieceTaskPoller.
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:51 +08:00
c38bba2612
feat: merge same tasks in daemon ( #977 )
...
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:49 +08:00
38178804e8
feat: add daemon metrics support ( #960 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:48 +08:00
7ba341fc5c
fix: infinitely get pieces when piece num is invalid ( #926 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:47 +08:00
cb4202319e
feat: make idgen package public ( #931 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:47 +08:00
6134277c5c
chore: support multi daemons e2e test ( #896 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:44 +08:00
456e776bfd
correct metadata spell ( #884 )
...
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
c1d84fb56d
validate grpc model ( #825 )
...
* base validate
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:40 +08:00
d79a2a6877
move RPC code definition to proto file ( #829 )
...
* move rpc code to proto
* replace Cdn to CDN
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:38 +08:00
806f2ef11d
feat: calculate piece metadata digest ( #787 )
...
* feat: calculate piece metadata digest
* fix: back source not work in midway
* chore: add partial back source test
* chore: optimize digest log
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:35 +08:00
860af67773
chore: optimize stream peer task ( #763 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:33 +08:00
77bb686cbe
fix: client panic ( #719 )
...
* fix: client panic
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: update ginkgo version
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: avoid send failed piece hang
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:29 +08:00
6c73629016
fix: client goroutine and fd leak ( #713 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:29 +08:00
89fec9d5c0
feat: avoid report peer result fail due to context cancel & add backsource tracer ( #606 )
...
Signed-off-by: santong <244372610@qq.com>
2023-06-28 17:31:43 +08:00
60bee061d2
feat: client back source ( #579 )
...
* feat: client back source
Signed-off-by: santong <244372610@qq.com>
2023-06-28 17:31:43 +08:00
0ed79cb56c
Optimize backsource logic and report peer result ( #589 )
...
* chore: optimize backsource logic and report peer result
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: optimize peer task success logic
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: add back source trace
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:39 +08:00
7292fad950
feat: add time out when register ( #588 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:39 +08:00
9afb52b11e
fix: wrong content length in proxy
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:36 +08:00
dbf583524c
chore: optimize peer task report function ( #543 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:35 +08:00
1efd2f6cef
feat: Add distribute Schedule Tracer & Refactor scheduler ( #537 )
...
* check same cdn list avoid updating client too frequent
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:31:35 +08:00
2120051ab0
feature: enable grpc tracing ( #531 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:34 +08:00
fdf2b688fa
feat: back source when no available peers or scheduler error ( #521 )
...
* feature: back source when no available peers
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:34 +08:00
0df064202e
Fix: proxy for stress testing tool ( #507 )
...
* chore: fix proxy for stress testing tool
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:33 +08:00
a15ce877c7
fix: scheduler concurrent dead lock ( #509 )
...
* fix scheduler concurrent dead lock
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:31:33 +08:00
27103e4b4e
feat: rename manager grpc ( #510 )
...
* feat: rename manager grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:26 +08:00
c4d306a6c3
fix: dead lock when pt.failedPieceCh is full ( #466 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:30:28 +08:00
9ea9f7192e
feature: optimize failed reason not set ( #446 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:30:26 +08:00
777e98784e
feature: export peer.TaskManager for embedding dragonfly in custom binary ( #434 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:30:19 +08:00
5898f1d808
feature: optimize error message for proxy ( #428 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:28:03 +08:00
6cce5a9872
Feature/code spec ( #336 )
...
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:24:44 +08:00
5b9c3033f8
Support peer task data multiplex ( #347 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:23:35 +08:00
19129f53e0
Refactor the storage function module ( #289 )
...
* feat: refactor cdn storage module
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:22:42 +08:00
8198af07a1
fix: change peerPacketReady to buffer channel ( #256 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:22:38 +08:00
f9b4d77bb2
Fix golang lint ( #249 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:38 +08:00
7e595898e5
Fix lint ( #244 )
...
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: lint hostIP
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: error check
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: nil pointer
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename taskID
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename peerID
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: taskID
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: licenses
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:37 +08:00
f592032f72
Refactor cmd ( #217 )
...
* refator:cmd and path
Signed-off-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
2023-06-28 17:22:37 +08:00
48bb920806
daemon: add add additional peer id for some logs ( #205 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:21:33 +08:00
3955a7d716
daemon: update default timeout and add context for downloading piece ( #190 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:21:33 +08:00
9528f67101
daemon: record failed code when unfinished and event for scheduler ( #176 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:21:33 +08:00
0af8817c1f
Merge branch main-rc to branch main
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: 李玉海 <yuhai.lyh@alibaba-inc.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
Co-authored-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
Co-authored-by: 加菲 <garfieldhu@aliyun.com>
2023-06-28 17:20:20 +08:00
Jim Ma