dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,907 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

63 Commits

Author SHA1 Message Date
Gaius b3f23f71a3
feat: update dy7.io/api to v2 (#2558) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-07-20 23:17:57 +08:00
Jim Ma 9137707139
feat: support breakpoint resume for running tasks (#2457) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:58 +08:00
Gaius 79024c87ca
refactor: parse http range (#2071) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 91454c8a60
feat: remove callsystem and pattern (#1925) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:17 +08:00
Jim Ma 7224801bae
feat: refactor peertask option (#1600) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:41 +08:00
Jim Ma cf7debf9c4
feat: back source error detail (#1437) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:34 +08:00
Gaius 3b426d2c54
feat: rename client/clientutil to client/util (#1420) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 64f5e9357b
refactor: scheduler announce task (#1407) 
* feat: scheduler announce normal task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cid

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:30 +08:00
Jim Ma 823c722c3e
feat: support response header (#1292) 
* feat: support response header via ExtendAttribute
* chore: update http pass through header
* chore: update ExtendAttribute in cdn downloader

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:19 +08:00
Jim Ma 429252a071
feat: implement pattern in client daemon (#1231) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:19 +08:00
Jim Ma 49f9ebde68
feat: implement client seed mode (#1247) 
* feat: implement client seed mode

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:18 +08:00
Jim Ma 60a1d9e53e
chore: print client stream task error log (#1210) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:10 +08:00
Jim Ma 9f4b86cb2f
fix: client break error (#1190) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:08 +08:00
Jim Ma 42046c0845
chore: optimize stream peer task (#1186) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:08 +08:00
Jim Ma 98e95e693c
feat: implement bidirectional fetch pieces (#1165) 
* feat: implement grpc server side sync pieces

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:06 +08:00
Jim Ma ac4c4bb912
feat: reuse partial completed task (#1107) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:00 +08:00
Jim Ma 4597f74283
feat: merge ranged request storage into parent (#1078) 
* feat: merge ranged request storage into parent

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:59 +08:00
Jim Ma 6fc2bb4de2
chore: add content length for fast stream peer task (#1061) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:56 +08:00
Jim Ma f80c75efdb
Feature: prefetch ranged requests (#1053) 
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:56 +08:00
Jim Ma e9f824e0a7
chore: optimize defer and test (#1010) 
* chore: optimize defer and test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: random test failed

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:53 +08:00
Jim Ma c00f07c2de
Extract storage driver into peer tasks (#998) 
* chore: extract storage instead load every time

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: gofmt

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
Jim Ma 6f76c74838
chore: clarify daemon interface (#991) 
1. Rename FilePeerTask to FileTask.
2. Rename StreamPeerTask to StreamTask.
3. Extract pulling piece task logic to pieceTaskPoller.

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma c38bba2612
feat: merge same tasks in daemon (#977) 
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:49 +08:00
Jim Ma 38178804e8
feat: add daemon metrics support (#960) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma 7ba341fc5c
fix: infinitely get pieces when piece num is invalid (#926) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:47 +08:00
Gaius cb4202319e
feat: make idgen package public (#931) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:47 +08:00
Jim Ma 6134277c5c
chore: support multi daemons e2e test (#896) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:44 +08:00
sunwp 456e776bfd
correct metadata spell (#884) 
Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:42 +08:00
sunwp c1d84fb56d
validate grpc model (#825) 
* base validate

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:40 +08:00
sunwp d79a2a6877
move RPC code definition to proto file (#829) 
* move rpc code to proto
* replace Cdn to CDN

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:38 +08:00
Jim Ma 806f2ef11d
feat: calculate piece metadata digest (#787) 
* feat: calculate piece metadata digest
* fix: back source not work in midway
* chore: add partial back source test
* chore: optimize digest log

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:35 +08:00
Jim Ma 860af67773
chore: optimize stream peer task (#763) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:33 +08:00
Jim Ma 77bb686cbe
fix: client panic (#719) 
* fix: client panic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: update ginkgo version

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: avoid send failed piece hang

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:29 +08:00
Jim Ma 6c73629016
fix: client goroutine and fd leak (#713) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:29 +08:00
sunwp 89fec9d5c0
feat: avoid report peer result fail due to context cancel & add backsource tracer (#606) 
Signed-off-by: santong <244372610@qq.com>
 2023-06-28 17:31:43 +08:00
sunwp 60bee061d2
feat: client back source (#579) 
* feat: client back source

Signed-off-by: santong <244372610@qq.com>
 2023-06-28 17:31:43 +08:00
Jim Ma 0ed79cb56c
Optimize backsource logic and report peer result (#589) 
* chore: optimize backsource logic and report peer result

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize peer task success logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: add back source trace

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:39 +08:00
Jim Ma 7292fad950
feat: add time out when register (#588) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:39 +08:00
Jim Ma 9afb52b11e
fix: wrong content length in proxy 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:36 +08:00
Jim Ma dbf583524c
chore: optimize peer task report function (#543) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:35 +08:00
sunwp 1efd2f6cef
feat: Add distribute Schedule Tracer & Refactor scheduler (#537) 
* check same cdn list avoid updating client too frequent

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:31:35 +08:00
Jim Ma 2120051ab0
feature: enable grpc tracing (#531) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:34 +08:00
Jim Ma fdf2b688fa
feat: back source when no available peers or scheduler error (#521) 
* feature: back source when no available peers

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:34 +08:00
Jim Ma 0df064202e
Fix: proxy for stress testing tool (#507) 
* chore: fix proxy for stress testing tool

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:33 +08:00
sunwp a15ce877c7
fix: scheduler concurrent dead lock (#509) 
* fix scheduler concurrent dead lock

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:31:33 +08:00
Gaius 27103e4b4e
feat: rename manager grpc (#510) 
* feat: rename manager grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:31:26 +08:00
Jim Ma c4d306a6c3
fix: dead lock when pt.failedPieceCh is full (#466) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:30:28 +08:00
Jim Ma 9ea9f7192e
feature: optimize failed reason not set (#446) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:30:26 +08:00