dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
684 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

34 Commits

Author SHA1 Message Date
Jim Ma 98e95e693c
feat: implement bidirectional fetch pieces (#1165) 
* feat: implement grpc server side sync pieces

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:06 +08:00
Jim Ma ac4c4bb912
feat: reuse partial completed task (#1107) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:00 +08:00
Jim Ma 4597f74283
feat: merge ranged request storage into parent (#1078) 
* feat: merge ranged request storage into parent

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:59 +08:00
Jim Ma 5478414751
feat: add prefetch metric in client (#1068) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:58 +08:00
Jim Ma f80c75efdb
Feature: prefetch ranged requests (#1053) 
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:56 +08:00
Jim Ma 6e636b1125
chore: optimize metrics and trace in daemon (#1022) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:53 +08:00
Jim Ma a0049b17ae
chore: register to scheduler after updated running tasks (#1016) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:53 +08:00
Jim Ma e9f824e0a7
chore: optimize defer and test (#1010) 
* chore: optimize defer and test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: random test failed

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:53 +08:00
Jim Ma c00f07c2de
Extract storage driver into peer tasks (#998) 
* chore: extract storage instead load every time

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: gofmt

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
Jim Ma c3951e0cb3
feat: trace https proxy request (#996) 
* feat: trace https proxy request

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: daemon test

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma 6f76c74838
chore: clarify daemon interface (#991) 
1. Rename FilePeerTask to FileTask.
2. Rename StreamPeerTask to StreamTask.
3. Extract pulling piece task logic to pieceTaskPoller.

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma 17ff9c17be
fix: send piece result error not handled (#987) 
* fix: send piece result error not handled
* chore: optimize daemon cancel logic
* chore: remove redundant log and ctx.Done check
* chore: handle piece download with 404 status

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:50 +08:00
Jim Ma 37b332a366
test: dump goroutine in e2e (#980) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:50 +08:00
Jim Ma c38bba2612
feat: merge same tasks in daemon (#977) 
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:49 +08:00
Jim Ma 38178804e8
feat: add daemon metrics support (#960) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma 7ba341fc5c
fix: infinitely get pieces when piece num is invalid (#926) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:47 +08:00
Eng Zer Jun f579d3c9ce
refactor: move from io/ioutil to io and os packages (#906) 
* chore: run `go fmt ./...`

This commit synchronizes `//go:build` lines with `// +build` lines.

Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2023-06-28 17:32:45 +08:00
sunwp 456e776bfd
correct metadata spell (#884) 
Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:42 +08:00
sunwp c1d84fb56d
validate grpc model (#825) 
* base validate

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:40 +08:00
Jim Ma 806f2ef11d
feat: calculate piece metadata digest (#787) 
* feat: calculate piece metadata digest
* fix: back source not work in midway
* chore: add partial back source test
* chore: optimize digest log

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:35 +08:00
Jim Ma 6c73629016
fix: client goroutine and fd leak (#713) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:29 +08:00
sunwp 60bee061d2
feat: client back source (#579) 
* feat: client back source

Signed-off-by: santong <244372610@qq.com>
 2023-06-28 17:31:43 +08:00
Jim Ma af99b88616
feat: support limit from dfget client (#578) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:39 +08:00
Jim Ma dbf583524c
chore: optimize peer task report function (#543) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:35 +08:00
Jim Ma 2120051ab0
feature: enable grpc tracing (#531) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:34 +08:00
Jim Ma fdf2b688fa
feat: back source when no available peers or scheduler error (#521) 
* feature: back source when no available peers

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:34 +08:00
Jim Ma 0df064202e
Fix: proxy for stress testing tool (#507) 
* chore: fix proxy for stress testing tool

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:33 +08:00
Jim Ma 777e98784e
feature: export peer.TaskManager for embedding dragonfly in custom binary (#434) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:30:19 +08:00
Jim Ma f6694a87ec
Support reuse tiny peer task (#425) 
* feature: support reuse tiny peer task

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feature: reuse context when store tiny peer task

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:28:03 +08:00
sunwp 6cce5a9872
Feature/code spec (#336) 
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:24:44 +08:00
Jim Ma 5b9c3033f8
Support peer task data multiplex (#347) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:23:35 +08:00
Gaius f9b4d77bb2
Fix golang lint (#249) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:22:38 +08:00
Jim Ma 48bb920806
daemon: add add additional peer id for some logs (#205) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:21:33 +08:00
Jim Ma 0af8817c1f
Merge branch main-rc to branch main 
Signed-off-by: Jim Ma <majinjing3@gmail.com>

Co-authored-by: 李玉海 <yuhai.lyh@alibaba-inc.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
Co-authored-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
Co-authored-by: 加菲 <garfieldhu@aliyun.com>
 2023-06-28 17:20:20 +08:00