dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
2,186 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

71 Commits

Author SHA1 Message Date
Jim Ma 8c9ff386b1
fix: tiny piece digest reader (#3164) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2024-04-02 14:23:12 +08:00
Jim Ma 09b11a280a
chore: optimize stream task id (#2983) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-12-29 16:16:06 +08:00
Jim Ma 839e476ecb
chore: optimize sync pieces error report (#2925) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-12-14 11:23:15 +08:00
Gaius 67c36f62b3
feat: remove concurrent_piece_count in scheduler (#2942) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-12-12 16:39:16 +08:00
Gaius 12fc630612
fix: if scheduler has no seed peer, return error in preheating (#2835) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-10-30 16:45:39 +08:00
Gaius 05a66805cc
fix: directories created via os.MkdirAll are not checked for permissions (#2613) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-08-09 13:52:26 +08:00
Jim Ma 1b9d2b03d7
fix: improper use strings.TrimLeft (#2603) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-08-07 17:55:27 +08:00
Gaius b3f23f71a3
feat: update dy7.io/api to v2 (#2558) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-07-20 23:17:57 +08:00
embroede 8c905d9b8d
feat: enable configuration of some directory modes for dfdaemon (#2340) 
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:57 +08:00
Jim Ma f5aff5e27f
feat: enhance daemon health check (#2130) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:35 +08:00
Gaius 79024c87ca
refactor: parse http range (#2071) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 1666a17827
feat: remove NetTopology from scheduler and manager (#2007) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:25 +08:00
Gaius e83fb85216
feat: add v2 verison of the grpc to scheduler (#1999) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:24 +08:00
Jim Ma 7cb802aeb7
feat: remove legacy peers support (#1939) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Gaius 91454c8a60
feat: remove callsystem and pattern (#1925) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:17 +08:00
Jim Ma 0a8f5118c9
feat: support reregister peer task in client (#1876) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:13 +08:00
Jim Ma 55d5308b19
fix: open end range in concurrent back source (#1764) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:03 +08:00
greenhandatsjtu bc8d623b3d
feat: add traffic shaper for download tasks (#1654) 
Signed-off-by: greenhandatsjtu <sunhengke@sjtu.edu.cn>
 2023-06-28 17:33:59 +08:00
Jim Ma c5f4ca50fa
fix: grpc download tidy file error (#1697) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:57 +08:00
Jim Ma 40300990de
feat: support download empty file (#1686) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:56 +08:00
Jim Ma 7224801bae
feat: refactor peertask option (#1600) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma ecfe467585
feat: add client grpc dial timeout (#1599) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Gaius 272f38f567
feat: grpc add ratelimit (#1572) 
feat: grpc ratelimit

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:41 +08:00
Gaius ed4c381383
feat: rename steal peers to candidate peers (#1476) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:37 +08:00
Gaius 3b426d2c54
feat: rename client/clientutil to client/util (#1420) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius a2511cb945
feat: rewrite interface{} to any (#1419) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 0222649b88
feat: update namely/protoc-all image version to 1.47_0 (#1418) 
* feat: update namely/protoc-all image version to 1.47_0

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate mock file

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 15d3274224
feat: rename digest func and add new digest func (#1405) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius 7a30db3fba
refactor: pkg util (#1402) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius 5763ee86bd
feat: add seed peer metrics (#1342) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:23 +08:00
Gaius 2a0314e45b
refactor: scheduler grpc (#1310) 
* refactor: scheduler grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate protoc

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:20 +08:00
Jim Ma 823c722c3e
feat: support response header (#1292) 
* feat: support response header via ExtendAttribute
* chore: update http pass through header
* chore: update ExtendAttribute in cdn downloader

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:19 +08:00
Jim Ma 49f9ebde68
feat: implement client seed mode (#1247) 
* feat: implement client seed mode

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:18 +08:00
Jim Ma 3682b26647
feat: in tree plugin (#1276) 
* feat: in tree plugin

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:16 +08:00
Gaius 076ba20b51
feat: move dfnet to pkg dir (#1265) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:15 +08:00
Eryu Guan 65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache (#1227) 
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.

For example:
- add a file into cache system
  dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file

- check if a file exists in cache system
  dfcache stat --cid testid --local  # only check local cache
  dfcache stat --cid testid          # check other peers as well

- export/download a file from cache system
  dfcache export --cid testid -O /path/to/output

- delete a file from cache system, both local cache and P2P network
  dfcache delete -i testid -t testtag

Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius 3fdde9b891
fix: make generate (#1228) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:11 +08:00
Jim Ma 10da5fa0d6
feat: implement grpc client side sync pieces (#1167) 
* feat: implement grpc client side sync pieces

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: client unit test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: report sync piece error

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize pieceTaskSyncManager cancel

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:07 +08:00
Jim Ma 4597f74283
feat: merge ranged request storage into parent (#1078) 
* feat: merge ranged request storage into parent

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:59 +08:00
Jim Ma f80c75efdb
Feature: prefetch ranged requests (#1053) 
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:56 +08:00
Jim Ma a0049b17ae
chore: register to scheduler after updated running tasks (#1016) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:53 +08:00
Jim Ma e9f824e0a7
chore: optimize defer and test (#1010) 
* chore: optimize defer and test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: random test failed

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:53 +08:00
Jim Ma c00f07c2de
Extract storage driver into peer tasks (#998) 
* chore: extract storage instead load every time

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: gofmt

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
Gaius 43d769b0b4
fix: register task with size scope (#1003) 
* feat: register task with size scope

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: fix peer manager data race

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:52 +08:00
Jim Ma 0c03af1979
fix: when scheduler is not available, replace the scheduler client (#999) 
* fix: when scheduler is not available, just replace the scheduler client in single peer task

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma c3951e0cb3
feat: trace https proxy request (#996) 
* feat: trace https proxy request

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: daemon test

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma c87be98799
chore: parameterize tests in peer task (#994) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma 88278be464
fix: total pieces count not set cause digest invalid (#992) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00