Commit Graph

94 Commits

Author SHA1 Message Date
Jim Ma 47c6cdd8cc
chore: optimize recursive download log (#2785)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-10 14:43:45 +08:00
guangwu a3f2aae26c
chore: remove refs to deprecated io/ioutil (#2754)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-09-26 10:09:57 +08:00
Gaius 170a7f2186
feat: change file mode when export task from local in dfcache (#2701)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-09-05 19:59:11 +08:00
Gaius 5b7ea5da2a
Revert "feat: replace fmt.Sprintf with net.JoinHostPort" (#2649)
Revert "feat: replace fmt.Sprintf with net.JoinHostPort (#2642)"

This reverts commit a6863f9b19.
2023-08-17 17:21:50 +08:00
Gaius a6863f9b19
feat: replace fmt.Sprintf with net.JoinHostPort (#2642)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-16 16:45:14 +08:00
Jim Ma 0eaf429c4e
chore: skip export to exist file (#2637)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:05:17 +08:00
Gaius 05a66805cc
fix: directories created via os.MkdirAll are not checked for permissions (#2613)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-09 13:52:26 +08:00
Jim Ma a64140ff72
fix: potential nil panic (#2602)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:55:03 +08:00
Gaius b3f23f71a3
feat: update dy7.io/api to v2 (#2558)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-20 23:17:57 +08:00
Jim Ma afb47f6729
chore: check grpc peer info for download service (#2385)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 047a55fb84
feat: add handleRegisterSeedPeerRequest to AnnouncePeer in service v2 (#2147)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:36 +08:00
Jim Ma f5aff5e27f
feat: enhance daemon health check (#2130)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:35 +08:00
Gaius 79024c87ca
refactor: parse http range (#2071)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:30 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:29 +08:00
Gaius 7dc3c826f2
feat: add v2 version of dfdaemon client (#2050)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:29 +08:00
Gaius 1666a17827
feat: remove NetTopology from scheduler and manager (#2007)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:25 +08:00
Gaius c89234e976
feat: add manager v2 api (#1990)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:23 +08:00
Jim Ma 56e668bb0d
chore: optimize download log (#1944)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:19 +08:00
Jim Ma f402824618
fix: download context cancelled (#1942)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:19 +08:00
Gaius 731ecc1234
feat: change yaml field type from string to net.IP (#1929)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:18 +08:00
Gaius 91454c8a60
feat: remove callsystem and pattern (#1925)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:17 +08:00
Jim Ma 3fe342dabc
fix: recursive download always return none error (#1841)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:10 +08:00
Jim Ma 1c24c3fc8c
fix: expire header timezone (#1840)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:10 +08:00
Jim Ma ba58047871
feat: optional save list metadata to p2p (#1822)
* feat: save list metadata to p2p
* chore: optimize rpc download worker logic
* feat: add expire header reclaim logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:09 +08:00
Gaius 5c25a325c6
feat: seed peer reuses traffic (#1825)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
Jim Ma 8749845aae
chore: make SendMsg in doRecursiveDownload safe (#1806)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:07 +08:00
Jim Ma 0e83c7f58f
feat: add otel trace in log (#1804)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:07 +08:00
Jim Ma 2a7e576778
chore: add list log in rpc download (#1802)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:07 +08:00
Jim Ma 876c2ae17e
feat: support split running tasks (#1794)
* feat: support split running tasks

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: go tests

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: add split runnging tasks e2e config

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:06 +08:00
Jim Ma 55d5308b19
fix: open end range in concurrent back source (#1764)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:03 +08:00
Jim Ma 491b309620
feat: support concurrent recursive download (#1714)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:59 +08:00
Gaius 9f098362fb
refactor: pkg basic (#1712)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:58 +08:00
Gaius 7cc842d64d
test: remove test main (#1710)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:58 +08:00
zhaoshang fda6002aac
test: add test for daemon rpcserver (#1704)
Signed-off-by: zhaoshang <zhaoshangsjtu@linux.alibaba.com>
2023-06-28 17:33:58 +08:00
Jim Ma c5f4ca50fa
fix: grpc download tidy file error (#1697)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:57 +08:00
Jim Ma 73de99353d
feat: empty file e2e (#1687)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:56 +08:00
Jim Ma f4d34e56ed
chore: update download rpc check (#1684)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:56 +08:00
Jim Ma 7bceca51bc
chore: update debug info (#1617)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:51 +08:00
Gaius 43d8478507
feat: add tls policy to scheduler grpc server (#1616)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Jim Ma 98fb1fc427
feat: add grpc mux transport (#1602)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:49 +08:00
Jim Ma 717ac99ef5
feat: daemon support auto issue certificate (#1586)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Gaius ad553d949b
feat: grpc dial adds context (#1594)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:47 +08:00
Gaius f631263d25
refactor: dfnet package (#1578)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius a6a44269b4
refactor: dfdaemon client and remove rpc connection pool (#1576)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 272f38f567
feat: grpc add ratelimit (#1572)
feat: grpc ratelimit

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 5bcb53f0a9
feat: grpc consistent hashing (#1554)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: bigerous <bigerous@qq.com>
2023-06-28 17:33:44 +08:00
Jim Ma f6506115a7
feat: support grpc recursive download (#1518)
* feat: daemon grpc recursive download
* fix: init storage error

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:42 +08:00
Jim Ma 4984cdfbbc
fix: check same peer id for sync pieces (#1525)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:42 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:41 +08:00
Gaius f9d2736ae8
feat: scheduler merge end of piece and piece from seed peer (#1474)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:37 +08:00