Commit Graph

84 Commits

Author SHA1 Message Date
Jim Ma ba58047871
feat: optional save list metadata to p2p (#1822)
* feat: save list metadata to p2p
* chore: optimize rpc download worker logic
* feat: add expire header reclaim logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:09 +08:00
Jim Ma 876c2ae17e
feat: support split running tasks (#1794)
* feat: support split running tasks

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: go tests

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: add split runnging tasks e2e config

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:06 +08:00
cuidajun 502c01f9aa
fix: daemon don't leaveHost when keepStorage=true (#1790)
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
2023-06-28 17:34:06 +08:00
cuidajun 536b039462
feat: daemon call leaveHost when exit (#1788)
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>

Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
2023-06-28 17:34:06 +08:00
Jim Ma 491b309620
feat: support concurrent recursive download (#1714)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:59 +08:00
Gaius 9573d5a659
fix: docker compose config (#1713)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:59 +08:00
greenhandatsjtu bc8d623b3d
feat: add traffic shaper for download tasks (#1654)
Signed-off-by: greenhandatsjtu <sunhengke@sjtu.edu.cn>
2023-06-28 17:33:59 +08:00
Gaius c9755aabfc
feat: stop grpc client (#1671)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:55 +08:00
Gaius b6c395a6cb
feat: add cert spec to security configuration (#1621)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:51 +08:00
Gaius 13580f89f2
refactor: dfpath for certify cache dir (#1618)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Gaius 43d8478507
feat: add tls policy to scheduler grpc server (#1616)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Jim Ma 9050230926
feat: refactor grpc credential (#1612)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Gaius 346cc0649e
feat: add tls policy constants (#1610)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:49 +08:00
Jim Ma 98fb1fc427
feat: add grpc mux transport (#1602)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:49 +08:00
Gaius 9ab33635c5
feat: manager init cert for grpc server (#1603)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:49 +08:00
Jim Ma 7224801bae
feat: refactor peertask option (#1600)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Jim Ma ecfe467585
feat: add client grpc dial timeout (#1599)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Jim Ma c25cfdd651
feat: support multiple certify cache (#1598)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Jim Ma 717ac99ef5
feat: daemon support auto issue certificate (#1586)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Gaius ad553d949b
feat: grpc dial adds context (#1594)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:47 +08:00
Gaius 2c48148ac8
fix: dfdaemon can not shutdown (#1580)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius a6a44269b4
refactor: dfdaemon client and remove rpc connection pool (#1576)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 272f38f567
feat: grpc add ratelimit (#1572)
feat: grpc ratelimit

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 69a51808d2
feat: refresh dynconfig addresses when grpc requests unavailable (#1571)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 631654f98b
feat: manager client add context (#1562)
Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:45 +08:00
Gaius 5bcb53f0a9
feat: grpc consistent hashing (#1554)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: bigerous <bigerous@qq.com>
2023-06-28 17:33:44 +08:00
Gaius 2ce864fbf6
fix: scheduler and manager tracing (#1551)
fix: scheduler and manager trace

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:44 +08:00
Jim Ma baa64bb0a3
feat: add seed trace (#1549)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:44 +08:00
cuidajun 1258175366
fix daemon socket remove error (#1529) 2023-06-28 17:33:42 +08:00
Jim Ma 4984cdfbbc
fix: check same peer id for sync pieces (#1525)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:42 +08:00
Gaius e42186acb7
feat: add daemon-socket-path (#1521)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:42 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:41 +08:00
Gaius 0bd894adad
fix: dfdaemon unix socket (#1489)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:38 +08:00
Gaius 7d3c9d539d
refactor: cache key for peer (#1483)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:38 +08:00
Jim Ma 9f360c1b03
feat: reload proxy option (#1443)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:35 +08:00
Jim Ma 776a0fef42
feat: concurrent multiple pieces back source (#1426)
* feat: concurrent multiple pieces back source
* chore: update http source client

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:33 +08:00
Gaius 3b426d2c54
feat: rename client/clientutil to client/util (#1420)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
Gaius ad36eb64a7
feat: remove github/pkg/errors package (#1416)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:31 +08:00
Gaius 30313a4caa
feat: client add create object storage (#1409)
* feat: add create object storage rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
Gaius 64f5e9357b
refactor: scheduler announce task (#1407)
* feat: scheduler announce normal task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cid

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
Gaius ad827170a4
feat: dfdaemon upload and object storage service add middlewares (#1404)
* feat: dfdaemon upload and object storage service add middlewares

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:29 +08:00
Gaius 230966cf8f
feat: dfdaemon add GetObject rest api (#1398)
* feat: dfdaemon add GetObject rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:28 +08:00
Gaius 171c0d54ee
feat: dfdaemon add object storage rest api (#1390)
* feat: dfdaemon add object storage rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:28 +08:00
Gaius 4889a37886
feat: replace gin-gonic/gin with gorilla/mux (#1389)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:28 +08:00
Jim Ma 3c6af10433
feat: add sync piece watchdog (#1272)
* feat: add sync piece watchdog

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:21 +08:00
Gaius 605865d3a9
feat: client add tls verify config (#1323)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:21 +08:00
Gaius 999f5919c6
feat: reduce dynconfig log (#1312)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:20 +08:00
Gaius 2a0314e45b
refactor: scheduler grpc (#1310)
* refactor: scheduler grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate protoc

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:20 +08:00
Gaius faa5e4e465
feat: add seed peer logic (#1302)
* feat: announce seed peer

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn logic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn job

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon change host uuid to host id

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: go generate mocks

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn compatibility

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change docker compose

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: reuse panic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: compatible with v2.0.3-beta.2

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:19 +08:00
Jim Ma 429252a071
feat: implement pattern in client daemon (#1231)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:19 +08:00