Commit Graph

104 Commits

Author SHA1 Message Date
embroede 8c905d9b8d
feat: enable configuration of some directory modes for dfdaemon (#2340)
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Gaius 37c588608a
feat: remove security domain (#2285)
* feat: remove security domain

Remove security domain in dfdaemon and scheduler.
Remove SecurityGroup and SecurityRule table in manager.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:46 +08:00
Gaius 2cda1def7f
feat: rename HostName to Hostname (#2205)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:40 +08:00
Jim Ma f5aff5e27f
feat: enhance daemon health check (#2130)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:35 +08:00
Gaius 1b9353ba8d
feat: add SyncNetworkTopology and SyncProbes to scheduler client (#2114)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:33 +08:00
Gaius f9759ca926
feat: dynconfig resolves addresses with host (#2109)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:33 +08:00
Gaius 1d1aec5e0d
refactor: support reload scheduler addresses for local Dynconfig in client (#2107)
feat: support reload scheduler addresses for local Dynconfig in client

Add OnNotify to client dynconfig, it will listen changes of the
local configuration.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:33 +08:00
Eryu Guan c10b6da498
feat: support reload scheduler addresses for local Dynconfig (#2091)
Dynconfig only supports refresh new configs from manager, but local
dynconfig is unable to reload scheduler addresses when local config file
is updated.

Now we introduce new Dynconfig method 'SetConfig()', which sets the
given DaemonOption to struct dynconfigLocal, and register it as a
watcher of WatchConfig(), which will reload config file periodically. So
that dynconfigLocal.GetResolveSchedulerAddrs() will get new scheduler
addresses.

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
2023-06-28 17:34:33 +08:00
Gaius 010396a0f8
feat: use v2 version of host id and add Addrs func to seed peer (#2086)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:31 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:29 +08:00
Jim Ma 7e69d8bed6
feat: support source client option (#2008)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:25 +08:00
Gaius 1666a17827
feat: remove NetTopology from scheduler and manager (#2007)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:25 +08:00
Gaius e83fb85216
feat: add v2 verison of the grpc to scheduler (#1999)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:24 +08:00
Gaius c89234e976
feat: add manager v2 api (#1990)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:23 +08:00
Gaius 008eeb980b
feat: add IPAddresses and DNSNames to sans of the cert (#1930)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:18 +08:00
Gaius 731ecc1234
feat: change yaml field type from string to net.IP (#1929)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:18 +08:00
Gaius 91454c8a60
feat: remove callsystem and pattern (#1925)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:17 +08:00
Gaius e1b89dc6f0
feat: add health check to service (#1889)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:14 +08:00
Gaius 2421ce88f4
feat: add announcer to dfdameon (#1852)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
Gaius f194f0961f
feat: scheduler adds announce host handler (#1843)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
Jim Ma ba58047871
feat: optional save list metadata to p2p (#1822)
* feat: save list metadata to p2p
* chore: optimize rpc download worker logic
* feat: add expire header reclaim logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:09 +08:00
Jim Ma 876c2ae17e
feat: support split running tasks (#1794)
* feat: support split running tasks

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: go tests

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: add split runnging tasks e2e config

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:06 +08:00
cuidajun 502c01f9aa
fix: daemon don't leaveHost when keepStorage=true (#1790)
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
2023-06-28 17:34:06 +08:00
cuidajun 536b039462
feat: daemon call leaveHost when exit (#1788)
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>

Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
2023-06-28 17:34:06 +08:00
Jim Ma 491b309620
feat: support concurrent recursive download (#1714)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:59 +08:00
Gaius 9573d5a659
fix: docker compose config (#1713)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:59 +08:00
greenhandatsjtu bc8d623b3d
feat: add traffic shaper for download tasks (#1654)
Signed-off-by: greenhandatsjtu <sunhengke@sjtu.edu.cn>
2023-06-28 17:33:59 +08:00
Gaius c9755aabfc
feat: stop grpc client (#1671)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:55 +08:00
Gaius b6c395a6cb
feat: add cert spec to security configuration (#1621)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:51 +08:00
Gaius 13580f89f2
refactor: dfpath for certify cache dir (#1618)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Gaius 43d8478507
feat: add tls policy to scheduler grpc server (#1616)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Jim Ma 9050230926
feat: refactor grpc credential (#1612)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Gaius 346cc0649e
feat: add tls policy constants (#1610)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:49 +08:00
Jim Ma 98fb1fc427
feat: add grpc mux transport (#1602)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:49 +08:00
Gaius 9ab33635c5
feat: manager init cert for grpc server (#1603)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:49 +08:00
Jim Ma 7224801bae
feat: refactor peertask option (#1600)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Jim Ma ecfe467585
feat: add client grpc dial timeout (#1599)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Jim Ma c25cfdd651
feat: support multiple certify cache (#1598)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Jim Ma 717ac99ef5
feat: daemon support auto issue certificate (#1586)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:48 +08:00
Gaius ad553d949b
feat: grpc dial adds context (#1594)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:47 +08:00
Gaius 2c48148ac8
fix: dfdaemon can not shutdown (#1580)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius a6a44269b4
refactor: dfdaemon client and remove rpc connection pool (#1576)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 272f38f567
feat: grpc add ratelimit (#1572)
feat: grpc ratelimit

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 69a51808d2
feat: refresh dynconfig addresses when grpc requests unavailable (#1571)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:46 +08:00
Gaius 631654f98b
feat: manager client add context (#1562)
Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:45 +08:00
Gaius 5bcb53f0a9
feat: grpc consistent hashing (#1554)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: bigerous <bigerous@qq.com>
2023-06-28 17:33:44 +08:00
Gaius 2ce864fbf6
fix: scheduler and manager tracing (#1551)
fix: scheduler and manager trace

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:44 +08:00
Jim Ma baa64bb0a3
feat: add seed trace (#1549)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:44 +08:00
cuidajun 1258175366
fix daemon socket remove error (#1529) 2023-06-28 17:33:42 +08:00
Jim Ma 4984cdfbbc
fix: check same peer id for sync pieces (#1525)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:42 +08:00