dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,653 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

379 Commits

Author SHA1 Message Date
Jim Ma bdfeb48f02
feat: daemon store file exclusive (#2465) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:59 +08:00
Jim Ma 9137707139
feat: support breakpoint resume for running tasks (#2457) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:58 +08:00
embroede 8c905d9b8d
feat: enable configuration of some directory modes for dfdaemon (#2340) 
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:57 +08:00
Gaius 146f82c453
feat: optimize announcer in scheduler and client (#2445) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:57 +08:00
Jim Ma afb47f6729
chore: check grpc peer info for download service (#2385) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:53 +08:00
Jim Ma 276333fceb
fix: SyncPieceViaHTTPS not work (#2329) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:49 +08:00
Gaius 2287adca60
fix: object downloads failed by dfstore when dfdaemon enabled concurrent (#2328) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:49 +08:00
Gaius e9cb38e851
feat: remove SyncNetworkTopology API (#2296) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:47 +08:00
Gaius 37c588608a
feat: remove security domain (#2285) 
* feat: remove security domain

Remove security domain in dfdaemon and scheduler.
Remove SecurityGroup and SecurityRule table in manager.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:46 +08:00
Jim Ma b38b5fcbeb
fix: client grpc dial non-block (#2261) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:45 +08:00
Jim Ma c82ebec8e0
chore: update traffic shaper log (#2227) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:42 +08:00
Jim Ma 74e8e65b0b
fix: traffic shaper record task not found (#2226) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:42 +08:00
Gaius 7a1ae0a2ee
test: add new metrics test to service (#2212) 
test: add new metric test to service

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:41 +08:00
Gaius 2cda1def7f
feat: rename HostName to Hostname (#2205) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:40 +08:00
Gaius 047a55fb84
feat: add handleRegisterSeedPeerRequest to AnnouncePeer in service v2 (#2147) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:36 +08:00
Jim Ma f5aff5e27f
feat: enhance daemon health check (#2130) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:35 +08:00
Gaius 670c892898
feat: implement v2 version of scheduler service (#2125) 
Implement StatTask, AnnounceHost and LeaveHost api in
scheduler service v2.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:34 +08:00
Gaius 1b9353ba8d
feat: add SyncNetworkTopology and SyncProbes to scheduler client (#2114) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Gaius f9759ca926
feat: dynconfig resolves addresses with host (#2109) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Gaius 1d1aec5e0d
refactor: support reload scheduler addresses for local Dynconfig in client (#2107) 
feat: support reload scheduler addresses for local Dynconfig in client

Add OnNotify to client dynconfig, it will listen changes of the
local configuration.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Eryu Guan c10b6da498
feat: support reload scheduler addresses for local Dynconfig (#2091) 
Dynconfig only supports refresh new configs from manager, but local
dynconfig is unable to reload scheduler addresses when local config file
is updated.

Now we introduce new Dynconfig method 'SetConfig()', which sets the
given DaemonOption to struct dynconfigLocal, and register it as a
watcher of WatchConfig(), which will reload config file periodically. So
that dynconfigLocal.GetResolveSchedulerAddrs() will get new scheduler
addresses.

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:33 +08:00
cuidajun e8822c144b
test: remove random test in pieceDispatcherTest (#2106) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:33 +08:00
Eryu Guan fab5609d3d
feat: don't GC task if expire time is 0 (#2102) 
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:32 +08:00
Jim Ma d7a7e69916
chore: remove unused MarkInvalid in daemon (#2101) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:32 +08:00
Eryu Guan 3352deff39
fix: unregister task from scheduler in storage.deleteTask (#2100) 
On graceful shutdown we will call forceGC() to delete all cached tasks
if 'keepStorage' is false. And we rely on deleteTask() to delete the
given task from both local storage and scheduler.

But commit 86a6030c8f ("feat: unregister failed task storage (#1717)")
changed MarkReclaim() to MarkInvalid() in deleteTask(), and resulted in
task leak in scheduler.

Fix it by calling MarkReclaim() in deleteTask().

Fixes: 86a6030c8f ("feat: unregister failed task storage (#1717)")

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:32 +08:00
Gaius 010396a0f8
feat: use v2 version of host id and add Addrs func to seed peer (#2086) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 07976961c6
refactor: package digest (#2085) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Jim Ma cc42334e4d
fix: backsource first piece timeout (#2083) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 79024c87ca
refactor: parse http range (#2071) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 7dc3c826f2
feat: add v2 version of dfdaemon client (#2050) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius d4b55c7287
refactor: announcer and dynconfig with v2 verison of the manager grpc (#2037) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:27 +08:00
cuidajun 304448009f
refactor: piece_dispatcher considering score of parent peer (#1978) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:26 +08:00
Jim Ma 7e69d8bed6
feat: support source client option (#2008) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:25 +08:00
Gaius 1666a17827
feat: remove NetTopology from scheduler and manager (#2007) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:25 +08:00
Jim Ma 2c8ecac6e2
fix: dferror not convert (#2001) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:24 +08:00
Gaius e83fb85216
feat: add v2 verison of the grpc to scheduler (#1999) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:24 +08:00
cuidajun acdb05ccba
fix: daemon recognize Code_SchedForbidden (#1994) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:24 +08:00
Gaius c89234e976
feat: add manager v2 api (#1990) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:23 +08:00
cuidajun 81865e6ba7
fix: client bitMap extend capacity (#1973) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:22 +08:00
Gaius e4fc6d5c85
feat: add S3ForcePathStyle to object storage (#1976) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:22 +08:00
Jim Ma 004d237537
feat: corrupt data check (#1946) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:20 +08:00
Jim Ma 9c8d82e294
feat: create synchronizers concurrently (#1941) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Jim Ma 56e668bb0d
chore: optimize download log (#1944) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Jim Ma f402824618
fix: download context cancelled (#1942) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Jim Ma 7cb802aeb7
feat: remove legacy peers support (#1939) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Gaius c3882ccfc6
fix: peer keepalive with manager (#1940) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:19 +08:00
Gaius 008eeb980b
feat: add IPAddresses and DNSNames to sans of the cert (#1930) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius 731ecc1234
feat: change yaml field type from string to net.IP (#1929) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Jim Ma ff79c3148a
feat: random pieces download (#1918) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:18 +08:00