dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,653 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

446 Commits

Author SHA1 Message Date
Jim Ma bdfeb48f02
feat: daemon store file exclusive (#2465) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:59 +08:00
dlut_xz 8a8d6abaae
feat: move probe interval from scheduler config to client config (#2462) 
Signed-off-by: XZ <834756128@qq.com>
 2023-06-28 17:34:58 +08:00
Jim Ma bb29a234f4
chore: update dfget recursive log (#2459) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:58 +08:00
Jim Ma 9137707139
feat: support breakpoint resume for running tasks (#2457) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:58 +08:00
embroede 8c905d9b8d
feat: enable configuration of some directory modes for dfdaemon (#2340) 
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:57 +08:00
Gaius 146f82c453
feat: optimize announcer in scheduler and client (#2445) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:57 +08:00
Gaius b5c5b5b2e6
feat: implement Enqueue and AverageRTT in probes.go (#2393) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:53 +08:00
Jim Ma afb47f6729
chore: check grpc peer info for download service (#2385) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:53 +08:00
Jim Ma 276333fceb
fix: SyncPieceViaHTTPS not work (#2329) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:49 +08:00
Gaius 2287adca60
fix: object downloads failed by dfstore when dfdaemon enabled concurrent (#2328) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:49 +08:00
Gaius e9cb38e851
feat: remove SyncNetworkTopology API (#2296) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:47 +08:00
Gaius 37c588608a
feat: remove security domain (#2285) 
* feat: remove security domain

Remove security domain in dfdaemon and scheduler.
Remove SecurityGroup and SecurityRule table in manager.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:46 +08:00
Gaius 558d8527e5
fix: local dynconfig panic in Notify (#2266) 
Fix local dynconfig panic in Notify and
add scheduler addr test in config.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:45 +08:00
Gaius 6ad8db588c
feat: local dynconfig notifies data in client (#2264) 
Dfdaemon will fail the health check if there is no available scheduler address.
So in the case of local dynconfig, the address of scheduler should be passed.
For dfdaemon health check, please refer to https://github.com/dragonflyoss/Dragonfly2/pull/2130.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:45 +08:00
Jim Ma b38b5fcbeb
fix: client grpc dial non-block (#2261) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:45 +08:00
Jim Ma 94a7c7639f
feat: update resource director (#2243) 
* feat: update resource director
* chore: add digest check for oras

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:44 +08:00
Jim Ma 144335ffb1
feat: add authinfo injector (#2149) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:43 +08:00
Jim Ma c82ebec8e0
chore: update traffic shaper log (#2227) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:42 +08:00
Jim Ma 74e8e65b0b
fix: traffic shaper record task not found (#2226) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:42 +08:00
Gaius 7a1ae0a2ee
test: add new metrics test to service (#2212) 
test: add new metric test to service

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:41 +08:00
Gaius 2cda1def7f
feat: rename HostName to Hostname (#2205) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:40 +08:00
Gaius bf0a18230e
feat: change default value of dynconfig cache (#2203) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:40 +08:00
Gaius 047a55fb84
feat: add handleRegisterSeedPeerRequest to AnnouncePeer in service v2 (#2147) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:36 +08:00
Jim Ma f5aff5e27f
feat: enhance daemon health check (#2130) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:35 +08:00
Gaius c672e75ed8
test: add Validate test to scheduler config (#2129) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:35 +08:00
Gaius 6ae36555be
test: refactor client validate ut (#2126) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:34 +08:00
Gaius 670c892898
feat: implement v2 version of scheduler service (#2125) 
Implement StatTask, AnnounceHost and LeaveHost api in
scheduler service v2.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:34 +08:00
Eryu Guan a9df345d5b
test: add unit tests for DaemonConfig.Validate() (#2119) 
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:34 +08:00
Eryu Guan bbeba99da5
fix: validate daemon gcInterval config (#2118) 
If gcInterval is set to 0s or negative value, dfdaemon will panic as

  panic: non-positive interval for NewTicker

  goroutine 132 [running]:
  time.NewTicker(0x0?)

Fix it by validating it as well.

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:34 +08:00
Gaius 1b9353ba8d
feat: add SyncNetworkTopology and SyncProbes to scheduler client (#2114) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Gaius f9759ca926
feat: dynconfig resolves addresses with host (#2109) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Gaius 1d1aec5e0d
refactor: support reload scheduler addresses for local Dynconfig in client (#2107) 
feat: support reload scheduler addresses for local Dynconfig in client

Add OnNotify to client dynconfig, it will listen changes of the
local configuration.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Eryu Guan c10b6da498
feat: support reload scheduler addresses for local Dynconfig (#2091) 
Dynconfig only supports refresh new configs from manager, but local
dynconfig is unable to reload scheduler addresses when local config file
is updated.

Now we introduce new Dynconfig method 'SetConfig()', which sets the
given DaemonOption to struct dynconfigLocal, and register it as a
watcher of WatchConfig(), which will reload config file periodically. So
that dynconfigLocal.GetResolveSchedulerAddrs() will get new scheduler
addresses.

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:33 +08:00
cuidajun e8822c144b
test: remove random test in pieceDispatcherTest (#2106) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:33 +08:00
Eryu Guan fab5609d3d
feat: don't GC task if expire time is 0 (#2102) 
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:32 +08:00
Jim Ma d7a7e69916
chore: remove unused MarkInvalid in daemon (#2101) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:32 +08:00
Eryu Guan 3352deff39
fix: unregister task from scheduler in storage.deleteTask (#2100) 
On graceful shutdown we will call forceGC() to delete all cached tasks
if 'keepStorage' is false. And we rely on deleteTask() to delete the
given task from both local storage and scheduler.

But commit 86a6030c8f ("feat: unregister failed task storage (#1717)")
changed MarkReclaim() to MarkInvalid() in deleteTask(), and resulted in
task leak in scheduler.

Fix it by calling MarkReclaim() in deleteTask().

Fixes: 86a6030c8f ("feat: unregister failed task storage (#1717)")

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:34:32 +08:00
Gaius 010396a0f8
feat: use v2 version of host id and add Addrs func to seed peer (#2086) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 07976961c6
refactor: package digest (#2085) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Jim Ma cc42334e4d
fix: backsource first piece timeout (#2083) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 79024c87ca
refactor: parse http range (#2071) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 7dc3c826f2
feat: add v2 version of dfdaemon client (#2050) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius d4b55c7287
refactor: announcer and dynconfig with v2 verison of the manager grpc (#2037) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:27 +08:00
cuidajun 304448009f
refactor: piece_dispatcher considering score of parent peer (#1978) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:26 +08:00
Jim Ma 7e69d8bed6
feat: support source client option (#2008) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:25 +08:00
Gaius 1666a17827
feat: remove NetTopology from scheduler and manager (#2007) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:25 +08:00
Jim Ma 2c8ecac6e2
fix: dferror not convert (#2001) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:24 +08:00
Jim Ma e8b96ada11
fix: dfstore typo (#2000) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:24 +08:00
Gaius e83fb85216
feat: add v2 verison of the grpc to scheduler (#1999) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:24 +08:00