Commit Graph

414 Commits

Author SHA1 Message Date
Gaius 0ffc55b6d4
refactor: trainer server module (#2486)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:35:00 +08:00
dlut_xz 8a8d6abaae
feat: move probe interval from scheduler config to client config (#2462)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:58 +08:00
Gaius 5342fb97fc
fix: Interval in SyncProbesResponse (#2466)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:58 +08:00
Gaius 98aee9640d
feat: implement SyncProbes api in scheduler grpc service (#2449)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: XZ <834756128@qq.com>
Co-authored-by: dlut_xz <52518280+fcgxz2003@users.noreply.github.com>
2023-06-28 17:34:58 +08:00
Gaius c984444d4b
test: add TestAnnouncer_Serve test
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Gaius 38b550c666
feat: remove dirty file
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Gaius c8583237e4
fix: announcer in scheduler (#2451)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Gaius 8634495f2c
test: optimize announcer in scheduler (#2448)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Min 4ef338d5da
test: announcer adds tests (#2377)
Signed-off-by: MinH-09 <2107139596@qq.com>
2023-06-28 17:34:57 +08:00
Gaius 146f82c453
feat: optimize announcer in scheduler and client (#2445)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Gaius 7f4d4d2ec0
feat: change DefaultProbeInterval to 20 minute (#2440)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:56 +08:00
Gaius 691e884d35
feat: remove useless fields in network topology (#2439)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:56 +08:00
Gaius f610ab9c0d
test: add ParseProbedCountKeyInScheduler and Snapshot tests (#2438)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:56 +08:00
Gaius 274c6d04e6
feat: support to collect and snapshot in network topology (#2429)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:56 +08:00
dlut_xz 7fbe7553f9
test: optimize network topology and probes unit tests (#2425)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:56 +08:00
dlut_xz e6049d2362
test: add delete host unit tests (#2424)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:55 +08:00
Gaius a3d417a69c
feat: change Dequeue to private func (#2420)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:55 +08:00
Gaius f325ae67a2
fix: delete host in network topology (#2417)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:55 +08:00
Gaius 57c29ee51a
feat: remove redis Pipelined in network topology (#2416)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:55 +08:00
Gaius 989f658544
feat: optimize network topology comment (#2415)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:55 +08:00
dlut_xz 5bc465f7d8
test: add probes and network topology unit tests (#2414)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:54 +08:00
Gaius 9999ce51b6
feat: add ProbedAt to network topology (#2413)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:54 +08:00
Gaius 5794cfc02c
refactor: network topology package (#2412)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:54 +08:00
Gaius b4e71f586d
test: optimize network topology and probes tests (#2409)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:54 +08:00
dlut_xz 8630740952
test: add probes and network topology unit tests (#2390)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:54 +08:00
Gaius b5c5b5b2e6
feat: implement Enqueue and AverageRTT in probes.go (#2393)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius a44c084f28
feat: handle context in triggerSeedPeerTask (#2392)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 730dcbf2bc
feat: optimize field name of ProbeConfig (#2391)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 8de3594a8b
feat: scheduler supports to disable redis (#2389)
feat: scheduler support to disable redis

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 3ad8ddd4b5
refactor: probes package in network topology (#2382)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius e1113c17c5
refactor: network topology package in scheduler (#2380)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:52 +08:00
Gaius 81817666e5
feat: move redis key to pkg/redis package (#2378)
Network Topology adds functions to new redis key and move the manager's redis key to pkg/redis.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:52 +08:00
dlut_xz deab6df2c2
feat: add network topology package (#2364)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:52 +08:00
Gaius 320097b44d
feat: add announceToTrainer in scheduler (#2371)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:52 +08:00
Gaius 701a26a8d6
feat: hide sensitive information in log (#2369)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius 25edc40937
fix: evaluate after filter (#2363)
Since the final length of the filter is the candidateParentLimit used,
the parents after the filter is the returned parents.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius ddcea685e7
fix: when bufferSize is zero, storage can not write data to file (#2366)
When bufferSize is zero, createDownload and createNetworkTopology
can not write data to file, because buffer is empty.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius 71f13df1f0
feat: remove traffic_type in DownloadPeerDuration metric (#2357)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Tianxi f2130509d1
feat: add traffic type of peer task download duration (#2349)
Co-authored-by: li.tianxi <li.tianxi@hellogroup.com>
2023-06-28 17:34:51 +08:00
Gaius b97c0cb0ae
feat: remove deprecated field in manager and scheduler (#2345)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:50 +08:00
dlut_xz 557046d0d4
feat: add database config and move redis to it (#2338)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:50 +08:00
Gaius 65ec5b2906
feat: rename createRecord to createDownloadRecord (#2306)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:48 +08:00
Gaius e9cb38e851
feat: remove SyncNetworkTopology API (#2296)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:47 +08:00
dlut_xz 91853edb07
feat: add network topology storage interface (#2286)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:47 +08:00
Gaius e34b66f58c
fix: redis validation in scheduler config (#2287)
When Redis is in cluster mode, only DB 0 can be used.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:47 +08:00
dlut_xz 0c7ccb7517
feat: add network topology and probes storage structs (#2254)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:46 +08:00
Gaius 37c588608a
feat: remove security domain (#2285)
* feat: remove security domain

Remove security domain in dfdaemon and scheduler.
Remove SecurityGroup and SecurityRule table in manager.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:46 +08:00
dlut_xz 8d29390b51
feat: rename Record to Download in storage (#2253)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:45 +08:00
dlut_xz c5549c82e3
fix: modify the traversal condition for Items (#2250)
Signed-off-by: XZ <834756128@qq.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:44 +08:00
Gaius 65bf673d2d
fix: ip and hostname params in FindSchedulerClusters (#2249)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:44 +08:00