dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,653 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

414 Commits

Author SHA1 Message Date
dlut_xz 734ea4c522
feat: add CreatedAt function (#2244) 
Signed-off-by: XZ <834756128@qq.com>
 2023-06-28 17:34:44 +08:00
MinH-09 fa59bde001
feat: add train interval and trainer addresses (#2223) 
Signed-off-by: MinH-09 <2107139596@qq.com>
 2023-06-28 17:34:43 +08:00
Gaius 2765aa8c35
fix: fsm events failed when register task (#2225) 
fix: fsm events failed when register task.

When there is a high concurrency of registration tasks,
it can cause error InTransitionError, because of looplab/fsm library
uses optimistic locking to implement state transitions,
refer to https://github.com/looplab/fsm/blob/main/fsm.go#L312.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:42 +08:00
dlut_xz 0a9cd254f4
feat: add probes struct (#2190) 
Signed-off-by: XZ <834756128@qq.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:42 +08:00
MinH-09 1994765fce
feat: add trainer config in scheduler (#2214) 
Signed-off-by: MinH-09 <2107139596@qq.com>
 2023-06-28 17:34:41 +08:00
Gaius 7a1ae0a2ee
test: add new metrics test to service (#2212) 
test: add new metric test to service

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:41 +08:00
Gaius 2cda1def7f
feat: rename HostName to Hostname (#2205) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:40 +08:00
Gaius bf0a18230e
feat: change default value of dynconfig cache (#2203) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:40 +08:00
Garen Wen 0f66064a3a
feat: add advertise port (#2156) 
Co-authored-by: wenmanxiang <wenmanxiang@baidu.com>
 2023-06-28 17:34:39 +08:00
Gaius 224b951e63
fix: stat DownloadPeerCount and DownloadPieceCount (#2180) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:38 +08:00
Gaius 6d13b4a871
feat: add metrics to service v2 (#2153) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:36 +08:00
Gaius 17649a069c
feat: implement announce peer (#2150) 
feat: implement announce peer api

Add handleDownloadPeerStartedRequest, handleDownloadPeerBackToSourceStartedRequest,
handleDownloadPeerFinishedRequest, handleDownloadPeerBackToSourceFinishedRequest,
handleDownloadPeerFailedRequest, handleDownloadPeerBackToSourceFailedRequest,
handleDownloadPieceFinishedRequest, handleDownloadPieceBackToSourceFinishedRequest,
handleDownloadPieceFailedRequest and handleDownloadPieceBackToSourceFailedRequest,
to AnnouncePeer service api.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:36 +08:00
Gaius ff63681e88
feat: add handleRegisterSeedPeerRequest to service v2 in scheduler (#2148) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:36 +08:00
Gaius 047a55fb84
feat: add handleRegisterSeedPeerRequest to AnnouncePeer in service v2 (#2147) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:36 +08:00
Gaius b0e6089783
feat: change ScheduleCandidateParentsForNormalPeer implement (#2133) 
Change ScheduleCandidateParentsForNormalPeer implement.
Remove TaskStateLeave event in task GC.
Implement AnnouncePeer in service v2.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:35 +08:00
Gaius c672e75ed8
test: add Validate test to scheduler config (#2129) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:35 +08:00
Gaius 670c892898
feat: implement v2 version of scheduler service (#2125) 
Implement StatTask, AnnounceHost and LeaveHost api in
scheduler service v2.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:34 +08:00
Gaius 7306aec508
feat: correct grpc error code and implement StatPeer and LeavePeer (#2115) 
Correct grpc error from codes.Unknow to codes.Internal in gorm
operation. Implement StatPeer and LeavePeer interface in v2 version of
grpc.

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:34 +08:00
Gaius 1b9353ba8d
feat: add SyncNetworkTopology and SyncProbes to scheduler client (#2114) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Gaius f9759ca926
feat: dynconfig resolves addresses with host (#2109) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:33 +08:00
Gaius 58034e4b1f
refactor: scheduling with v2 grpc (#2104) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:32 +08:00
Gaius da8c45f2a8
feat: add host ttl to scheduler (#2089) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius dacf1f01a6
feat: rename scheduler package to scheduling (#2087) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 010396a0f8
feat: use v2 version of host id and add Addrs func to seed peer (#2086) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 59856bda21
refactor: type of digest in task (#2084) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 042bb64ce4
refactor: task.SizeScope with v2 grpc in scheduler (#2082) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
dlut_xz 91c9d83b6a
feat: add networkTopology configuration to scheduler (#2070) 
Signed-off-by: XZ <834756128@qq.com>
 2023-06-28 17:34:31 +08:00
Gaius 286885733a
feat: remove training configuration in scheduler (#2081) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 840c599d04
refactor: task piece with v2 grpc (#2080) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius d8f36a44be
feat: change piece size to length (#2079) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius a5685582ce
refactor: resource task with v2 version of grpc (#2078) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius 79024c87ca
refactor: parse http range (#2071) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius 09713e547c
fix: peer GC clear all peers when peer's count large than PeerCountLimitForTask (#2061) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 951705da98
feat: change PeerCountLimitForTask to 1000 (#2059) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 0f5470b5eb
feat: update task type from v1 to v2 (#2053) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius e1dd1efca1
feat: add AnnouncePeers to task in resource (#2051) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 29e57d1129
feat: add DownloadTask to seed peer resource (#2048) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:28 +08:00
Gaius ad1956fb45
feat: init AnnouncePeerStream of the peer (#2040) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:28 +08:00
Gaius 51cc31b574
refactor: peer resource with v2 version of the grpc (#2039) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:28 +08:00
Gaius d4b55c7287
refactor: announcer and dynconfig with v2 verison of the manager grpc (#2037) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:27 +08:00
Gaius 2b9aa7bf83
refactor: resource host without scheduler v1 definition (#2036) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:27 +08:00
Gaius eb65cd76f6
feat: add directed graph to pkg (#2014) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:26 +08:00
Gaius b6f0435011
feat: change peer's piece type in resource (#2012) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:25 +08:00
Gaius c1066e60f4
feat: change ok to loaded in loading func (#2010) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:25 +08:00
Gaius 1666a17827
feat: remove NetTopology from scheduler and manager (#2007) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:25 +08:00
Gaius e83fb85216
feat: add v2 verison of the grpc to scheduler (#1999) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:24 +08:00
Gaius c89234e976
feat: add manager v2 api (#1990) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:23 +08:00
cuidajun 5764104fe0
feat: add fail_code in scheduler's DownloadFailureCount metric (#1981) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:23 +08:00
cuidajun 2833353d0d
fix:scheduler's health client not close (#1975) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:22 +08:00
Gaius 1e783bd155
fix: context of trigger seed peer (#1971) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:22 +08:00
Gaius d01a742b6f
feat: update fsm stable api (#1938) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius 008eeb980b
feat: add IPAddresses and DNSNames to sans of the cert (#1930) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius 731ecc1234
feat: change yaml field type from string to net.IP (#1929) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius af76512b2a
feat: update version guage metrics (#1927) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius ffa80c1eb9
refactor: dynconfig without Unmarshal (#1926) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius 73ebf093bb
fix: application not found (#1924) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:17 +08:00
cuidajun 88f8f974ab
feat: client support 'priority' parameter (#1911) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:17 +08:00
Gaius c20c457abd
feat: handle application not found (#1913) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius 473c5d33f7
feat: update priority api (#1912) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius f120c8778b
feat: support redis sentinal (#1910) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius da40946f17
feat: storage collects upload piece count, peer cost and error details (#1907) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius 9cd2aa48c9
feat: priority of the register parameter is higher than parameter of the application (#1906) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius d64a6d8e29
feat: trigger task with priority (#1904) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius 481df276a3
feat: dynconfig adds list application in scheduler (#1903) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:15 +08:00
Gaius 63b4a868dc
refactor: back-to-source configuration (#1895) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:15 +08:00
Gaius 0627ad6f5c
feat: add backSourceCount validation (#1892) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Gaius e1b89dc6f0
feat: add health check to service (#1889) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Gaius 8749814dee
feat: add the timeout of downloading piece to scheduler (#1880) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Gaius 42ae0cec68
feat: if the scheduler cannot find the peer, then return Code_SchedReregister to dfdaemon (#1875) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:13 +08:00
Gaius 5276523f11
feat: store parent information (#1867) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:13 +08:00
Gaius a9f10bd678
feat: remove MainParent from peer and add IsPieceBackToSource to piece 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:13 +08:00
cuidajun 51683c2fd9
Fix: [scheduler] destPeer keepalive when downloaded by other peer (#1865) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:12 +08:00
Gaius 3af4cc7a0f
feat: scheduler supports storage config (#1864) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:12 +08:00
Gaius 4f91332992
feat: store peer download information (#1863) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:12 +08:00
walleliu 91f67603bb
add username when create redis client (#1862) 2023-06-28 17:34:12 +08:00
Gaius a1b65218db
feat: optimize gc package (#1855) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:12 +08:00
Gaius ef6ad12587
feat: add announcer to scheduler (#1854) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:12 +08:00
Gaius 2421ce88f4
feat: add announcer to dfdameon (#1852) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Gaius f194f0961f
feat: scheduler adds announce host handler (#1843) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Gaius 02543c781d
feat: optimize scope size is error (#1831) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:10 +08:00
Gaius 0f8b75808a
feat: add timeout grpc and job (#1830) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:10 +08:00
Gaius 42223d994f
feat: optimize peer log (#1828) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius 3da67311ea
feat: optimize preheat (#1824) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius b1c049db46
feat: returns an scheduling error if the peer state is not PeerStateRunning (#1821) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius 073c84dd71
feat: optimize peer gc (#1819) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:08 +08:00
Gaius 83ec42cf25
feat: peer.UpdateAt needs to be updated during download process (#1818) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:08 +08:00
Gaius 96910887b2
feat: statistical the traffic of reused peer (#1816) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:08 +08:00
Gaius 4bd7d413f2
feat: add workHome and pluginDir to configuration (#1807) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:08 +08:00
Gaius d1363f20e7
fix: leave host (#1803) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:07 +08:00
Gaius abf1c43a4e
feat: add leave host logger (#1801) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:07 +08:00
Gaius 3587ba9537
feat: scheduler's record adds ParentUploadCount and ParentUploadFailedCount (#1795) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
Gaius 7e24a85efa
feat: add calculateParentHostUploadSuccessScore to scheduler (#1789) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
Gaius 446fcc9fd7
feat: add LeaveHost handler (#1780) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
Gaius 9014c21bec
refactor: scheduler registers task (#1766) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:04 +08:00
Gaius 481f685214
feat: available peer includes state is PeerStatePending (#1756) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:03 +08:00
Gaius 14f88993ae
feat: peer will back-to-source when task switch state failed (#1754) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:02 +08:00
Gaius 61a6e545e5
feat: change FilterParentRangeLimit validation (#1752) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:02 +08:00
Gaius bbd418542e
feat: task state is TaskStateRunning can be registered (#1751) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:02 +08:00
Gaius 036e6e56fb
feat: overwrite task url and url meta (#1740) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:01 +08:00