dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,653 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

46 Commits

Author SHA1 Message Date
Gaius 58034e4b1f
refactor: scheduling with v2 grpc (#2104) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:32 +08:00
Gaius 59856bda21
refactor: type of digest in task (#2084) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 042bb64ce4
refactor: task.SizeScope with v2 grpc in scheduler (#2082) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:31 +08:00
Gaius 840c599d04
refactor: task piece with v2 grpc (#2080) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius d8f36a44be
feat: change piece size to length (#2079) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius a5685582ce
refactor: resource task with v2 version of grpc (#2078) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Gaius 951705da98
feat: change PeerCountLimitForTask to 1000 (#2059) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 0f5470b5eb
feat: update task type from v1 to v2 (#2053) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius e1dd1efca1
feat: add AnnouncePeers to task in resource (#2051) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:29 +08:00
Gaius 51cc31b574
refactor: peer resource with v2 version of the grpc (#2039) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:28 +08:00
Gaius eb65cd76f6
feat: add directed graph to pkg (#2014) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:26 +08:00
Gaius c1066e60f4
feat: change ok to loaded in loading func (#2010) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:25 +08:00
Gaius d01a742b6f
feat: update fsm stable api (#1938) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius d64a6d8e29
feat: trigger task with priority (#1904) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius e1b89dc6f0
feat: add health check to service (#1889) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Gaius 8749814dee
feat: add the timeout of downloading piece to scheduler (#1880) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Gaius f194f0961f
feat: scheduler adds announce host handler (#1843) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Gaius 42223d994f
feat: optimize peer log (#1828) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius 7e24a85efa
feat: add calculateParentHostUploadSuccessScore to scheduler (#1789) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
Gaius 9014c21bec
refactor: scheduler registers task (#1766) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:04 +08:00
Gaius 481f685214
feat: available peer includes state is PeerStatePending (#1756) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:03 +08:00
Gaius bbd418542e
feat: task state is TaskStateRunning can be registered (#1751) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:02 +08:00
Gaius 500f767073
feat: remove task and host gc ttl (#1735) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:01 +08:00
Gaius 04dabff01e
fix: content length is zero when task succeed (#1732) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:00 +08:00
Gaius 327cf4493e
feat: add TaskStateLeave to task (#1728) 
Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:00 +08:00
Jim Ma 73de99353d
feat: empty file e2e (#1687) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:56 +08:00
Gaius a1a4dcdc5a
fix: task CanBackToSource func (#1663) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:54 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:41 +08:00
Gaius 7c2ee7858b
refactor: set and dag with generics (#1490) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:39 +08:00
Gaius 46deb4fccf
Use dag to represent peer scheduling topology (#1473) 
* feat: use dag to represent peer scheduling topology

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:37 +08:00
Gaius 8cf344992f
feat: if peer back-to-source failed, return source metadata. (#1444) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:34 +08:00
Gaius a2511cb945
feat: rewrite interface{} to any (#1419) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 64f5e9357b
refactor: scheduler announce task (#1407) 
* feat: scheduler announce normal task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cid

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:30 +08:00
Gaius 3fa7c7be75
feat: scheduler handles seed peer failed (#1325) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:21 +08:00
Gaius 2a0314e45b
refactor: scheduler grpc (#1310) 
* refactor: scheduler grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate protoc

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:20 +08:00
Gaius 9c1e05983a
feat: scheduler add seed peer (#1298) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:18 +08:00
Gaius 8724f49e07
fix: scheduler typo (#1297) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:18 +08:00
Gaius ca8961ed09
refactor: scheduler task SizeScope (#1287) 
* refactor: scheduler task SizeScope

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:17 +08:00
Gaius fe7b5fe0b4
fix: error message (#1255) 
* fix: error message

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:14 +08:00
Eryu Guan 65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache (#1227) 
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.

For example:
- add a file into cache system
  dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file

- check if a file exists in cache system
  dfcache stat --cid testid --local  # only check local cache
  dfcache stat --cid testid          # check other peers as well

- export/download a file from cache system
  dfcache export --cid testid -O /path/to/output

- delete a file from cache system, both local cache and P2P network
  dfcache delete -i testid -t testtag

Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius f5d372bab4
feat: if the number of failed peers in the task is greater than FailedPeerCountLimit, then scheduler notifies running peers of failure (#1211) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:10 +08:00
Gaius 557425249c
feat: optimize peer range (#1095) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:59 +08:00
Gaius 65ad608d58
feat: If cdn only updates IP, set cdn peers state to PeerStateLeave (#1038) 
* feat: If cdn only updates IP, set cdn peers state to PeerStateLeave

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:54 +08:00
Gaius 61310b55d5
feat: when cdn peer is failed, peer should be back-to-source (#1005) 
* feat: schdule peer with cdn failed

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: when cdn peer is failed, peer back-to-source

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: callback

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change scheduler default config

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:53 +08:00
Gaius 97c1eb65e7
Refactor scheduler service (#958) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:50 +08:00