dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,016 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

64 Commits

Author SHA1 Message Date
Jim Ma 743712e479
chore: add source error metrics (#1560) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:49 +08:00
cuidajun 242fb2bc37
add application-level statistics (#1559) 
Signed-off-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:45 +08:00
Gaius 5bcb53f0a9
feat: grpc consistent hashing (#1554) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:44 +08:00
Jim Ma 25e929142f
chore: optimize source error log (#1553) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:44 +08:00
Gaius 2ce864fbf6
fix: scheduler and manager tracing (#1551) 
fix: scheduler and manager trace

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:44 +08:00
Gaius 4bf14f57bc
feat: schedulers create main peer record (#1539) 
feat: scheduler create main peer record

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:43 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:41 +08:00
Gaius 7c2ee7858b
refactor: set and dag with generics (#1490) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:39 +08:00
Gaius 46deb4fccf
Use dag to represent peer scheduling topology (#1473) 
* feat: use dag to represent peer scheduling topology

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:37 +08:00
Gaius c6a019560d
feat: proxy add defaultTag field (#1462) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:36 +08:00
Gaius 8cf344992f
feat: if peer back-to-source failed, return source metadata. (#1444) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:34 +08:00
Gaius a2511cb945
feat: rewrite interface{} to any (#1419) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 30313a4caa
feat: client add create object storage (#1409) 
* feat: add create object storage rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:30 +08:00
Gaius 64f5e9357b
refactor: scheduler announce task (#1407) 
* feat: scheduler announce normal task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cid

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:30 +08:00
Gaius 7a30db3fba
refactor: pkg util (#1402) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius f688d341bd
feat: remove cdn (#1401) 
* feat: remove cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* docs: generate swagger

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius 3fa7c7be75
feat: scheduler handles seed peer failed (#1325) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:21 +08:00
Gaius 5707d86e6b
feat: scheduler register interface return task type (#1318) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:21 +08:00
Gaius 2a0314e45b
refactor: scheduler grpc (#1310) 
* refactor: scheduler grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate protoc

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:20 +08:00
Gaius faa5e4e465
feat: add seed peer logic (#1302) 
* feat: announce seed peer

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn logic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn job

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon change host uuid to host id

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: go generate mocks

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn compatibility

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change docker compose

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: reuse panic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: compatible with v2.0.3-beta.2

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:19 +08:00
Gaius 9c1e05983a
feat: scheduler add seed peer (#1298) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:18 +08:00
Gaius 8724f49e07
fix: scheduler typo (#1297) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:18 +08:00
Gaius ca8961ed09
refactor: scheduler task SizeScope (#1287) 
* refactor: scheduler task SizeScope

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:17 +08:00
Gaius 4497de122a
fix: legacy cdn peer (#1283) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:17 +08:00
Gaius 913b534668
fix: tree infinite loop (#1271) 
* fix: peer tree infinite loop

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: peer add Ancestors func

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:16 +08:00
Gaius 7f113e0dc2
feat: add AnnounceTask and StatTask metrics (#1256) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:14 +08:00
Gaius fe7b5fe0b4
fix: error message (#1255) 
* fix: error message

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:14 +08:00
Gaius 9c539fd425
test: AnnounceTask and StatTask (#1254) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:14 +08:00
Eryu Guan 65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache (#1227) 
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.

For example:
- add a file into cache system
  dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file

- check if a file exists in cache system
  dfcache stat --cid testid --local  # only check local cache
  dfcache stat --cid testid          # check other peers as well

- export/download a file from cache system
  dfcache export --cid testid -O /path/to/output

- delete a file from cache system, both local cache and P2P network
  dfcache delete -i testid -t testtag

Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius f26a63b96e
feat: registerTask returns to the task in time (#1250) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius b2c9577e7b
Add csv storage to scheduler (#1234) 
* feat: scheduler add csv storage for peer data
* feat: add storage configuration to scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius 913535ad7a
feat: add type to DownloadFailureCount (#1212) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:10 +08:00
Gaius f5d372bab4
feat: if the number of failed peers in the task is greater than FailedPeerCountLimit, then scheduler notifies running peers of failure (#1211) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:10 +08:00
Gaius bc1b807832
feat: optimize get available task (#1208) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:10 +08:00
Gaius e33736121f
feat: scheduler add block peers set (#1202) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:09 +08:00
Gaius 603db21e17
feat: scheduler add default biz tag (#1164) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:06 +08:00
Gaius 7adee4c2d9
feat: host remove peer (#1161) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:05 +08:00
Gaius b61865e2a9
feat: scheduler trace trigger cdn (#1147) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:04 +08:00
Gaius 7d30dd86ae
feat: add scheduler traffic metrics (#1143) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:03 +08:00
Gaius 5223cc1ce8
feat: add scheduler metrics (#1139) 
* feat: add scheduler metrics

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate validate for url meta

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:03 +08:00
Gaius addcf04df0
feat: task state for register (#1132) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:02 +08:00
Gaius 58aff0c231
feat: scheduler change piece cost from nanosecond to millisecond (#1119) 
* feat: change picec cost from nanoseconds to milliseconds

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler service

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:02 +08:00
Gaius e940af77ca
fix: scheduler piece cost time (#1118) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:01 +08:00
Gaius c5428e0657
feat: optimize scheduler log (#1114) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:01 +08:00
Gaius 0326f781f4
feat: optimize scheduler (#1106) 
* feat: optimize scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:00 +08:00
Gaius 27fcc904c0
fix: when peer state is PeerStateSucceeded, return size scope is small (#1103) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:00 +08:00
Gaius 557425249c
feat: optimize peer range (#1095) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:59 +08:00
Gaius 060429c675
feat: scheduler blocks cdn (#1079) 
* feat: scheduler blocks cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:57 +08:00
Gaius d52782ff38
feat: job trigger cdn by resource (#1076) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:57 +08:00
Gaius 6d58de0ef1
refactor: scheduler grpc server (#1046) 
* refactor: scheduler grpc server

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate mock file

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler package

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:55 +08:00