7c2ee7858b
refactor: set and dag with generics ( #1490 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:39 +08:00
46deb4fccf
Use dag to represent peer scheduling topology ( #1473 )
...
* feat: use dag to represent peer scheduling topology
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:37 +08:00
c6a019560d
feat: proxy add defaultTag field ( #1462 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:36 +08:00
8cf344992f
feat: if peer back-to-source failed, return source metadata. ( #1444 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:34 +08:00
a2511cb945
feat: rewrite interface{} to any ( #1419 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
30313a4caa
feat: client add create object storage ( #1409 )
...
* feat: add create object storage rest api
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
64f5e9357b
refactor: scheduler announce task ( #1407 )
...
* feat: scheduler announce normal task
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cid
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
f688d341bd
feat: remove cdn ( #1401 )
...
* feat: remove cdn
Signed-off-by: Gaius <gaius.qi@gmail.com>
* docs: generate swagger
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:29 +08:00
2a0314e45b
refactor: scheduler grpc ( #1310 )
...
* refactor: scheduler grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: generate protoc
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:20 +08:00
faa5e4e465
feat: add seed peer logic ( #1302 )
...
* feat: announce seed peer
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn logic
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn job
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dfdaemon change host uuid to host id
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: go generate mocks
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn compatibility
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: change docker compose
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: reuse panic
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: compatible with v2.0.3-beta.2
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:19 +08:00
9c1e05983a
feat: scheduler add seed peer ( #1298 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:18 +08:00
ca8961ed09
refactor: scheduler task SizeScope ( #1287 )
...
* refactor: scheduler task SizeScope
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:17 +08:00
4497de122a
fix: legacy cdn peer ( #1283 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:17 +08:00
913b534668
fix: tree infinite loop ( #1271 )
...
* fix: peer tree infinite loop
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: peer add Ancestors func
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:16 +08:00
9c539fd425
test: AnnounceTask and StatTask ( #1254 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:14 +08:00
65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache ( #1227 )
...
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.
For example:
- add a file into cache system
dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file
- check if a file exists in cache system
dfcache stat --cid testid --local # only check local cache
dfcache stat --cid testid # check other peers as well
- export/download a file from cache system
dfcache export --cid testid -O /path/to/output
- delete a file from cache system, both local cache and P2P network
dfcache delete -i testid -t testtag
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
f26a63b96e
feat: registerTask returns to the task in time ( #1250 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
b2c9577e7b
Add csv storage to scheduler ( #1234 )
...
* feat: scheduler add csv storage for peer data
* feat: add storage configuration to scheduler
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
f5d372bab4
feat: if the number of failed peers in the task is greater than FailedPeerCountLimit, then scheduler notifies running peers of failure ( #1211 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:10 +08:00
bc1b807832
feat: optimize get available task ( #1208 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:10 +08:00
e33736121f
feat: scheduler add block peers set ( #1202 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:09 +08:00
603db21e17
feat: scheduler add default biz tag ( #1164 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:06 +08:00
eb408d136f
test: scheduler service handlePieceFail ( #1146 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
7d30dd86ae
feat: add scheduler traffic metrics ( #1143 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
5223cc1ce8
feat: add scheduler metrics ( #1139 )
...
* feat: add scheduler metrics
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: generate validate for url meta
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
addcf04df0
feat: task state for register ( #1132 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:02 +08:00
58aff0c231
feat: scheduler change piece cost from nanosecond to millisecond ( #1119 )
...
* feat: change picec cost from nanoseconds to milliseconds
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler service
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:02 +08:00
0326f781f4
feat: optimize scheduler ( #1106 )
...
* feat: optimize scheduler
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:00 +08:00
27fcc904c0
fix: when peer state is PeerStateSucceeded, return size scope is small ( #1103 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:00 +08:00
81ea759393
test: scheduler handle failed piece ( #1084 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:58 +08:00
060429c675
feat: scheduler blocks cdn ( #1079 )
...
* feat: scheduler blocks cdn
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:57 +08:00
d52782ff38
feat: job trigger cdn by resource ( #1076 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:57 +08:00
6d58de0ef1
refactor: scheduler grpc server ( #1046 )
...
* refactor: scheduler grpc server
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: generate mock file
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler package
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:55 +08:00
65ad608d58
feat: If cdn only updates IP, set cdn peers state to PeerStateLeave ( #1038 )
...
* feat: If cdn only updates IP, set cdn peers state to PeerStateLeave
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:54 +08:00
5479b3a53e
feat: stream send error code ( #986 )
...
* feat: change task and peer ttl (#984 )
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: send error code
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add retry interval
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dfdaemon report successful piece before end of piece
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: update submodule version
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:51 +08:00
97c1eb65e7
Refactor scheduler service ( #958 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:50 +08:00
Gaius