473c5d33f7
feat: update priority api ( #1912 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:16 +08:00
da40946f17
feat: storage collects upload piece count, peer cost and error details ( #1907 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:16 +08:00
9cd2aa48c9
feat: priority of the register parameter is higher than parameter of the application ( #1906 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:16 +08:00
d64a6d8e29
feat: trigger task with priority ( #1904 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:16 +08:00
481df276a3
feat: dynconfig adds list application in scheduler ( #1903 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:15 +08:00
63b4a868dc
refactor: back-to-source configuration ( #1895 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:15 +08:00
42ae0cec68
feat: if the scheduler cannot find the peer, then return Code_SchedReregister to dfdaemon ( #1875 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:13 +08:00
4f91332992
feat: store peer download information ( #1863 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:12 +08:00
f194f0961f
feat: scheduler adds announce host handler ( #1843 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
7e24a85efa
feat: add calculateParentHostUploadSuccessScore to scheduler ( #1789 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:06 +08:00
446fcc9fd7
feat: add LeaveHost handler ( #1780 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:06 +08:00
9014c21bec
refactor: scheduler registers task ( #1766 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:04 +08:00
481f685214
feat: available peer includes state is PeerStatePending ( #1756 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:03 +08:00
14f88993ae
feat: peer will back-to-source when task switch state failed ( #1754 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:02 +08:00
bbd418542e
feat: task state is TaskStateRunning can be registered ( #1751 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:02 +08:00
036e6e56fb
feat: overwrite task url and url meta ( #1740 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:01 +08:00
500f767073
feat: remove task and host gc ttl ( #1735 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:01 +08:00
04dabff01e
fix: content length is zero when task succeed ( #1732 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:00 +08:00
327cf4493e
feat: add TaskStateLeave to task ( #1728 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:00 +08:00
f354ee3861
refactor: manager and scheduler config ( #1701 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:57 +08:00
73de99353d
feat: empty file e2e ( #1687 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:56 +08:00
242fb2bc37
add application-level statistics ( #1559 )
...
Signed-off-by: bigerous <bigerous@qq.com>
2023-06-28 17:33:45 +08:00
4bf14f57bc
feat: schedulers create main peer record ( #1539 )
...
feat: scheduler create main peer record
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:43 +08:00
d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api ( #1515 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:41 +08:00
7c2ee7858b
refactor: set and dag with generics ( #1490 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:39 +08:00
46deb4fccf
Use dag to represent peer scheduling topology ( #1473 )
...
* feat: use dag to represent peer scheduling topology
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:37 +08:00
c6a019560d
feat: proxy add defaultTag field ( #1462 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:36 +08:00
8cf344992f
feat: if peer back-to-source failed, return source metadata. ( #1444 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:34 +08:00
a2511cb945
feat: rewrite interface{} to any ( #1419 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
30313a4caa
feat: client add create object storage ( #1409 )
...
* feat: add create object storage rest api
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
64f5e9357b
refactor: scheduler announce task ( #1407 )
...
* feat: scheduler announce normal task
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cid
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
f688d341bd
feat: remove cdn ( #1401 )
...
* feat: remove cdn
Signed-off-by: Gaius <gaius.qi@gmail.com>
* docs: generate swagger
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:29 +08:00
2a0314e45b
refactor: scheduler grpc ( #1310 )
...
* refactor: scheduler grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: generate protoc
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:20 +08:00
faa5e4e465
feat: add seed peer logic ( #1302 )
...
* feat: announce seed peer
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn logic
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn job
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dfdaemon change host uuid to host id
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: go generate mocks
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn compatibility
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: change docker compose
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: reuse panic
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: compatible with v2.0.3-beta.2
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:19 +08:00
9c1e05983a
feat: scheduler add seed peer ( #1298 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:18 +08:00
ca8961ed09
refactor: scheduler task SizeScope ( #1287 )
...
* refactor: scheduler task SizeScope
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:17 +08:00
4497de122a
fix: legacy cdn peer ( #1283 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:17 +08:00
913b534668
fix: tree infinite loop ( #1271 )
...
* fix: peer tree infinite loop
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: peer add Ancestors func
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:16 +08:00
9c539fd425
test: AnnounceTask and StatTask ( #1254 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:14 +08:00
65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache ( #1227 )
...
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.
For example:
- add a file into cache system
dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file
- check if a file exists in cache system
dfcache stat --cid testid --local # only check local cache
dfcache stat --cid testid # check other peers as well
- export/download a file from cache system
dfcache export --cid testid -O /path/to/output
- delete a file from cache system, both local cache and P2P network
dfcache delete -i testid -t testtag
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
f26a63b96e
feat: registerTask returns to the task in time ( #1250 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
b2c9577e7b
Add csv storage to scheduler ( #1234 )
...
* feat: scheduler add csv storage for peer data
* feat: add storage configuration to scheduler
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
f5d372bab4
feat: if the number of failed peers in the task is greater than FailedPeerCountLimit, then scheduler notifies running peers of failure ( #1211 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:10 +08:00
bc1b807832
feat: optimize get available task ( #1208 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:10 +08:00
e33736121f
feat: scheduler add block peers set ( #1202 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:09 +08:00
603db21e17
feat: scheduler add default biz tag ( #1164 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:06 +08:00
eb408d136f
test: scheduler service handlePieceFail ( #1146 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
7d30dd86ae
feat: add scheduler traffic metrics ( #1143 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
5223cc1ce8
feat: add scheduler metrics ( #1139 )
...
* feat: add scheduler metrics
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: generate validate for url meta
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
addcf04df0
feat: task state for register ( #1132 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:02 +08:00
Gaius