dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
869 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

55 Commits

Author SHA1 Message Date
Gaius c6a019560d
feat: proxy add defaultTag field (#1462) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:36 +08:00
Gaius 8cf344992f
feat: if peer back-to-source failed, return source metadata. (#1444) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:34 +08:00
Gaius a2511cb945
feat: rewrite interface{} to any (#1419) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 30313a4caa
feat: client add create object storage (#1409) 
* feat: add create object storage rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:30 +08:00
Gaius 64f5e9357b
refactor: scheduler announce task (#1407) 
* feat: scheduler announce normal task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cid

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:30 +08:00
Gaius 7a30db3fba
refactor: pkg util (#1402) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius f688d341bd
feat: remove cdn (#1401) 
* feat: remove cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* docs: generate swagger

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius 3fa7c7be75
feat: scheduler handles seed peer failed (#1325) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:21 +08:00
Gaius 5707d86e6b
feat: scheduler register interface return task type (#1318) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:21 +08:00
Gaius 2a0314e45b
refactor: scheduler grpc (#1310) 
* refactor: scheduler grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate protoc

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:20 +08:00
Gaius faa5e4e465
feat: add seed peer logic (#1302) 
* feat: announce seed peer

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn logic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn job

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon change host uuid to host id

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: go generate mocks

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn compatibility

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change docker compose

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: reuse panic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: compatible with v2.0.3-beta.2

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:19 +08:00
Gaius 9c1e05983a
feat: scheduler add seed peer (#1298) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:18 +08:00
Gaius 8724f49e07
fix: scheduler typo (#1297) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:18 +08:00
Gaius ca8961ed09
refactor: scheduler task SizeScope (#1287) 
* refactor: scheduler task SizeScope

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:17 +08:00
Gaius 4497de122a
fix: legacy cdn peer (#1283) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:17 +08:00
Gaius 913b534668
fix: tree infinite loop (#1271) 
* fix: peer tree infinite loop

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: peer add Ancestors func

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:16 +08:00
Gaius 7f113e0dc2
feat: add AnnounceTask and StatTask metrics (#1256) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:14 +08:00
Gaius fe7b5fe0b4
fix: error message (#1255) 
* fix: error message

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:14 +08:00
Gaius 9c539fd425
test: AnnounceTask and StatTask (#1254) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:14 +08:00
Eryu Guan 65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache (#1227) 
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.

For example:
- add a file into cache system
  dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file

- check if a file exists in cache system
  dfcache stat --cid testid --local  # only check local cache
  dfcache stat --cid testid          # check other peers as well

- export/download a file from cache system
  dfcache export --cid testid -O /path/to/output

- delete a file from cache system, both local cache and P2P network
  dfcache delete -i testid -t testtag

Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius f26a63b96e
feat: registerTask returns to the task in time (#1250) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius b2c9577e7b
Add csv storage to scheduler (#1234) 
* feat: scheduler add csv storage for peer data
* feat: add storage configuration to scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius 913535ad7a
feat: add type to DownloadFailureCount (#1212) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:10 +08:00
Gaius f5d372bab4
feat: if the number of failed peers in the task is greater than FailedPeerCountLimit, then scheduler notifies running peers of failure (#1211) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:10 +08:00
Gaius bc1b807832
feat: optimize get available task (#1208) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:10 +08:00
Gaius e33736121f
feat: scheduler add block peers set (#1202) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:09 +08:00
Gaius 603db21e17
feat: scheduler add default biz tag (#1164) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:06 +08:00
Gaius 7adee4c2d9
feat: host remove peer (#1161) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:05 +08:00
Gaius b61865e2a9
feat: scheduler trace trigger cdn (#1147) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:04 +08:00
Gaius 7d30dd86ae
feat: add scheduler traffic metrics (#1143) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:03 +08:00
Gaius 5223cc1ce8
feat: add scheduler metrics (#1139) 
* feat: add scheduler metrics

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate validate for url meta

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:03 +08:00
Gaius addcf04df0
feat: task state for register (#1132) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:02 +08:00
Gaius 58aff0c231
feat: scheduler change piece cost from nanosecond to millisecond (#1119) 
* feat: change picec cost from nanoseconds to milliseconds

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler service

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:02 +08:00
Gaius e940af77ca
fix: scheduler piece cost time (#1118) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:01 +08:00
Gaius c5428e0657
feat: optimize scheduler log (#1114) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:01 +08:00
Gaius 0326f781f4
feat: optimize scheduler (#1106) 
* feat: optimize scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:00 +08:00
Gaius 27fcc904c0
fix: when peer state is PeerStateSucceeded, return size scope is small (#1103) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:00 +08:00
Gaius 557425249c
feat: optimize peer range (#1095) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:59 +08:00
Gaius 060429c675
feat: scheduler blocks cdn (#1079) 
* feat: scheduler blocks cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:57 +08:00
Gaius d52782ff38
feat: job trigger cdn by resource (#1076) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:57 +08:00
Gaius 6d58de0ef1
refactor: scheduler grpc server (#1046) 
* refactor: scheduler grpc server

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate mock file

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler package

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:55 +08:00
Gaius cb2f9298ce
fix: cdn trigger peer error (#1035) 
* feat: If cdn only updates IP, set cdn peers state to PeerStateLeave

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:54 +08:00
Gaius 65ad608d58
feat: If cdn only updates IP, set cdn peers state to PeerStateLeave (#1038) 
* feat: If cdn only updates IP, set cdn peers state to PeerStateLeave

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:54 +08:00
Gaius f8ebd597e2
feat: task mutex replace sync kmutex (#1000) 
* feat: trace https proxy request (#996)

* feat: trace https proxy request

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: daemon test

Signed-off-by: Jim Ma <majinjing3@gmail.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove sync kmutex and add task mutex

Signed-off-by: Gaius <gaius.qi@gmail.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
Gaius 5479b3a53e
feat: stream send error code (#986) 
* feat: change task and peer ttl (#984)

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: send error code

Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add retry interval

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon report successful piece before end of piece

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update submodule version

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:51 +08:00
Gaius 97c1eb65e7
Refactor scheduler service (#958) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:50 +08:00
sunwp 7b8aaf6b31
Refactor scheduler (#369) 
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:30:29 +08:00
Jim Ma 777e98784e
feature: export peer.TaskManager for embedding dragonfly in custom binary (#434) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:30:19 +08:00
sunwp 6cce5a9872
Feature/code spec (#336) 
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:24:44 +08:00
Gaius f9b4d77bb2
Fix golang lint (#249) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:22:38 +08:00