dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,907 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

109 Commits

Author SHA1 Message Date
Guangwen Feng 229b942bea
test: add unit test for MarshalResponse (#2683) 
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
 2023-08-29 23:27:47 +08:00
Gaius fe28ba4c8e
feat: merge sync peer with peer table in manager (#2668) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-08-24 10:28:08 +08:00
Gaius dfde8bdce6
feat: add sync peer job for scheduler (#2663) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-08-22 19:58:27 +08:00
Gaius 05a66805cc
fix: directories created via os.MkdirAll are not checked for permissions (#2613) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-08-09 13:52:26 +08:00
Gaius b3f23f71a3
feat: update dy7.io/api to v2 (#2558) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-07-20 23:17:57 +08:00
Gaius c7e43be4e1
feat: add training service (#2543) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-07-12 17:19:25 +08:00
Gaius a7f3c7c0b9
feat: add GNNModelIDV1 and MLPModelIDV1 in idgen (#2540) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-07-12 14:32:41 +08:00
Gaius 0ffc55b6d4
refactor: trainer server module (#2486) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:35:00 +08:00
dlut_xz e20bb9743e
feat: add trainer cmd and trainer service (#2479) 
Signed-off-by: XZ <834756128@qq.com>
 2023-06-28 17:34:59 +08:00
ZouYu bc4ca6270b
test: add unit test for ComputePieceCount (#2401) 
Signed-off-by: Zou Yu <zouy.fnst@fujitsu.com>
 2023-06-28 17:34:54 +08:00
Gaius b80b4acbe4
feat: optimize job new in internal (#2341) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:49 +08:00
Gaius a5685582ce
refactor: resource task with v2 version of grpc (#2078) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:30 +08:00
Jim Ma 2c8ecac6e2
fix: dferror not convert (#2001) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:24 +08:00
Jim Ma 0546fe4898
chore: create log dir (#1947) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:20 +08:00
Gaius ffa80c1eb9
refactor: dynconfig without Unmarshal (#1926) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
walleliu fa11e30b99
fix machinery redis broker address format err (#1916) 2023-06-28 17:34:17 +08:00
Gaius f120c8778b
feat: support redis sentinal (#1910) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius d64a6d8e29
feat: trigger task with priority (#1904) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:16 +08:00
Gaius 10ae70db79
feat: change log rotate size (#1879) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Gaius 2421ce88f4
feat: add announcer to dfdameon (#1852) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Gaius 42223d994f
feat: optimize peer log (#1828) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius d2f98d1429
feat: optimize preheat log (#1827) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
cuidajun 3b1e805ce6
chore: add timestamp to stdout&stderr (#1781) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:05 +08:00
Gaius 27d8cf51db
feat: gin logger rotation (#1749) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:02 +08:00
Gaius 29d4452277
feat: support redis cluster (#1667) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:54 +08:00
Gaius 13580f89f2
refactor: dfpath for certify cache dir (#1618) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Jim Ma 98fb1fc427
feat: add grpc mux transport (#1602) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:49 +08:00
Gaius 5bcb53f0a9
feat: grpc consistent hashing (#1554) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:44 +08:00
Gaius 439ad71230
feat: gc removes logrus (#1548) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:44 +08:00
Gaius c216872b5d
feat: store preheat result (#1516) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:41 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:41 +08:00
Jim Ma 776a0fef42
feat: concurrent multiple pieces back source (#1426) 
* feat: concurrent multiple pieces back source
* chore: update http source client

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:33 +08:00
Gaius a2511cb945
feat: rewrite interface{} to any (#1419) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 0222649b88
feat: update namely/protoc-all image version to 1.47_0 (#1418) 
* feat: update namely/protoc-all image version to 1.47_0

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate mock file

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius ad36eb64a7
feat: remove github/pkg/errors package (#1416) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:31 +08:00
Gaius f688d341bd
feat: remove cdn (#1401) 
* feat: remove cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* docs: generate swagger

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius 233aac29db
feat: redirect stdout and stderr to file (#1399) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius 866eafe2c7
feat: remove preheat tag validate with required (#1363) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:26 +08:00
Gaius e2eb30bc96
feat: change pattern from cdn to seed peer and remove kustomize shell (#1345) 
* feat: change pattern from cdn to seed peer and remove kustomize shell

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:24 +08:00
Gaius faa5e4e465
feat: add seed peer logic (#1302) 
* feat: announce seed peer

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn logic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn job

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon change host uuid to host id

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: go generate mocks

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn compatibility

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change docker compose

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: reuse panic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: compatible with v2.0.3-beta.2

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:19 +08:00
Gaius 076ba20b51
feat: move dfnet to pkg dir (#1265) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:15 +08:00
Eryu Guan 65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache (#1227) 
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.

For example:
- add a file into cache system
  dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file

- check if a file exists in cache system
  dfcache stat --cid testid --local  # only check local cache
  dfcache stat --cid testid          # check other peers as well

- export/download a file from cache system
  dfcache export --cid testid -O /path/to/output

- delete a file from cache system, both local cache and P2P network
  dfcache delete -i testid -t testtag

Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Jim Ma 517ed6c51e
fix: client sync piece panic (#1246) 
* fix: client sync piece panic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize sync piece log

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: daemon panic when register fail

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:13 +08:00
yxxhero 7167c01d0f
add header check for ClientOption (#1174) 
Signed-off-by: yxxhero <aiopsclub@163.com>
 2023-06-28 17:33:07 +08:00
sunwp 78591d0309
Optimized the log creation (#1058) 
* optimize logger

Signed-off-by: sunwp <244372610@qq.com>

* resolve rpc server logger panic

Signed-off-by: sunwp <244372610@qq.com>

* resolve rpc server logger panic

Signed-off-by: sunwp <244372610@qq.com>

* reset levels nil to release init method level

Signed-off-by: sunwp <244372610@qq.com>

* golang lint

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:33:06 +08:00
Jim Ma 98e95e693c
feat: implement bidirectional fetch pieces (#1165) 
* feat: implement grpc server side sync pieces

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:06 +08:00
sunwp 266da57866
Reduce default RotateMaxBackups count (#1162) 
* reduce default RotateMaxBackups count

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:33:05 +08:00
sunwp 08d5f10517
feat: set grpc logger verbosity from env variable (#1111) 
Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:33:00 +08:00
Jim Ma f80c75efdb
Feature: prefetch ranged requests (#1053) 
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:56 +08:00
Jim Ma cc5a43c73f
feat: support change console log level (#1055) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:56 +08:00