0ffc55b6d4
refactor: trainer server module ( #2486 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:35:00 +08:00
e20bb9743e
feat: add trainer cmd and trainer service ( #2479 )
...
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:59 +08:00
bc4ca6270b
test: add unit test for ComputePieceCount ( #2401 )
...
Signed-off-by: Zou Yu <zouy.fnst@fujitsu.com>
2023-06-28 17:34:54 +08:00
b80b4acbe4
feat: optimize job new in internal ( #2341 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:49 +08:00
a5685582ce
refactor: resource task with v2 version of grpc ( #2078 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:30 +08:00
2c8ecac6e2
fix: dferror not convert ( #2001 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:24 +08:00
0546fe4898
chore: create log dir ( #1947 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:20 +08:00
ffa80c1eb9
refactor: dynconfig without Unmarshal ( #1926 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:18 +08:00
fa11e30b99
fix machinery redis broker address format err ( #1916 )
2023-06-28 17:34:17 +08:00
f120c8778b
feat: support redis sentinal ( #1910 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:16 +08:00
d64a6d8e29
feat: trigger task with priority ( #1904 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:16 +08:00
10ae70db79
feat: change log rotate size ( #1879 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:14 +08:00
2421ce88f4
feat: add announcer to dfdameon ( #1852 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
42223d994f
feat: optimize peer log ( #1828 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
d2f98d1429
feat: optimize preheat log ( #1827 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
3b1e805ce6
chore: add timestamp to stdout&stderr ( #1781 )
...
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
2023-06-28 17:34:05 +08:00
27d8cf51db
feat: gin logger rotation ( #1749 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:02 +08:00
29d4452277
feat: support redis cluster ( #1667 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:54 +08:00
13580f89f2
refactor: dfpath for certify cache dir ( #1618 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
98fb1fc427
feat: add grpc mux transport ( #1602 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:49 +08:00
5bcb53f0a9
feat: grpc consistent hashing ( #1554 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: bigerous <bigerous@qq.com>
2023-06-28 17:33:44 +08:00
439ad71230
feat: gc removes logrus ( #1548 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:44 +08:00
c216872b5d
feat: store preheat result ( #1516 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:41 +08:00
d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api ( #1515 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:41 +08:00
776a0fef42
feat: concurrent multiple pieces back source ( #1426 )
...
* feat: concurrent multiple pieces back source
* chore: update http source client
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:33 +08:00
a2511cb945
feat: rewrite interface{} to any ( #1419 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
0222649b88
feat: update namely/protoc-all image version to 1.47_0 ( #1418 )
...
* feat: update namely/protoc-all image version to 1.47_0
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: generate mock file
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
ad36eb64a7
feat: remove github/pkg/errors package ( #1416 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:31 +08:00
f688d341bd
feat: remove cdn ( #1401 )
...
* feat: remove cdn
Signed-off-by: Gaius <gaius.qi@gmail.com>
* docs: generate swagger
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:29 +08:00
233aac29db
feat: redirect stdout and stderr to file ( #1399 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:29 +08:00
866eafe2c7
feat: remove preheat tag validate with required ( #1363 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:26 +08:00
e2eb30bc96
feat: change pattern from cdn to seed peer and remove kustomize shell ( #1345 )
...
* feat: change pattern from cdn to seed peer and remove kustomize shell
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:24 +08:00
faa5e4e465
feat: add seed peer logic ( #1302 )
...
* feat: announce seed peer
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn logic
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn job
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dfdaemon change host uuid to host id
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: go generate mocks
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove cdn compatibility
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: change docker compose
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: reuse panic
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: compatible with v2.0.3-beta.2
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:19 +08:00
076ba20b51
feat: move dfnet to pkg dir ( #1265 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:15 +08:00
65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache ( #1227 )
...
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.
For example:
- add a file into cache system
dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file
- check if a file exists in cache system
dfcache stat --cid testid --local # only check local cache
dfcache stat --cid testid # check other peers as well
- export/download a file from cache system
dfcache export --cid testid -O /path/to/output
- delete a file from cache system, both local cache and P2P network
dfcache delete -i testid -t testtag
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
517ed6c51e
fix: client sync piece panic ( #1246 )
...
* fix: client sync piece panic
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: optimize sync piece log
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: daemon panic when register fail
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:13 +08:00
7167c01d0f
add header check for ClientOption ( #1174 )
...
Signed-off-by: yxxhero <aiopsclub@163.com>
2023-06-28 17:33:07 +08:00
78591d0309
Optimized the log creation ( #1058 )
...
* optimize logger
Signed-off-by: sunwp <244372610@qq.com>
* resolve rpc server logger panic
Signed-off-by: sunwp <244372610@qq.com>
* resolve rpc server logger panic
Signed-off-by: sunwp <244372610@qq.com>
* reset levels nil to release init method level
Signed-off-by: sunwp <244372610@qq.com>
* golang lint
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:06 +08:00
98e95e693c
feat: implement bidirectional fetch pieces ( #1165 )
...
* feat: implement grpc server side sync pieces
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:06 +08:00
266da57866
Reduce default RotateMaxBackups count ( #1162 )
...
* reduce default RotateMaxBackups count
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:05 +08:00
08d5f10517
feat: set grpc logger verbosity from env variable ( #1111 )
...
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:00 +08:00
f80c75efdb
Feature: prefetch ranged requests ( #1053 )
...
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:56 +08:00
cc5a43c73f
feat: support change console log level ( #1055 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:56 +08:00
009fbc6db1
Feature: change log level in-flight ( #1023 )
...
* feat: change log level in flight
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:54 +08:00
97c1eb65e7
Refactor scheduler service ( #958 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:50 +08:00
0b21f66714
feat: remove stat log from scheduler ( #946 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:47 +08:00
a1ab42790f
Optimaze GC log ( #923 )
...
* optimaze gc log
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:47 +08:00
7965ed70fe
Set totalPieceCount when the file length is available ( #930 )
...
* set totalPieceCount when the file length is available
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:47 +08:00
cb4202319e
feat: make idgen package public ( #931 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:47 +08:00
1cde964259
feat: make dfpath public ( #929 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:46 +08:00
Gaius