Commit Graph

625 Commits

Author SHA1 Message Date
sunwp 67d6fb970f
Fix empty client output path (#1159)
* If the client output parameter is not specified, the client cannot automatically obtain the target file path

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:06 +08:00
Jim Ma 98e95e693c
feat: implement bidirectional fetch pieces (#1165)
* feat: implement grpc server side sync pieces

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:06 +08:00
Gaius 603db21e17
feat: scheduler add default biz tag (#1164)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:06 +08:00
Jim Ma 010887ab66
feat: optimize proxy performance (#1137)
* feat: optimize back source trace and logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: fix keepalive performance

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize generate peer id and transport performance

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: io.Copy call superfluous sendfile, then increase network latency

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:05 +08:00
sunwp f3c1080dd9
Optimize CDN GC Debug log (#1163)
* remove CDN gc Debug log

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:05 +08:00
Gaius 7adee4c2d9
feat: host remove peer (#1161)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:05 +08:00
sunwp 266da57866
Reduce default RotateMaxBackups count (#1162)
* reduce default RotateMaxBackups count

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:05 +08:00
Gaius b3103f48ed
test: cache expire interval (#1160)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:05 +08:00
Gaius 4e670035ed
feat: change reschdule config (#1158)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:05 +08:00
Gaius 6aa27db850
chore: update go mod (#1156)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:05 +08:00
Gaius 9beaf03fdf
chore: add makefile note (#1155)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:05 +08:00
crasybest 3279173f5e
Update dfdaemon.md (#1131)
Change dfget start command
2023-06-28 17:33:05 +08:00
Gaius 4ab498837c
test: add scheduler constructSuccessPeerPacket case (#1154)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
Gaius 1bdac7e063
feat: update git submodule (#1153)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
sunwp 8bd1ac713c
Remove residues left by E2E tests (#1142)
* clean E2E test log

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:04 +08:00
sunwp ff134d0a48
fix daemon scheduler option validate bug (#1152)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:04 +08:00
Gaius 1ea700f421
feat: scheduler metrics add default value of biz tag (#1151)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
Gaius ef6e3665e6
feat: add user update interface and rename rest to service (#1148)
* feat: add user interface and rename rest to service

Signed-off-by: Gaius <gaius.qi@gmail.com>

* docs: api reference

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
Gaius b61865e2a9
feat: scheduler trace trigger cdn (#1147)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
Gaius eb408d136f
test: scheduler service handlePieceFail (#1146)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:04 +08:00
sunwp 2a33ff6005
lower cdn fullGC threshould (#1144)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:03 +08:00
Gaius 7d30dd86ae
feat: add scheduler traffic metrics (#1143)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
Gaius 98e23e4be7
feat: update otel package version and fix otelgrpc goroutine leak (#1141)
* feat: update otel version

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: grpc.WithInsecure is deprecated

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
sunwp 44dd19ec12
Extend conn GC time (#1130)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:03 +08:00
Gaius 07bae54063
chore: change scheduler config (#1140)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
Gaius 5223cc1ce8
feat: add scheduler metrics (#1139)
* feat: add scheduler metrics

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate validate for url meta

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
AnJia de4c628089
fix: docker-compose run with mac throw error (#1134)
Signed-off-by: 赵安家 <anjia0532@gmail.com>
2023-06-28 17:33:03 +08:00
sunwp a9be3408e7
resolve forget unlock resource lack lock (#1138)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:03 +08:00
Gaius 8f19c11f54
feat: scheduler remove inactive host (#1135)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:03 +08:00
Gaius addcf04df0
feat: task state for register (#1132)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:02 +08:00
Jim Ma bb73b45b46
fix: wrong md5 sign in cdn (#1126)
* fix: wrong md5 sign in cdn

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* Extract the pieces sorting rule to the GetProgress method

Signed-off-by: sunwp <244372610@qq.com>

* cal sign notice

Signed-off-by: sunwp <244372610@qq.com>

Co-authored-by: sunwp <244372610@qq.com>
2023-06-28 17:33:02 +08:00
Gaius 7d778d7b95
feat: change grpc client keepalive config (#1125)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:02 +08:00
sunwp 4a752a47da
fix concurrent piece map panic in cdn (#1121)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:02 +08:00
Jim Ma 84eaf402c3
chore: fast back source when get pieces task failed (#1123)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:02 +08:00
AnJia d939824c5f
fix: docker-compose preheat pending (#1124)
Signed-off-by: 赵安家 <anjia0532@gmail.com>
2023-06-28 17:33:02 +08:00
sunwp 17e3f933c4
get piece tasks log (#974)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:02 +08:00
sunwp bd29967c39
use os.stat replace custom stat function (#1109)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:02 +08:00
Gaius 58aff0c231
feat: scheduler change piece cost from nanosecond to millisecond (#1119)
* feat: change picec cost from nanoseconds to milliseconds

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler service

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:02 +08:00
Jim Ma ebc1aff720
feat: support health probe in daemon (#1120)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:01 +08:00
Gaius e940af77ca
fix: scheduler piece cost time (#1118)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:01 +08:00
sunwp 0b0dd27ff8
add CDN piece download cost (#966)
Signed-off-by: sunwp <244372610@qq.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:01 +08:00
Gaius 8ba4a12fcd
feat: when peer downloads finished, peer deletes parent (#1116)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:01 +08:00
Gaius a95edf5e0d
feat: change source client dialer config (#1115)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:01 +08:00
Gaius c5428e0657
feat: optimize scheduler log (#1114)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:01 +08:00
Jim Ma 977f095c49
chore: optimize reuse logic (#1110)
* chore: optimize reuse logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:01 +08:00
Gaius a5f385cd99
feat: remove needless manager grpc proxy (#1113)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:00 +08:00
sunwp 08d5f10517
feat: set grpc logger verbosity from env variable (#1111)
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:33:00 +08:00
Gaius f16c5592ab
feat: change back-to-source timeout (#1112)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:00 +08:00
Gaius 0326f781f4
feat: optimize scheduler (#1106)
* feat: optimize scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:00 +08:00
Jim Ma ac4c4bb912
feat: reuse partial completed task (#1107)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:00 +08:00