Commit Graph

23 Commits

Author SHA1 Message Date
Jim Ma c38bba2612
feat: merge same tasks in daemon (#977)
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:49 +08:00
Jim Ma beaf4ce18d
feat: update source.Response and source client interface (#945)
* feat: update source.Response and source client interface

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:48 +08:00
Jim Ma 7ba341fc5c
fix: infinitely get pieces when piece num is invalid (#926)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:47 +08:00
Eng Zer Jun f579d3c9ce
refactor: move from io/ioutil to io and os packages (#906)
* chore: run `go fmt ./...`

This commit synchronizes `//go:build` lines with `// +build` lines.

Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-28 17:32:45 +08:00
sunwp 46105b0986
unify back source interface (#877)
* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* metaData Spelling correction

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs unit test

Signed-off-by: sunwp <244372610@qq.com>

* reset fqdn

Signed-off-by: sunwp <244372610@qq.com>

* new seed task with header

Signed-off-by: sunwp <244372610@qq.com>

* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* add unit tests & rename CheckRespCode -> CheckResponseCode & rename GetLastModifiedMillis -> GetLastModified

Signed-off-by: sunwp <244372610@qq.com>

* add request test

Signed-off-by: sunwp <244372610@qq.com>

* golang lint

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:42 +08:00
Gaius f6ce817b72
feat: move dfnet to internal (#862)
* feat: move dfnet to internal

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename dfutils to util

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
sunwp 44730a2e46
unify piece size (#854)
* unify piece size

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:40 +08:00
sunwp c1d84fb56d
validate grpc model (#825)
* base validate

Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:40 +08:00
Jim Ma 3d63ab2ca3
chore: update golang import lint (#780)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:34 +08:00
sunwp 359d11367e
chore: rename cdnsystem to cdn (#626)
Signed-off-by: santong <244372610@qq.com>
2023-06-28 17:31:46 +08:00
Jim Ma af99b88616
feat: support limit from dfget client (#578)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:39 +08:00
Jim Ma 2120051ab0
feature: enable grpc tracing (#531)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:34 +08:00
Jim Ma 94e35fa9df
chore: optimize compute piece size function (#528)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:34 +08:00
Jim Ma fdf2b688fa
feat: back source when no available peers or scheduler error (#521)
* feature: back source when no available peers

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:31:34 +08:00
Gaius 27103e4b4e
feat: rename manager grpc (#510)
* feat: rename manager grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:26 +08:00
lengrongfu 4751a92db6
feat: CDN implementation supports HDFS type storage (#420)
Signed-off-by: lengrongfu <lengrongfu@meituan.com>
2023-06-28 17:30:27 +08:00
Jim Ma 777e98784e
feature: export peer.TaskManager for embedding dragonfly in custom binary (#434)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:30:19 +08:00
Jim Ma 5175e8009a
Fix: infinite recursion in MkDirAll (#358)
* fix: infinite recursion in MkDirAll

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* disable manager test, enable cdn test

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:27:51 +08:00
sunwp 6cce5a9872
Feature/code spec (#336)
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:24:44 +08:00
Jim Ma 5b9c3033f8
Support peer task data multiplex (#347)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:23:35 +08:00
sunwp 155ee02a54
Add CDN unit-tests (#338)
* feat: refactor cdn storage module & remove unused ctx & cdn test

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:22:43 +08:00
Jim Ma 3955a7d716
daemon: update default timeout and add context for downloading piece (#190)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:21:33 +08:00
Jim Ma 0af8817c1f
Merge branch main-rc to branch main
Signed-off-by: Jim Ma <majinjing3@gmail.com>

Co-authored-by: 李玉海 <yuhai.lyh@alibaba-inc.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
Co-authored-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
Co-authored-by: 加菲 <garfieldhu@aliyun.com>
2023-06-28 17:20:20 +08:00