dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
843 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

48 Commits

Author SHA1 Message Date
Gaius f68ce30f26
feat: remove errors package (#1434) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:33 +08:00
Jim Ma 776a0fef42
feat: concurrent multiple pieces back source (#1426) 
* feat: concurrent multiple pieces back source
* chore: update http source client

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:33 +08:00
Gaius a2511cb945
feat: rewrite interface{} to any (#1419) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 0222649b88
feat: update namely/protoc-all image version to 1.47_0 (#1418) 
* feat: update namely/protoc-all image version to 1.47_0

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate mock file

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius ad36eb64a7
feat: remove github/pkg/errors package (#1416) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:31 +08:00
Gaius 7a30db3fba
refactor: pkg util (#1402) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Gaius f688d341bd
feat: remove cdn (#1401) 
* feat: remove cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* docs: generate swagger

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:29 +08:00
Jim Ma 823c722c3e
feat: support response header (#1292) 
* feat: support response header via ExtendAttribute
* chore: update http pass through header
* chore: update ExtendAttribute in cdn downloader

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:19 +08:00
Jim Ma 3682b26647
feat: in tree plugin (#1276) 
* feat: in tree plugin

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:16 +08:00
Jim Ma 010887ab66
feat: optimize proxy performance (#1137) 
* feat: optimize back source trace and logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: fix keepalive performance

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize generate peer id and transport performance

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: io.Copy call superfluous sendfile, then increase network latency

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:05 +08:00
Gaius a95edf5e0d
feat: change source client dialer config (#1115) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:01 +08:00
Gaius f16c5592ab
feat: change back-to-source timeout (#1112) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:00 +08:00
Jim Ma 4597f74283
feat: merge ranged request storage into parent (#1078) 
* feat: merge ranged request storage into parent

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:59 +08:00
Jim Ma c38bba2612
feat: merge same tasks in daemon (#977) 
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:49 +08:00
sunwp 25f8cd56dd
Fix hdfs write (#948) 
* rename reader to limitedReader

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:49 +08:00
Jim Ma beaf4ce18d
feat: update source.Response and source client interface (#945) 
* feat: update source.Response and source client interface

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
sunwp 0a152e01e9
Fix HDFS File reader (#940) 
* remove unused hdfs writeTo method

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:48 +08:00
Jim Ma 062687e4c7
feat: support recursive download in dfget (#932) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:47 +08:00
Jim Ma 705c5ed342
fix: plugin dir is empty (#922) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:46 +08:00
Jim Ma e24b1621e4
docs: update plugin docs (#921) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:46 +08:00
Eng Zer Jun f579d3c9ce
refactor: move from io/ioutil to io and os packages (#906) 
* chore: run `go fmt ./...`

This commit synchronizes `//go:build` lines with `// +build` lines.

Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
 2023-06-28 17:32:45 +08:00
Jim Ma 10ec7d2bbc
fix: go plugin test build error (#912) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:45 +08:00
sunwp eaca95d24e
Load source plugins (#894) 
* unit test

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:44 +08:00
sunwp 0096c613b0
Fix hdfs test (#902) 
* unit test

Signed-off-by: sunwp <244372610@qq.com>

* fix hdfs range test

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:44 +08:00
sunwp 23747be9d3
unit test (#898) 
Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:44 +08:00
Gaius c2c721bc78
refactor: dfpath pkg (#879) 
* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: path

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: plugin

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:43 +08:00
sunwp 929fe7db2d
Fix load source plugins (#892) 
* load source plugins

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:43 +08:00
Jim Ma ff3a542fde
chore: update UnknownSourceFileLen (#888) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:43 +08:00
sunwp 46105b0986
unify back source interface (#877) 
* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* metaData Spelling correction

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs compile

Signed-off-by: sunwp <244372610@qq.com>

* hdfs unit test

Signed-off-by: sunwp <244372610@qq.com>

* reset fqdn

Signed-off-by: sunwp <244372610@qq.com>

* new seed task with header

Signed-off-by: sunwp <244372610@qq.com>

* unify back source interface

Signed-off-by: sunwp <244372610@qq.com>

* add unit tests & rename CheckRespCode -> CheckResponseCode & rename GetLastModifiedMillis -> GetLastModified

Signed-off-by: sunwp <244372610@qq.com>

* add request test

Signed-off-by: sunwp <244372610@qq.com>

* golang lint

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:42 +08:00
Jim Ma 0ad1bf9674
Fix: source plugin not loaded (#811) 
* fix: source plugin not loaded

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:36 +08:00
Jim Ma 806f2ef11d
feat: calculate piece metadata digest (#787) 
* feat: calculate piece metadata digest
* fix: back source not work in midway
* chore: add partial back source test
* chore: optimize digest log

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:35 +08:00
Jim Ma 3d63ab2ca3
chore: update golang import lint (#780) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:34 +08:00
Gaius 52cf43a31d
feat: clear hashcircler and maputils package (#768) 
* feat: clear hashcircler and maputils package

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:33 +08:00
sunwp 359d11367e
chore: rename cdnsystem to cdn (#626) 
Signed-off-by: santong <244372610@qq.com>
 2023-06-28 17:31:46 +08:00
sunwp 376219a78b
feat: beautify scheduler & CDN log (#618) 
* feat: beautify scheduler & CDN log

Signed-off-by: santong <244372610@qq.com>
 2023-06-28 17:31:45 +08:00
zzy987 15c37f99e7
chore: add copyright (#593) 
* chore: add copyright

Signed-off-by: zzy987 <67889264+zzy987@users.noreply.github.com>

* chore: ci

Signed-off-by: zzy987 <67889264+zzy987@users.noreply.github.com>
 2023-06-28 17:31:42 +08:00
Gaius f65275d896
feat: skip verify when back to source (#586) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:31:39 +08:00
sunwp 2d8eb59005
Adjust package structure (#539) 
* feat: package structure adjust

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:31:36 +08:00
Jim Ma fdf2b688fa
feat: back source when no available peers or scheduler error (#521) 
* feature: back source when no available peers

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:31:34 +08:00
sunwp 7b8aaf6b31
Refactor scheduler (#369) 
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:30:29 +08:00
lengrongfu 4751a92db6
feat: CDN implementation supports HDFS type storage (#420) 
Signed-off-by: lengrongfu <lengrongfu@meituan.com>
 2023-06-28 17:30:27 +08:00
zzy987 402f1eb397
fix: log specification (#452) 
Signed-off-by: 张哲溢 <zhangzheyi@zhangzheyideMacBook-Pro.local>

Co-authored-by: 张哲溢 <zhangzheyi@zhangzheyideMacBook-Pro.local>
 2023-06-28 17:30:27 +08:00
sunwp 158ea2ec4a
fix: wrong cache header (#423) 
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:28:03 +08:00
Jim Ma 2d825f5a96
feat: Add plugin support for resource (#291) 
* feature: add plugin support for resource

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: skip link for mockgen import

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feature: add generate in Makefile

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: remove unused package

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: remove unused test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* feature: remove unused import package

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:28:01 +08:00
sunwp f344a0c566
Repair wrong piece info (#382) 
* cdn error definition

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>

* fix: write wrong piece info

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:27:59 +08:00
Jim Ma 5175e8009a
Fix: infinite recursion in MkDirAll (#358) 
* fix: infinite recursion in MkDirAll

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* disable manager test, enable cdn test

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:27:51 +08:00
sunwp 6cce5a9872
Feature/code spec (#336) 
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:24:44 +08:00
sunwp 155ee02a54
Add CDN unit-tests (#338) 
* feat: refactor cdn storage module & remove unused ctx & cdn test

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:22:43 +08:00