dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,653 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

379 Commits

Author SHA1 Message Date
Gaius af76512b2a
feat: update version guage metrics (#1927) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius 91454c8a60
feat: remove callsystem and pattern (#1925) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:17 +08:00
cuidajun 88f8f974ab
feat: client support 'priority' parameter (#1911) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:17 +08:00
Gaius e1b89dc6f0
feat: add health check to service (#1889) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Jim Ma f528df2b0a
chore: optimize reregister (#1888) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:14 +08:00
Jim Ma 0a8f5118c9
feat: support reregister peer task in client (#1876) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:13 +08:00
Gaius cf50824526
feat: change announcer validation (#1869) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:13 +08:00
Gaius ef6ad12587
feat: add announcer to scheduler (#1854) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:12 +08:00
Gaius 2421ce88f4
feat: add announcer to dfdameon (#1852) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Gaius f194f0961f
feat: scheduler adds announce host handler (#1843) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Jim Ma 7e90f43a53
chore: update dst peer log (#1844) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:11 +08:00
Jim Ma 3fe342dabc
fix: recursive download always return none error (#1841) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:10 +08:00
Jim Ma 1c24c3fc8c
fix: expire header timezone (#1840) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:10 +08:00
Jim Ma ba58047871
feat: optional save list metadata to p2p (#1822) 
* feat: save list metadata to p2p
* chore: optimize rpc download worker logic
* feat: add expire header reclaim logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius 5c25a325c6
feat: seed peer reuses traffic (#1825) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Jim Ma f581d10208
chore: daemon avoid alway open metadata files (#1823) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius 3da67311ea
feat: optimize preheat (#1824) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Jim Ma 9378ce410a
chore: close out of use client grpc conn (#1817) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:08 +08:00
Jim Ma 6fd389c27a
fix: otel goroutine leak (#1815) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:08 +08:00
Jim Ma 8749845aae
chore: make SendMsg in doRecursiveDownload safe (#1806) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:07 +08:00
Jim Ma 0e83c7f58f
feat: add otel trace in log (#1804) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:07 +08:00
Jim Ma 2a7e576778
chore: add list log in rpc download (#1802) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:07 +08:00
Jim Ma 876c2ae17e
feat: support split running tasks (#1794) 
* feat: support split running tasks

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: go tests

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: add split runnging tasks e2e config

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:06 +08:00
Gaius 0c1cf2619d
feat: add download header log (#1793) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
cuidajun 502c01f9aa
fix: daemon don't leaveHost when keepStorage=true (#1790) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:06 +08:00
cuidajun 536b039462
feat: daemon call leaveHost when exit (#1788) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>

Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:06 +08:00
Gaius 446fcc9fd7
feat: add LeaveHost handler (#1780) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
Jim Ma 32eaba3ffb
fix: did not call scheduler leave tasks in forceGC (#1782) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:05 +08:00
Jim Ma 7ab0a979cd
chore: check reuse file (#1765) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:03 +08:00
Jim Ma 55d5308b19
fix: open end range in concurrent back source (#1764) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:03 +08:00
Gaius 649b652a4f
refactor: obs of objectstorage pkg (#1762) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:03 +08:00
Jim Ma 0bca9dd549
chore: update roundtrip log (#1750) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:02 +08:00
Gaius 27d8cf51db
feat: gin logger rotation (#1749) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:02 +08:00
Jim Ma 2d46f6424a
feat: update source temporary error logic (#1739) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:01 +08:00
Jim Ma ef61df0027
chore: make lru cache safe (#1737) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:01 +08:00
cuidajun c6119d6ce0
fix: backsource temporary error judgement (#1726) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>

Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:00 +08:00
Jim Ma c9bfe9eb08
feat: unregister failed task storage (#1717) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:00 +08:00
cuidajun 9dd9bc5c9f
chore: change disk usage debug log format to decimal (#1727) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:00 +08:00
Jim Ma 491b309620
feat: support concurrent recursive download (#1714) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:59 +08:00
Gaius 9573d5a659
fix: docker compose config (#1713) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:59 +08:00
greenhandatsjtu bc8d623b3d
feat: add traffic shaper for download tasks (#1654) 
Signed-off-by: greenhandatsjtu <sunhengke@sjtu.edu.cn>
 2023-06-28 17:33:59 +08:00
Gaius 9f098362fb
refactor: pkg basic (#1712) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:58 +08:00
Gaius 7cc842d64d
test: remove test main (#1710) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:58 +08:00
zhaoshang fda6002aac
test: add test for daemon rpcserver (#1704) 
Signed-off-by: zhaoshang <zhaoshangsjtu@linux.alibaba.com>
 2023-06-28 17:33:58 +08:00
Jim Ma 89376d6c4e
feat: ipv6 support (#1685) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:57 +08:00
Jim Ma c5f4ca50fa
fix: grpc download tidy file error (#1697) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:57 +08:00
Jim Ma 73de99353d
feat: empty file e2e (#1687) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:56 +08:00
Jim Ma f4d34e56ed
chore: update download rpc check (#1684) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:56 +08:00
Jim Ma 40300990de
feat: support download empty file (#1686) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:56 +08:00
Gaius c9755aabfc
feat: stop grpc client (#1671) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:55 +08:00
Gaius 198613a357
chore: add disable seed peer action (#1653) 
* chore: add disable seed peer action
* fix: concurrent back source

Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:54 +08:00
Jim Ma 7bceca51bc
chore: update debug info (#1617) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:51 +08:00
Gaius b6c395a6cb
feat: add cert spec to security configuration (#1621) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:51 +08:00
Jim Ma 56ac9d1801
feat: support mutate all proxy requests (#1623) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 13580f89f2
refactor: dfpath for certify cache dir (#1618) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 43d8478507
feat: add tls policy to scheduler grpc server (#1616) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Jim Ma 9050230926
feat: refactor grpc credential (#1612) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 346cc0649e
feat: add tls policy constants (#1610) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:49 +08:00
Jim Ma 98fb1fc427
feat: add grpc mux transport (#1602) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:49 +08:00
Gaius 9ab33635c5
feat: manager init cert for grpc server (#1603) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:49 +08:00
Jim Ma 7224801bae
feat: refactor peertask option (#1600) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma ecfe467585
feat: add client grpc dial timeout (#1599) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma c25cfdd651
feat: support multiple certify cache (#1598) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma 717ac99ef5
feat: daemon support auto issue certificate (#1586) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Gaius ad553d949b
feat: grpc dial adds context (#1594) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:47 +08:00
Gaius 499d2fb886
feat: remove golang +build tag (#1585) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:47 +08:00
Gaius 2c48148ac8
fix: dfdaemon can not shutdown (#1580) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius f631263d25
refactor: dfnet package (#1578) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius a6a44269b4
refactor: dfdaemon client and remove rpc connection pool (#1576) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius 272f38f567
feat: grpc add ratelimit (#1572) 
feat: grpc ratelimit

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius 69a51808d2
feat: refresh dynconfig addresses when grpc requests unavailable (#1571) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius 631654f98b
feat: manager client add context (#1562) 
Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:45 +08:00
cuidajun 242fb2bc37
add application-level statistics (#1559) 
Signed-off-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:45 +08:00
Gaius 5bcb53f0a9
feat: grpc consistent hashing (#1554) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:44 +08:00
Jim Ma 25e929142f
chore: optimize source error log (#1553) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:44 +08:00
Gaius 2ce864fbf6
fix: scheduler and manager tracing (#1551) 
fix: scheduler and manager trace

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:44 +08:00
Jim Ma baa64bb0a3
feat: add seed trace (#1549) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:44 +08:00
Gaius 740696a171
feat: when the request has a range header, object storage is no need to to calculate md5 (#1534) 
feat: when the request has a range header, object storage is no need to calculate md5

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:42 +08:00
cuidajun 1258175366
fix daemon socket remove error (#1529) 2023-06-28 17:33:42 +08:00
Jim Ma f6506115a7
feat: support grpc recursive download (#1518) 
* feat: daemon grpc recursive download
* fix: init storage error

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:42 +08:00
Jim Ma 4984cdfbbc
fix: check same peer id for sync pieces (#1525) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:42 +08:00
Gaius e42186acb7
feat: add daemon-socket-path (#1521) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:42 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:41 +08:00
Jim Ma ddfdb24dbf
fix: auto switch to concurrent back source (#1507) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:40 +08:00
greenhandatsjtu acf4922866
feat: auto switch to concurrent back source based on download speed (#1494) 
Signed-off-by: greenhandatsjtu <sunhengke@sjtu.edu.cn>
 2023-06-28 17:33:40 +08:00
Jim Ma 3b539f9abf
fix: wait first peer packet fail (#1500) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:39 +08:00
cuidajun de35b14db9
fix: one piece task sometimes backsource after succeed (#1499) 
Signed-off-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:39 +08:00
Gaius 0bd894adad
fix: dfdaemon unix socket (#1489) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:38 +08:00
Jim Ma c921c265c5
fix: init storage error (#1486) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:38 +08:00
Jim Ma 59060ef277
fix: back source error (#1485) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>

Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:38 +08:00
Gaius 7d3c9d539d
refactor: cache key for peer (#1483) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:38 +08:00
Gaius ed4c381383
feat: rename steal peers to candidate peers (#1476) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:37 +08:00
Gaius f9d2736ae8
feat: scheduler merge end of piece and piece from seed peer (#1474) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:37 +08:00
cuidajun abec6d3a84
fix: upload_manager write header in time (#1471) 
Signed-off-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:37 +08:00
Gaius 4d71217572
feat: generate dag mock 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:36 +08:00
cuidajun c6654cb4ae
fix: upload_manager write header immediately when it is ready (#1466) 
Signed-off-by: bigerous <bigerous@qq.com>
 2023-06-28 17:33:36 +08:00
1037husterljx 31032e05aa
fix [4000] digest not set error (#1465) 
Signed-off-by: ljx373327 <ljx373327@alibaba-inc.com>

Co-authored-by: ljx373327 <ljx373327@alibaba-inc.com>
 2023-06-28 17:33:36 +08:00
Gaius c6a019560d
feat: proxy add defaultTag field (#1462) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:36 +08:00
Gaius 601993aa46
feat: use os.PathSeparator to generate object key 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:35 +08:00
Gaius 812ff5557b
fix: metrics reduces labels (#1457) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:35 +08:00