dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,653 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

446 Commits

Author SHA1 Message Date
Gaius d71c90f138
fix: manager typo (#1995) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:24 +08:00
cuidajun acdb05ccba
fix: daemon recognize Code_SchedForbidden (#1994) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:24 +08:00
Gaius c89234e976
feat: add manager v2 api (#1990) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:23 +08:00
cuidajun 81865e6ba7
fix: client bitMap extend capacity (#1973) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:22 +08:00
Gaius e4fc6d5c85
feat: add S3ForcePathStyle to object storage (#1976) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:22 +08:00
cuidajun 2833353d0d
fix:scheduler's health client not close (#1975) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:22 +08:00
Jim Ma 004d237537
feat: corrupt data check (#1946) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:20 +08:00
Jim Ma 9c8d82e294
feat: create synchronizers concurrently (#1941) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Jim Ma 56e668bb0d
chore: optimize download log (#1944) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Jim Ma f402824618
fix: download context cancelled (#1942) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Jim Ma 7cb802aeb7
feat: remove legacy peers support (#1939) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:19 +08:00
Gaius c3882ccfc6
fix: peer keepalive with manager (#1940) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:19 +08:00
Gaius 008eeb980b
feat: add IPAddresses and DNSNames to sans of the cert (#1930) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius 731ecc1234
feat: change yaml field type from string to net.IP (#1929) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Jim Ma ff79c3148a
feat: random pieces download (#1918) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius af76512b2a
feat: update version guage metrics (#1927) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius ffa80c1eb9
refactor: dynconfig without Unmarshal (#1926) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:18 +08:00
Gaius 91454c8a60
feat: remove callsystem and pattern (#1925) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:17 +08:00
cuidajun 88f8f974ab
feat: client support 'priority' parameter (#1911) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:17 +08:00
Gaius e1b89dc6f0
feat: add health check to service (#1889) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:14 +08:00
Jim Ma f528df2b0a
chore: optimize reregister (#1888) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:14 +08:00
Jim Ma 0a8f5118c9
feat: support reregister peer task in client (#1876) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:13 +08:00
Gaius cf50824526
feat: change announcer validation (#1869) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:13 +08:00
Gaius a1b65218db
feat: optimize gc package (#1855) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:12 +08:00
Gaius ef6ad12587
feat: add announcer to scheduler (#1854) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:12 +08:00
Gaius 2421ce88f4
feat: add announcer to dfdameon (#1852) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Gaius 43b9c39149
feat: when dfdaemon disable object storage, dynconfig can't fetch manager (#1845) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Gaius f194f0961f
feat: scheduler adds announce host handler (#1843) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:11 +08:00
Jim Ma 7e90f43a53
chore: update dst peer log (#1844) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:11 +08:00
Jim Ma 3fe342dabc
fix: recursive download always return none error (#1841) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:10 +08:00
Jim Ma 1c24c3fc8c
fix: expire header timezone (#1840) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:10 +08:00
Gaius ca9afc5b9d
feat: update concurrent-map version (#1837) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:10 +08:00
Gaius 0f8b75808a
feat: add timeout grpc and job (#1830) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:10 +08:00
Jim Ma ba58047871
feat: optional save list metadata to p2p (#1822) 
* feat: save list metadata to p2p
* chore: optimize rpc download worker logic
* feat: add expire header reclaim logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:09 +08:00
Jim Ma 5040af8b4f
feat: add s3 resource client and recursive e2e test (#1826) 
* feat: add s3 resource client
* chore: add minio server and recursive e2e test
* chore: add download grpc test tool

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius 5c25a325c6
feat: seed peer reuses traffic (#1825) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Jim Ma f581d10208
chore: daemon avoid alway open metadata files (#1823) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:09 +08:00
Gaius 3da67311ea
feat: optimize preheat (#1824) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:09 +08:00
Jim Ma 9378ce410a
chore: close out of use client grpc conn (#1817) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:08 +08:00
Jim Ma 6fd389c27a
fix: otel goroutine leak (#1815) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:08 +08:00
Gaius 4bd7d413f2
feat: add workHome and pluginDir to configuration (#1807) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:08 +08:00
Jim Ma 8749845aae
chore: make SendMsg in doRecursiveDownload safe (#1806) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:07 +08:00
Jim Ma 0e83c7f58f
feat: add otel trace in log (#1804) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:07 +08:00
Jim Ma 2a7e576778
chore: add list log in rpc download (#1802) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:07 +08:00
Jim Ma 876c2ae17e
feat: support split running tasks (#1794) 
* feat: support split running tasks

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: go tests

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: add split runnging tasks e2e config

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: actions

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize get running tasks logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:06 +08:00
Gaius 0c1cf2619d
feat: add download header log (#1793) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
cuidajun 502c01f9aa
fix: daemon don't leaveHost when keepStorage=true (#1790) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:06 +08:00
cuidajun 536b039462
feat: daemon call leaveHost when exit (#1788) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>

Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:06 +08:00
Gaius 446fcc9fd7
feat: add LeaveHost handler (#1780) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:06 +08:00
Jim Ma 32eaba3ffb
fix: did not call scheduler leave tasks in forceGC (#1782) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:05 +08:00
Jim Ma 7ab0a979cd
chore: check reuse file (#1765) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:03 +08:00
Jim Ma 55d5308b19
fix: open end range in concurrent back source (#1764) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:03 +08:00
Gaius 649b652a4f
refactor: obs of objectstorage pkg (#1762) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:03 +08:00
Jim Ma 0bca9dd549
chore: update roundtrip log (#1750) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:02 +08:00
Gaius 27d8cf51db
feat: gin logger rotation (#1749) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:02 +08:00
Jim Ma 2d46f6424a
feat: update source temporary error logic (#1739) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:01 +08:00
Jim Ma ef61df0027
chore: make lru cache safe (#1737) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:01 +08:00
cuidajun c6119d6ce0
fix: backsource temporary error judgement (#1726) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>

Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:00 +08:00
Gaius 327cf4493e
feat: add TaskStateLeave to task (#1728) 
Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:34:00 +08:00
Jim Ma c9bfe9eb08
feat: unregister failed task storage (#1717) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:34:00 +08:00
cuidajun 9dd9bc5c9f
chore: change disk usage debug log format to decimal (#1727) 
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
 2023-06-28 17:34:00 +08:00
Gaius 58f7bbbb9f
refactor: idgen pkg (#1715) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:59 +08:00
Jim Ma 491b309620
feat: support concurrent recursive download (#1714) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:59 +08:00
Gaius 9573d5a659
fix: docker compose config (#1713) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:59 +08:00
greenhandatsjtu bc8d623b3d
feat: add traffic shaper for download tasks (#1654) 
Signed-off-by: greenhandatsjtu <sunhengke@sjtu.edu.cn>
 2023-06-28 17:33:59 +08:00
Gaius 9f098362fb
refactor: pkg basic (#1712) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:58 +08:00
Gaius 7cc842d64d
test: remove test main (#1710) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:58 +08:00
zhaoshang fda6002aac
test: add test for daemon rpcserver (#1704) 
Signed-off-by: zhaoshang <zhaoshangsjtu@linux.alibaba.com>
 2023-06-28 17:33:58 +08:00
Jim Ma 89376d6c4e
feat: ipv6 support (#1685) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:57 +08:00
Jim Ma c5f4ca50fa
fix: grpc download tidy file error (#1697) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:57 +08:00
Jim Ma 73de99353d
feat: empty file e2e (#1687) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:56 +08:00
Jim Ma f4d34e56ed
chore: update download rpc check (#1684) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:56 +08:00
Jim Ma 40300990de
feat: support download empty file (#1686) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:56 +08:00
Gaius c9755aabfc
feat: stop grpc client (#1671) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:55 +08:00
Gaius 198613a357
chore: add disable seed peer action (#1653) 
* chore: add disable seed peer action
* fix: concurrent back source

Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:54 +08:00
Gaius 3764af25f3
feat: change dfdaemon rate limit (#1661) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:54 +08:00
Gaius da06e48d3e
feat: change storage strategy to simple (#1658) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:53 +08:00
Jim Ma 563f892b0d
feat: add missing client version for ListSchedulers (#1657) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:53 +08:00
Jim Ma 7bceca51bc
chore: update debug info (#1617) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:51 +08:00
Gaius b6c395a6cb
feat: add cert spec to security configuration (#1621) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:51 +08:00
Jim Ma 56ac9d1801
feat: support mutate all proxy requests (#1623) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 4d9261af7d
feat: check whether scheduler is in the same cluster (#1620) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 13580f89f2
refactor: dfpath for certify cache dir (#1618) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 43d8478507
feat: add tls policy to scheduler grpc server (#1616) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 7fff3537f2
feat: resolver addr add ServerName (#1614) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Jim Ma 9050230926
feat: refactor grpc credential (#1612) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:50 +08:00
Gaius 346cc0649e
feat: add tls policy constants (#1610) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:49 +08:00
Jim Ma 98fb1fc427
feat: add grpc mux transport (#1602) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:49 +08:00
Gaius 9ab33635c5
feat: manager init cert for grpc server (#1603) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:49 +08:00
Jim Ma 7224801bae
feat: refactor peertask option (#1600) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma c5a8512cb6
feat: add common serialize package (#1601) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma ecfe467585
feat: add client grpc dial timeout (#1599) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma c25cfdd651
feat: support multiple certify cache (#1598) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Jim Ma 717ac99ef5
feat: daemon support auto issue certificate (#1586) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:48 +08:00
Gaius ad553d949b
feat: grpc dial adds context (#1594) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:47 +08:00
Gaius 499d2fb886
feat: remove golang +build tag (#1585) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:47 +08:00
Gaius 2c48148ac8
fix: dfdaemon can not shutdown (#1580) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius 4a3d342a94
feat: dynconfig refresh and notify listeners (#1579) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius f631263d25
refactor: dfnet package (#1578) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00
Gaius a6a44269b4
refactor: dfdaemon client and remove rpc connection pool (#1576) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:46 +08:00