Commit Graph

414 Commits

Author SHA1 Message Date
Jim Ma ccacc3ec3a
feat: proxy graceful shutdown (#2802)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-18 13:47:56 +08:00
Gaius b71cbc94cf
feat: add comment about not checking object storage close (#2789)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-11 10:38:54 +08:00
Gaius 61f63e2e8f
fix: task length did not match range length (#2787)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 18:28:59 +08:00
Jim Ma 47c6cdd8cc
chore: optimize recursive download log (#2785)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-10 14:43:45 +08:00
Yong-An9 e847992c90
fixed close file twice issue when using dfstore upload large file (#2784)
Co-authored-by: Yong.An <thomas.an@icloudsky.com>
2023-10-10 11:35:09 +08:00
Gaius d275bfb302
feat: change error handle in object storage (#2777)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 10:43:22 +08:00
Sasha Melentyev eeea18b16e
chore(client/daemon): cleanup, and add noTLSConfig (#2783)
chore(client/daemon): cleanup, and add noTLSConfig instead nil for better readable

Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2023-10-10 10:42:53 +08:00
pbannykh c9c7100e77
fix: set download header log level debug (#2770)
Signed-off-by: bannykh <bannykh@tochka.com>
2023-10-08 10:56:34 +08:00
guangwu a3f2aae26c
chore: remove refs to deprecated io/ioutil (#2754)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-09-26 10:09:57 +08:00
Jim Ma 9eb15b218d
fix: overhead gc when DiskGCThreshold not set (#2750)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-09-21 21:04:17 +08:00
Gaius 170a7f2186
feat: change file mode when export task from local in dfcache (#2701)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-09-05 19:59:11 +08:00
tan ding c29edd28fd
Fix issues in the objectstorage based on JuiceFS use cases. (#2648)
Signed-off-by: XDTD <1355582364@qq.com>
2023-08-30 11:20:00 +08:00
Gaius 85b446746f
feat: initialize object storage client in the New func (#2682)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-29 15:16:03 +08:00
Guangwen Feng a7f6320a7e
test: add unit test for PieceDownloader.Error (#2667)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-24 10:31:08 +08:00
Gaius fe28ba4c8e
feat: merge sync peer with peer table in manager (#2668)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-24 10:28:08 +08:00
Gaius f3b92900af
feat: peer announces scheduler cluster id to scheduler (#2652)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-21 22:00:05 +08:00
Gaius 5b7ea5da2a
Revert "feat: replace fmt.Sprintf with net.JoinHostPort" (#2649)
Revert "feat: replace fmt.Sprintf with net.JoinHostPort (#2642)"

This reverts commit a6863f9b19.
2023-08-17 17:21:50 +08:00
Gaius a6863f9b19
feat: replace fmt.Sprintf with net.JoinHostPort (#2642)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-16 16:45:14 +08:00
Jim Ma 0eaf429c4e
chore: skip export to exist file (#2637)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:05:17 +08:00
Jim Ma 0e04a56177
chore: check task id (#2636)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:04:50 +08:00
Jim Ma 554b27c373
fix: add traffic (#2634)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:04:03 +08:00
Gaius 05a66805cc
fix: directories created via os.MkdirAll are not checked for permissions (#2613)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-09 13:52:26 +08:00
Gaius bd140f1104
refactor: file close error
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-08 12:08:15 +08:00
Jim Ma 342438eae6
chore: optimize unhandled file close error (#2599)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-08 11:48:29 +08:00
Jim Ma 1b9d2b03d7
fix: improper use strings.TrimLeft (#2603)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:55:27 +08:00
Jim Ma a64140ff72
fix: potential nil panic (#2602)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:55:03 +08:00
Jim Ma dcea13aa47
chore: use subtle compare to verify proxy auth (#2601)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:54:39 +08:00
Gaius 029983008a
refactor: support for JuiceFS objectStorage implementation (#2579)
Refer to PR #2578.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-31 14:00:03 +08:00
tan ding f1fbfdeb81
feat: provide support for JuiceFS objectStorage implementation (#2578) 2023-07-31 10:44:21 +08:00
Guangwen Feng d2aaa0835f
test: add unit tests for PieceDownloader (#2570)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-07-26 20:41:29 +08:00
Gaius b3f23f71a3
feat: update dy7.io/api to v2 (#2558)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-20 23:17:57 +08:00
MisakaCloud 512ad52c73
fix: also add ca to RootCAs (#2516)
Signed-off-by: MisakaCloud <xjdzch@126.com>
2023-07-06 17:28:16 +08:00
dlut_xz c5fd00e0e4
test: add client daemon network topology unit tests (#2490)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:35:00 +08:00
Gaius a0d14c58b8
feat: add network topology to daemon (#2489)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:35:00 +08:00
dlut_xz 883496cb14
feat: implement probe interface in client daemon (#2473)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:35:00 +08:00
Jim Ma bdfeb48f02
feat: daemon store file exclusive (#2465)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:59 +08:00
Jim Ma 9137707139
feat: support breakpoint resume for running tasks (#2457)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:58 +08:00
embroede 8c905d9b8d
feat: enable configuration of some directory modes for dfdaemon (#2340)
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Gaius 146f82c453
feat: optimize announcer in scheduler and client (#2445)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:57 +08:00
Jim Ma afb47f6729
chore: check grpc peer info for download service (#2385)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:53 +08:00
Jim Ma 276333fceb
fix: SyncPieceViaHTTPS not work (#2329)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:49 +08:00
Gaius 2287adca60
fix: object downloads failed by dfstore when dfdaemon enabled concurrent (#2328)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:49 +08:00
Gaius e9cb38e851
feat: remove SyncNetworkTopology API (#2296)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:47 +08:00
Gaius 37c588608a
feat: remove security domain (#2285)
* feat: remove security domain

Remove security domain in dfdaemon and scheduler.
Remove SecurityGroup and SecurityRule table in manager.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:46 +08:00
Jim Ma b38b5fcbeb
fix: client grpc dial non-block (#2261)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:45 +08:00
Jim Ma c82ebec8e0
chore: update traffic shaper log (#2227)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:42 +08:00
Jim Ma 74e8e65b0b
fix: traffic shaper record task not found (#2226)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:42 +08:00
Gaius 7a1ae0a2ee
test: add new metrics test to service (#2212)
test: add new metric test to service

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:41 +08:00
Gaius 2cda1def7f
feat: rename HostName to Hostname (#2205)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:40 +08:00
Gaius 047a55fb84
feat: add handleRegisterSeedPeerRequest to AnnouncePeer in service v2 (#2147)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:36 +08:00