Commit Graph

495 Commits

Author SHA1 Message Date
Jim Ma ccacc3ec3a
feat: proxy graceful shutdown (#2802)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-18 13:47:56 +08:00
Yong-An9 d6e426fae7
Optimized dfstore upload file performance (#2803)
Co-authored-by: Yong.An <thomas.an@icloudsky.com>
Co-authored-by: thomas <thomas@ThomasAns-MacBook-Pro.local>
2023-10-17 11:34:23 +08:00
Sasha Melentyev d189e18026
chore: gofmt -w -r 'interface{} -> any' . (#2790)
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2023-10-11 10:39:34 +08:00
Gaius b71cbc94cf
feat: add comment about not checking object storage close (#2789)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-11 10:38:54 +08:00
Gaius 61f63e2e8f
fix: task length did not match range length (#2787)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 18:28:59 +08:00
Jim Ma 47c6cdd8cc
chore: optimize recursive download log (#2785)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-10 14:43:45 +08:00
Yong-An9 e847992c90
fixed close file twice issue when using dfstore upload large file (#2784)
Co-authored-by: Yong.An <thomas.an@icloudsky.com>
2023-10-10 11:35:09 +08:00
Gaius d275bfb302
feat: change error handle in object storage (#2777)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 10:43:22 +08:00
Sasha Melentyev eeea18b16e
chore(client/daemon): cleanup, and add noTLSConfig (#2783)
chore(client/daemon): cleanup, and add noTLSConfig instead nil for better readable

Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2023-10-10 10:42:53 +08:00
Jim Ma 237eb9e505
chore: optimize dynconfig error log (#2771)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-08 15:17:40 +08:00
pbannykh c9c7100e77
fix: set download header log level debug (#2770)
Signed-off-by: bannykh <bannykh@tochka.com>
2023-10-08 10:56:34 +08:00
guangwu a3f2aae26c
chore: remove refs to deprecated io/ioutil (#2754)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-09-26 10:09:57 +08:00
Jim Ma 9eb15b218d
fix: overhead gc when DiskGCThreshold not set (#2750)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-09-21 21:04:17 +08:00
Guangwen Feng dd2c285be5
test: add unit test for Notify (#2748)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-09-21 12:07:44 +08:00
Gaius 19ad719f86
feat: convert limit in dfstore (#2745)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-09-19 11:45:23 +08:00
Guangwen Feng da1fc92827
test: add unit test for Convert (#2725)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-09-13 11:32:08 +08:00
lilongfeng 8fb8b7337a
Remove useless constant values (#2721)
Signed-off-by: 李龙峰 <lilongfeng@inspur.com>
2023-09-12 17:51:00 +08:00
Gaius 170a7f2186
feat: change file mode when export task from local in dfcache (#2701)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-09-05 19:59:11 +08:00
Gaius 9c9959fe3a
feat: optimize GetObjectMetadatasInput validate in dfstore (#2686)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-30 16:31:35 +08:00
tan ding c29edd28fd
Fix issues in the objectstorage based on JuiceFS use cases. (#2648)
Signed-off-by: XDTD <1355582364@qq.com>
2023-08-30 11:20:00 +08:00
Gaius 85b446746f
feat: initialize object storage client in the New func (#2682)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-29 15:16:03 +08:00
Guangwen Feng a7f6320a7e
test: add unit test for PieceDownloader.Error (#2667)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-24 10:31:08 +08:00
Guangwen Feng d61eecd8d8
test: add unit test for DfgetConfig.Validate (#2666)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-24 10:30:25 +08:00
Gaius fe28ba4c8e
feat: merge sync peer with peer table in manager (#2668)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-24 10:28:08 +08:00
Guangwen Feng 63392f2d29
test: add unit test for Dfget.parseHeader (#2665)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-23 14:19:29 +08:00
Gaius f3b92900af
feat: peer announces scheduler cluster id to scheduler (#2652)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-21 22:00:05 +08:00
Gaius 5b7ea5da2a
Revert "feat: replace fmt.Sprintf with net.JoinHostPort" (#2649)
Revert "feat: replace fmt.Sprintf with net.JoinHostPort (#2642)"

This reverts commit a6863f9b19.
2023-08-17 17:21:50 +08:00
Gaius 7d5ebae1f1
test: rename Test_Validate to TestDfstoreConfig_Validate (#2644)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-16 16:49:04 +08:00
Guangwen Feng ffd32a5bdf
test: add unit tests for dfstore (#2643)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-16 16:46:21 +08:00
Gaius a6863f9b19
feat: replace fmt.Sprintf with net.JoinHostPort (#2642)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-16 16:45:14 +08:00
Jim Ma 0eaf429c4e
chore: skip export to exist file (#2637)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:05:17 +08:00
Jim Ma 0e04a56177
chore: check task id (#2636)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:04:50 +08:00
Jim Ma 554b27c373
fix: add traffic (#2634)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:04:03 +08:00
Gaius 9e1bb52f2b
feat: net.JoinHostPort replace fmt.Sprintf (#2622)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-11 14:03:01 +08:00
Gaius 05a66805cc
fix: directories created via os.MkdirAll are not checked for permissions (#2613)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-09 13:52:26 +08:00
Gaius bd140f1104
refactor: file close error
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-08 12:08:15 +08:00
Jim Ma 342438eae6
chore: optimize unhandled file close error (#2599)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-08 11:48:29 +08:00
Jim Ma 1b9d2b03d7
fix: improper use strings.TrimLeft (#2603)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:55:27 +08:00
Jim Ma a64140ff72
fix: potential nil panic (#2602)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:55:03 +08:00
Jim Ma dcea13aa47
chore: use subtle compare to verify proxy auth (#2601)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:54:39 +08:00
Gaius 029983008a
refactor: support for JuiceFS objectStorage implementation (#2579)
Refer to PR #2578.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-31 14:00:03 +08:00
tan ding f1fbfdeb81
feat: provide support for JuiceFS objectStorage implementation (#2578) 2023-07-31 10:44:21 +08:00
Jim Ma 3044b867bd
chore: clean temporary file when backsource error (#2575)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-07-28 14:44:53 +08:00
Guangwen Feng d2aaa0835f
test: add unit tests for PieceDownloader (#2570)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-07-26 20:41:29 +08:00
Gaius b3f23f71a3
feat: update dy7.io/api to v2 (#2558)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-20 23:17:57 +08:00
MisakaCloud 512ad52c73
fix: also add ca to RootCAs (#2516)
Signed-off-by: MisakaCloud <xjdzch@126.com>
2023-07-06 17:28:16 +08:00
dlut_xz c5fd00e0e4
test: add client daemon network topology unit tests (#2490)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:35:00 +08:00
Gaius a0d14c58b8
feat: add network topology to daemon (#2489)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:35:00 +08:00
dlut_xz 883496cb14
feat: implement probe interface in client daemon (#2473)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:35:00 +08:00
Jim Ma bdfeb48f02
feat: daemon store file exclusive (#2465)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:59 +08:00