Commit Graph

508 Commits

Author SHA1 Message Date
Jim Ma 09b11a280a
chore: optimize stream task id (#2983)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-12-29 16:16:06 +08:00
Jim Ma e329bdbb39
feat: back to source with piece group(multiple pieces) (#2826)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-12-27 15:54:21 +08:00
Jim Ma 839e476ecb
chore: optimize sync pieces error report (#2925)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-12-14 11:23:15 +08:00
Gaius 67c36f62b3
feat: remove concurrent_piece_count in scheduler (#2942)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-12 16:39:16 +08:00
Gaius f8d538b0f1
feat: dfget sets data dir and cache dir (#2931)
feat: dfget sets data and cache dir

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-07 20:48:30 +08:00
lishaokai da67241dbf
fix the wrong word (#2914)
Signed-off-by: lishaokai1995 <lishaokai1995@126.com>
2023-12-04 21:39:09 +08:00
Jim Ma 9b251a8092
feat: enable watch dog by default (#2905)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-11-29 21:08:19 +08:00
Gaius 9608eff3bf
feat: avoid hot resolve in grpc (#2884)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-15 22:23:43 +08:00
Gaius 3557eb5453
feat: add getObjectStorageMetadata to dfstore (#2869)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-09 15:28:47 +08:00
Gaius cc417f1f11
feat: add MaxScheduleCount and GetSeedPeers (#2868)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-09 13:49:09 +08:00
lixin18 6f50872dc0
Update objectstorage.go (#2854)
instence --> instance
2023-11-09 12:22:16 +08:00
Jim Ma 3dc2e468a2
chore: split concurrentDownloadSource (#2842) 2023-10-31 16:51:46 +08:00
Gaius 12fc630612
fix: if scheduler has no seed peer, return error in preheating (#2835)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-30 16:45:39 +08:00
Jim Ma ccacc3ec3a
feat: proxy graceful shutdown (#2802)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-18 13:47:56 +08:00
Yong-An9 d6e426fae7
Optimized dfstore upload file performance (#2803)
Co-authored-by: Yong.An <thomas.an@icloudsky.com>
Co-authored-by: thomas <thomas@ThomasAns-MacBook-Pro.local>
2023-10-17 11:34:23 +08:00
Sasha Melentyev d189e18026
chore: gofmt -w -r 'interface{} -> any' . (#2790)
Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2023-10-11 10:39:34 +08:00
Gaius b71cbc94cf
feat: add comment about not checking object storage close (#2789)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-11 10:38:54 +08:00
Gaius 61f63e2e8f
fix: task length did not match range length (#2787)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 18:28:59 +08:00
Jim Ma 47c6cdd8cc
chore: optimize recursive download log (#2785)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-10 14:43:45 +08:00
Yong-An9 e847992c90
fixed close file twice issue when using dfstore upload large file (#2784)
Co-authored-by: Yong.An <thomas.an@icloudsky.com>
2023-10-10 11:35:09 +08:00
Gaius d275bfb302
feat: change error handle in object storage (#2777)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 10:43:22 +08:00
Sasha Melentyev eeea18b16e
chore(client/daemon): cleanup, and add noTLSConfig (#2783)
chore(client/daemon): cleanup, and add noTLSConfig instead nil for better readable

Signed-off-by: Sasha Melentyev <sasha@melentyev.io>
2023-10-10 10:42:53 +08:00
Jim Ma 237eb9e505
chore: optimize dynconfig error log (#2771)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-10-08 15:17:40 +08:00
pbannykh c9c7100e77
fix: set download header log level debug (#2770)
Signed-off-by: bannykh <bannykh@tochka.com>
2023-10-08 10:56:34 +08:00
guangwu a3f2aae26c
chore: remove refs to deprecated io/ioutil (#2754)
Signed-off-by: guoguangwu <guoguangwu@magic-shield.com>
2023-09-26 10:09:57 +08:00
Jim Ma 9eb15b218d
fix: overhead gc when DiskGCThreshold not set (#2750)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-09-21 21:04:17 +08:00
Guangwen Feng dd2c285be5
test: add unit test for Notify (#2748)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-09-21 12:07:44 +08:00
Gaius 19ad719f86
feat: convert limit in dfstore (#2745)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-09-19 11:45:23 +08:00
Guangwen Feng da1fc92827
test: add unit test for Convert (#2725)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-09-13 11:32:08 +08:00
lilongfeng 8fb8b7337a
Remove useless constant values (#2721)
Signed-off-by: 李龙峰 <lilongfeng@inspur.com>
2023-09-12 17:51:00 +08:00
Gaius 170a7f2186
feat: change file mode when export task from local in dfcache (#2701)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-09-05 19:59:11 +08:00
Gaius 9c9959fe3a
feat: optimize GetObjectMetadatasInput validate in dfstore (#2686)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-30 16:31:35 +08:00
tan ding c29edd28fd
Fix issues in the objectstorage based on JuiceFS use cases. (#2648)
Signed-off-by: XDTD <1355582364@qq.com>
2023-08-30 11:20:00 +08:00
Gaius 85b446746f
feat: initialize object storage client in the New func (#2682)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-29 15:16:03 +08:00
Guangwen Feng a7f6320a7e
test: add unit test for PieceDownloader.Error (#2667)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-24 10:31:08 +08:00
Guangwen Feng d61eecd8d8
test: add unit test for DfgetConfig.Validate (#2666)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-24 10:30:25 +08:00
Gaius fe28ba4c8e
feat: merge sync peer with peer table in manager (#2668)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-24 10:28:08 +08:00
Guangwen Feng 63392f2d29
test: add unit test for Dfget.parseHeader (#2665)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-23 14:19:29 +08:00
Gaius f3b92900af
feat: peer announces scheduler cluster id to scheduler (#2652)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-21 22:00:05 +08:00
Gaius 5b7ea5da2a
Revert "feat: replace fmt.Sprintf with net.JoinHostPort" (#2649)
Revert "feat: replace fmt.Sprintf with net.JoinHostPort (#2642)"

This reverts commit a6863f9b19.
2023-08-17 17:21:50 +08:00
Gaius 7d5ebae1f1
test: rename Test_Validate to TestDfstoreConfig_Validate (#2644)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-16 16:49:04 +08:00
Guangwen Feng ffd32a5bdf
test: add unit tests for dfstore (#2643)
Signed-off-by: Guangwen Feng <fenggw-fnst@fujitsu.com>
2023-08-16 16:46:21 +08:00
Gaius a6863f9b19
feat: replace fmt.Sprintf with net.JoinHostPort (#2642)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-16 16:45:14 +08:00
Jim Ma 0eaf429c4e
chore: skip export to exist file (#2637)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:05:17 +08:00
Jim Ma 0e04a56177
chore: check task id (#2636)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:04:50 +08:00
Jim Ma 554b27c373
fix: add traffic (#2634)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-15 23:04:03 +08:00
Gaius 9e1bb52f2b
feat: net.JoinHostPort replace fmt.Sprintf (#2622)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-11 14:03:01 +08:00
Gaius 05a66805cc
fix: directories created via os.MkdirAll are not checked for permissions (#2613)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-09 13:52:26 +08:00
Gaius bd140f1104
refactor: file close error
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-08 12:08:15 +08:00
Jim Ma 342438eae6
chore: optimize unhandled file close error (#2599)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-08 11:48:29 +08:00