Commit Graph

39 Commits

Author SHA1 Message Date
Gaius 3557eb5453
feat: add getObjectStorageMetadata to dfstore (#2869)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-09 15:28:47 +08:00
lixin18 6f50872dc0
Update objectstorage.go (#2854)
instence --> instance
2023-11-09 12:22:16 +08:00
Gaius b71cbc94cf
feat: add comment about not checking object storage close (#2789)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-11 10:38:54 +08:00
Gaius 61f63e2e8f
fix: task length did not match range length (#2787)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 18:28:59 +08:00
Yong-An9 e847992c90
fixed close file twice issue when using dfstore upload large file (#2784)
Co-authored-by: Yong.An <thomas.an@icloudsky.com>
2023-10-10 11:35:09 +08:00
Gaius d275bfb302
feat: change error handle in object storage (#2777)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-10-10 10:43:22 +08:00
tan ding c29edd28fd
Fix issues in the objectstorage based on JuiceFS use cases. (#2648)
Signed-off-by: XDTD <1355582364@qq.com>
2023-08-30 11:20:00 +08:00
Gaius 85b446746f
feat: initialize object storage client in the New func (#2682)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-29 15:16:03 +08:00
Gaius 5b7ea5da2a
Revert "feat: replace fmt.Sprintf with net.JoinHostPort" (#2649)
Revert "feat: replace fmt.Sprintf with net.JoinHostPort (#2642)"

This reverts commit a6863f9b19.
2023-08-17 17:21:50 +08:00
Gaius a6863f9b19
feat: replace fmt.Sprintf with net.JoinHostPort (#2642)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-16 16:45:14 +08:00
Gaius bd140f1104
refactor: file close error
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-08 12:08:15 +08:00
Jim Ma 342438eae6
chore: optimize unhandled file close error (#2599)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-08 11:48:29 +08:00
Jim Ma 1b9d2b03d7
fix: improper use strings.TrimLeft (#2603)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-08-07 17:55:27 +08:00
Gaius 029983008a
refactor: support for JuiceFS objectStorage implementation (#2579)
Refer to PR #2578.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-31 14:00:03 +08:00
tan ding f1fbfdeb81
feat: provide support for JuiceFS objectStorage implementation (#2578) 2023-07-31 10:44:21 +08:00
Gaius b3f23f71a3
feat: update dy7.io/api to v2 (#2558)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-20 23:17:57 +08:00
Gaius 2287adca60
fix: object downloads failed by dfstore when dfdaemon enabled concurrent (#2328)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:49 +08:00
Gaius 79024c87ca
refactor: parse http range (#2071)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:30 +08:00
Gaius f7a399b47a
feat: add v2 version of the idgen (#2056)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:29 +08:00
Gaius e4fc6d5c85
feat: add S3ForcePathStyle to object storage (#1976)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:22 +08:00
Gaius 731ecc1234
feat: change yaml field type from string to net.IP (#1929)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:18 +08:00
Jim Ma 55d5308b19
fix: open end range in concurrent back source (#1764)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:03 +08:00
Gaius 740696a171
feat: when the request has a range header, object storage is no need to to calculate md5 (#1534)
feat: when the request has a range header, object storage is no need to calculate md5

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:42 +08:00
Gaius d79f6405ff
feat: replace grpc package with https://github.com/dragonflyoss/api (#1515)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:41 +08:00
Gaius 601993aa46
feat: use os.PathSeparator to generate object key
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:35 +08:00
Gaius 812ff5557b
fix: metrics reduces labels (#1457)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:35 +08:00
Gaius 93a8ebb4bc
feat: add dfstore command (#1441)
* feat: add dfstore command

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:34 +08:00
Gaius 567e8e978c
feat: use put object action (#1422)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
Gaius 3b426d2c54
feat: rename client/clientutil to client/util (#1420)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
Gaius 0222649b88
feat: update namely/protoc-all image version to 1.47_0 (#1418)
* feat: update namely/protoc-all image version to 1.47_0

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate mock file

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
Gaius ad36eb64a7
feat: remove github/pkg/errors package (#1416)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:31 +08:00
Gaius 1d7cd9dd12
feat: add dfstore client interface (#1415)
* feat: add dfstore client interface

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:31 +08:00
Gaius ec2a72d872
feat: import object to seed peer with max replicas (#1413)
* feat: create object storage with max replicas

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: import object storage with max replicas

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add strings.Unique func

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:31 +08:00
Gaius 4ad950a8e3
feat: object storage add filter field (#1412)
* feat: object storage add filter field

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: object storage add logger

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: GetObjectMetadata return isExist

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
Gaius 2504f28cd0
feat: dfdaemon add destroyObject rest api (#1410)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
Gaius 30313a4caa
feat: client add create object storage (#1409)
* feat: add create object storage rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:30 +08:00
Gaius ad827170a4
feat: dfdaemon upload and object storage service add middlewares (#1404)
* feat: dfdaemon upload and object storage service add middlewares

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:29 +08:00
Gaius 230966cf8f
feat: dfdaemon add GetObject rest api (#1398)
* feat: dfdaemon add GetObject rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:28 +08:00
Gaius 171c0d54ee
feat: dfdaemon add object storage rest api (#1390)
* feat: dfdaemon add object storage rest api

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:28 +08:00