Commit Graph

83 Commits

Author SHA1 Message Date
Gaius 02eddfbd10
refactor(dragonfly-client-util): request and pool logic (#1370)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-23 18:18:16 +08:00
Gaius 79fa36ca97
refactor(downloader): streamline download concurrency handling (#1341)
* style(error): update error message formatting

Signed-off-by: Gaius <gaius.qi@gmail.com>

* refactor(downloader): streamline download concurrency handling

Signed-off-by: Gaius <gaius.qi@gmail.com>

---------

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-09 15:57:00 +08:00
Gaius 2738d590d5
chore(dependencies): update project dependencies and version (#1332)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-04 09:02:42 +00:00
Gaius 826dad6d93
feat(object_storage): add security token to OSS initialization (#1328)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-02 10:02:57 +00:00
Gaius 2070c8156e
chore(dependencies): update dependencies and remove unused metakeys (#1322)
* chore(dependencies): update dependencies and remove unused metakeys

Signed-off-by: Gaius <gaius.qi@gmail.com>

* build(deps): update dependencies and remove unused features

Signed-off-by: Gaius <gaius.qi@gmail.com>

---------

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-02 10:27:20 +08:00
Chlins Zhang 190dead698
feat: add the X-Dragonfly-Proxy-Status header to the response (#1299)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2025-08-22 18:03:51 +08:00
Chlins Zhang a756f50daa
fix: replace inspect with inspect_err for error logging (#1298)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2025-08-21 18:38:06 +08:00
Gaius 578fc8d147
feat: add conditional piece content reading based on need_piece_content flag (#1296)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-21 11:49:19 +08:00
Gaius 90eeaebe7c
feat: replace glibc DNS resolver with hickory-dns in reqwest (#1293)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-19 23:18:00 +08:00
Gaius e415df936d
feat: enable console subscriber layer for tracing spawn tasks on `127.0.0.1:6669` when log level is TRACE (#1275)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-06 06:58:47 +00:00
Gaius 52b263ac66
feat: Disable compression in HTTP client configuration (#1273)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-05 16:58:43 +08:00
Gaius f0c983093a
feat: add Range header to ensure Content-Length is returned in response headers (#1263)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-04 06:25:58 +00:00
Gaius b30993eef0
refactor(dfget): Improve logic and error handling in `filter_entries` (#1258)
- Updated function signature to use references (`&Url`, `&[String]`) for efficiency.
- Improved error handling with detailed `ValidationError` messages instead of generic `UnexpectedResponse`.
- Renamed `rel_path_to_entry` to `entries_by_relative_path` for better clarity.
- Replaced `Vec` with `HashSet` for filtered entries to avoid duplicates.
- Simplified parent directory path construction using `join("")`.
- Enhanced doc comments to clearly describe functionality and behavior.
- Streamlined pattern compilation and iteration using `iter()`.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-07-25 14:09:34 +00:00
Cyrus cca88b3eea
feat: add include-files argrument in the downloaded directory (#1247)
* add include-files arg

Signed-off-by: LunaWhispers <yangmuyucs@gmail.com>

* fix regular expression matching

Signed-off-by: LunaWhispers <yangmuyucs@gmail.com>

* fix lint

Signed-off-by: LunaWhispers <yangmuyucs@gmail.com>

---------

Signed-off-by: LunaWhispers <yangmuyucs@gmail.com>
2025-07-25 17:13:57 +08:00
Gaius aedbc2ceb0
fix: logging message in BackendFactory to reference correct backend_plugin_dir variable (#1237)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-07-18 11:25:41 +08:00
Cyrus 9c3c0930af
feat: Create a blacklist of backends that do not support directory downloads (#1224)
Create a blacklist of backends that do not support path downloads

Signed-off-by: LunaWhispers <yangmuyucs@gmail.com>
2025-07-08 15:09:20 +08:00
dependabot[bot] a0a347eda4
chore(deps): Bump wiremock from 0.6.3 to 0.6.4 (#1201)
Bumps [wiremock](https://github.com/LukeMathWalker/wiremock-rs) from 0.6.3 to 0.6.4.
- [Changelog](https://github.com/LukeMathWalker/wiremock-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/LukeMathWalker/wiremock-rs/compare/v0.6.3...v0.6.4)

---
updated-dependencies:
- dependency-name: wiremock
  dependency-version: 0.6.4
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-24 10:59:28 +08:00
dependabot[bot] ee5cf8f642
chore(deps): Bump libloading from 0.8.7 to 0.8.8 (#1174)
Bumps [libloading](https://github.com/nagisa/rust_libloading) from 0.8.7 to 0.8.8.
- [Commits](https://github.com/nagisa/rust_libloading/commits)

---
updated-dependencies:
- dependency-name: libloading
  dependency-version: 0.8.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-06-11 10:46:26 +08:00
Gaius 7819702b67
feat(tracing): align resource attributes with OpenTelemetry semantic conventions (#1164)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-05-29 00:00:12 +08:00
Gaius 3959bb9330
feat: rename --log-to-stdout to --console (#1163)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-05-27 15:31:14 +08:00
Gaius 7b1cdbe1f1
feat: add --log-to-stdout for command (#1162)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-05-27 14:15:56 +08:00
dependabot[bot] 3ce1e2ca42
chore(deps): Bump libloading from 0.8.6 to 0.8.7 (#1143)
Bumps [libloading](https://github.com/nagisa/rust_libloading) from 0.8.6 to 0.8.7.
- [Commits](https://github.com/nagisa/rust_libloading/compare/0.8.6...0.8.7)

---
updated-dependencies:
- dependency-name: libloading
  dependency-version: 0.8.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-05-12 20:54:27 +08:00
Gaius f3fd5f46c4
feat(dragonfly-client-backend): change retry times for backend (#1129)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-04-29 16:39:28 +08:00
KennyMcCormick ad335784fe
fix: modify ut logic which is incorrect under macOS (#1099)
Signed-off-by: cormick <cormick1080@gmail.com>
2025-04-16 20:53:27 +08:00
Gaius f1a9f0af55
feat: remove unnecessary dependencies (#1085)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-04-10 17:03:04 +08:00
dependabot[bot] b9b784289b
chore(deps): Bump wiremock from 0.6.2 to 0.6.3 (#1044)
Bumps [wiremock](https://github.com/LukeMathWalker/wiremock-rs) from 0.6.2 to 0.6.3.
- [Changelog](https://github.com/LukeMathWalker/wiremock-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/LukeMathWalker/wiremock-rs/compare/v0.6.2...v0.6.3)

---
updated-dependencies:
- dependency-name: wiremock
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-03-18 11:47:37 +08:00
Gaius 3d91c09b0a
feat: support transfer piece content from dfdaemon (#926)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-03 17:15:50 +08:00
dependabot[bot] c835aa6a2e
chore(deps): Bump libloading from 0.8.5 to 0.8.6 (#890)
Bumps [libloading](https://github.com/nagisa/rust_libloading) from 0.8.5 to 0.8.6.
- [Commits](https://github.com/nagisa/rust_libloading/compare/0.8.5...0.8.6)

---
updated-dependencies:
- dependency-name: libloading
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-12-09 20:16:15 +08:00
Gaius d8f0931ca5
feat: add gzip, deflate, brotli, zstd compressed for reqwest (#886)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-12-05 18:06:55 +08:00
Gaius 6f4fbdd47d
chore: exclude hdfs for building
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-12-04 16:21:04 +08:00
Gaius 4be92a6a38
chore: change crate-type for hdfs examples
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-12-04 15:13:50 +08:00
Gaius 064a3f005c
feat: add retry middleware for http backend (#872)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-29 20:09:26 +08:00
Gaius 62c62f7ec6
feat: add sse4.2 target feature for building (#867)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-26 18:07:50 +08:00
Gaius e7740f46ed
feat: use CRC-32-Castagnoli algorithm with hardware acceleration (#866)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-26 10:32:35 +08:00
Gaius 84f0b30ca9
feat: add timeout for hdfs backend (#857) 2024-11-20 18:44:21 +08:00
Gaius a19cc89605
feat: reuse http client for object storage in backend (#856)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-20 17:43:49 +08:00
Gaius 2fd672cb7a
feat: reuse http client for backend (#855)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-20 16:50:43 +08:00
Gaius 9c6d16a5ff
refactor: support hdfs for backend (#852)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-19 14:46:23 +08:00
Chongzhi Deng 8ce9f23d4c
dfget supports HDFS (#838)
Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
2024-11-19 13:37:48 +08:00
dependabot[bot] aecc949a43
chore(deps): Bump futures from 0.3.30 to 0.3.31 (#848)
Bumps [futures](https://github.com/rust-lang/futures-rs) from 0.3.30 to 0.3.31.
- [Release notes](https://github.com/rust-lang/futures-rs/releases)
- [Changelog](https://github.com/rust-lang/futures-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/futures-rs/compare/0.3.30...0.3.31)

---
updated-dependencies:
- dependency-name: futures
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-11-18 20:28:39 +08:00
Gaius fbca1897c6
feat: remove unnecessary logs (#844)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-13 22:48:41 +08:00
Gaius feb38d7f3d
feat: load certs in dfdaemon config (#779)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-10-15 21:10:15 +08:00
Gaius f911d7e24d
refactor: unit tests of the self-signed certs in http backend (#778)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-10-15 11:32:52 +08:00
Lzzzt 6e26222e79
test: add unit tests for dragonfly-backend::http (#770)
Signed-off-by: Lzzzt <liuzitao0123@gmail.com>
2024-10-15 10:35:29 +08:00
Gaius e2209a8e61
refactor: change comments to rust style (#742)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-20 20:15:51 +08:00
dependabot[bot] 25ad9c245a
chore(deps): Bump wiremock from 0.6.1 to 0.6.2 (#728)
Bumps [wiremock](https://github.com/LukeMathWalker/wiremock-rs) from 0.6.1 to 0.6.2.
- [Changelog](https://github.com/LukeMathWalker/wiremock-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/LukeMathWalker/wiremock-rs/compare/v0.6.1...v0.6.2)

---
updated-dependencies:
- dependency-name: wiremock
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-09-10 13:39:55 +08:00
Gaius ad1d783af6
test: set plugin ext to .dylib in macos (#722)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-06 15:46:57 +08:00
Gaius b73c15398b
feat: optimize make_need_fields_message macro (#721)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-06 15:02:47 +08:00
Lzzzt 40bcafca40
test: Add unit tests for object storage (#714)
Signed-off-by: lzzzt <liuzitao0123@gmail.com>
2024-09-06 13:50:58 +08:00
Gaius a00b2015f3
refactor: backend tests in lib.rs (#706)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-08-29 22:55:12 +08:00