Commit Graph

65 Commits

Author SHA1 Message Date
Chlins Zhang 677049c769
refactor(request-sdk): move seed-peer filtering to server side (#1377)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2025-09-25 19:56:54 +08:00
LXDgit2018 74a6f04b8c
feat: Add QUIC Client and Server Implementation for Piece Download in dragonfly-client-storage (#1376)
Signed-off-by: LXDgit2018 <1289504283@qq.com>
2025-09-25 11:31:20 +08:00
Gaius 7cf2e3d800
feat(export): add overwrite option for output files (#1374)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-24 21:33:42 +08:00
Gaius b9817dc279
fix(dfget): get wrong entries and add tests for entry retrieval functions (#1373)
* feat(grpc): add new DfdaemonDownloadClient creation method

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test(dfget): add tests for entry retrieval functions

Signed-off-by: Gaius <gaius.qi@gmail.com>

---------

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-24 12:05:38 +08:00
Gaius 02eddfbd10
refactor(dragonfly-client-util): request and pool logic (#1370)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-23 18:18:16 +08:00
Chlins Zhang 826aeabf08
feat: add the request sdk in dragonfly-client-util (#1333)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2025-09-23 10:58:41 +08:00
dependabot[bot] d3dda551b7
chore(deps): Bump rustix from 1.0.8 to 1.1.2 (#1351)
Bumps [rustix](https://github.com/bytecodealliance/rustix) from 1.0.8 to 1.1.2.
- [Release notes](https://github.com/bytecodealliance/rustix/releases)
- [Changelog](https://github.com/bytecodealliance/rustix/blob/main/CHANGES.md)
- [Commits](https://github.com/bytecodealliance/rustix/compare/v1.0.8...v1.1.2)

---
updated-dependencies:
- dependency-name: rustix
  dependency-version: 1.1.2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-09-16 14:43:43 +08:00
Chlins Zhang 8ff694f431
refactor: move the shutdown crate to the util package (#1344)
Signed-off-by: chlins <chlins.zhang@gmail.com>
2025-09-11 14:15:42 +08:00
Gaius 79fa36ca97
refactor(downloader): streamline download concurrency handling (#1341)
* style(error): update error message formatting

Signed-off-by: Gaius <gaius.qi@gmail.com>

* refactor(downloader): streamline download concurrency handling

Signed-off-by: Gaius <gaius.qi@gmail.com>

---------

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-09 15:57:00 +08:00
Gaius af354b7e59
fix(fs): handle unsupported fallocate errors gracefully (#1340)
* fix(fs): handle unsupported fallocate errors gracefully

Signed-off-by: Gaius <gaius.qi@gmail.com>

* chore(dependencies): update dragonfly-client versions to 1.0.17

Signed-off-by: Gaius <gaius.qi@gmail.com>

---------

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-09-09 15:02:16 +08:00
Cyrus e252c8d503
fix: set persistent_cache_task_id to 64 bytes (#1316)
set persistent_cache_task_id to 64 byte

Signed-off-by: LunaWhispers <yangmuyucs@gmail.com>
2025-08-28 20:31:45 +08:00
Gaius e05270e598
feat: get network speed for scheduler (#1279)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-11 22:24:16 +08:00
Gaius e1ae65a48d
chore(ci/Dockerfile): add grpcurl for dfdaemon container (#1276)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-06 08:26:15 +00:00
Gaius 7796ee7342
chore(deps): remove unused dependencies (#1274)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-06 11:11:15 +08:00
dependabot[bot] 52307b64cc
chore(deps): Bump rustix from 1.0.5 to 1.0.8 (#1243)
Bumps [rustix](https://github.com/bytecodealliance/rustix) from 1.0.5 to 1.0.8.
- [Release notes](https://github.com/bytecodealliance/rustix/releases)
- [Changelog](https://github.com/bytecodealliance/rustix/blob/main/CHANGES.md)
- [Commits](https://github.com/bytecodealliance/rustix/compare/v1.0.5...v1.0.8)

---
updated-dependencies:
- dependency-name: rustix
  dependency-version: 1.0.8
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2025-07-22 16:45:02 +08:00
Gaius 23fa1ba3b7
feat: verify digest when file is downloaded (#1208)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-06-30 14:32:07 +08:00
Gaius 7819702b67
feat(tracing): align resource attributes with OpenTelemetry semantic conventions (#1164)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-05-29 00:00:12 +08:00
Gaius 604a9451da
feat: add task type to ID generation for Standard and PersistentCache tasks (#1149)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-05-14 09:38:31 +00:00
Gaius a2ba39026e
feat: support fallocate for creating task (#1119)
* feat: support fallocate and filling zero for creating task

* feat: support fallocate and filling zero for creating task

---------

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-04-29 11:43:42 +08:00
Gaius 7950aa5ab3
feat: handle Interrupted error for io (#1112)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-04-25 10:48:30 +08:00
Gaius c21159037a
feat(dragonfly-client): add support for content-based task ID generation in Dragonfly client (#1111)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-04-24 20:54:46 +08:00
KennyMcCormick 75c0c1f5a0
chroe: fix typo in ut (#1106)
Signed-off-by: cormick <cormick1080@gmail.com>
2025-04-22 04:34:21 +00:00
Gaius 3299a2aa20
refactor: add network interface utilities and enhance upload rate limiting (#1091)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-04-14 16:57:45 +08:00
Gaius f1a9f0af55
feat: remove unnecessary dependencies (#1085)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-04-10 17:03:04 +08:00
Gaius 0f04e07797
feat: add certs cache for generating self-signed certs (#1025)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-03-07 14:25:19 +08:00
Gaius 68bc6df1b2
feat: change crc to crc32fast (#1023)
* feat: change crc to crc32fast

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update

Signed-off-by: Gaius <gaius.qi@gmail.com>

---------

Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-03-06 13:58:03 +08:00
Gaius 5b0da70938
feat(dragonfly-client-util): remove wyhash package (#1001)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-02-25 15:14:44 +08:00
Gaius 259ebc0d52
feat: add piece length for calculating task id (#994)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-02-24 18:13:41 +08:00
Gaius 04c71e6e2b
feat: calculate digest of the persistent cache piece to check the integrity of the metadata (#980)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-02-14 12:00:53 +08:00
KennyMcCormick 32c99b711e
add checksum in download request to verify its consistency (#979)
Signed-off-by: cormick <cormick1080@gmail.com>
2025-02-14 10:37:57 +08:00
Gaius d4096db1e2
feat: support download persistent cache task from parents (#950)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-21 21:18:22 +08:00
Gaius bb86c3b63b
feat: generate persistent task id by wyhash (#946)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-17 20:02:38 +08:00
Gaius f4c991c34d
feat: replace crc32c with crc-rc (#934)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-08 21:20:07 +08:00
Gaius 6d9cdbf968
feat: remove crc32 iso3309 from write piece func (#933)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-07 20:29:28 +08:00
Gaius 9fd770ffab
feat: add cache store for proxy (#870)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-28 22:27:09 +08:00
Gaius e7740f46ed
feat: use CRC-32-Castagnoli algorithm with hardware acceleration (#866)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-26 10:32:35 +08:00
Gaius fbca1897c6
feat: remove unnecessary logs (#844)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-13 22:48:41 +08:00
Gaius db7b878b5a
feat: optimize convert headermap to hashmap (#841)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-13 17:21:23 +08:00
KennyMcCormick 6e0a859856
test: add ut for client-util/tls (#827) 2024-11-05 17:43:36 +08:00
KennyMcCormick 344271a842
test: add ut for client-util (#826) 2024-11-05 12:54:17 +08:00
Gaius b09f9d3e61
feat: exclude digest when calculating task id (#803)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-10-24 22:52:46 +08:00
Gaius e396706adf
feat: add basic auth for HTTP proxy (#785)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-10-18 19:11:19 +08:00
Gaius feb38d7f3d
feat: load certs in dfdaemon config (#779)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-10-15 21:10:15 +08:00
Gaius f911d7e24d
refactor: unit tests of the self-signed certs in http backend (#778)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-10-15 11:32:52 +08:00
Gaius f536bab788
feat: rename cache task to persistent cache task (#753)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-27 17:32:43 +08:00
Gaius e2209a8e61
refactor: change comments to rust style (#742)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-20 20:15:51 +08:00
Gaius a00b2015f3
refactor: backend tests in lib.rs (#706)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-08-29 22:55:12 +08:00
Gaius b4c0e817d6
feat: add tracing flame for dfdaemon (#694)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-08-28 10:34:24 +08:00
Gaius adc2bd6ff4
feat: optimize calculate file by crc32 (#685)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-08-22 22:37:34 +08:00
McCormickKneey b6569886f0
add crc32 algorithm with few tests (#684)
Signed-off-by: cormick <cormick1080@gmail.com>
2024-08-22 21:27:03 +08:00