Commit Graph

94 Commits

Author SHA1 Message Date
Gaius 7056342013
feat: add insecure_skip_verify and certificate_chain for PreheatImage
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-08-01 14:11:42 +08:00
Gaius cd8b9986c1
feat(golang/proto): add PreheatImage and StatImage for scheduler
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-07-29 19:39:46 +08:00
this is my name 80c539463b
feat: add Cache Task mechanism (#512)
* # Description
This pull request introduces APIs for the new Cache Task mechanism, enabling support for memory cache operations. Additionally, by refactoring the Download model and removing the load_to_cache field, the Task component now focuses exclusively on disk I/O operations, while the Cache Task handles memory cache operations. Finally, the version number of the dragonfly-api package has been incremented.

## Key Changes

### common.proto
- Added a new enum type Cache to TaskType to identify CacheTask.
- Introduced a new structure CachePeer to represent peers handling cache tasks.
- Added a new structure CacheTask to define metadata for cache tasks.
- Removed the load_to_cache field from the Download structure, as Tasks will no longer handle cache operations.

### dfdaemon.proto
- Added three RPC calls (DownloadCacheTask, StatCacheTask, DeleteCacheTask) to the DfdaemonUpload and DfdaemonDownload services for cache task management.
- Added two RPC calls (SyncCachePieces, DownloadCachePiece) to the DfdaemonUpload service for cache piece synchronization.
- Introduced 9 new message types to support requests and responses for CacheTask downloads.

### scheduler.proto
- Added five RPC calls to the Scheduler service: AnnounceCachePeer, StatCachePeer, DeleteCachePeer for cache peer management. StatCacheTask, DeleteCacheTask for cache task management.
- Introduced 16 new message types to cover the full lifecycle state transitions of cache tasks.

## Version Update
- Incremented the version of the dragonfly-api package from 2.1.47 to 2.1.48 in Cargo.toml to reflect the new changes.

## Related Issue

## Motivation and Context
- Separation of Responsibilities: Tasks now focus exclusively on disk I/O operations, while CacheTask independently handles memory cache management.

Signed-off-by: fu220 <2863318196@qq.com>

* Changed Download and introduced CacheDownload.

- Added more detailed descriptions for the CACHE type in TaskType, while clarifying that the STANDARD type interacts exclusively with disk operations.
- Removed the load_to_cache field and related information from the Download structure.
- Introduced CacheDownload for handling CacheTask downloads.

Signed-off-by: fu220 <2863318196@qq.com>

* Removed CacheDownload and directly merged into the Request

Signed-off-by: fu220 <2863318196@qq.com>

* Allow large_enum_variant in RegisterCachePeerRequest

Signed-off-by: fu220 <2863318196@qq.com>

---------

Signed-off-by: fu220 <2863318196@qq.com>
2025-07-24 18:27:23 +08:00
Gaius b040351547
feat: add validation for piece_length (#465)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-02-24 15:52:00 +08:00
Gaius 8d312b5628
feat: change validation of the ttl in UploadPersistentCacheTaskStartedRequest (#459)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-02-12 17:05:38 +08:00
Gaius 7fde34770f
feat: add persistent field for RegisterPersistentCachePeerRequest
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-26 15:12:45 +08:00
Gaius 1920f90700
feat: remove ReschedulePeerFailedRequest and ReschedulePersistentCachePeerFailedRequest
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-21 20:40:33 +08:00
Gaius 214460fd27
feat: add ReschedulePeerFailedRequest and ReschedulePersistentCachePeerFailedRequest (#448)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-21 12:38:09 +08:00
Gaius b9b98411e4
feat: remove digest field in PersistentCacheTask (#443)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-17 11:58:26 +08:00
Gaius 13a45c2c30
feat: remove host_id and task_id in RegisterPersistentCachePeerRequest (#442)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-16 15:32:33 +08:00
Gaius 352bcc2611
feat: set digest to optional for persistent cache task
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-15 14:15:43 +08:00
Gaius 05f657ba25
feat: set output optional for presistent cache (#436)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-03 10:48:02 +08:00
Gaius 8ac708ed9c
fix: partten for crc32 (#420)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-29 19:26:27 +08:00
Gaius f79271fd78
feat: support castagnoli for crc32 (#419)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-25 21:55:42 +08:00
Gaius 5934c78ace
feat: add unknow message for errordetails (#417)
feat: add unknown message for errordetails

Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-11-21 14:49:15 +08:00
Gaius 8cbd5239ad
feat: add task info for UploadPersistentCacheTaskStarted (#408)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-10-30 11:08:27 +08:00
Gaius a65d57a10b
feat: remove sync probes api (#401)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-30 17:13:44 +08:00
Gaius 28a7989601
feat: rename cache task to persistent cache task (#394)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-26 15:34:55 +08:00
Gaius ba358f22e9
feat: remove AnnouncePeers api in scheduler (#392)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-25 21:43:00 +08:00
Chongzhi Deng dbf4bbae2b
feat: add ListHosts proto (#389)
Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
2024-09-23 22:45:41 +08:00
Gaius 304da3a284
fix: wrong name in normal cache task and empty cache task (#353)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-07-17 18:23:59 +08:00
Chongzhi Deng 32065ec25b
feat: add AnnouncePeers proto and fix typo (#343)
feat: add AnnouncePeers proto

1. add AnnouncePeers api
2. fix typo of dfdaemon

Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
2024-07-10 19:51:57 +08:00
Gaius 370d016656
feat: add host_id to UploadCacheTask (#340)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-07-08 17:10:39 +08:00
Gaius 56393b235d
feat: add id for UploadCacheTaskFinishedRequest and UploadCacheTaskFailedRequest (#339)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-07-08 16:46:13 +08:00
Gaius 7998f59978
feat: change UploadCacheTask message (#338)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-07-08 12:36:22 +08:00
Gaius 1bbcef2694
feat: change UploadCacheTaskRequest type to stream (#335)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-07-05 18:17:12 +08:00
Chongzhi Deng 7431d58884
feat: add interval field to AnnounceHostRequest (#330)
Signed-off-by: BruceAko <chongzhi@hust.edu.cn>
2024-06-28 18:42:37 +08:00
Gaius 183280e924
fix: ttl validation format
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-06-18 14:45:53 +08:00
Gaius e1e59d1696
feat: add announce proto of the cache task in scheduler (#327)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-06-17 22:06:34 +08:00
Gaius 110fe0ebc7
feat: change leave to delete in scheduler messages (#325)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-06-17 15:30:39 +08:00
Gaius 7beb7a448f
feat: rename DownloadTask message to Download (#324)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-06-17 15:07:40 +08:00
Gaius 39fa1f1591
feat: add cache task proto (#323)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-06-17 14:54:30 +08:00
Gaius 74581bea67
feat: add message of the backend error detail (#319)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-05-28 18:41:09 +08:00
Gaius ab1743c359
feat: add leave task api (#313)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-05-16 21:32:38 +08:00
Gaius 6bb1bedc70
feat: add error details to api (#283)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-02-28 12:42:22 +08:00
Gaius 9a73b42a15
feat: response http code and status (#280)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-02-27 14:21:37 +08:00
Gaius 16e69e4e9f
feat: remove rate limit in download message (#249)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-01-11 15:45:30 +08:00
Gaius ce44d9501d
feat: change candidate_parent_ids to candidate_parents (#246)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-01-10 10:40:57 +08:00
Gaius 55a239ecc0
feat: remove RegisterSeedPeerRequest in AnnouncePeer (#243)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-25 17:58:21 +08:00
Gaius ac0a2a42a0
feat: remove concurrent_piece_count in scheduler (#228)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-12 14:48:04 +08:00
Gaius 300b38f803
feat: add optional to piece_number in DownloadPieceFailedRequest (#224)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-05 11:14:42 +08:00
Gaius f94caaafdc
feat: remove SyncPiecesFailedRequest in scheduler (#223)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-29 15:06:10 +08:00
Gaius 667eb7320d
feat: change length from int to uint (#220)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-20 15:03:51 +08:00
Gaius b53f8ac93c
feat: change download piece failed message (#210)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-14 22:15:45 +08:00
Gaius 6261753124
feat: add RescheduleRequest message to AnnouncePeerRequest (#209)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-14 21:05:42 +08:00
Gaius cfb441c3e0
feat: remove TinyTaskResponse and SmallTaskResponse message (#208)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-14 14:23:56 +08:00
Gaius b8791ffca0
feat: add SchedulerClusterId to host message in v2 proto (#167)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-21 13:33:46 +08:00
Gaius acf9a4e4f4
feat: add scheduler cluster id in AnnounceHostRequest (#166)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-08-21 11:47:16 +08:00
Gaius 2825951636
feat: add content to piece message (#138)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-13 15:23:27 +08:00
Gaius c1bc8f3ccf
feat: remove ProbeInterval in SyncProbesResponse (#128)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-15 10:43:51 +08:00