Commit Graph

37 Commits

Author SHA1 Message Date
fu220 8a5885e249 # Description
This pull request introduces APIs for the new Cache Task mechanism, enabling support for memory cache operations. Additionally, by refactoring the Download model and removing the load_to_cache field, the Task component now focuses exclusively on disk I/O operations, while the Cache Task handles memory cache operations. Finally, the version number of the dragonfly-api package has been incremented.

## Key Changes

### common.proto
- Added a new enum type Cache to TaskType to identify CacheTask.
- Introduced a new structure CachePeer to represent peers handling cache tasks.
- Added a new structure CacheTask to define metadata for cache tasks.
- Removed the load_to_cache field from the Download structure, as Tasks will no longer handle cache operations.

### dfdaemon.proto
- Added three RPC calls (DownloadCacheTask, StatCacheTask, DeleteCacheTask) to the DfdaemonUpload and DfdaemonDownload services for cache task management.
- Added two RPC calls (SyncCachePieces, DownloadCachePiece) to the DfdaemonUpload service for cache piece synchronization.
- Introduced 9 new message types to support requests and responses for CacheTask downloads.

### scheduler.proto
- Added five RPC calls to the Scheduler service: AnnounceCachePeer, StatCachePeer, DeleteCachePeer for cache peer management. StatCacheTask, DeleteCacheTask for cache task management.
- Introduced 16 new message types to cover the full lifecycle state transitions of cache tasks.

## Version Update
- Incremented the version of the dragonfly-api package from 2.1.47 to 2.1.48 in Cargo.toml to reflect the new changes.

## Related Issue

## Motivation and Context
- Separation of Responsibilities: Tasks now focus exclusively on disk I/O operations, while CacheTask independently handles memory cache management.

Signed-off-by: fu220 <2863318196@qq.com>
2025-07-22 09:52:58 +00:00
Gaius 20d9c4bc90
feat: add ListTaskEntries for downloading directory (#505)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-07-08 11:45:19 +08:00
Gaius 8e1782b9d8
feat: add ExchangeIBVerbsQueuePairEndpoint to DfdaemonUpload
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-03-25 18:52:55 +08:00
Gaius b7701531cf
feat: add ExchangeIBVerbsQueuePairEndpoint for DfdaemonUpload (#474)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-03-25 18:34:27 +08:00
Gaius 6bd235200f
feat: add IbVerbsConnection service to support connection between ibverbs peers (#473)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-03-24 20:16:07 +08:00
Gaius be1ca95b4c
feat: remove piece_length in Task (#466)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-02-26 11:43:43 +08:00
Gaius c8e0b2526f
feat: add UpdatePersistentCacheTask api
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-26 19:06:56 +08:00
Gaius 2c761170dd
feat: add DeletePersistentCacheTask for UploadDfdaemon
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-26 13:26:42 +08:00
Gaius c8398377dd
feat: remove DeletePersistentCacheTask in DfdaemonDownload (#452)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-26 11:06:44 +08:00
Gaius 05b74d50e3
feat: remove update_persistent_cache_task api (#451)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-24 17:52:18 +08:00
Gaius 2a23158326
feat: remove response from UpdatePersistentCacheTask
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-22 11:13:39 +08:00
Gaius d4e497d037
feat: add UpdatePersistentCacheTask to upload server (#450)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-22 10:36:51 +08:00
Gaius c7f8c2a600
feat: add UpdatePersistentCacheTask for setting the value of persistent (#449)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-21 22:14:01 +08:00
Gaius 161f9d810f
feat: add DownloadPersistentCachePiece and SyncPersistentCachePieces api (#447)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-20 12:08:33 +08:00
Gaius 76b2eac2d0
feat: add ReadPersistentCacheTask and WritePersistentCacheTask message (#441)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2025-01-15 12:14:58 +08:00
baowj 190d05d9bf
feat: add method SyncHost to service DfdaemonUpload (#431)
Signed-off-by: baowj <bwj_678@qq.com>
2024-12-31 16:48:34 +08:00
Gaius 28a7989601
feat: rename cache task to persistent cache task (#394)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-09-26 15:34:55 +08:00
Gaius 75e49e0ee4
feat: add DeleteTaskRequest and StatCacheTask to DfdaemonUpload (#370)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-08-13 12:41:35 +08:00
Gaius 110fe0ebc7
feat: change leave to delete in scheduler messages (#325)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-06-17 15:30:39 +08:00
Gaius 39fa1f1591
feat: add cache task proto (#323)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-06-17 14:54:30 +08:00
Gaius 9f7fd897b8
feat: change TriggerDownloadTask to DownloadTask with stream (#302)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-04-16 20:00:32 +08:00
Gaius 4ca58f1c92
feat: replace LeaveHostRequest with empty (#271)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-02-06 22:24:03 +08:00
Gaius b9e871c8cc
feat: add leave host to dfdaemon (#270)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2024-02-06 22:09:54 +08:00
Gaius a71c5bd200
feat: remove TriggerDownloadTaskResponse in TriggerDownloadTask (#238)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-19 18:01:48 +08:00
Gaius 2292870068
feat: add TriggerDownloadTask api for seed peer (#235)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-19 15:04:57 +08:00
Gaius 79249ee02f
feat: add DownloadTask to DfdaemonUpload (#232)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-18 21:23:19 +08:00
Gaius 5425d1bb1b
feat: dfdaemon is divided into download and upload (#231)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-18 12:32:26 +08:00
Gaius d333a929ee
feat: add sync pieces api (#227)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-07 22:06:29 +08:00
Gaius 011a20f6ed
feat: add DownloadPiece api (#225)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-12-06 17:43:34 +08:00
Gaius 8fdd8c81c1
feat: remove request stream in SyncPieces (#214)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-16 15:40:57 +08:00
Gaius 6d865929d3
feat: add stream response to DownloadTask (#212)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-15 17:23:09 +08:00
Gaius db4b1f4325
feat: add GetPieceNumbers api in dfdaemon_grpc (#203)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-11-01 18:13:33 +08:00
Gaius aae0807767
feat: update verison to v2 (#147)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-07-20 21:56:21 +08:00
Gaius 602aa22af8
refactor: download metadata in common (#66)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-02-10 18:13:16 +08:00
Gaius b2e4c03aac
feat: add DownloadTask api (#65)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-02-06 22:05:03 +08:00
Gaius b0bebd2958
fix: grpc namespace conflict (#54)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-01-10 12:24:27 +08:00
Gaius 58fdcc6ff5
feat: add ImportTask, ExportTask and DeleteTask api (#50)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2022-12-21 14:10:08 +08:00