On graceful shutdown we will call forceGC() to delete all cached tasks
if 'keepStorage' is false. And we rely on deleteTask() to delete the
given task from both local storage and scheduler.
But commit 86a6030c8f ("feat: unregister failed task storage (#1717)")
changed MarkReclaim() to MarkInvalid() in deleteTask(), and resulted in
task leak in scheduler.
Fix it by calling MarkReclaim() in deleteTask().
Fixes: 86a6030c8f ("feat: unregister failed task storage (#1717)")
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
* feat: save list metadata to p2p
* chore: optimize rpc download worker logic
* feat: add expire header reclaim logic
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: add disable seed peer action
* fix: concurrent back source
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
* feat: update namely/protoc-all image version to 1.47_0
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: generate mock file
Signed-off-by: Gaius <gaius.qi@gmail.com>
fix: seed peer did not send done seed result
fix: unknown length back source seed did not send content length to other peers
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feat: support response header via ExtendAttribute
* chore: update http pass through header
* chore: update ExtendAttribute in cdn downloader
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.
For example:
- add a file into cache system
dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file
- check if a file exists in cache system
dfcache stat --cid testid --local # only check local cache
dfcache stat --cid testid # check other peers as well
- export/download a file from cache system
dfcache export --cid testid -O /path/to/output
- delete a file from cache system, both local cache and P2P network
dfcache delete -i testid -t testtag
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
* fix: client sync piece panic
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: optimize sync piece log
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: daemon panic when register fail
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feat: optimize back source trace and logic
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: fix keepalive performance
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: optimize generate peer id and transport performance
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: io.Copy call superfluous sendfile, then increase network latency
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: extract storage instead load every time
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: gofmt
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Jim Ma