* feat: support response header via ExtendAttribute
* chore: update http pass through header
* chore: update ExtendAttribute in cdn downloader
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.
For example:
- add a file into cache system
dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file
- check if a file exists in cache system
dfcache stat --cid testid --local # only check local cache
dfcache stat --cid testid # check other peers as well
- export/download a file from cache system
dfcache export --cid testid -O /path/to/output
- delete a file from cache system, both local cache and P2P network
dfcache delete -i testid -t testtag
Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
* feat: implement grpc client side sync pieces
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: client unit test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: report sync piece error
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: optimize pieceTaskSyncManager cancel
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: extract storage instead load every time
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: gofmt
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feat: register task with size scope
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: fix peer manager data race
Signed-off-by: Gaius <gaius.qi@gmail.com>
1. Rename FilePeerTask to FileTask.
2. Rename StreamPeerTask to StreamTask.
3. Extract pulling piece task logic to pieceTaskPoller.
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: run `go fmt ./...`
This commit synchronizes `//go:build` lines with `// +build` lines.
Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* feat: calculate piece metadata digest
* fix: back source not work in midway
* chore: add partial back source test
* chore: optimize digest log
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: use atomic to avoid data race
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: temporary disable cdnsystem&manager test for CI
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: update .github/workflows/ci.yaml and codecov.yml
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: update ratelimiter test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: update dynconfig test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: ratelimiter and file utils test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: ratelimiter and queue test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: httputil test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feat: remove httputils
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
Gaius