dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,653 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

31 Commits

Author SHA1 Message Date
Gaius 0222649b88
feat: update namely/protoc-all image version to 1.47_0 (#1418) 
* feat: update namely/protoc-all image version to 1.47_0

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate mock file

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:32 +08:00
Gaius 64f5e9357b
refactor: scheduler announce task (#1407) 
* feat: scheduler announce normal task

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cid

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:30 +08:00
Gaius 5763ee86bd
feat: add seed peer metrics (#1342) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:23 +08:00
Gaius 2a0314e45b
refactor: scheduler grpc (#1310) 
* refactor: scheduler grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: generate protoc

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:20 +08:00
Jim Ma 823c722c3e
feat: support response header (#1292) 
* feat: support response header via ExtendAttribute
* chore: update http pass through header
* chore: update ExtendAttribute in cdn downloader

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:19 +08:00
Jim Ma 49f9ebde68
feat: implement client seed mode (#1247) 
* feat: implement client seed mode

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:18 +08:00
Gaius 076ba20b51
feat: move dfnet to pkg dir (#1265) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:15 +08:00
Eryu Guan 65fa8a1680
feat: define and implement new dfdaemon APIs to make dragonfly2 work as a distributed cache (#1227) 
Introduce a new dfcache command which uses dfcache SDK interface to
interactive with dfdaemon and operate on P2P cache system.

For example:
- add a file into cache system
  dfcache import --cid sha256:xxxxxx --tag testtag /path/to/file

- check if a file exists in cache system
  dfcache stat --cid testid --local  # only check local cache
  dfcache stat --cid testid          # check other peers as well

- export/download a file from cache system
  dfcache export --cid testid -O /path/to/output

- delete a file from cache system, both local cache and P2P network
  dfcache delete -i testid -t testtag

Signed-off-by: Gaius <gaius.qi@gmail.com>
Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:13 +08:00
Gaius 3fdde9b891
fix: make generate (#1228) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:11 +08:00
Gaius 04bd2ea383
feat: scheduler addresses log (#1183) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:33:08 +08:00
Jim Ma 10da5fa0d6
feat: implement grpc client side sync pieces (#1167) 
* feat: implement grpc client side sync pieces

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: client unit test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: report sync piece error

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* chore: optimize pieceTaskSyncManager cancel

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:07 +08:00
Jim Ma 98e95e693c
feat: implement bidirectional fetch pieces (#1165) 
* feat: implement grpc server side sync pieces

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:06 +08:00
Jim Ma ac4c4bb912
feat: reuse partial completed task (#1107) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:33:00 +08:00
Jim Ma 4597f74283
feat: merge ranged request storage into parent (#1078) 
* feat: merge ranged request storage into parent

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:59 +08:00
Jim Ma f80c75efdb
Feature: prefetch ranged requests (#1053) 
1. implement prefetch ranged requests
2. optimize exact http code in transport
3. simplify reuse peer task logic
4. reuse peer task for ranged request size error
5. fix data race for peer task storage

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:56 +08:00
sunwp 25832aa3d2
cdn rpc server test (#1020) 
Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:55 +08:00
Jim Ma c00f07c2de
Extract storage driver into peer tasks (#998) 
* chore: extract storage instead load every time

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: gofmt

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
Jim Ma 6f76c74838
chore: clarify daemon interface (#991) 
1. Rename FilePeerTask to FileTask.
2. Rename StreamPeerTask to StreamTask.
3. Extract pulling piece task logic to pieceTaskPoller.

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma c38bba2612
feat: merge same tasks in daemon (#977) 
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:49 +08:00
sunwp 456e776bfd
correct metadata spell (#884) 
Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:42 +08:00
Gaius f6ce817b72
feat: move dfnet to internal (#862) 
* feat: move dfnet to internal

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename dfutils to util

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:41 +08:00
Gaius 72213eb1d4
feat: dfdaemon get scheduler list dynamically from manager (#812) 
* feat: dfdaemon get scheduler list dynamically from manager

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:37 +08:00
Jim Ma 806f2ef11d
feat: calculate piece metadata digest (#787) 
* feat: calculate piece metadata digest
* fix: back source not work in midway
* chore: add partial back source test
* chore: optimize digest log

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:35 +08:00
Jim Ma 777e98784e
feature: export peer.TaskManager for embedding dragonfly in custom binary (#434) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:30:19 +08:00
Gaius 5653f1b800
Refactor manager (#363) 
* feat: handler

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add handlers

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add models and service

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: models

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: migrate

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: database

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: create

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: CDN CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: cdn CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: scheduler CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: security group CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: cdn instances CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: associated CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: id type

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: association

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: rest

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: health and prometheus

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove swagger

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: ginprometheus

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: gorm naming

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename host to hostname

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename host to hostname

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add manager grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: security_group

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: security_group

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add grpc internal get cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: cache time

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: database mysql

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager init

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: init db

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: init db

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: register

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: tag

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: keepAlive

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: keepAlive

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: lint

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: typo

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: typo

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: typo

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove println

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:28:00 +08:00
sunwp 6cce5a9872
Feature/code spec (#336) 
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:24:44 +08:00
Jim Ma 5b9c3033f8
Support peer task data multiplex (#347) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:23:35 +08:00
sunwp 155ee02a54
Add CDN unit-tests (#338) 
* feat: refactor cdn storage module & remove unused ctx & cdn test

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:22:43 +08:00
Gaius f9b4d77bb2
Fix golang lint (#249) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:22:38 +08:00
sunwp a89e915aee
feature: remove redundant configurations (#216) 
* feature: remove redundant configurations

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
 2023-06-28 17:22:29 +08:00
Jim Ma 0af8817c1f
Merge branch main-rc to branch main 
Signed-off-by: Jim Ma <majinjing3@gmail.com>

Co-authored-by: 李玉海 <yuhai.lyh@alibaba-inc.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
Co-authored-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
Co-authored-by: 加菲 <garfieldhu@aliyun.com>
 2023-06-28 17:20:20 +08:00