dragonflyoss
/
dragonfly
mirror of https://github.com/dragonflyoss/dragonfly.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
1,907 Commits 5 Branches 281 Tags 98 MiB
Commit Graph

1907 Commits

Author SHA1 Message Date
Jim Ma e9f824e0a7
chore: optimize defer and test (#1010) 
* chore: optimize defer and test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: random test failed

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:53 +08:00
sunwp 537667cfaa
add cdn totalpieces (#1014) 
Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:52 +08:00
Gaius 6fee32b0f8
chore: workflow add test timeout (#1011) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:52 +08:00
Gaius 69b56e9713
feat: add actions job timout (#1008) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:52 +08:00
Jim Ma c00f07c2de
Extract storage driver into peer tasks (#998) 
* chore: extract storage instead load every time

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: test

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: gofmt

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
sunwp c9b2c0acf8
fix: calculate piece MD5 sign when last piece download (#1006) 
* fix: calculate piece md5 sign when last piece download

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:52 +08:00
Gaius 43d769b0b4
fix: register task with size scope (#1003) 
* feat: register task with size scope

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: fix peer manager data race

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:52 +08:00
Gaius 5359ac2e75
feat: set peer state to running when scope size is SizeScope_TINY (#1004) 
* chore: sync docker-compose scheduler config (#1001)

Signed-off-by: Jim Ma <majinjing3@gmail.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: set peer state to running when scope size is SizeScope_TINY

Signed-off-by: Gaius <gaius.qi@gmail.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
Gaius 90f77cdc4e
feat: update submodule charts (#1002) 
* feat: add pprof config
* feat: update submodule helm charts

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:52 +08:00
Gaius f8ebd597e2
feat: task mutex replace sync kmutex (#1000) 
* feat: trace https proxy request (#996)

* feat: trace https proxy request

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: daemon test

Signed-off-by: Jim Ma <majinjing3@gmail.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove sync kmutex and add task mutex

Signed-off-by: Gaius <gaius.qi@gmail.com>

Co-authored-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:52 +08:00
Jim Ma 9a8540d211
chore: sync docker-compose scheduler config (#1001) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Gaius 5479b3a53e
feat: stream send error code (#986) 
* feat: change task and peer ttl (#984)

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: send error code

Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add retry interval

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon report successful piece before end of piece

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update submodule version

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma 0c03af1979
fix: when scheduler is not available, replace the scheduler client (#999) 
* fix: when scheduler is not available, just replace the scheduler client in single peer task

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma c3951e0cb3
feat: trace https proxy request (#996) 
* feat: trace https proxy request

Signed-off-by: Jim Ma <majinjing3@gmail.com>

* fix: daemon test

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma c87be98799
chore: parameterize tests in peer task (#994) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma 88278be464
fix: total pieces count not set cause digest invalid (#992) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma 6f76c74838
chore: clarify daemon interface (#991) 
1. Rename FilePeerTask to FileTask.
2. Rename StreamPeerTask to StreamTask.
3. Extract pulling piece task logic to pieceTaskPoller.

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:51 +08:00
Gaius 02913a5442
feat: add scheduler host gc (#989) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:51 +08:00
guoguodan 3b6966c585
feat: update typo in local_storage.go (#955) 2023-06-28 17:32:51 +08:00
Gaius 91c15f01d1
feat: update charts submodule version (#985) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:51 +08:00
Jim Ma 17ff9c17be
fix: send piece result error not handled (#987) 
* fix: send piece result error not handled
* chore: optimize daemon cancel logic
* chore: remove redundant log and ctx.Done check
* chore: handle piece download with 404 status

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:50 +08:00
yxxhero 2d8d449bcf
fix markdown lint error (#988) 
Signed-off-by: yxxhero <aiopsclub@163.com>
 2023-06-28 17:32:50 +08:00
Gaius 54d6542aa6
feat: change task and peer ttl (#984) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:50 +08:00
yihongfei 0868c7a6f7
Ffix: typo in Makefile (#975) 2023-06-28 17:32:50 +08:00
Jim Ma cb36b79dd1
feat: when write last piece, generate digest (#982) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:50 +08:00
Gaius c3263bb7fa
fix: scheduler config typo (#983) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:50 +08:00
Gaius 97c1eb65e7
Refactor scheduler service (#958) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:50 +08:00
Jim Ma 37b332a366
test: dump goroutine in e2e (#980) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:50 +08:00
Jim Ma c38bba2612
feat: merge same tasks in daemon (#977) 
* feat: merge same tasks in daemon
* test: add tiny file test in daemon
* fix: when validate digest disabled, did not store metadata

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:49 +08:00
lilongfeng 174d7d664e
fix typo in CHANGELOG.md (#976) 
Signed-off-by: 李龙峰 <lilongfeng@inspur.com>
 2023-06-28 17:32:49 +08:00
sunwp 25f8cd56dd
Fix hdfs write (#948) 
* rename reader to limitedReader

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:49 +08:00
Gaius c95d835c6b
feat: if cdn is deleted, clear cdn related information (#967) 
* feat: if cdn is deleted, clear cdn related information

Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:49 +08:00
Gaius b94102db47
chore: change docker.pkg.github.com to ghcr.io (#973) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:49 +08:00
Jim Ma a54ae4fae4
feat: add default DiskGCThresholdPercent and ignore it when is 0 (#971) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:49 +08:00
Jim Ma 6ec8b48dd3
fix: schedulers send invalid direct piece (#970) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:49 +08:00
Christian Bourgeois 0706fc959e
feat: improve redirect to allow url rewrite (#969) 
* feat: improve redirect to allow url rewrite

Signed-off-by: momiji <teq1uila-free01@yahoo.fr>
 2023-06-28 17:32:49 +08:00
Christian Bourgeois b4cc13db41
feat: Add useProxies to registryMirror allowing to mirror more anything (#965) 
* feat: add useProxies to registryMirror to allow mirroring anything

Signed-off-by: momiji <teq1uila-free01@yahoo.fr>
 2023-06-28 17:32:49 +08:00
Gaius 9f75cabc34
feat: change metrics port to 8000 (#964) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma aee13a46ea
chore: copy e2e proxy log to artifact (#962) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma 38178804e8
feat: add daemon metrics support (#960) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Eryu Guan e893fe9359
fix: use 'parent' as mainPeer in PeerPacket in removePeerFromCurrentTree() (#957) 
We've removed 'peer' from current tree, and ScheduleChildren() set
'parent' as parent of peers in 'children', so we should inform children
that 'parent' is the destination peer, not the removed 'peer'.

Fixes: #956

Signed-off-by: Eryu Guan <eguan@linux.alibaba.com>
 2023-06-28 17:32:48 +08:00
Jim Ma ec55989716
chore: add version metric (#954) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma b7f8ad4e20
feat: support disk usage gc in client (#953) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma b5b7b75786
docs: update plugin doc (#951) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma 8d6e4044a4
chore: optimize back source update digest logic (#950) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
Jim Ma beaf4ce18d
feat: update source.Response and source client interface (#945) 
* feat: update source.Response and source client interface

Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:48 +08:00
sunwp 0a152e01e9
Fix HDFS File reader (#940) 
* remove unused hdfs writeTo method

Signed-off-by: sunwp <244372610@qq.com>
 2023-06-28 17:32:48 +08:00
Gaius 0b21f66714
feat: remove stat log from scheduler (#946) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:47 +08:00
Jim Ma 062687e4c7
feat: support recursive download in dfget (#932) 
Signed-off-by: Jim Ma <majinjing3@gmail.com>
 2023-06-28 17:32:47 +08:00
Gaius 01d022e85b
fix: size scope empty (#941) 
Signed-off-by: Gaius <gaius.qi@gmail.com>
 2023-06-28 17:32:47 +08:00