Commit Graph

92 Commits

Author SHA1 Message Date
Gaius 7acb092706
fix: scheduler config validation (#1274)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:16 +08:00
Gaius b2c9577e7b
Add csv storage to scheduler (#1234)
* feat: scheduler add csv storage for peer data
* feat: add storage configuration to scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
Gaius 7a5857d389
feat: change scheduler and cdn listen (#1205)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:10 +08:00
Gaius 4e670035ed
feat: change reschdule config (#1158)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:05 +08:00
Gaius 0326f781f4
feat: optimize scheduler (#1106)
* feat: optimize scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:00 +08:00
Gaius e0a494e8a8
feat: when scheduler blocks cdn, resource does not initialize cdn (#1081)
* feat: when scheduler blocks cdn, resource does not initialize cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: scheduler remove manager enable config

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:58 +08:00
Gaius 060429c675
feat: scheduler blocks cdn (#1079)
* feat: scheduler blocks cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:57 +08:00
Gaius 61310b55d5
feat: when cdn peer is failed, peer should be back-to-source (#1005)
* feat: schdule peer with cdn failed

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: when cdn peer is failed, peer back-to-source

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: callback

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change scheduler default config

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:53 +08:00
Gaius 5479b3a53e
feat: stream send error code (#986)
* feat: change task and peer ttl (#984)

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: send error code

Signed-off-by: Gaius <gaius.qi@gmail.com>

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add retry interval

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon report successful piece before end of piece

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update submodule version

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:51 +08:00
Gaius 02913a5442
feat: add scheduler host gc (#989)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:51 +08:00
Gaius 54d6542aa6
feat: change task and peer ttl (#984)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:50 +08:00
Gaius c3263bb7fa
fix: scheduler config typo (#983)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:50 +08:00
Gaius 97c1eb65e7
Refactor scheduler service (#958)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:50 +08:00
Gaius c2c721bc78
refactor: dfpath pkg (#879)
* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: path

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: plugin

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
Gaius 0cb14903b2
feat: scheduler and cdn report fqdn to manager (#818)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:37 +08:00
Gaius d01eef6b41
refactor: scheduler evaluator (#805)
* refactor: scheduler evaluator

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:36 +08:00
Jim Ma 2ee0585ff3
feat: support traffic metrics by peer host (#776)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:34 +08:00
Gaius 8388cab8ef
feat: load limit (#739)
* feat: load limit
* feat: default load

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:30 +08:00
Gaius 466912aa48
docs: scheduler config (#698)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:25 +08:00
zzy987 36a32460d5
docs: Update documents (#595)
* docs: Add zh docs (#592)
Co-authored-by: santong <244372610@qq.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: yxxhero <11087727+yxxhero@users.noreply.github.com>
2023-06-28 17:31:44 +08:00
sunwp 60bee061d2
feat: client back source (#579)
* feat: client back source

Signed-off-by: santong <244372610@qq.com>
2023-06-28 17:31:43 +08:00
Gaius 79662a3bb4
feat: start process (#572)
* feat: server start process

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:38 +08:00
sunwp 720873c7f9
Reload CDN client (#566)
* optimaze dynamic cdn hosts

Signed-off-by: santong <244372610@qq.com>

* feat: set default cdn loadLimit

Signed-off-by: santong <244372610@qq.com>
2023-06-28 17:31:37 +08:00
Gaius 66e3ca538f
feat: optimize scheduler start server (#558)
* feat: optimize scheduler start server

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:37 +08:00
Gaius 962a171bcd
feat: rename task to job (#544)
* feat: rename task to job

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:35 +08:00
sunwp 1efd2f6cef
feat: Add distribute Schedule Tracer & Refactor scheduler (#537)
* check same cdn list avoid updating client too frequent

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:31:35 +08:00
Gaius f27770af32
Feature: Add manager charts with submodule (#525)
* chore: add charts submodule

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:35 +08:00
Gaius a187d7d7fc
feat: add task manager (#490)
* feat: add task manager

Signed-off-by: Gaius <gaius.qi@gmail.com>

Co-authored-by: zzy987 <67889264+zzy987@users.noreply.github.com>
2023-06-28 17:31:33 +08:00
sunwp a15ce877c7
fix: scheduler concurrent dead lock (#509)
* fix scheduler concurrent dead lock

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:31:33 +08:00
sunwp a6a92e75d0
fix: scheduler pick candidate and associate child encounter dead lock (#500)
* fix dead lock

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:30:30 +08:00
sunwp 6c8cf122f8
Rescheduler parent/children later when no candidates and add schedule log (#497)
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:30:29 +08:00
sunwp 39ee24b0bf
feat: Extract peer event processing function (#489)
* feat: Extract peer event processing function

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:30:29 +08:00
sunwp 7b8aaf6b31
Refactor scheduler (#369)
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:30:29 +08:00
Gaius 5a5195806b
feat: optimize scheduler dynconfig (#480)
* feat: optimize scheduler dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:30:29 +08:00
Gaius a56fe4f579
feat: add host info for scheduler and cdn (#457)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:30:27 +08:00
Jim Ma ef5e1756a1
fix: update DynconfigOptions typo (#390)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:28:00 +08:00
Gaius 5653f1b800
Refactor manager (#363)
* feat: handler

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add handlers

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add models and service

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: models

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: migrate

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: database

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: create

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: CDN CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: cdn CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: scheduler CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: security group CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: cdn instances CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: associated CURD

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: id type

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: association

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: rest

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: health and prometheus

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove swagger

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: ginprometheus

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: gorm naming

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename host to hostname

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename host to hostname

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add manager grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: security_group

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: security_group

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add grpc internal get cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc cache

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: cache time

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: database mysql

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager init

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: update scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: init db

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: init db

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: register

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: tag

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: add

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: grpc

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: keepAlive

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: keepAlive

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: lint

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: typo

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: typo

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: typo

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove println

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:28:00 +08:00
Gaius 7726b0f621
Dynconfig in schduler gets the cdn configuration from the directory (#304)
* feat: add dynconfig get cdn infos from directory

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: use dynconfig.LocalSourceType

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:41 +08:00
Gaius 93d638c806
Refactor scheduler's dynconfig (#228)
* feat: scheduler's dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: cdn manager use dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove file

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler cdn manager

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: scheduler dynconfig log

* feat: scheduler dynconfig observers

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove scheduler dynconfig logger

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler mocks pkg rename mocks

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove reflect.DeepEqual

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager client

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: scheduler start

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: scheduler dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dynconfig watch server

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: scheduler config

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: dynconfig

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: manager CDN hosts

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: rename scheduler dynconfig Start func

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dynconfig type

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: manager keepalive

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dynconfig netAddrs

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: scheduler config validate

Signed-off-by: Gaius <gaius.qi@gmail.com>

Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:22:37 +08:00
加菲 f592032f72
Refactor cmd (#217)
* refator:cmd and path 

Signed-off-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
2023-06-28 17:22:37 +08:00
sunwp a89e915aee
feature: remove redundant configurations (#216)
* feature: remove redundant configurations

Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:22:29 +08:00
Jim Ma 0af8817c1f
Merge branch main-rc to branch main
Signed-off-by: Jim Ma <majinjing3@gmail.com>

Co-authored-by: 李玉海 <yuhai.lyh@alibaba-inc.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
Co-authored-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
Co-authored-by: 加菲 <garfieldhu@aliyun.com>
2023-06-28 17:20:20 +08:00