94fde2608c
refactor: scheduler register task ( #924 )
...
* refactor: scheduler register task
Signed-off-by: Gaius <gaius.qi@gmail.com>
* refactor: scheduler register task
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:46 +08:00
e252ef5880
Adjust CDN module package ( #893 )
...
* adjust CDN code package structure
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:46 +08:00
f579d3c9ce
refactor: move from io/ioutil to io and os packages ( #906 )
...
* chore: run `go fmt ./...`
This commit synchronizes `//go:build` lines with `// +build` lines.
Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
* refactor: move from io/ioutil to io and os packages
The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil . This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-28 17:32:45 +08:00
a3659820f0
test: idgen peer id ( #913 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:45 +08:00
9800a211da
feat: add data directory ( #910 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:44 +08:00
c2c721bc78
refactor: dfpath pkg ( #879 )
...
* feat: dfpath
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dfpath
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: path
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: plugin
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
f6ce817b72
feat: move dfnet to internal ( #862 )
...
* feat: move dfnet to internal
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename dfutils to util
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:41 +08:00
44730a2e46
unify piece size ( #854 )
...
* unify piece size
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:40 +08:00
d13042bfb5
feat: plugin log ( #843 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:39 +08:00
d79a2a6877
move RPC code definition to proto file ( #829 )
...
* move rpc code to proto
* replace Cdn to CDN
Signed-off-by: sunwp <244372610@qq.com>
2023-06-28 17:32:38 +08:00
8042aea64c
feat: support list plugin ( #819 )
...
* feat: support list plugin
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* chore: optmize plugin output
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:37 +08:00
0cb14903b2
feat: scheduler and cdn report fqdn to manager ( #818 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:37 +08:00
72213eb1d4
feat: dfdaemon get scheduler list dynamically from manager ( #812 )
...
* feat: dfdaemon get scheduler list dynamically from manager
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:37 +08:00
0ad1bf9674
Fix: source plugin not loaded ( #811 )
...
* fix: source plugin not loaded
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:36 +08:00
61f9f83b28
fix: manager typo and cdn peer id ( #809 )
...
* fix: manager typo and cdn peer id
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:36 +08:00
d01eef6b41
refactor: scheduler evaluator ( #805 )
...
* refactor: scheduler evaluator
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:36 +08:00
558782af00
feat: add idgen peer id ( #800 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:35 +08:00
e837ae9bdf
chore: add lint errcheck and fix errcheck( #766 )
...
* feat: add lint errcheck and fix errcheck
Signed-off-by: Gaius <gaius.qi@gmail.com>
* replace assert to require
Signed-off-by: 孙伟鹏 <weipeng.swp@alibaba-inc.com>
Co-authored-by: 孙伟鹏 <weipeng.swp@alibaba-inc.com>
2023-06-28 17:32:35 +08:00
3d63ab2ca3
chore: update golang import lint ( #780 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:32:34 +08:00
1dccea01ce
feat: add jobs api ( #751 )
...
* feat: add jobs api
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:32 +08:00
03774882fd
feat: add job logger ( #740 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:31 +08:00
061e25c3e5
feat: preheat cluster ( #731 )
...
* feat: preheat cluster
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:30 +08:00
466912aa48
docs: scheduler config ( #698 )
...
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:25 +08:00
b4e58e3c08
refactor: scheduler supervisor ( #655 )
...
* refactor: scheduler supervisor
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:49 +08:00
60bee061d2
feat: client back source ( #579 )
...
* feat: client back source
Signed-off-by: santong <244372610@qq.com>
2023-06-28 17:31:43 +08:00
79662a3bb4
feat: start process ( #572 )
...
* feat: server start process
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:38 +08:00
962a171bcd
feat: rename task to job ( #544 )
...
* feat: rename task to job
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:35 +08:00
3657d8f8c4
feature: optimize manager project layout ( #540 )
...
* feat: optimize manager project layout
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add file license
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:35 +08:00
1efd2f6cef
feat: Add distribute Schedule Tracer & Refactor scheduler ( #537 )
...
* check same cdn list avoid updating client too frequent
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:31:35 +08:00
f27770af32
Feature: Add manager charts with submodule ( #525 )
...
* chore: add charts submodule
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:35 +08:00
39ce5d98c0
let cdn support sha256 ( #517 )
...
cdn support sha256 digest
2023-06-28 17:31:34 +08:00
a187d7d7fc
feat: add task manager ( #490 )
...
* feat: add task manager
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: zzy987 <67889264+zzy987@users.noreply.github.com>
2023-06-28 17:31:33 +08:00
a15ce877c7
fix: scheduler concurrent dead lock ( #509 )
...
* fix scheduler concurrent dead lock
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:31:33 +08:00
27103e4b4e
feat: rename manager grpc ( #510 )
...
* feat: rename manager grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:31:26 +08:00
9ffa58c8f6
feat: Add schedule log ( #495 )
...
* feat: add bad node log
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:30:29 +08:00
7b8aaf6b31
Refactor scheduler ( #369 )
...
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:30:29 +08:00
5a5195806b
feat: optimize scheduler dynconfig ( #480 )
...
* feat: optimize scheduler dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:30:29 +08:00
b24f9dc0bd
feat: add searcher to scheduler cluster ( #462 )
...
* feat: add searcher
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:30:28 +08:00
eda5839eb8
Fix client dfget bug and support digest ( #432 )
...
Signed-off-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
2023-06-28 17:30:26 +08:00
777e98784e
feature: export peer.TaskManager for embedding dragonfly in custom binary ( #434 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:30:19 +08:00
2d825f5a96
feat: Add plugin support for resource ( #291 )
...
* feature: add plugin support for resource
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: skip link for mockgen import
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: add generate in Makefile
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: remove unused package
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: remove unused test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: remove unused import package
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:28:01 +08:00
5653f1b800
Refactor manager ( #363 )
...
* feat: handler
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add handlers
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add models and service
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: models
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: migrate
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: database
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: create
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: CDN CURD
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: cdn CURD
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: scheduler CURD
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: security group CURD
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: cdn instances CURD
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: associated CURD
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: id type
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: association
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: rest
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: health and prometheus
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove swagger
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: ginprometheus
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add cache
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: gorm naming
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename host to hostname
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename host to hostname
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add manager grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: security_group
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: security_group
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: manager grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add grpc internal get cdn
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: config
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: cache
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc cache
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc cache
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: cache time
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: database mysql
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: manager init
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: update scheduler
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: config
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: config
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: init db
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: init db
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: register
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: tag
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: add
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: manager
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: config
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: grpc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: keepAlive
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: keepAlive
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler config
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: typo
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: typo
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: typo
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: cdn
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: cdn
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove println
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: manager
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:28:00 +08:00
cab5ed6d2e
Feature/dfget param ( #373 )
...
* update image for arch
Signed-off-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
* feature: adjust dfget grpc param
Signed-off-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
2023-06-28 17:27:59 +08:00
5175e8009a
Fix: infinite recursion in MkDirAll ( #358 )
...
* fix: infinite recursion in MkDirAll
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* disable manager test, enable cdn test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:27:51 +08:00
6cce5a9872
Feature/code spec ( #336 )
...
* Modify the code according to https://github.com/uber-go/guide/blob/master/style.md
Signed-off-by: santong <weipeng.swp@alibaba-inc.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:24:44 +08:00
fb85383a08
fix: when load config from cdn directory in dynconfig, skip sub directories ( #310 )
...
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:22:41 +08:00
7726b0f621
Dynconfig in schduler gets the cdn configuration from the directory ( #304 )
...
* feat: add dynconfig get cdn infos from directory
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: use dynconfig.LocalSourceType
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:41 +08:00
c25792bbb8
Added GC public library ( #267 )
...
* feat: gc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: init gc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dynconfig validate
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: generate gc mocks
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: gc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: gc
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: add gc run timeout
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove return pointer
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove NewWithOptions
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove gomock finish
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:39 +08:00
a777c8882c
Fix: use atomic to avoid data race in client ( #254 )
...
* fix: use atomic to avoid data race
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: temporary disable cdnsystem&manager test for CI
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: update .github/workflows/ci.yaml and codecov.yml
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: update ratelimiter test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feature: update dynconfig test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: ratelimiter and file utils test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: ratelimiter and queue test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* fix: httputil test
Signed-off-by: Jim Ma <majinjing3@gmail.com>
* feat: remove httputils
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:38 +08:00
7e595898e5
Fix lint ( #244 )
...
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: lint hostIP
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: error check
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: nil pointer
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename taskID
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename peerID
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: taskID
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: licenses
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: golangci-lint
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:22:37 +08:00
93d638c806
Refactor scheduler's dynconfig ( #228 )
...
* feat: scheduler's dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: cdn manager use dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler config
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove file
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler cdn manager
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: scheduler dynconfig log
* feat: scheduler dynconfig observers
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove scheduler dynconfig logger
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler mocks pkg rename mocks
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: remove reflect.DeepEqual
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: manager client
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: scheduler start
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: scheduler dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dynconfig watch server
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: scheduler config
Signed-off-by: Gaius <gaius.qi@gmail.com>
* test: dynconfig
Signed-off-by: Gaius <gaius.qi@gmail.com>
* fix: manager CDN hosts
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: rename scheduler dynconfig Start func
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dynconfig type
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: manager keepalive
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: dynconfig netAddrs
Signed-off-by: Gaius <gaius.qi@gmail.com>
* feat: scheduler config validate
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: santong <weipeng.swp@alibaba-inc.com>
2023-06-28 17:22:37 +08:00
f592032f72
Refactor cmd ( #217 )
...
* refator:cmd and path
Signed-off-by: zuozheng.hzz <zuozheng.hzz@alibaba-inc.com>
2023-06-28 17:22:37 +08:00
Gaius