Commit Graph

67 Commits

Author SHA1 Message Date
dlut_xz 8a8d6abaae
feat: move probe interval from scheduler config to client config (#2462)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:58 +08:00
Gaius 730dcbf2bc
feat: optimize field name of ProbeConfig (#2391)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 8de3594a8b
feat: scheduler supports to disable redis (#2389)
feat: scheduler support to disable redis

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 320097b44d
feat: add announceToTrainer in scheduler (#2371)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:52 +08:00
dlut_xz 557046d0d4
feat: add database config and move redis to it (#2338)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:50 +08:00
Gaius e34b66f58c
fix: redis validation in scheduler config (#2287)
When Redis is in cluster mode, only DB 0 can be used.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:47 +08:00
MinH-09 fa59bde001
feat: add train interval and trainer addresses (#2223)
Signed-off-by: MinH-09 <2107139596@qq.com>
2023-06-28 17:34:43 +08:00
dlut_xz 0a9cd254f4
feat: add probes struct (#2190)
Signed-off-by: XZ <834756128@qq.com>
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:42 +08:00
MinH-09 1994765fce
feat: add trainer config in scheduler (#2214)
Signed-off-by: MinH-09 <2107139596@qq.com>
2023-06-28 17:34:41 +08:00
Garen Wen 0f66064a3a
feat: add advertise port (#2156)
Co-authored-by: wenmanxiang <wenmanxiang@baidu.com>
2023-06-28 17:34:39 +08:00
Gaius c672e75ed8
test: add Validate test to scheduler config (#2129)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:35 +08:00
Gaius da8c45f2a8
feat: add host ttl to scheduler (#2089)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:31 +08:00
dlut_xz 91c9d83b6a
feat: add networkTopology configuration to scheduler (#2070)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:31 +08:00
Gaius 286885733a
feat: remove training configuration in scheduler (#2081)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:31 +08:00
Gaius a5685582ce
refactor: resource task with v2 version of grpc (#2078)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:30 +08:00
Gaius 1666a17827
feat: remove NetTopology from scheduler and manager (#2007)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:25 +08:00
Gaius 008eeb980b
feat: add IPAddresses and DNSNames to sans of the cert (#1930)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:18 +08:00
Gaius 731ecc1234
feat: change yaml field type from string to net.IP (#1929)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:18 +08:00
Gaius f120c8778b
feat: support redis sentinal (#1910)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:16 +08:00
Gaius 63b4a868dc
refactor: back-to-source configuration (#1895)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:15 +08:00
Gaius 8749814dee
feat: add the timeout of downloading piece to scheduler (#1880)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:14 +08:00
Gaius 4bd7d413f2
feat: add workHome and pluginDir to configuration (#1807)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:08 +08:00
Gaius 500f767073
feat: remove task and host gc ttl (#1735)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:01 +08:00
Gaius f354ee3861
refactor: manager and scheduler config (#1701)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:57 +08:00
Jim Ma 89376d6c4e
feat: ipv6 support (#1685)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
Co-authored-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:57 +08:00
Gaius 13caf6f0c2
refactor: listenIP and advertiseIP (#1694)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:56 +08:00
Gaius 29d4452277
feat: support redis cluster (#1667)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:54 +08:00
Gaius b6c395a6cb
feat: add cert spec to security configuration (#1621)
Signed-off-by: Gaius <gaius.qi@gmail.com>
Co-authored-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:33:51 +08:00
Gaius 43d8478507
feat: add tls policy to scheduler grpc server (#1616)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:50 +08:00
Gaius 066fc3f3af
feat: add default metrics address (#1595)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:47 +08:00
Gaius 8a874964b7
feat: scheduler set workhome (#1493)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:39 +08:00
Gaius 527990620b
refactor: scheduler training configuration
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:38 +08:00
Gaius 70ed392263
feat: scheduler add data dir (#1453)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:35 +08:00
1037husterljx 3eccba8e21
Added unit tests on configuration (#1430)
Signed-off-by: ljx373327 <ljx373327@alibaba-inc.com>

Co-authored-by: ljx373327 <ljx373327@alibaba-inc.com>
2023-06-28 17:33:33 +08:00
Gaius a2511cb945
feat: rewrite interface{} to any (#1419)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:32 +08:00
Gaius 7a30db3fba
refactor: pkg util (#1402)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:29 +08:00
Gaius faa5e4e465
feat: add seed peer logic (#1302)
* feat: announce seed peer

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn logic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn job

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfdaemon change host uuid to host id

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: go generate mocks

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: remove cdn compatibility

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change docker compose

Signed-off-by: Gaius <gaius.qi@gmail.com>

* fix: reuse panic

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: compatible with v2.0.3-beta.2

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:19 +08:00
Gaius 9c1e05983a
feat: scheduler add seed peer (#1298)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:18 +08:00
Gaius b2c9577e7b
Add csv storage to scheduler (#1234)
* feat: scheduler add csv storage for peer data
* feat: add storage configuration to scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:13 +08:00
Gaius 7a5857d389
feat: change scheduler and cdn listen (#1205)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:10 +08:00
Gaius 0326f781f4
feat: optimize scheduler (#1106)
* feat: optimize scheduler

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:33:00 +08:00
Gaius e0a494e8a8
feat: when scheduler blocks cdn, resource does not initialize cdn (#1081)
* feat: when scheduler blocks cdn, resource does not initialize cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: scheduler remove manager enable config

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:58 +08:00
Gaius 060429c675
feat: scheduler blocks cdn (#1079)
* feat: scheduler blocks cdn

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:57 +08:00
Gaius 61310b55d5
feat: when cdn peer is failed, peer should be back-to-source (#1005)
* feat: schdule peer with cdn failed

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: when cdn peer is failed, peer back-to-source

Signed-off-by: Gaius <gaius.qi@gmail.com>

* test: callback

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: change scheduler default config

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:53 +08:00
Gaius 02913a5442
feat: add scheduler host gc (#989)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:51 +08:00
Gaius 97c1eb65e7
Refactor scheduler service (#958)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:50 +08:00
Eng Zer Jun f579d3c9ce
refactor: move from io/ioutil to io and os packages (#906)
* chore: run `go fmt ./...`

This commit synchronizes `//go:build` lines with `// +build` lines.

Reference: https://go.googlesource.com/proposal/+/master/design/draft-gobuild.md
Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>

* refactor: move from io/ioutil to io and os packages

The io/ioutil package has been deprecated as of Go 1.16, see
https://golang.org/doc/go1.16#ioutil. This commit replaces the existing
io/ioutil functions with their new definitions in io and os packages.

Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>
2023-06-28 17:32:45 +08:00
Gaius c2c721bc78
refactor: dfpath pkg (#879)
* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: dfpath

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: path

Signed-off-by: Gaius <gaius.qi@gmail.com>

* feat: plugin

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:43 +08:00
Gaius d01eef6b41
refactor: scheduler evaluator (#805)
* refactor: scheduler evaluator

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:32:36 +08:00
Gaius e837ae9bdf
chore: add lint errcheck and fix errcheck(#766)
* feat: add lint errcheck and fix errcheck

Signed-off-by: Gaius <gaius.qi@gmail.com>

* replace assert to require

Signed-off-by: 孙伟鹏 <weipeng.swp@alibaba-inc.com>

Co-authored-by: 孙伟鹏 <weipeng.swp@alibaba-inc.com>
2023-06-28 17:32:35 +08:00