Commit Graph

1653 Commits

Author SHA1 Message Date
Gaius 57c29ee51a
feat: remove redis Pipelined in network topology (#2416)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:55 +08:00
Gaius 989f658544
feat: optimize network topology comment (#2415)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:55 +08:00
dlut_xz 5bc465f7d8
test: add probes and network topology unit tests (#2414)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:54 +08:00
Gaius 9999ce51b6
feat: add ProbedAt to network topology (#2413)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:54 +08:00
Gaius 5794cfc02c
refactor: network topology package (#2412)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:54 +08:00
Gaius b4e71f586d
test: optimize network topology and probes tests (#2409)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:54 +08:00
ZouYu bc4ca6270b
test: add unit test for ComputePieceCount (#2401)
Signed-off-by: Zou Yu <zouy.fnst@fujitsu.com>
2023-06-28 17:34:54 +08:00
dlut_xz 8630740952
test: add probes and network topology unit tests (#2390)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:54 +08:00
dependabot[bot] de8abcf563
chore(deps): bump github.com/montanaflynn/stats from 0.7.0 to 0.7.1 (#2407)
Bumps [github.com/montanaflynn/stats](https://github.com/montanaflynn/stats) from 0.7.0 to 0.7.1.
- [Release notes](https://github.com/montanaflynn/stats/releases)
- [Changelog](https://github.com/montanaflynn/stats/blob/master/CHANGELOG.md)
- [Commits](https://github.com/montanaflynn/stats/compare/v0.7.0...v0.7.1)

---
updated-dependencies:
- dependency-name: github.com/montanaflynn/stats
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:54 +08:00
dependabot[bot] 9fff4c29fd
chore(deps): bump github.com/mdlayher/vsock from 1.2.0 to 1.2.1 (#2405)
Bumps [github.com/mdlayher/vsock](https://github.com/mdlayher/vsock) from 1.2.0 to 1.2.1.
- [Release notes](https://github.com/mdlayher/vsock/releases)
- [Changelog](https://github.com/mdlayher/vsock/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mdlayher/vsock/compare/v1.2.0...v1.2.1)

---
updated-dependencies:
- dependency-name: github.com/mdlayher/vsock
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:54 +08:00
dependabot[bot] 204f71df5b
chore(deps): bump github.com/stretchr/testify from 1.8.2 to 1.8.3 (#2408)
Bumps [github.com/stretchr/testify](https://github.com/stretchr/testify) from 1.8.2 to 1.8.3.
- [Release notes](https://github.com/stretchr/testify/releases)
- [Commits](https://github.com/stretchr/testify/compare/v1.8.2...v1.8.3)

---
updated-dependencies:
- dependency-name: github.com/stretchr/testify
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:54 +08:00
dependabot[bot] 647dcd5574
chore(deps): bump go.uber.org/atomic from 1.10.0 to 1.11.0 (#2404)
Bumps [go.uber.org/atomic](https://github.com/uber-go/atomic) from 1.10.0 to 1.11.0.
- [Release notes](https://github.com/uber-go/atomic/releases)
- [Changelog](https://github.com/uber-go/atomic/blob/master/CHANGELOG.md)
- [Commits](https://github.com/uber-go/atomic/compare/v1.10.0...v1.11.0)

---
updated-dependencies:
- dependency-name: go.uber.org/atomic
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:54 +08:00
Gaius b5c5b5b2e6
feat: implement Enqueue and AverageRTT in probes.go (#2393)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius a44c084f28
feat: handle context in triggerSeedPeerTask (#2392)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 730dcbf2bc
feat: optimize field name of ProbeConfig (#2391)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
dlut_xz fd4422a044
test: add unit tests in pkg/redis (#2384)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:53 +08:00
Gaius 8de3594a8b
feat: scheduler supports to disable redis (#2389)
feat: scheduler support to disable redis

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Jim Ma afb47f6729
chore: check grpc peer info for download service (#2385)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius a91797c9e9
test: add slice packege tests (#2386)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
周丽贞 c487853f93
fix: call MakeNamespaceKeyInScheduler function error (#2383)
Signed-off-by: zhoulizhen <2313464078@qq.com>
2023-06-28 17:34:53 +08:00
Gaius 3ad8ddd4b5
refactor: probes package in network topology (#2382)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius 72ac0c0adf
feat: add Reverse function to slice (#2381)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:53 +08:00
Gaius e1113c17c5
refactor: network topology package in scheduler (#2380)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:52 +08:00
dlut_xz f0b52945bf
fix: package declaration error (#2379)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:52 +08:00
Gaius 81817666e5
feat: move redis key to pkg/redis package (#2378)
Network Topology adds functions to new redis key and move the manager's redis key to pkg/redis.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:52 +08:00
dlut_xz deab6df2c2
feat: add network topology package (#2364)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:52 +08:00
dependabot[bot] aadf52279b
chore(deps): bump go.opentelemetry.io/otel/exporters/jaeger from 1.14.0 to 1.15.1 (#2376)
chore(deps): bump go.opentelemetry.io/otel/exporters/jaeger

Bumps [go.opentelemetry.io/otel/exporters/jaeger](https://github.com/open-telemetry/opentelemetry-go) from 1.14.0 to 1.15.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.14.0...v1.15.1)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/jaeger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:52 +08:00
dependabot[bot] 994ccdbb3d
chore(deps): bump gorm.io/driver/mysql from 1.5.0 to 1.5.1 (#2374)
Bumps [gorm.io/driver/mysql](https://github.com/go-gorm/mysql) from 1.5.0 to 1.5.1.
- [Commits](https://github.com/go-gorm/mysql/compare/v1.5.0...v1.5.1)

---
updated-dependencies:
- dependency-name: gorm.io/driver/mysql
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:52 +08:00
dependabot[bot] 31654775f2
chore(deps): bump golang.org/x/oauth2 from 0.7.0 to 0.8.0 (#2372)
Bumps [golang.org/x/oauth2](https://github.com/golang/oauth2) from 0.7.0 to 0.8.0.
- [Commits](https://github.com/golang/oauth2/compare/v0.7.0...v0.8.0)

---
updated-dependencies:
- dependency-name: golang.org/x/oauth2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:52 +08:00
dependabot[bot] c3114ff1e9
chore(deps): bump github.com/huaweicloud/huaweicloud-sdk-go-obs from 3.22.11+incompatible to 3.23.4+incompatible (#2373)
chore(deps): bump github.com/huaweicloud/huaweicloud-sdk-go-obs

Bumps [github.com/huaweicloud/huaweicloud-sdk-go-obs](https://github.com/huaweicloud/huaweicloud-sdk-go-obs) from 3.22.11+incompatible to 3.23.4+incompatible.
- [Release notes](https://github.com/huaweicloud/huaweicloud-sdk-go-obs/releases)
- [Commits](https://github.com/huaweicloud/huaweicloud-sdk-go-obs/compare/v3.22.11...v3.23.4)

---
updated-dependencies:
- dependency-name: github.com/huaweicloud/huaweicloud-sdk-go-obs
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:52 +08:00
Gaius 320097b44d
feat: add announceToTrainer in scheduler (#2371)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:52 +08:00
ZouYu 5f9438f831
test: add test case "new dfpath by dataDir" (#2368)
Signed-off-by: Zou Yu <zouy.fnst@fujitsu.com>
2023-06-28 17:34:52 +08:00
Gaius f5aacf3c9b
chore: change gorm-adaptor version to v3.5.0 (#2370)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius 701a26a8d6
feat: hide sensitive information in log (#2369)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius 25edc40937
fix: evaluate after filter (#2363)
Since the final length of the filter is the candidateParentLimit used,
the parents after the filter is the returned parents.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius ddcea685e7
fix: when bufferSize is zero, storage can not write data to file (#2366)
When bufferSize is zero, createDownload and createNetworkTopology
can not write data to file, because buffer is empty.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius d66126f6a4
test: improve timeout in recursive download (#2367)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius 6339b14ed9
feat: replace net dial with grpc health check in client (#2361)
Replace net dial with grpc health check in manager grpc client
and security grpc client. Make grpc health check consistent.

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Gaius 71f13df1f0
feat: remove traffic_type in DownloadPeerDuration metric (#2357)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:51 +08:00
Tianxi f2130509d1
feat: add traffic type of peer task download duration (#2349)
Co-authored-by: li.tianxi <li.tianxi@hellogroup.com>
2023-06-28 17:34:51 +08:00
dependabot[bot] f1d6560358
chore(deps): bump golang.org/x/crypto from 0.8.0 to 0.9.0 (#2355)
Bumps [golang.org/x/crypto](https://github.com/golang/crypto) from 0.8.0 to 0.9.0.
- [Commits](https://github.com/golang/crypto/compare/v0.8.0...v0.9.0)

---
updated-dependencies:
- dependency-name: golang.org/x/crypto
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:51 +08:00
dependabot[bot] a1e1a6f7d9
chore(deps): bump github.com/casbin/gorm-adapter/v3 from 3.16.1 to 3.17.0 (#2353)
chore(deps): bump github.com/casbin/gorm-adapter/v3

Bumps [github.com/casbin/gorm-adapter/v3](https://github.com/casbin/gorm-adapter) from 3.16.1 to 3.17.0.
- [Release notes](https://github.com/casbin/gorm-adapter/releases)
- [Changelog](https://github.com/casbin/gorm-adapter/blob/master/.releaserc.json)
- [Commits](https://github.com/casbin/gorm-adapter/compare/v3.16.1...v3.17.0)

---
updated-dependencies:
- dependency-name: github.com/casbin/gorm-adapter/v3
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:50 +08:00
dependabot[bot] d0159b0dfb
chore(deps): bump go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc from 0.39.0 to 0.41.1 (#2352)
chore(deps): bump go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc

Bumps [go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc](https://github.com/open-telemetry/opentelemetry-go-contrib) from 0.39.0 to 0.41.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go-contrib/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go-contrib/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go-contrib/compare/zpages/v0.39.0...zpages/v0.41.1)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:50 +08:00
lilongfeng d43e5351ad
misspelling (#2351)
Signed-off-by: 李龙峰 <lilongfeng@inspur.com>
2023-06-28 17:34:50 +08:00
Gaius 2edfca4e5e
feat: change DefaultServerPort to 9090 in trainer (#2348)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:50 +08:00
Gaius 99dda4e684
chore: checkout code first in CI (#2347)
setup-go has built-in functionality for caching and restoring go modules and build outputs. The action defaults to
search for the dependency file - go.sum in the repository root, and uses its hash as a part of the cache key.
To use the cache, we should first checkout code. Otherwise setup-go can't find go.sum and use/restore cache.
Related: https://github.com/actions/setup-go#caching-dependency-files-and-build-outputs

Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:50 +08:00
Yadong Ding 1b0105fc6e
chore: checkout code first in CI (#2346)
chore: checkout code first

setup-go has a built-in functionality for caching and restoring go modules and build outputs.
The action defaults to search for the dependency file - go.sum in the repository root, and uses its hash as a part of the cache key.
To use the cache, we should first checkout code. Otherwise action can't find go.sum.

Signed-off-by: Yadong Ding <ding_yadong@foxmail.com>
2023-06-28 17:34:50 +08:00
Gaius b97c0cb0ae
feat: remove deprecated field in manager and scheduler (#2345)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:50 +08:00
Gaius 600ccdbd57
chore: update redis config in docker compose and update helm chart version (#2344)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:50 +08:00
dlut_xz 557046d0d4
feat: add database config and move redis to it (#2338)
Signed-off-by: XZ <834756128@qq.com>
2023-06-28 17:34:50 +08:00