Commit Graph

1907 Commits

Author SHA1 Message Date
Gaius cf50824526
feat: change announcer validation (#1869)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:13 +08:00
Gaius 45ad287250
feat: add mysql read and write timeout (#1868)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:13 +08:00
Gaius 5276523f11
feat: store parent information (#1867)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:13 +08:00
Gaius a9f10bd678
feat: remove MainParent from peer and add IsPieceBackToSource to piece
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:13 +08:00
Gaius 73439f6f22
chore: add Kuaishou to ADOPTERS.md (#1866)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:12 +08:00
cuidajun 51683c2fd9
Fix: [scheduler] destPeer keepalive when downloaded by other peer (#1865)
Signed-off-by: bigerous <cuidajun.cdj@alibaba-inc.com>
2023-06-28 17:34:12 +08:00
Gaius 3af4cc7a0f
feat: scheduler supports storage config (#1864)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:12 +08:00
Gaius 4f91332992
feat: store peer download information (#1863)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:12 +08:00
walleliu 91f67603bb
add username when create redis client (#1862) 2023-06-28 17:34:12 +08:00
Gaius 68172eee24
feat: manager changes filterParentLimit value (#1859)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:12 +08:00
Gaius a1b65218db
feat: optimize gc package (#1855)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:12 +08:00
Gaius ef6ad12587
feat: add announcer to scheduler (#1854)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:12 +08:00
dependabot[bot] 5b104d4c83
chore(deps): bump github.com/aws/aws-sdk-go from 1.44.121 to 1.44.143 (#1853)
Bumps [github.com/aws/aws-sdk-go](https://github.com/aws/aws-sdk-go) from 1.44.121 to 1.44.143.
- [Release notes](https://github.com/aws/aws-sdk-go/releases)
- [Commits](https://github.com/aws/aws-sdk-go/compare/v1.44.121...v1.44.143)

---
updated-dependencies:
- dependency-name: github.com/aws/aws-sdk-go
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:12 +08:00
dependabot[bot] 6a4c57be60
chore(deps): bump github.com/prometheus/client_golang from 1.13.0 to 1.14.0 (#1851)
chore(deps): bump github.com/prometheus/client_golang

Bumps [github.com/prometheus/client_golang](https://github.com/prometheus/client_golang) from 1.13.0 to 1.14.0.
- [Release notes](https://github.com/prometheus/client_golang/releases)
- [Changelog](https://github.com/prometheus/client_golang/blob/main/CHANGELOG.md)
- [Commits](https://github.com/prometheus/client_golang/compare/v1.13.0...v1.14.0)

---
updated-dependencies:
- dependency-name: github.com/prometheus/client_golang
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:12 +08:00
dependabot[bot] 81883536c5
chore(deps): bump github.com/aliyun/aliyun-oss-go-sdk from 2.2.5+incompatible to 2.2.6+incompatible (#1849)
chore(deps): bump github.com/aliyun/aliyun-oss-go-sdk

Bumps [github.com/aliyun/aliyun-oss-go-sdk](https://github.com/aliyun/aliyun-oss-go-sdk) from 2.2.5+incompatible to 2.2.6+incompatible.
- [Release notes](https://github.com/aliyun/aliyun-oss-go-sdk/releases)
- [Changelog](https://github.com/aliyun/aliyun-oss-go-sdk/blob/master/CHANGELOG.md)
- [Commits](https://github.com/aliyun/aliyun-oss-go-sdk/compare/v2.2.5...v2.2.6)

---
updated-dependencies:
- dependency-name: github.com/aliyun/aliyun-oss-go-sdk
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:11 +08:00
dependabot[bot] e6e7ba966d
chore(deps): bump github.com/bits-and-blooms/bitset from 1.3.3 to 1.4.0 (#1848)
Bumps [github.com/bits-and-blooms/bitset](https://github.com/bits-and-blooms/bitset) from 1.3.3 to 1.4.0.
- [Release notes](https://github.com/bits-and-blooms/bitset/releases)
- [Commits](https://github.com/bits-and-blooms/bitset/compare/v1.3.3...v1.4.0)

---
updated-dependencies:
- dependency-name: github.com/bits-and-blooms/bitset
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:11 +08:00
dependabot[bot] ba789a20f2
chore(deps): bump k8s.io/component-base from 0.25.3 to 0.25.4 (#1847)
Bumps [k8s.io/component-base](https://github.com/kubernetes/component-base) from 0.25.3 to 0.25.4.
- [Release notes](https://github.com/kubernetes/component-base/releases)
- [Commits](https://github.com/kubernetes/component-base/compare/v0.25.3...v0.25.4)

---
updated-dependencies:
- dependency-name: k8s.io/component-base
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:11 +08:00
Gaius 2421ce88f4
feat: add announcer to dfdameon (#1852)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
Gaius 43b9c39149
feat: when dfdaemon disable object storage, dynconfig can't fetch manager (#1845)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
Gaius fc0eccaf74
feat: optimize manager log (#1846)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
Gaius f194f0961f
feat: scheduler adds announce host handler (#1843)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
Jim Ma 7e90f43a53
chore: update dst peer log (#1844)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:11 +08:00
Gaius bdf5cd54c5
feat: call all nodes in consistent hashing and reuse grpc connection (#1842)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:11 +08:00
Jim Ma 3fe342dabc
fix: recursive download always return none error (#1841)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:10 +08:00
Jim Ma 1c24c3fc8c
fix: expire header timezone (#1840)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:10 +08:00
dependabot[bot] 3582a10b91
chore(deps): bump github.com/onsi/gomega from 1.23.0 to 1.24.1 (#1832)
Bumps [github.com/onsi/gomega](https://github.com/onsi/gomega) from 1.23.0 to 1.24.1.
- [Release notes](https://github.com/onsi/gomega/releases)
- [Changelog](https://github.com/onsi/gomega/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/gomega/compare/v1.23.0...v1.24.1)

---
updated-dependencies:
- dependency-name: github.com/onsi/gomega
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:10 +08:00
Jim Ma ea02bbefde
chore: update e2e test (#1839)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:10 +08:00
Jim Ma f6d8d5c6b2
chore: remove unused code (#1838)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:10 +08:00
dependabot[bot] 900835f277
chore(deps): bump github.com/onsi/ginkgo/v2 from 2.4.0 to 2.5.0 (#1836)
Bumps [github.com/onsi/ginkgo/v2](https://github.com/onsi/ginkgo) from 2.4.0 to 2.5.0.
- [Release notes](https://github.com/onsi/ginkgo/releases)
- [Changelog](https://github.com/onsi/ginkgo/blob/master/CHANGELOG.md)
- [Commits](https://github.com/onsi/ginkgo/compare/v2.4.0...v2.5.0)

---
updated-dependencies:
- dependency-name: github.com/onsi/ginkgo/v2
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:10 +08:00
dependabot[bot] f2bcc5c47a
chore(deps): bump github.com/mdlayher/vsock from 1.1.1 to 1.2.0 (#1834)
Bumps [github.com/mdlayher/vsock](https://github.com/mdlayher/vsock) from 1.1.1 to 1.2.0.
- [Release notes](https://github.com/mdlayher/vsock/releases)
- [Changelog](https://github.com/mdlayher/vsock/blob/main/CHANGELOG.md)
- [Commits](https://github.com/mdlayher/vsock/compare/v1.1.1...v1.2.0)

---
updated-dependencies:
- dependency-name: github.com/mdlayher/vsock
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:10 +08:00
Gaius ca9afc5b9d
feat: update concurrent-map version (#1837)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:10 +08:00
Gaius 02543c781d
feat: optimize scope size is error (#1831)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:10 +08:00
Gaius 0f8b75808a
feat: add timeout grpc and job (#1830)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:10 +08:00
Jim Ma fdd189e661
chore: enable cache list metadata e2e (#1829)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:09 +08:00
Gaius 42223d994f
feat: optimize peer log (#1828)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
Jim Ma ba58047871
feat: optional save list metadata to p2p (#1822)
* feat: save list metadata to p2p
* chore: optimize rpc download worker logic
* feat: add expire header reclaim logic

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:09 +08:00
Jim Ma 5040af8b4f
feat: add s3 resource client and recursive e2e test (#1826)
* feat: add s3 resource client
* chore: add minio server and recursive e2e test
* chore: add download grpc test tool

Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:09 +08:00
Gaius d2f98d1429
feat: optimize preheat log (#1827)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
Gaius 5c25a325c6
feat: seed peer reuses traffic (#1825)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
Jim Ma f581d10208
chore: daemon avoid alway open metadata files (#1823)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:09 +08:00
Gaius 3da67311ea
feat: optimize preheat (#1824)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
Gaius b1c049db46
feat: returns an scheduling error if the peer state is not PeerStateRunning (#1821)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:09 +08:00
Gaius 073c84dd71
feat: optimize peer gc (#1819)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:08 +08:00
Gaius 83ec42cf25
feat: peer.UpdateAt needs to be updated during download process (#1818)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:08 +08:00
Jim Ma 9378ce410a
chore: close out of use client grpc conn (#1817)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:08 +08:00
Gaius 96910887b2
feat: statistical the traffic of reused peer (#1816)
Signed-off-by: Gaius <gaius.qi@gmail.com>
2023-06-28 17:34:08 +08:00
Jim Ma 6fd389c27a
fix: otel goroutine leak (#1815)
Signed-off-by: Jim Ma <majinjing3@gmail.com>
2023-06-28 17:34:08 +08:00
dependabot[bot] b440be333e
chore(deps): bump go.opentelemetry.io/otel/exporters/jaeger from 1.10.0 to 1.11.1 (#1813)
chore(deps): bump go.opentelemetry.io/otel/exporters/jaeger

Bumps [go.opentelemetry.io/otel/exporters/jaeger](https://github.com/open-telemetry/opentelemetry-go) from 1.10.0 to 1.11.1.
- [Release notes](https://github.com/open-telemetry/opentelemetry-go/releases)
- [Changelog](https://github.com/open-telemetry/opentelemetry-go/blob/main/CHANGELOG.md)
- [Commits](https://github.com/open-telemetry/opentelemetry-go/compare/v1.10.0...v1.11.1)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/otel/exporters/jaeger
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:08 +08:00
dependabot[bot] 7a486afa1b
chore(deps): bump github.com/shirou/gopsutil/v3 from 3.22.9 to 3.22.10 (#1812)
Bumps [github.com/shirou/gopsutil/v3](https://github.com/shirou/gopsutil) from 3.22.9 to 3.22.10.
- [Release notes](https://github.com/shirou/gopsutil/releases)
- [Commits](https://github.com/shirou/gopsutil/compare/v3.22.9...v3.22.10)

---
updated-dependencies:
- dependency-name: github.com/shirou/gopsutil/v3
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:08 +08:00
dependabot[bot] 6fd1233dad
chore(deps): bump go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin from 0.36.1 to 0.36.4 (#1811)
chore(deps): bump go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin

Bumps [go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin](https://github.com/gin-gonic/gin) from 0.36.1 to 0.36.4.
- [Release notes](https://github.com/gin-gonic/gin/releases)
- [Changelog](https://github.com/gin-gonic/gin/blob/master/CHANGELOG.md)
- [Commits](https://github.com/gin-gonic/gin/commits)

---
updated-dependencies:
- dependency-name: go.opentelemetry.io/contrib/instrumentation/github.com/gin-gonic/gin/otelgin
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <support@github.com>

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2023-06-28 17:34:08 +08:00