Commit Graph

61 Commits

Author SHA1 Message Date
lhy1024 9738245505
pd client: add valid function for pd client config (#1699)
Signed-off-by: lhy1024 <admin@liudos.us>
2025-07-07 06:56:24 +00:00
glorv 1430158810
resource_control: support collecting cross AZ traffic in ru consumption (#1669)
Signed-off-by: glorv <glorvs@163.com>
2025-06-25 21:28:54 +00:00
zyguan 96b6487fac
*: support async batch get (#1646)
ref tikv/client-go#1586

Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-06-16 02:21:36 +00:00
王超 d1ec4b4f67
txn: handle undetermined error in client go (#1642)
close tikv/client-go#1641

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2025-05-20 07:00:04 +00:00
zyguan 7cca6713e9
tikv: disable health-feedback in next-gen (#1635)
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-04-22 12:39:19 +00:00
yibin 157f083989
Fix backoff lose info when forked (#1627)
ref pingcap/tidb#60271

Signed-off-by: yibin87 <huyibin@pingcap.com>
2025-04-21 02:21:14 +00:00
crazycs e7894a7b27
config: change grpc-keepalive-timeout from uint to float64, for support millisecond timeout (#1461)
Signed-off-by: crazycs520 <crazycs520@gmail.com>

Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
2024-09-11 04:15:06 +00:00
you06 c810ed88fb
region_request: remove backoff for stale read (#1423)
* remove stale read backoff

Signed-off-by: you06 <you1474600@gmail.com>

* sort imports

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-08-15 10:09:19 +08:00
zyguan 4c6b2171b2
improve batch efficiency for high throughput workloads (#1411)
* improve batch efficiency by multiple attempts

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* add support for aggressive batching

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* add some metrics for batch client

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fetch more requests according to recent wait head durs

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* add experimental batch options

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* optimize and refactor

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* some minor updates

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix the metric of head arrival interval

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* a minor update

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* some minor fixes

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* update according to the spec

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix the ut

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix batch condition

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* add some doc comments

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* address https://github.com/tikv/client-go/pull/1411#discussion_r1714086744

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* rename some vars according to the comments

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* add more comments

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-08-13 12:55:44 +08:00
crazycs 4a72526f6c
add enable-replica-selector-v2 config back but mark it was deprecated (#1374)
* add enable-replica-selector-v2 config back but mark it was deprecated

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* update comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-06-26 14:42:48 +08:00
crazycs 0cc1c5239d
Remove old replica selector (#1265)
* remove EnableReplicaSelectorV2 config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove replicaSelectorV1

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add replica flag to reduce struct size

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove contextPatcher

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* use option.preferLeader

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-27 17:11:02 +08:00
crazycs c82e921992
fix issue of doesn't fast fail request when no available connections (#1339)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-13 19:07:04 +08:00
Liqi Geng 44b2944b64
Add grpc window size config and change the default value
Signed-off-by: gengliqi <gengliqiii@gmail.com>
2024-03-19 23:01:50 +08:00
crazycs 0606e74e8e
enable replica-selector-v2 by default (#1229)
* refine replica selector v2 config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-15 15:42:30 +08:00
crazycs 8d6a95f73d
replica selector refactor (#1142)
* init

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add ReplicaSelector interface

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add replica_selector_v2, todo: fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix all test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test in another way to compatible with old version about stale-read request

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* tiny refactor

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refactor to remove duplicate code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine onServerIsBusy region error handing logic

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support forwarding by proxy

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support busyThreshold and tiny reractor code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix proxy bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test and tiny refactor

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add some test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* tidy refine

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test in v1

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add backoff-cnt check and timeout test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add benchmark test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make test stale

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix golangci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix race test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more comprehensive enumeration testing

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix accessFollower with label retry bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test and fix some bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test and add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine onNotLeader logic when new leader is not available

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine calculateScore logic, if the replica already tried, decrease the score

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine logic: replica-read request with mixed strategy and with label, should be able to retry all remain replicas

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* prefer-leader for mixed read with label for non stale-read req

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test case

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* resolve conflict

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove some duplicate test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refactor test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add busy_threshold test and fix bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix proxy bug and add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add learner test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove old test and refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* move test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* use new score calculation by flag bit

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make test stable

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment,license and fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix race test timeout

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add flagNotAttemp in score

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-11 15:40:44 +08:00
crazycs 50c4085d0a
reduce unnecessary tikvServerBusy backoff when able to try next replica (#1184)
* reduce unnecessary tikvServerBusy backoff when

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:38:29 +08:00
crazycs 6f9550fda3
stale read request shoudn't retry leader if leader is already tried (#1174)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-23 18:01:14 +08:00
tongjian 824302acd0
client: Cache tikv request in tidb client side (#1098)
* impl priority queue

Signed-off-by: bufferflies <1045931706@qq.com>

* replace priority queue

Signed-off-by: bufferflies <1045931706@qq.com>

* cache request in tidb side

Signed-off-by: bufferflies <1045931706@qq.com>

* fix gosimple

Signed-off-by: bufferflies <1045931706@qq.com>

* impl priority

Signed-off-by: bufferflies <1045931706@qq.com>

* pass ut

Signed-off-by: bufferflies <1045931706@qq.com>

* add

Signed-off-by: bufferflies <1045931706@qq.com>

* remove request if the request has been canceled

Signed-off-by: bufferflies <1045931706@qq.com>

* remove request if it has been canceled

Signed-off-by: bufferflies <1045931706@qq.com>

* add comment for cancel

Signed-off-by: bufferflies <1045931706@qq.com>

* not make the loop is busy

Signed-off-by: bufferflies <1045931706@qq.com>

* lint

Signed-off-by: bufferflies <1045931706@qq.com>

* revert busy loop

Signed-off-by: bufferflies <1045931706@qq.com>

* add unit test

Signed-off-by: bufferflies <1045931706@qq.com>

* not limit ehigh prioirty test

Signed-off-by: bufferflies <1045931706@qq.com>

* pass lint

Signed-off-by: bufferflies <1045931706@qq.com>

* support all

Signed-off-by: bufferflies <1045931706@qq.com>

* add comment

Signed-off-by: bufferflies <1045931706@qq.com>

* squash

Signed-off-by: bufferflies <1045931706@qq.com>

* revert all to All

Signed-off-by: bufferflies <1045931706@qq.com>

* remove index from entry

Signed-off-by: bufferflies <1045931706@qq.com>

* make fail reasons more clear

Signed-off-by: bufferflies <1045931706@qq.com>

---------

Signed-off-by: bufferflies <1045931706@qq.com>
2024-02-21 14:54:33 +08:00
ekexium 70c148e84e
ErrQueryInterrupted with parameters (#1124)
* feat: ErrQueryInterrupted with parameters

Signed-off-by: ekexium <eke@fastmail.com>

* Revert "Revert "fix: check kill signal against 0 (#1102)" (#1129)"

This reverts commit 3480b5ed7c.

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-02-02 10:26:02 +08:00
Weizhen Wang e6f5a45b00
fix misspell (#1135)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-01-25 11:09:10 +08:00
Weizhen Wang f90605363e
support grpc's shard buffer pool (#1132)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-01-24 17:57:47 +08:00
zyguan e79e8008ce
fix the issue that health check may set liveness wrongly (#1127)
* fix the issue that health check may set liveness wrongly

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix lint issue

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix rawkv ut

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix data race

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* use getStore instead of accessing storeMu directly

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* make TestAccessFollowerAfter1TiKVDown stable

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* make TestBackoffErrorType stable

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* address comments

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2024-01-24 15:11:01 +08:00
MyonKeminta 3480b5ed7c
Revert "fix: check kill signal against 0 (#1102)" (#1129)
This reverts commit 057c479dd8.

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-01-23 13:54:05 +08:00
Weizhen Wang 1fd589dbf2
Change MinTLSVersion to TLSv1.2 (#1116)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-01-16 12:05:09 +08:00
ekexium 057c479dd8
fix: check kill signal against 0 (#1102)
* fix: check kill signal against 0, so that all kill signals will not be ignored.

Signed-off-by: ekexium <eke@fastmail.com>

* refactor: extract method checkKilled()

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-01-10 15:48:27 +08:00
Weizhen Wang 85e66083bc
*: set default tls.MinVersion as tls.VersionTLS10 (#1088)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-12-25 10:10:32 +08:00
Smilencer 6ba909c4ad
config: enable user configuration for retry strategy. (#1055)
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 14:54:10 +08:00
Spade A e540aa3b96
make cop request timeout a config paramter (#865)
* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

* update

Signed-off-by: Spade A <u6748471@anu.edu.au>

---------

Signed-off-by: Spade A <u6748471@anu.edu.au>
2023-07-11 15:58:55 +08:00
Smilencer f64d5dd79b
update dco (#731)
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-04-27 14:45:04 +08:00
Yongbo Jiang 3f7860f109
region_request: handle resource group error before Backoff (#719)
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-03-02 13:40:57 +08:00
Yongbo Jiang e2da552702
*: Update pd client (#722)
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-02-28 17:15:02 +08:00
disksing 9b3ecc1dca
*: update pd client (#694)
Signed-off-by: disksing <i@disksing.com>
2023-02-07 12:00:04 +08:00
JmPotato f24b7314ff Refine the code
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2023-01-19 20:31:38 +08:00
JmPotato 6ee294fedf Introduce the resource control client
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2023-01-18 17:14:24 +08:00
David f313ddf58d
reformat codec and add keyspace support (#649)
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
2023-01-10 15:15:33 +08:00
Jack Yu 6bf6951325
use global variable instead of config (#458)
Signed-off-by: Jack Yu <jackysp@gmail.com>
2022-03-29 17:20:50 +08:00
Jack Yu 8489c3e8c3
config,txnkv: make txnCommitBatchSize adjustable by config (#457)
Signed-off-by: Jack Yu <jackysp@gmail.com>
2022-03-28 16:37:38 +08:00
Sandy Xu b0d9bb8821
Bump go version to 1.16 (#379)
* Revert "use ioutil.ReadFile to release dependency of Go 1.16 (#217)"

This reverts commit aadf3cf627.

Signed-off-by: Sandy Xu <sandy@juicedata.io>

* use go 1.16 for examples

Signed-off-by: Sandy Xu <sandy@juicedata.io>
2021-11-18 06:06:13 -08:00
disksing 9ec50224be
*: replace pingcap/errors with pkg/errors (#357)
Signed-off-by: disksing <i@disksing.com>
2021-11-01 11:59:52 +08:00
disksing c425022782
*: remove use of errors.Trace (#353)
Signed-off-by: disksing <i@disksing.com>
2021-10-28 16:25:58 +08:00
lysu 4fc565e203
config: move resolve-lock-lite-threshold to tikv-client segment (#297) 2021-09-02 14:23:07 +08:00
lysu eb7e4884a8
config: support config resolve lock lite threshold (#291)
Signed-off-by: lysu <sulifx@gmail.com>
2021-09-02 05:31:00 +00:00
disksing b7d1044eb5
config: remove unused utility (#269)
Signed-off-by: disksing <i@disksing.com>
2021-08-16 11:55:36 +08:00
disksing 288c649636
fix license (#267)
Signed-off-by: disksing <i@disksing.com>
2021-08-15 01:48:01 +08:00
Sandy Xu aadf3cf627
use ioutil.ReadFile to release dependency of Go 1.16 (#217)
Signed-off-by: Sandy Xu <sandy@juicedata.io>
2021-07-09 05:25:06 +00:00
Yilin Chen 394d9735f8
add global async commit and 1pc config (#216)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-07-08 11:36:09 +08:00
disksing d0a98864a6
*: move logutil to internal directory (#207)
Signed-off-by: disksing <i@disksing.com>
2021-07-05 15:15:41 +08:00
disksing 28ad1f94e6
util: add flag to control all failpoints (#176)
Signed-off-by: disksing <i@disksing.com>
2021-06-26 21:46:26 +08:00
disksing a0410b45bb
introduce goleak (#172)
Signed-off-by: disksing <i@disksing.com>
2021-06-25 12:13:38 +08:00
tison 090d6eb694
config: replace pingcap/check with testify (#136)
Signed-off-by: tison <wander4096@gmail.com>
2021-06-22 13:13:47 +08:00