Commit Graph

32 Commits

Author SHA1 Message Date
mittalrishabh d632a4c0b2
retry on leader for region not available (#1769)
Signed-off-by: rishabh_mittal <rishabh.mittal@airbnb.com>

Co-authored-by: rishabh_mittal <rishabh.mittal@airbnb.com>
2025-10-22 02:12:46 +00:00
you06 6fbcc3dbc5
replica selector: do not send replica-read to leader (#1719)
close tikv/client-go#1718

Signed-off-by: you06 <you1474600@gmail.com>
2025-07-22 09:01:56 +00:00
you06 9b1e2a6652
snapshot: set `ReplicaRead` to false when `ReplicaReadType` fallbacks to `ReplicaReadLeader` (#1663)
ref pingcap/tidb#61745

Signed-off-by: you06 <you1474600@gmail.com>
2025-06-18 01:59:54 +00:00
you06 924198a868
Fix stale read metrics (#1649)
close tikv/client-go#1648

Signed-off-by: you06 <you1474600@gmail.com>
2025-06-14 02:02:13 +00:00
zyguan ff15611bb9
locate: implement SendReqAsync for RegionRequestSender (#1618)
ref tikv/client-go#1586

Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-05-22 11:51:40 +00:00
王超 d1ec4b4f67
txn: handle undetermined error in client go (#1642)
close tikv/client-go#1641

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2025-05-20 07:00:04 +00:00
mittalrishabh ad5b9e27f3
retry stale read as stale if leader is not accessible (#1566)
Signed-off-by: rishabh_mittal <mittalrishabh@gmail.com>
2025-02-05 03:01:03 +00:00
MyonKeminta 06d7f4b923
Try to validate read ts for all RPC requests (#1513)
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-12-09 09:49:30 +00:00
mittalrishabh c65273eeb1
retry as stale read on replica (#1509)
Signed-off-by: rishabh_mittal <mittalrishabh@gmail.com>
2024-12-02 07:37:11 +00:00
crazycs 0cc1c5239d
Remove old replica selector (#1265)
* remove EnableReplicaSelectorV2 config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove replicaSelectorV1

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add replica flag to reduce struct size

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove contextPatcher

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* use option.preferLeader

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-27 17:11:02 +08:00
MyonKeminta 31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV (#1287)
* Support actively requesting update health feedback information by calling RPC to TiKV

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update kvproto; follow the protocol change; and add more comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix build

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add integration tests

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* tune the frequency of active updating

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update kvproto

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Revert changes to examples

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove unnecessary empty line

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
crazycs cf37e5e03e
skip TestReplicaReadAccessPathByGenError because it's unstable and slow (#1302)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-19 15:24:53 +08:00
zyguan 36c8d2c668
region_cache: extract store related fields to store cache (#1279)
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-04-18 21:37:35 +08:00
crazycs f9591044f1
reverts PR #988 to fix transaction is undetermined cause by region invalid error (#1298)
* Revert "dco (#988)"

This reverts commit 4c2ae43454.

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-16 21:44:58 +08:00
crazycs 2bd95773ce
fix bug of not invalidate store when tikv store is down in replica-selector-v2 with enable forwarding (#1273)
* fix bug of enable-forwarding

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-08 14:51:17 +08:00
MyonKeminta 642a09bef1
Make slow store filtering the highest priority in replica selector v2 (#1267)
* Add some logs

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Make slow store filtering the highest priority in replica selector v2

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add non stale read case to TestMultiReplicaInOneAZ

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Enrich the multi replcia in one AZ case but it failed...

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update test to adapt the fix on master branch

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove TestMultiReplicaInOneAZ

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-04-08 14:23:38 +08:00
crazycs 8fc819c1ca
fix issue replica selector v2 not compatible with v1 in mixed mode (#1269)
* fix issue replica selector v2 not compatible with v1 in mixed mode

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add prefer-leader test case

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-08 11:18:55 +08:00
zyguan 05aaba6cc6
tikvrpc: avoid data race on `XxxRequest.Context` (#1242)
* tikvrpc: avoid data race on `XxxRequest.Context`

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix grammar of codegen comment

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* address comments

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* check diff of go generate

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix a typo

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-03-22 15:07:37 +08:00
zyguan 08aa706635
test: make TestReplicaReadAccessPathByGenError stable (#1243)
* test: make TestReplicaReadAccessPathByGenError stable

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* reduce maxAccessErrCnt

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-03-22 11:54:22 +08:00
crazycs 98a7df8f41
fix issue read req timeout bug cause by pr#1223 (#1232)
* fix issue read req timeout bug cause by pr#1223

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* implement Cause

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix race test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 19:58:42 +09:00
crazycs 0606e74e8e
enable replica-selector-v2 by default (#1229)
* refine replica selector v2 config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-15 15:42:30 +08:00
crazycs 0416875f55
shouldn't try leader first when leader's store is slow in prefer-leader strategy in replicaSelectorV2 (#1215)
* shouldn't try leader first when leader's store is slow in prefer-leader strategy in replicaSelectorV2

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix bug and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-12 19:59:37 +08:00
crazycs 8d6a95f73d
replica selector refactor (#1142)
* init

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add ReplicaSelector interface

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add replica_selector_v2, todo: fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix all test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test in another way to compatible with old version about stale-read request

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* tiny refactor

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refactor to remove duplicate code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine onServerIsBusy region error handing logic

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support forwarding by proxy

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support busyThreshold and tiny reractor code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix proxy bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test and tiny refactor

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add some test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* tidy refine

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test in v1

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add backoff-cnt check and timeout test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add benchmark test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make test stale

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix golangci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix race test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more comprehensive enumeration testing

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix accessFollower with label retry bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test and fix some bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test and add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine onNotLeader logic when new leader is not available

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine calculateScore logic, if the replica already tried, decrease the score

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine logic: replica-read request with mixed strategy and with label, should be able to retry all remain replicas

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* prefer-leader for mixed read with label for non stale-read req

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test case

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* resolve conflict

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove some duplicate test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refactor test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add busy_threshold test and fix bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix proxy bug and add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add learner test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove old test and refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* move test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* use new score calculation by flag bit

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make test stable

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment,license and fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix race test timeout

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add flagNotAttemp in score

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-11 15:40:44 +08:00
crazycs 50c4085d0a
reduce unnecessary tikvServerBusy backoff when able to try next replica (#1184)
* reduce unnecessary tikvServerBusy backoff when

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:38:29 +08:00
crazycs 02cd637b9b
fix some issue of tryIdleReplica strategy (#1203)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:01:59 +08:00
crazycs ea5a5c4ba6
Don't invalid region in accessFollower when set user-defined req time out and meet DeadLineExceededErr (#1192)
* Don't invalid region in accessFollower when set user-defined req timeout and meet DeadLineExceededErr

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* rename func

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-28 21:38:34 +08:00
MyonKeminta 03bbadb277
Support receving health feedback (#1153)
* Register health feedback handler in batch client

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update protocol

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Collect the health status and record to the stores

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove the filter which looks not necessary

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add metrics

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix checkAndUpdateStoreHealthStatus panicking

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add logs

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Merge SetHealthFeedbackHandler to Client

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try another pattern: separated callback registry

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try region cache eventListener pattern

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add tests

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Test receiving froim callback

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add tests to region cache

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove unnecessary debug log

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* remove replace of tidb repo

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* fix build

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update comments; fix lint

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add the isSlow method of Store back

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* remove unused method for now to make lint happy

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
crazycs a04bbfe5dc
Don't invalid region in tryFollowers when set user-defined req timeout and meet DeadLineExceededErr (#1182)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-02-27 14:02:23 +08:00
crazycs 87064b8038
replica-read request with mixed strategy and with label, should be able to retry all remain replicas (#1183)
* replica-read request with mixed strategy and with label, should be able to retry all remain replicas

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 21:29:12 +08:00
crazycs 190f0cce53
don't retry same replica twice in a round, but for stale read, we can retry DataIsNotReady replica by replica-read (#1181)
* don't retry same replica twice in a round, but for stale read, we can retry DataIsNotReady replica by replica-read

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove duplicate test case

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove old duplicate test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 14:52:53 +08:00
crazycs 93fff7cec5
refine onNotLeader logic when new leader is not available (#1178)
* refine onNotLeader logic when new leader is not available #1169

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine logic and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test case for ReplicaReadLeader with kv_read_timeout

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix leader has deadlineErr and retry it again issue, and add test for it

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 10:19:09 +08:00
crazycs 6f9550fda3
stale read request shoudn't retry leader if leader is already tried (#1174)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-23 18:01:14 +08:00