a05a5382fb
region_cache: reload region cache when flag `needDelayedReloadReady` is set ( #1738 )
...
ref pingcap/tidb#62933
Signed-off-by: you06 <you1474600@gmail.com>
2025-08-21 08:12:08 +00:00
017cde67a5
region_cache: allow only one inflight updating buckets for each region ( #1723 )
...
close tikv/client-go#1722
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-07-30 15:55:16 +00:00
2f847d5b0f
region cache: limit key ranges sent to PD in one `BatchScanRegions` gRPC ( #1712 )
...
close tikv/client-go#1704
Signed-off-by: you06 <you1474600@gmail.com>
2025-07-11 05:07:48 +00:00
84e019eae5
region cache: do not invalidate the valid region cache when new region cache is loaded ( #1698 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2025-07-07 06:38:42 +00:00
ff126df68a
Upgrade the PD client to adopt the latest refactor ( #1525 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-12-20 05:23:27 +00:00
06d7f4b923
Try to validate read ts for all RPC requests ( #1513 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-12-09 09:49:30 +00:00
691e80ae0e
region cache: retry scan or batch scan regions when returned region has no leader ( #1480 )
...
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-10-23 02:31:20 +00:00
f0f57f28c4
region cache: batch find regions by key ranges from cache ( #1410 )
...
* try
add benchmark
Signed-off-by: wshwsh12 <793703860@qq.com>
* impl
Signed-off-by: wshwsh12 <793703860@qq.com>
* time
Signed-off-by: wshwsh12 <793703860@qq.com>
* fix ut
Signed-off-by: wshwsh12 <793703860@qq.com>
* address comments
Signed-off-by: wshwsh12 <793703860@qq.com>
* startkey
Signed-off-by: wshwsh12 <793703860@qq.com>
* add comments
Signed-off-by: wshwsh12 <793703860@qq.com>
---------
Signed-off-by: wshwsh12 <793703860@qq.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-08-09 15:36:57 +09:00
cd64e24de8
fix issue that store's liveness may incorrectly marked as unreachable when the store restarts with label changed ( #1407 )
...
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix ci
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-08-01 19:02:26 +08:00
d73cc1ed65
region cache: check if the pd returned regions covers the ranges ( #1377 )
...
* add reproduce test
Signed-off-by: you06 <you1474600@gmail.com>
* check if the pd returned regions covers the ranges
Signed-off-by: you06 <you1474600@gmail.com>
* handle limit
Signed-off-by: you06 <you1474600@gmail.com>
* fix lint
Signed-off-by: you06 <you1474600@gmail.com>
* add TODO for func
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb for integration test
Signed-off-by: you06 <you1474600@gmail.com>
* rename gap detection function
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* add half bounded cases
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-07-03 17:58:01 +08:00
0206a3c142
region cache: fallback to ScanRegions when BatchScanRegions receive unimplement error ( #1378 )
...
* add fallback
Signed-off-by: you06 <you1474600@gmail.com>
* fallback to ScanRegions when BatchScanRegions receive unimplement error
Signed-off-by: you06 <you1474600@gmail.com>
* add comment & test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-07-03 15:38:53 +08:00
14f3d2b5b7
region cache: add `BatchLocateKeyRanges` interface ( #1367 )
...
* add batch scan interface
Signed-off-by: you06 <you1474600@gmail.com>
fix bug
Signed-off-by: you06 <you1474600@gmail.com>
copy encoded ranges
Signed-off-by: you06 <you1474600@gmail.com>
fix panic
Signed-off-by: you06 <you1474600@gmail.com>
add batch scan region test
Signed-off-by: you06 <you1474600@gmail.com>
fix log
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader option
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader optino
Signed-off-by: you06 <you1474600@gmail.com>
add comment
Signed-off-by: you06 <you1474600@gmail.com>
deprecate ScanRegions
Signed-off-by: you06 <you1474600@gmail.com>
update kvproto & pd client
Signed-off-by: you06 <you1474600@gmail.com>
update go.mod
Signed-off-by: you06 <you1474600@gmail.com>
fix need leader
Signed-off-by: you06 <you1474600@gmail.com>
fix integration test
Signed-off-by: you06 <you1474600@gmail.com>
fix lint
Signed-off-by: you06 <you1474600@gmail.com>
export BatchLocateKeyRangesOpt
Signed-off-by: you06 <you1474600@gmail.com>
update integration test's tidb
Signed-off-by: you06 <you1474600@gmail.com>
refactor BatchLocateKeyRanges and add comments
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb of integration test
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-06-24 16:06:06 +08:00
31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV ( #1287 )
...
* Support actively requesting update health feedback information by calling RPC to TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto; follow the protocol change; and add more comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add integration tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* tune the frequency of active updating
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Revert changes to examples
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary empty line
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
36c8d2c668
region_cache: extract store related fields to store cache ( #1279 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-04-18 21:37:35 +08:00
09b120cdf7
region_cache: fix issue that LocateKey may returns a wrong region ( #1299 )
...
* region_cache: fix issue that LocateKey may returns a wrong region
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-04-18 20:14:18 +08:00
642a09bef1
Make slow store filtering the highest priority in replica selector v2 ( #1267 )
...
* Add some logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Make slow store filtering the highest priority in replica selector v2
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add non stale read case to TestMultiReplicaInOneAZ
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Enrich the multi replcia in one AZ case but it failed...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update test to adapt the fix on master branch
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove TestMultiReplicaInOneAZ
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-04-08 14:23:38 +08:00
03bbadb277
Support receving health feedback ( #1153 )
...
* Register health feedback handler in batch client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update protocol
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Collect the health status and record to the stores
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove the filter which looks not necessary
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix checkAndUpdateStoreHealthStatus panicking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Merge SetHealthFeedbackHandler to Client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try another pattern: separated callback registry
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try region cache eventListener pattern
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test receiving froim callback
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests to region cache
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary debug log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove replace of tidb repo
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update comments; fix lint
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add the isSlow method of Store back
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove unused method for now to make lint happy
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
e72c4cd474
enhance background job management of region cache ( #1171 )
...
* enhance background job management of region cache
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race in ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* refine scheduleWithTrigger
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-26 11:26:25 +08:00
8b3b01e8d7
introduce a random jitter to region cache ttl ( #1148 )
...
* introduce a random jitter to region cache ttl
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* refactor searching cached region
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* observe load region by reason
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-02 20:13:23 +08:00
6e501a142d
improve region reload strategy ( #1122 )
...
* refine region reload strategy
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race in ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix another data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* access store.epoch atomic
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* re-implement async reload by sync flags
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* a minor optimization
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* Update internal/locate/region_cache.go
Co-authored-by: ekexium <eke@fastmail.com>
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* rename async-reload to delayed-reload
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-01-31 10:14:07 +08:00
e79e8008ce
fix the issue that health check may set liveness wrongly ( #1127 )
...
* fix the issue that health check may set liveness wrongly
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix lint issue
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix rawkv ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* use getStore instead of accessing storeMu directly
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* make TestAccessFollowerAfter1TiKVDown stable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* make TestBackoffErrorType stable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2024-01-24 15:11:01 +08:00
5d0ae57f22
region_cache: restrict access to store related fields ( #1106 )
...
* region_cache: restrict access to store related fields
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix a typo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* resolve a todo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* resolve an another todo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* limit the use of store api
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-01-22 21:39:56 +08:00
db2e85c463
fix intersected stale check ( #1090 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-25 09:53:55 +08:00
1a33acab05
region_cache: allow pd follower handle region api ( #1072 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-20 16:42:14 +08:00
85ca0a4a3f
region_cache: check epoch before insert ( #1079 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-12-20 14:05:16 +08:00
8deef63a94
region_cache: no need to metrics when inserting for scan regions ( #1077 )
...
* set false when scan region
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
* set false when scan region
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
---------
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-18 14:57:25 +08:00
6659170644
make `ContainsByEnd` return `true` when the key is empty and region's right bound is inf ( #1052 )
...
Signed-off-by: hillium <yujuncen@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 15:27:42 +08:00
6ba909c4ad
config: enable user configuration for retry strategy. ( #1055 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 14:54:10 +08:00
cad3142206
schedule async reload for region that has unavailable tiflash peers to avoid load un-balance issue ( #1029 )
...
Signed-off-by: xufei <xufei@pingcap.com>
Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>
2023-10-25 10:24:11 +08:00
ff39b4af09
error: handle bucket version not match ( #918 )
2023-08-22 03:25:20 -06:00
85fc8f3375
reload region cache when store is resolved from invalid status ( #843 )
...
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-07-14 13:27:14 +08:00
f64d5dd79b
update dco ( #731 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-04-27 14:45:04 +08:00
39cd84957f
fix fmt
...
Signed-off-by: guo-shaoge <shaoge1994@163.com>
2023-03-06 18:05:17 +08:00
bce56a5831
ReadMode: introduce AutoFollowerRead mechanism by adding a new `ReadMode == PreferLeader` ( #671 )
...
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
2023-02-07 16:42:52 +08:00
78470adc3d
Rewind after cacheGC reaches the end ( #678 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2023-01-30 15:45:29 +08:00
a2964b7151
Add a background region cache GC goroutine ( #664 )
...
* Add a background region cache GC goroutine
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* Change interval to 1s and regionPerRound to 50
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2023-01-11 16:47:55 +08:00
f313ddf58d
reformat codec and add keyspace support ( #649 )
...
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
2023-01-10 15:15:33 +08:00
c20405f345
support witness peers ( #633 )
...
Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
2023-01-09 20:53:19 +08:00
91be9c6ce6
Introduce the FlashbackNotPrepared error ( #606 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2022-10-17 17:26:35 +08:00
e425355a10
*: introduce the non-retryable error FlashbackInProgress ( #579 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2022-09-06 14:28:21 +08:00
025596b7a2
Clear intersecting regions in the cache when inserting a region ( #566 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-08-15 17:47:24 +08:00
86d51ba7eb
*: update google/btree to improve performance ( #547 )
...
* *: update google/btree to improve performance
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* update ci
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2022-07-18 16:02:14 +08:00
2be4a58dbc
region_cache: rename LocateBucketV2 to LocateBucket and let the old LocateBucket be the internal function of the new LocateBucket ( #496 )
...
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
2022-05-13 15:26:07 +08:00
3705989fa1
Wrap locateBucket to handle with edges ( #486 )
...
Signed-off-by: SpadeA-Tang <u6748471@anu.edu.au>
2022-05-07 15:34:05 +08:00
7e34d88af3
region_cache: support buckets ( #439 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2022-03-09 13:52:39 +08:00
f43cc3acc0
region_cache: more comments and minor refactor ( #427 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2022-02-23 16:53:24 +08:00
06c3868595
select replica for replica read requests using ReplicaSelector ( #285 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-09-02 02:10:36 +00:00
b440ea2f58
select replica for forward requests using RegionRequestSender ( #264 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-08-16 19:27:02 +08:00
288c649636
fix license ( #267 )
...
Signed-off-by: disksing <i@disksing.com>
2021-08-15 01:48:01 +08:00
d859acfa07
locate: don't backoff if fails to decode region key ( #263 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-08-10 17:35:00 +08:00
you06