09b120cdf7
region_cache: fix issue that LocateKey may returns a wrong region ( #1299 )
...
* region_cache: fix issue that LocateKey may returns a wrong region
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-04-18 20:14:18 +08:00
642a09bef1
Make slow store filtering the highest priority in replica selector v2 ( #1267 )
...
* Add some logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Make slow store filtering the highest priority in replica selector v2
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add non stale read case to TestMultiReplicaInOneAZ
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Enrich the multi replcia in one AZ case but it failed...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update test to adapt the fix on master branch
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove TestMultiReplicaInOneAZ
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-04-08 14:23:38 +08:00
abc9bb2878
region_cache: filter out unresolved stores when GetTiFlashStores ( #1262 )
...
* region_cache: filter out unresolved stores when GetTiFlashStores
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-04-04 12:33:34 +08:00
9a37a0a77f
txn: attach cluster id in kv request ( #1208 )
...
* attach cluster id in kv requests
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* update go mod in integration test
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-03-08 18:59:12 +08:00
eb8f254882
region_cache: move store related code to a standalone file ( #1196 )
...
* region_cache: move store related code to a standalone file
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* use std atomic instead of uber's atomic
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-07 09:58:17 +08:00
3625fc04d0
fix bug introduced by #1200 ( #1204 )
...
Signed-off-by: SeaRise <hhssearise@foxmail.com>
2024-03-05 16:55:06 +08:00
2b14d14975
Refine locate_key_range ( #1200 )
...
Signed-off-by: SeaRise <hhssearise@foxmail.com>
Co-authored-by: disksing <i@disksing.com>
2024-03-04 13:00:23 +08:00
50c4085d0a
reduce unnecessary tikvServerBusy backoff when able to try next replica ( #1184 )
...
* reduce unnecessary tikvServerBusy backoff when
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix lint
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code and add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add comment and refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add comment and refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix lint
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add more comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:38:29 +08:00
09ecb550d3
Add a new function LocateKeyRange ( #1189 )
...
Signed-off-by: SeaRise <hhssearise@foxmail.com>
2024-02-29 12:58:05 +08:00
03bbadb277
Support receving health feedback ( #1153 )
...
* Register health feedback handler in batch client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update protocol
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Collect the health status and record to the stores
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove the filter which looks not necessary
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix checkAndUpdateStoreHealthStatus panicking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Merge SetHealthFeedbackHandler to Client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try another pattern: separated callback registry
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try region cache eventListener pattern
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test receiving froim callback
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests to region cache
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary debug log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove replace of tidb repo
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update comments; fix lint
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add the isSlow method of Store back
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove unused method for now to make lint happy
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
e72c4cd474
enhance background job management of region cache ( #1171 )
...
* enhance background job management of region cache
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race in ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* refine scheduleWithTrigger
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-26 11:26:25 +08:00
6f9550fda3
stale read request shoudn't retry leader if leader is already tried ( #1174 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-23 18:01:14 +08:00
8b3b01e8d7
introduce a random jitter to region cache ttl ( #1148 )
...
* introduce a random jitter to region cache ttl
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* refactor searching cached region
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* observe load region by reason
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-02 20:13:23 +08:00
6e501a142d
improve region reload strategy ( #1122 )
...
* refine region reload strategy
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race in ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix another data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* access store.epoch atomic
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* re-implement async reload by sync flags
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* a minor optimization
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* Update internal/locate/region_cache.go
Co-authored-by: ekexium <eke@fastmail.com>
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* rename async-reload to delayed-reload
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-01-31 10:14:07 +08:00
e79e8008ce
fix the issue that health check may set liveness wrongly ( #1127 )
...
* fix the issue that health check may set liveness wrongly
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix lint issue
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix rawkv ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* use getStore instead of accessing storeMu directly
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* make TestAccessFollowerAfter1TiKVDown stable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* make TestBackoffErrorType stable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2024-01-24 15:11:01 +08:00
5d0ae57f22
region_cache: restrict access to store related fields ( #1106 )
...
* region_cache: restrict access to store related fields
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix a typo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* resolve a todo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* resolve an another todo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* limit the use of store api
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-01-22 21:39:56 +08:00
1a33acab05
region_cache: allow pd follower handle region api ( #1072 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-20 16:42:14 +08:00
85ca0a4a3f
region_cache: check epoch before insert ( #1079 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-12-20 14:05:16 +08:00
8deef63a94
region_cache: no need to metrics when inserting for scan regions ( #1077 )
...
* set false when scan region
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
* set false when scan region
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
---------
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-18 14:57:25 +08:00
6659170644
make `ContainsByEnd` return `true` when the key is empty and region's right bound is inf ( #1052 )
...
Signed-off-by: hillium <yujuncen@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 15:27:42 +08:00
6ba909c4ad
config: enable user configuration for retry strategy. ( #1055 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 14:54:10 +08:00
cad3142206
schedule async reload for region that has unavailable tiflash peers to avoid load un-balance issue ( #1029 )
...
Signed-off-by: xufei <xufei@pingcap.com>
Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>
2023-10-25 10:24:11 +08:00
a05564b183
log: Enhanced log key output ( #958 )
...
Signed-off-by: husharp <jinhao.hu@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-10-07 14:22:57 +08:00
4c2ae43454
dco ( #988 )
...
Signed-off-by: bufferflies <1045931706@qq.com>
2023-09-26 13:16:11 +08:00
fb8e4c0eac
add region cache state test & fix some issues of replica selector ( #910 ) ( #942 )
...
* add region cache state test & fix some issues of replica selector (#910 )
Signed-off-by: you06 <you1474600@gmail.com>
remove duplicate code
Signed-off-by: you06 <you1474600@gmail.com>
* remove comment
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
* fix flaky test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2023-09-12 12:36:53 +08:00
ba3de39920
*: fix panic log
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-09-11 14:38:07 +08:00
ff39b4af09
error: handle bucket version not match ( #918 )
2023-08-22 03:25:20 -06:00
44f5025f5a
fallback to follower when leader is busy ( #916 ) ( #923 )
...
* fallback to follower when leader is busy
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <cfzjywxk@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2023-08-11 21:36:47 +08:00
719e6456f7
use context logger as much as possible ( #908 )
...
* use context logger as much as possible
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-07-31 11:23:49 +08:00
9f94221c2d
add more log for diagnose ( #915 )
...
* add more log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add more log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add more log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-07-28 20:35:05 +08:00
85fc8f3375
reload region cache when store is resolved from invalid status ( #843 )
...
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-07-14 13:27:14 +08:00
51633ada95
region_cache: support check pending tiflash peer ( #821 )
...
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Co-authored-by: disksing <i@disksing.com>
2023-07-13 14:08:45 +08:00
5e3e38b270
format key in error message in method `scanRegions` ( #876 )
...
Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2023-07-10 15:42:05 +08:00
32eb2b0f88
use '%d' instead of '%q' for some int values in error message ( #875 )
...
Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2023-07-10 12:30:33 +08:00
c764d7bc8f
client-go: add some key range info to error when PD returned no region ( #862 )
...
Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2023-06-30 14:27:00 +08:00
f64d5dd79b
update dco ( #731 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-04-27 14:45:04 +08:00
415713766f
avoid dataIsNotReady error while retrying stale read on the leader ( #765 )
...
* avoid dataIsNotReady error while retrying stale read on the leader
Signed-off-by: artem_danilov <artem_danilov@airbnb.com>
* move StaleRead flag reset to retry section
Signed-off-by: artem_danilov <artem_danilov@airbnb.com>
* move all logic to #next and allow retry on the leader
Signed-off-by: artem_danilov <artem_danilov@airbnb.com>
---------
Signed-off-by: artem_danilov <artem_danilov@airbnb.com>
Co-authored-by: artem_danilov <artem_danilov@airbnb.com>
2023-04-19 10:48:52 +08:00
bb350d6ea8
fix data race ( #736 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-03-27 16:10:11 +08:00
884a634378
compatible batch copr with load-based replica read ( #739 )
...
* compatible batch copr with load-based replica read
Signed-off-by: you06 <you1474600@gmail.com>
* handle batch cop when server is busy
Signed-off-by: you06 <you1474600@gmail.com>
debug
Signed-off-by: you06 <you1474600@gmail.com>
debug
Signed-off-by: you06 <you1474600@gmail.com>
debug
Signed-off-by: you06 <you1474600@gmail.com>
remove debug
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* ci
Signed-off-by: you06 <you1474600@gmail.com>
* ci
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2023-03-17 11:26:22 +08:00
a7e3df4ab1
[Metrics] Supply extra metrics to monitor the flows under `prefer-leader` mode. ( #716 )
...
Signed-off-by: Lucasliang <nkcs_lykx@hotmail.com>
2023-03-13 18:08:39 +08:00
bd24a9e443
Merge branch 'master' into support_engine_role
2023-03-07 15:47:49 +08:00
8a8ef35790
support non-block read region ( #725 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2023-03-07 15:15:35 +08:00
39cd84957f
fix fmt
...
Signed-off-by: guo-shaoge <shaoge1994@163.com>
2023-03-06 18:05:17 +08:00
0d003d077b
region_cache: support <engine_role, write> for disaggregated tiflash
...
Signed-off-by: guo-shaoge <shaoge1994@163.com>
2023-03-06 17:53:57 +08:00
a27994e5aa
Support load-based replica read ( #675 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2023-02-22 15:35:43 +08:00
bce56a5831
ReadMode: introduce AutoFollowerRead mechanism by adding a new `ReadMode == PreferLeader` ( #671 )
...
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
2023-02-07 16:42:52 +08:00
3edbadb227
Increase consistent hash replicas to 200 for better balance ( #688 )
...
Signed-off-by: Wish <breezewish@outlook.com>
2023-02-06 11:19:46 +08:00
78470adc3d
Rewind after cacheGC reaches the end ( #678 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2023-01-30 15:45:29 +08:00
f2ff1ce65d
add tombstone check ( #660 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-01-12 13:23:02 +08:00
a2964b7151
Add a background region cache GC goroutine ( #664 )
...
* Add a background region cache GC goroutine
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* Change interval to 1s and regionPerRound to 50
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2023-01-11 16:47:55 +08:00
zyguan