eb8f254882
region_cache: move store related code to a standalone file ( #1196 )
...
* region_cache: move store related code to a standalone file
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* use std atomic instead of uber's atomic
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-07 09:58:17 +08:00
3625fc04d0
fix bug introduced by #1200 ( #1204 )
...
Signed-off-by: SeaRise <hhssearise@foxmail.com>
2024-03-05 16:55:06 +08:00
2b14d14975
Refine locate_key_range ( #1200 )
...
Signed-off-by: SeaRise <hhssearise@foxmail.com>
Co-authored-by: disksing <i@disksing.com>
2024-03-04 13:00:23 +08:00
50c4085d0a
reduce unnecessary tikvServerBusy backoff when able to try next replica ( #1184 )
...
* reduce unnecessary tikvServerBusy backoff when
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix lint
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code and add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add comment and refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add comment and refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix lint
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add more comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:38:29 +08:00
02cd637b9b
fix some issue of tryIdleReplica strategy ( #1203 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:01:59 +08:00
09ecb550d3
Add a new function LocateKeyRange ( #1189 )
...
Signed-off-by: SeaRise <hhssearise@foxmail.com>
2024-02-29 12:58:05 +08:00
ea5a5c4ba6
Don't invalid region in accessFollower when set user-defined req time out and meet DeadLineExceededErr ( #1192 )
...
* Don't invalid region in accessFollower when set user-defined req timeout and meet DeadLineExceededErr
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* rename func
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-28 21:38:34 +08:00
03bbadb277
Support receving health feedback ( #1153 )
...
* Register health feedback handler in batch client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update protocol
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Collect the health status and record to the stores
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove the filter which looks not necessary
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix checkAndUpdateStoreHealthStatus panicking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Merge SetHealthFeedbackHandler to Client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try another pattern: separated callback registry
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try region cache eventListener pattern
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test receiving froim callback
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests to region cache
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary debug log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove replace of tidb repo
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update comments; fix lint
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add the isSlow method of Store back
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove unused method for now to make lint happy
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
a04bbfe5dc
Don't invalid region in tryFollowers when set user-defined req timeout and meet DeadLineExceededErr ( #1182 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-02-27 14:02:23 +08:00
87064b8038
replica-read request with mixed strategy and with label, should be able to retry all remain replicas ( #1183 )
...
* replica-read request with mixed strategy and with label, should be able to retry all remain replicas
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 21:29:12 +08:00
190f0cce53
don't retry same replica twice in a round, but for stale read, we can retry DataIsNotReady replica by replica-read ( #1181 )
...
* don't retry same replica twice in a round, but for stale read, we can retry DataIsNotReady replica by replica-read
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* remove duplicate test case
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* remove old duplicate test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 14:52:53 +08:00
e72c4cd474
enhance background job management of region cache ( #1171 )
...
* enhance background job management of region cache
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race in ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* refine scheduleWithTrigger
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-26 11:26:25 +08:00
93fff7cec5
refine onNotLeader logic when new leader is not available ( #1178 )
...
* refine onNotLeader logic when new leader is not available #1169
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine logic and add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add more test case for ReplicaReadLeader with kv_read_timeout
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix leader has deadlineErr and retry it again issue, and add test for it
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code and comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 10:19:09 +08:00
6f9550fda3
stale read request shoudn't retry leader if leader is already tried ( #1174 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-23 18:01:14 +08:00
8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML ( #1114 )
...
* Support pipelined memdb.
Signed-off-by: you06 <you1474600@gmail.com>
* fix mutex for pipelined memdb
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
0f8c594090
fix data race in ut ( #1161 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-19 11:43:20 +08:00
8b3b01e8d7
introduce a random jitter to region cache ttl ( #1148 )
...
* introduce a random jitter to region cache ttl
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* refactor searching cached region
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* observe load region by reason
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-02 20:13:23 +08:00
70c148e84e
ErrQueryInterrupted with parameters ( #1124 )
...
* feat: ErrQueryInterrupted with parameters
Signed-off-by: ekexium <eke@fastmail.com>
* Revert "Revert "fix: check kill signal against 0 (#1102 )" (#1129 )"
This reverts commit 3480b5ed7c
2024-02-02 10:26:02 +08:00
6e501a142d
improve region reload strategy ( #1122 )
...
* refine region reload strategy
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race in ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix another data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* access store.epoch atomic
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* re-implement async reload by sync flags
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* a minor optimization
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* Update internal/locate/region_cache.go
Co-authored-by: ekexium <eke@fastmail.com>
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* rename async-reload to delayed-reload
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-01-31 10:14:07 +08:00
e79e8008ce
fix the issue that health check may set liveness wrongly ( #1127 )
...
* fix the issue that health check may set liveness wrongly
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix lint issue
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix rawkv ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix data race
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* use getStore instead of accessing storeMu directly
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* make TestAccessFollowerAfter1TiKVDown stable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* make TestBackoffErrorType stable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2024-01-24 15:11:01 +08:00
c9ecc6ed3b
fix the issue that prefer-leader doesn't try followers ( #1105 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-01-24 11:05:38 +08:00
3480b5ed7c
Revert "fix: check kill signal against 0 ( #1102 )" ( #1129 )
...
This reverts commit 057c479dd8
2024-01-23 13:54:05 +08:00
5d0ae57f22
region_cache: restrict access to store related fields ( #1106 )
...
* region_cache: restrict access to store related fields
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix a typo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* resolve a todo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* resolve an another todo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* limit the use of store api
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-01-22 21:39:56 +08:00
b8a65870d6
Check time spent on attempting RPC to avoid spending too much time on retrying ( #1117 )
...
* Check time spent on attempting RPC to avoid spending too much time on retrying
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Handle refreshRegionStore
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-01-18 20:48:43 +08:00
b1256130a5
Fix the issue that leader change with nea leader info cause invalidStore state and lead to unnecessary backoff ( #1115 )
...
* Fix the issue that leader change with nea leader info cause invalidStore
state and lead to unnecessary backoff
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-01-15 20:59:47 +08:00
057c479dd8
fix: check kill signal against 0 ( #1102 )
...
* fix: check kill signal against 0, so that all kill signals will not be ignored.
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: extract method checkKilled()
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-01-10 15:48:27 +08:00
dbea404a84
avoid too frequent logs ( #1100 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-01-02 14:52:58 +08:00
db2e85c463
fix intersected stale check ( #1090 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-25 09:53:55 +08:00
1a33acab05
region_cache: allow pd follower handle region api ( #1072 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-20 16:42:14 +08:00
85ca0a4a3f
region_cache: check epoch before insert ( #1079 )
...
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-12-20 14:05:16 +08:00
8deef63a94
region_cache: no need to metrics when inserting for scan regions ( #1077 )
...
* set false when scan region
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
* set false when scan region
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
---------
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-18 14:57:25 +08:00
0ff16620f6
pd client: uprade and fit get region option ( #1069 )
...
* fit get region option
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
* replace tidb
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
* replace tidb
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
---------
Signed-off-by: Cabinfever_B <cabinfeveroier@gmail.com>
2023-12-01 10:44:04 +08:00
812a791ab3
ci: make golangci happy ( #1066 )
...
Signed-off-by: husharp <jinhao.hu@pingcap.com>
2023-11-21 15:05:35 +08:00
6659170644
make `ContainsByEnd` return `true` when the key is empty and region's right bound is inf ( #1052 )
...
Signed-off-by: hillium <yujuncen@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 15:27:42 +08:00
6ba909c4ad
config: enable user configuration for retry strategy. ( #1055 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 14:54:10 +08:00
cad3142206
schedule async reload for region that has unavailable tiflash peers to avoid load un-balance issue ( #1029 )
...
Signed-off-by: xufei <xufei@pingcap.com>
Signed-off-by: xufei <xufeixw@mail.ustc.edu.cn>
2023-10-25 10:24:11 +08:00
476ea22ffb
fix request source tag unset ( #1025 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-10-20 14:26:25 +08:00
2eaf68e0cd
Fix batch client batchSendLoop panic ( #1021 )
...
* init
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* try to fix
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-10-17 15:44:06 +08:00
e70513e671
bugfix: fix invalid nil pointer when trying to record Store.SlownessStat. ( #1017 )
...
* bugfix: fix invalid nil pointer when trying to record Store.SlownessStat.
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
* Polish codes.
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
* Polish comments.
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
---------
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
2023-10-16 12:02:43 +08:00
07432ef6c0
refactor the func name to make it clear ( #1000 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-10-10 14:18:02 +08:00
a05564b183
log: Enhanced log key output ( #958 )
...
Signed-off-by: husharp <jinhao.hu@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-10-07 14:22:57 +08:00
4c2ae43454
dco ( #988 )
...
Signed-off-by: bufferflies <1045931706@qq.com>
2023-09-26 13:16:11 +08:00
44b0cf7aba
do not try leader if it's unreachable ( #971 )
...
* do not try leader if it's unreachable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix the flaky test
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2023-09-25 11:25:02 +08:00
39084386b3
*: configurable kv_read_timeout should not affect write request ( #978 )
...
* *: configurable kv_read_timeout should not affect write request
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine logic
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-09-18 11:17:51 +08:00
342301689f
*: fix issue of configure kv timeout not work when disable batch client ( #980 )
...
* *: fix issue of configure kv timeout not work when disable batch client
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-09-15 23:49:42 +08:00
001735b0b5
*: try follower when stale read request timeout ( #969 )
...
* fix stalre read retry logic when got timeout error
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-09-15 12:53:37 +08:00
79843ce227
Merge branch 'master' into add_log
2023-09-12 20:24:24 +08:00
fb8e4c0eac
add region cache state test & fix some issues of replica selector ( #910 ) ( #942 )
...
* add region cache state test & fix some issues of replica selector (#910 )
Signed-off-by: you06 <you1474600@gmail.com>
remove duplicate code
Signed-off-by: you06 <you1474600@gmail.com>
* remove comment
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
* fix flaky test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2023-09-12 12:36:53 +08:00
3216af2cb1
add cancel reason
...
Signed-off-by: bufferflies <1045931706@qq.com>
2023-09-12 11:59:38 +08:00
ba3de39920
*: fix panic log
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-09-11 14:38:07 +08:00
zyguan