Commit Graph

750 Commits

Author SHA1 Message Date
ekexium 87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB (#1212)
* feat: introduce GetLocal() for MemBuffer

Signed-off-by: ekexium <eke@fastmail.com>

* downgrade tools to 0.18.0

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set flushingMemDB = nil when an error is returned from a flush

Signed-off-by: ekexium <eke@fastmail.com>

* impl BatchGet for MemBuffer

Signed-off-by: you06 <you1474600@gmail.com>

test membuffer batch get

Signed-off-by: you06 <you1474600@gmail.com>

* add Prefetch & GetPrefetchCache

Signed-off-by: you06 <you1474600@gmail.com>

* cache multi Prefetch call

Signed-off-by: you06 <you1474600@gmail.com>

* add tests

Signed-off-by: you06 <you1474600@gmail.com>

* workaround for golang ci lint failure

Signed-off-by: you06 <you1474600@gmail.com>

* replace assert with require

Signed-off-by: you06 <you1474600@gmail.com>

* workaround golangci lint

Signed-off-by: you06 <you1474600@gmail.com>

* lint

Signed-off-by: you06 <you1474600@gmail.com>

* remove prefetch interface, pipelined memdb will cache batch get result

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb

Signed-off-by: you06 <you1474600@gmail.com>

* fix batch get cache when membuffer is empty

Signed-off-by: you06 <you1474600@gmail.com>

* fix returned delete value

Signed-off-by: you06 <you1474600@gmail.com>

* fix: handle resourceGroupTag of Flush

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set resource group tag for committer if it's pipelined

Signed-off-by: ekexium <eke@fastmail.com>

* Update internal/unionstore/pipelined_memdb.go

Co-authored-by: ekexium <eke@fastmail.com>

* remove prefetch interface

Signed-off-by: you06 <you1474600@gmail.com>

* release mutex

Signed-off-by: you06 <you1474600@gmail.com>

* flush wait to avoid race

Signed-off-by: you06 <you1474600@gmail.com>

* fix unstopped test

Signed-off-by: you06 <you1474600@gmail.com>

* set resource group tags for committer

Signed-off-by: ekexium <eke@fastmail.com>

* skip test due to tikv image not updated yet

Signed-off-by: you06 <you1474600@gmail.com>

* skip more test

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 11:36:05 +08:00
crazycs 0606e74e8e
enable replica-selector-v2 by default (#1229)
* refine replica selector v2 config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-15 15:42:30 +08:00
crazycs 5b6625f167
fix issue that no available connections cause by concurrency request limit bug (#1226)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-15 14:19:38 +08:00
ekexium 280cb0852e
feat: add pipeliend field in TxnInfo (#1228)
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-15 12:45:02 +08:00
Ping Yu 7bce6ea4df
Pick connection handling improvements from tidb-cse-7.5 (#1223)
* close on canceled (#1121)

Signed-off-by: Ping Yu <yuping@pingcap.com>

* Fix wait recvloop timeout (#1134)

* wip

Signed-off-by: Ping Yu <yuping@pingcap.com>

* CloseAddr with ver

Signed-off-by: Ping Yu <yuping@pingcap.com>

* fix ErrConn

Signed-off-by: Ping Yu <yuping@pingcap.com>

* fix ut

Signed-off-by: Ping Yu <yuping@pingcap.com>

* cleanup

Signed-off-by: Ping Yu <yuping@pingcap.com>

---------

Signed-off-by: Ping Yu <yuping@pingcap.com>

* Use correct address for CloseAddr (#1140)

* wip

Signed-off-by: Ping Yu <yuping@pingcap.com>

* CloseAddr with ver

Signed-off-by: Ping Yu <yuping@pingcap.com>

* fix ErrConn

Signed-off-by: Ping Yu <yuping@pingcap.com>

* fix ut

Signed-off-by: Ping Yu <yuping@pingcap.com>

* polish

Signed-off-by: Ping Yu <yuping@pingcap.com>

* polish

Signed-off-by: Ping Yu <yuping@pingcap.com>

* polish

Signed-off-by: Ping Yu <yuping@pingcap.com>

---------

Signed-off-by: Ping Yu <yuping@pingcap.com>

* rebase

Signed-off-by: Ping Yu <yuping@pingcap.com>

---------

Signed-off-by: Ping Yu <yuping@pingcap.com>
Co-authored-by: iosmanthus <dengliming@pingcap.com>
2024-03-14 16:30:36 +08:00
ekexium d7000ea557
feat: pipelined dml has its own max-txn-ttl of 24 hours (#1224)
* feat: pipelined dml has its own max-txn-ttl of 24 hours

Signed-off-by: ekexium <eke@fastmail.com>

* fix gh actions

Signed-off-by: ekexium <eke@fastmail.com>

* style: fix lint

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
2024-03-14 10:54:19 +08:00
ekexium d59fea5757
P-DML: for AlreadyExist error, attach the value from flushingMemDB (#1218)
* feat: for AlreadyExist error during Flush, attach the value from flushingMemDB to it

Signed-off-by: ekexium <eke@fastmail.com>

* style: apply comment suggestion

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-13 10:23:20 +08:00
crazycs 0416875f55
shouldn't try leader first when leader's store is slow in prefer-leader strategy in replicaSelectorV2 (#1215)
* shouldn't try leader first when leader's store is slow in prefer-leader strategy in replicaSelectorV2

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix bug and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-12 19:59:37 +08:00
MyonKeminta 3e419e68aa
Add hook after receiving rpc (#1211)
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-12 07:55:09 +08:00
you06 88fbc938cd
p-dml: fix unstopped resolve (#1210)
* fix unstopped resolve when the txn cross regions

Signed-off-by: you06 <you1474600@gmail.com>

* cancel context when flush done or abort

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-03-11 21:20:38 +08:00
Weizhen Wang b81a5ee7c4
*: upgrade grpc 1.62.1 (#1213)
* *: upgrade grpc 1.62.1

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* *: upgrade grpc 1.62.1

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

---------

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-03-11 16:37:22 +08:00
crazycs 8d6a95f73d
replica selector refactor (#1142)
* init

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add ReplicaSelector interface

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add replica_selector_v2, todo: fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix all test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test in another way to compatible with old version about stale-read request

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* tiny refactor

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refactor to remove duplicate code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine onServerIsBusy region error handing logic

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support forwarding by proxy

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* support busyThreshold and tiny reractor code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add config

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix proxy bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test and tiny refactor

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add some test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* tidy refine

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test in v1

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add backoff-cnt check and timeout test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add benchmark test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make test stale

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix golangci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix race test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more comprehensive enumeration testing

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix accessFollower with label retry bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test and fix some bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test and add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine onNotLeader logic when new leader is not available

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine calculateScore logic, if the replica already tried, decrease the score

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine logic: replica-read request with mixed strategy and with label, should be able to retry all remain replicas

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* prefer-leader for mixed read with label for non stale-read req

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test case

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* resolve conflict

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove some duplicate test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refactor test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add busy_threshold test and fix bug

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix proxy bug and add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add learner test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove old test and refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* move test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* use new score calculation by flag bit

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* make test stable

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment,license and fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix race test timeout

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add flagNotAttemp in score

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-11 15:40:44 +08:00
cfzjywxk 9a37a0a77f
txn: attach cluster id in kv request (#1208)
* attach cluster id in kv requests

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* update go mod in integration test

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

---------

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-03-08 18:59:12 +08:00
you06 af4f9a9b6e
fix data race when collapsing ResolveLockRequest (#1209)
* fix data race when collapsing ResolveLockRequest

Signed-off-by: you06 <you1474600@gmail.com>

* comment

Signed-off-by: you06 <you1474600@gmail.com>

* sleep to wait goroutine finish

Signed-off-by: you06 <you1474600@gmail.com>

* update comment

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-03-08 13:24:15 +08:00
zyguan eb8f254882
region_cache: move store related code to a standalone file (#1196)
* region_cache: move store related code to a standalone file

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* use std atomic instead of uber's atomic

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-07 09:58:17 +08:00
you06 c72236a601
adjust parameter for pipelined memdb by failpoint (#1205)
* adjust parameter for pipelined memdb by failpoint

Signed-off-by: you06 <you1474600@gmail.com>

* update comment

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-06 16:44:48 +08:00
cfzjywxk 83ec5f2f33
add ts information in the pre split path (#1206)
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-03-06 14:46:42 +08:00
SeaRise 3625fc04d0
fix bug introduced by #1200 (#1204)
Signed-off-by: SeaRise <hhssearise@foxmail.com>
2024-03-05 16:55:06 +08:00
SeaRise 2b14d14975
Refine locate_key_range (#1200)
Signed-off-by: SeaRise <hhssearise@foxmail.com>
Co-authored-by: disksing <i@disksing.com>
2024-03-04 13:00:23 +08:00
crazycs 50c4085d0a
reduce unnecessary tikvServerBusy backoff when able to try next replica (#1184)
* reduce unnecessary tikvServerBusy backoff when

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment and refine code

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix lint

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:38:29 +08:00
crazycs 02cd637b9b
fix some issue of tryIdleReplica strategy (#1203)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-03-04 12:01:59 +08:00
Ryan Leung bfab41501b
update pd client to the latest master (#1195)
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2024-02-29 15:58:03 +08:00
SeaRise 09ecb550d3
Add a new function LocateKeyRange (#1189)
Signed-off-by: SeaRise <hhssearise@foxmail.com>
2024-02-29 12:58:05 +08:00
crazycs ea5a5c4ba6
Don't invalid region in accessFollower when set user-defined req time out and meet DeadLineExceededErr (#1192)
* Don't invalid region in accessFollower when set user-defined req timeout and meet DeadLineExceededErr

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* rename func

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-28 21:38:34 +08:00
MyonKeminta 03bbadb277
Support receving health feedback (#1153)
* Register health feedback handler in batch client

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update protocol

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Collect the health status and record to the stores

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove the filter which looks not necessary

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add metrics

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix checkAndUpdateStoreHealthStatus panicking

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add logs

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Merge SetHealthFeedbackHandler to Client

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try another pattern: separated callback registry

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try region cache eventListener pattern

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add tests

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Test receiving froim callback

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add tests to region cache

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove unnecessary debug log

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* remove replace of tidb repo

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* fix build

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update comments; fix lint

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add the isSlow method of Store back

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* remove unused method for now to make lint happy

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
ystaticy a305dde809
Add get min ts implementation (#809) (#1187)
Signed-off-by: y_static_y@sina.com <y_static_y@sina.com>
Co-authored-by: Ryan Leung <rleungx@gmail.com>
2024-02-28 12:50:27 +08:00
crazycs a04bbfe5dc
Don't invalid region in tryFollowers when set user-defined req timeout and meet DeadLineExceededErr (#1182)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-02-27 14:02:23 +08:00
ystaticy da22d65e94
support gc v2 (#1180)
Signed-off-by: y_static_y@sina.com <y_static_y@sina.com>
2024-02-27 11:44:28 +08:00
crazycs 87064b8038
replica-read request with mixed strategy and with label, should be able to retry all remain replicas (#1183)
* replica-read request with mixed strategy and with label, should be able to retry all remain replicas

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* address comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 21:29:12 +08:00
crazycs 190f0cce53
don't retry same replica twice in a round, but for stale read, we can retry DataIsNotReady replica by replica-read (#1181)
* don't retry same replica twice in a round, but for stale read, we can retry DataIsNotReady replica by replica-read

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove duplicate test case

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* remove old duplicate test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 14:52:53 +08:00
zyguan e72c4cd474
enhance background job management of region cache (#1171)
* enhance background job management of region cache

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* add some comments

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* address comments

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix data race in ut

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix data race

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* refine scheduleWithTrigger

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* address the comment

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-26 11:26:25 +08:00
crazycs 93fff7cec5
refine onNotLeader logic when new leader is not available (#1178)
* refine onNotLeader logic when new leader is not available #1169

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine logic and add test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add more test case for ReplicaReadLeader with kv_read_timeout

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* add comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix leader has deadlineErr and retry it again issue, and add test for it

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine code and comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-26 10:19:09 +08:00
crazycs 6f9550fda3
stale read request shoudn't retry leader if leader is already tried (#1174)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-02-23 18:01:14 +08:00
you06 8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML (#1114)
* Support pipelined memdb.

Signed-off-by: you06 <you1474600@gmail.com>

* fix mutex for pipelined memdb

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
Weizhen Wang 8c13f6b5a6
*: upgrade grpc1.62 (#1168)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-22 13:15:55 +08:00
Ari Ekmekji 7bff6b6f32
Configurable low res tso update interval (#1154) (#1155)
* Configurable lowres tso update interval

Signed-off-by: Ari Ekmekji <aekmekji@gmail.com>

* Review comments

Signed-off-by: Ari Ekmekji <aekmekji@gmail.com>

* Comment for exported function

Signed-off-by: Ari Ekmekji <aekmekji@gmail.com>

---------

Signed-off-by: Ari Ekmekji <aekmekji@gmail.com>
Co-authored-by: Ari Ekmekji <ari.ekmekji@airbnb.com>
2024-02-22 09:56:00 +08:00
tongjian 824302acd0
client: Cache tikv request in tidb client side (#1098)
* impl priority queue

Signed-off-by: bufferflies <1045931706@qq.com>

* replace priority queue

Signed-off-by: bufferflies <1045931706@qq.com>

* cache request in tidb side

Signed-off-by: bufferflies <1045931706@qq.com>

* fix gosimple

Signed-off-by: bufferflies <1045931706@qq.com>

* impl priority

Signed-off-by: bufferflies <1045931706@qq.com>

* pass ut

Signed-off-by: bufferflies <1045931706@qq.com>

* add

Signed-off-by: bufferflies <1045931706@qq.com>

* remove request if the request has been canceled

Signed-off-by: bufferflies <1045931706@qq.com>

* remove request if it has been canceled

Signed-off-by: bufferflies <1045931706@qq.com>

* add comment for cancel

Signed-off-by: bufferflies <1045931706@qq.com>

* not make the loop is busy

Signed-off-by: bufferflies <1045931706@qq.com>

* lint

Signed-off-by: bufferflies <1045931706@qq.com>

* revert busy loop

Signed-off-by: bufferflies <1045931706@qq.com>

* add unit test

Signed-off-by: bufferflies <1045931706@qq.com>

* not limit ehigh prioirty test

Signed-off-by: bufferflies <1045931706@qq.com>

* pass lint

Signed-off-by: bufferflies <1045931706@qq.com>

* support all

Signed-off-by: bufferflies <1045931706@qq.com>

* add comment

Signed-off-by: bufferflies <1045931706@qq.com>

* squash

Signed-off-by: bufferflies <1045931706@qq.com>

* revert all to All

Signed-off-by: bufferflies <1045931706@qq.com>

* remove index from entry

Signed-off-by: bufferflies <1045931706@qq.com>

* make fail reasons more clear

Signed-off-by: bufferflies <1045931706@qq.com>

---------

Signed-off-by: bufferflies <1045931706@qq.com>
2024-02-21 14:54:33 +08:00
zyguan 0f8c594090
fix data race in ut (#1161)
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-19 11:43:20 +08:00
Weizhen Wang 4bb62b6db7
upgrade grpc (#1156)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-18 11:41:12 +08:00
cfzjywxk 11cb7985f0
txn: enable pessimistic region rollback (#1149)
* enable pessimistic region rollback

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* add key range parameters

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* change interface to use start and end keys

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

---------

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-02-05 15:11:26 +08:00
zyguan 8b3b01e8d7
introduce a random jitter to region cache ttl (#1148)
* introduce a random jitter to region cache ttl

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* refactor searching cached region

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* observe load region by reason

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* address the comment

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-02 20:13:23 +08:00
ekexium 70c148e84e
ErrQueryInterrupted with parameters (#1124)
* feat: ErrQueryInterrupted with parameters

Signed-off-by: ekexium <eke@fastmail.com>

* Revert "Revert "fix: check kill signal against 0 (#1102)" (#1129)"

This reverts commit 3480b5ed7c.

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-02-02 10:26:02 +08:00
zyguan 6e501a142d
improve region reload strategy (#1122)
* refine region reload strategy

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix data race in ut

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix another data race

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* access store.epoch atomic

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* re-implement async reload by sync flags

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* a minor optimization

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* fix ut

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* Update internal/locate/region_cache.go

Co-authored-by: ekexium <eke@fastmail.com>
Signed-off-by: zyguan <zhongyangguan@gmail.com>

* rename async-reload to delayed-reload

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-01-31 10:14:07 +08:00
cfzjywxk d2887d56ab
txn: use region pessimsitic lock rollback to speed up cleanup (#1125)
* use region pessimsitic lock rollback and clean for write-write conflict processing

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* format

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* debug ci failure patch, disable region pessimistic rollback

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

---------

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-01-31 09:53:00 +08:00
glorv 5945fe3920
Merge pull request #1143 from lance6716/remove-replace
integration_tests: upgrade tidb to remove replace
2024-01-26 16:27:41 +08:00
lance6716 30684f7d3c use new interface
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-26 13:53:07 +08:00
lance6716 d1b611cbe2 integration_tests: upgrade tidb to remove replace
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-26 13:48:55 +08:00
Weizhen Wang e6f5a45b00
fix misspell (#1135)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-01-25 11:09:10 +08:00
Weizhen Wang f90605363e
support grpc's shard buffer pool (#1132)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-01-24 17:57:47 +08:00
glorv 99ea0d4d9b
Merge pull request #1123 from lance6716/update-dep
dep: update pd client
2024-01-24 16:13:43 +08:00