tikv
/
client-go
mirror of https://github.com/tikv/client-go.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity
700 Commits 27 Branches 11 Tags 33 MiB
Commit Graph

141 Commits

Author SHA1 Message Date
ekexium 87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB (#1212) 
* feat: introduce GetLocal() for MemBuffer

Signed-off-by: ekexium <eke@fastmail.com>

* downgrade tools to 0.18.0

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set flushingMemDB = nil when an error is returned from a flush

Signed-off-by: ekexium <eke@fastmail.com>

* impl BatchGet for MemBuffer

Signed-off-by: you06 <you1474600@gmail.com>

test membuffer batch get

Signed-off-by: you06 <you1474600@gmail.com>

* add Prefetch & GetPrefetchCache

Signed-off-by: you06 <you1474600@gmail.com>

* cache multi Prefetch call

Signed-off-by: you06 <you1474600@gmail.com>

* add tests

Signed-off-by: you06 <you1474600@gmail.com>

* workaround for golang ci lint failure

Signed-off-by: you06 <you1474600@gmail.com>

* replace assert with require

Signed-off-by: you06 <you1474600@gmail.com>

* workaround golangci lint

Signed-off-by: you06 <you1474600@gmail.com>

* lint

Signed-off-by: you06 <you1474600@gmail.com>

* remove prefetch interface, pipelined memdb will cache batch get result

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb

Signed-off-by: you06 <you1474600@gmail.com>

* fix batch get cache when membuffer is empty

Signed-off-by: you06 <you1474600@gmail.com>

* fix returned delete value

Signed-off-by: you06 <you1474600@gmail.com>

* fix: handle resourceGroupTag of Flush

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set resource group tag for committer if it's pipelined

Signed-off-by: ekexium <eke@fastmail.com>

* Update internal/unionstore/pipelined_memdb.go

Co-authored-by: ekexium <eke@fastmail.com>

* remove prefetch interface

Signed-off-by: you06 <you1474600@gmail.com>

* release mutex

Signed-off-by: you06 <you1474600@gmail.com>

* flush wait to avoid race

Signed-off-by: you06 <you1474600@gmail.com>

* fix unstopped test

Signed-off-by: you06 <you1474600@gmail.com>

* set resource group tags for committer

Signed-off-by: ekexium <eke@fastmail.com>

* skip test due to tikv image not updated yet

Signed-off-by: you06 <you1474600@gmail.com>

* skip more test

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
 2024-03-16 11:36:05 +08:00
ekexium 280cb0852e
feat: add pipeliend field in TxnInfo (#1228) 
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
 2024-03-15 12:45:02 +08:00
ekexium d7000ea557
feat: pipelined dml has its own max-txn-ttl of 24 hours (#1224) 
* feat: pipelined dml has its own max-txn-ttl of 24 hours

Signed-off-by: ekexium <eke@fastmail.com>

* fix gh actions

Signed-off-by: ekexium <eke@fastmail.com>

* style: fix lint

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
 2024-03-14 10:54:19 +08:00
you06 88fbc938cd
p-dml: fix unstopped resolve (#1210) 
* fix unstopped resolve when the txn cross regions

Signed-off-by: you06 <you1474600@gmail.com>

* cancel context when flush done or abort

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
 2024-03-11 21:20:38 +08:00
you06 af4f9a9b6e
fix data race when collapsing ResolveLockRequest (#1209) 
* fix data race when collapsing ResolveLockRequest

Signed-off-by: you06 <you1474600@gmail.com>

* comment

Signed-off-by: you06 <you1474600@gmail.com>

* sleep to wait goroutine finish

Signed-off-by: you06 <you1474600@gmail.com>

* update comment

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
 2024-03-08 13:24:15 +08:00
you06 c72236a601
adjust parameter for pipelined memdb by failpoint (#1205) 
* adjust parameter for pipelined memdb by failpoint

Signed-off-by: you06 <you1474600@gmail.com>

* update comment

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
 2024-03-06 16:44:48 +08:00
cfzjywxk 83ec5f2f33
add ts information in the pre split path (#1206) 
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
 2024-03-06 14:46:42 +08:00
ystaticy da22d65e94
support gc v2 (#1180) 
Signed-off-by: y_static_y@sina.com <y_static_y@sina.com>
 2024-02-27 11:44:28 +08:00
you06 8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML (#1114) 
* Support pipelined memdb.

Signed-off-by: you06 <you1474600@gmail.com>

* fix mutex for pipelined memdb

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
 2024-02-23 10:24:44 +08:00
cfzjywxk 11cb7985f0
txn: enable pessimistic region rollback (#1149) 
* enable pessimistic region rollback

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* add key range parameters

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* change interface to use start and end keys

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

---------

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
 2024-02-05 15:11:26 +08:00
ekexium 70c148e84e
ErrQueryInterrupted with parameters (#1124) 
* feat: ErrQueryInterrupted with parameters

Signed-off-by: ekexium <eke@fastmail.com>

* Revert "Revert "fix: check kill signal against 0 (#1102)" (#1129)"

This reverts commit 3480b5ed7c.

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
 2024-02-02 10:26:02 +08:00
cfzjywxk d2887d56ab
txn: use region pessimsitic lock rollback to speed up cleanup (#1125) 
* use region pessimsitic lock rollback and clean for write-write conflict processing

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* format

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* debug ci failure patch, disable region pessimistic rollback

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

---------

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
 2024-01-31 09:53:00 +08:00
Weizhen Wang fd2fc84032
use slices.Sort to eliminate bounds check (#1128) 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
 2024-01-23 11:24:15 +08:00
YangKeao abda04c22a
export stats information about RPC count (#1110) 
Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>
 2024-01-12 15:16:19 +08:00
ekexium 2ea84a6c5a
feat: check kill signal in 2pc committer (#1084) 
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: disksing <i@disksing.com>
 2023-12-26 14:54:34 +08:00
Smilencer 6ba909c4ad
config: enable user configuration for retry strategy. (#1055) 
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
 2023-11-14 14:54:10 +08:00
cfzjywxk 07432ef6c0
refactor the func name to make it clear (#1000) 
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
 2023-10-10 14:18:02 +08:00
Weizhen Wang 18a8414120
*: fix data race at the aggressiveLockingDirty (#913) 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
 2023-10-08 14:40:27 +08:00
you06 295094e5b5
add retry info to request source (#953) 
* add retry info to request source

Signed-off-by: you06 <you1474600@gmail.com>

* handle upper layer retry

Signed-off-by: you06 <you1474600@gmail.com>

* stabilize test

Signed-off-by: you06 <you1474600@gmail.com>

* retry in 3 dimension

Signed-off-by: you06 <you1474600@gmail.com>

* record and restore req.ReadType

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
 2023-08-29 08:28:46 +08:00
crazycs 8a214402da
use tidb_kv_read_timeout as first kv request timeout (#919) 
* support tidb_kv_read_timeout as first round kv request timeout

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* fix ci

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* update comment

Signed-off-by: crazycs520 <crazycs520@gmail.com>

* refine test

Signed-off-by: crazycs520 <crazycs520@gmail.com>

---------

Signed-off-by: crazycs520 <crazycs520@gmail.com>
 2023-08-11 11:37:10 +08:00
Hangjie Mo 2f119351bd
*: add `SnapshotIterReverse` and make `iterReverse` supports `lowerBound` (#883) 
Signed-off-by: Jason Mo <mohangjie1995@gmail.com>
 2023-07-14 10:36:07 +08:00
ShuNing 178f6fa01a
add explicit request source type to label the external request like lightning/br (#868) 
Signed-off-by: nolouch <nolouch@gmail.com>
 2023-07-07 15:02:42 +08:00
MyonKeminta c0cf773917
Fix the issue that primary pessimistic lock may be left not cleared after GC (#866) 
* Fix the issue that primary pessimistic lock may be left not cleared after GC

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix mysteriously shown up thing that makes compilation failed

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test effectiveness (forgot to set txn2 to pessimistic txn); add more strict checks

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
 2023-07-04 15:17:05 +08:00
zyguan 04dc78e522
lock_resolver: let getTxnStatusFromLock return error when backoff timeout (#847) 
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: disksing <i@disksing.com>
 2023-06-21 13:07:44 +08:00
glorv 0b4b0ca00e
kvrpc: deduplicate chained intercept by name (#832) 
* dedup chained interceptor

Signed-off-by: glorv <glorvs@163.com>
 2023-06-13 20:21:06 -07:00
you06 11121c6913
Remove stale-read flag after resolving lock (#792) 
* remove stale-read when key-is-locked

Signed-off-by: you06 <you1474600@gmail.com>

* disable follower read for batch get

Signed-off-by: you06 <you1474600@gmail.com>

* the stale-read flag may already set to false

Signed-off-by: you06 <you1474600@gmail.com>

* fix batchget

Signed-off-by: you06 <you1474600@gmail.com>

* Reset busy-threshold when stale read fallback to leader read

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

* trigger CI

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
 2023-05-26 13:55:53 +08:00
zyguan 2b0667c65c
lock_resolver: handle pessimistic locks in BatchResolveLocks (#794) 
* lock_resolver: handle pessimistic locks in BatchResolveLocks

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test but failed on unistore because unistore's ScanLock doesnt return lock type

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: zyguan <zhongyangguan@gmail.com>

* Fix golangci

Signed-off-by: zyguan <zhongyangguan@gmail.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
 2023-05-11 12:26:01 +08:00
MyonKeminta 5c324b7c1e
lock_resolver: support verifying primary for check_txn_status (#777) 
* support verifying primary for check_txn_status

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update kvproto

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* add more failpoint usages

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update depencency and fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Do not skip for unistore; refine logs

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
 2023-05-08 16:18:26 +08:00
Connor 62cc95123c
Fix wrong resource group name for some requests (#788) 
* fix wrong resource group name

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
 2023-04-28 19:03:56 +08:00
disksing aee2f311a5
txn: fix goroutine leak (#784) 
Signed-off-by: disksing <i@disksing.com>
 2023-04-27 17:39:07 +08:00
Connor 35c1ee47c4
Set resource group penalty (#772) 
* Set resource group penalty 

Signed-off-by: Connor1996 <zbk602423539@gmail.com>
 2023-04-19 20:39:20 +08:00
ekexium a72e61dc64
log: print log if the assertion error is generated from client-go (#771) 
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: disksing <i@disksing.com>
 2023-04-19 17:42:17 +08:00
MyonKeminta 92db9f7b15
Support specifying checking forUpdateTS in prewrite (#756) 
* Support specifying checking forUpdateTS in prewrite

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix go mod in tests

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add successful cases to the test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update dependencies

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Check LockedWithConflictTS greater than requested ForUpdateTS

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
 2023-04-17 14:53:28 +08:00
Jack Yu c853ddc68c
metrics: add txn ttl lifetime (#766) 
Signed-off-by: Jack Yu <jackysp@gmail.com>
Co-authored-by: Shirly <AndreMouche@126.com>
 2023-04-14 13:59:29 +08:00
ekexium 67e56a956f
For KeyIsLocked error reported for timeout, if a lock is recently updated, don't try to resolve it. (#758) 
* update client-go; format

Signed-off-by: ekexium <eke@fastmail.com>

* feat: do not resolve lock if duration_to_last_updated is short

Signed-off-by: ekexium <eke@fastmail.com>

* adjust the threshold to 1200ms to allow small deviation

Signed-off-by: ekexium <eke@fastmail.com>

* fix: don't treat it as WriteConflict, simply retry

Signed-off-by: ekexium <eke@fastmail.com>

* update kvproto

Signed-off-by: ekexium <eke@fastmail.com>

* set the threshold to 300ms

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
 2023-04-13 16:14:28 +08:00
ekexium f3e87039d8
Finer `is_retry_request` settings (#763) 
* set is_retry_request only for requests that is possible to have undetermined errors

Signed-off-by: ekexium <eke@fastmail.com>

* rename tryTimes to retryTimes

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
 2023-04-10 10:39:43 +08:00
MyonKeminta ea13e97002
Add exitAggressiveLockingIfInapplicable function and call within mutex of lockKeys (#749) 
* Add exitAggressiveLockingIfInapplicable function and call within mutex of lockKeys

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* fix fmt

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
 2023-03-28 16:41:04 +08:00
JmPotato 9d950905d7
Introduce the RURuntimeStats (#732) 
Signed-off-by: JmPotato <ghzpotato@gmail.com>
 2023-03-16 10:19:36 +08:00
glorv c9119d02ce
support time detail v2 (#729) 
* support time detail v2

Signed-off-by: glorv <glorvs@163.com>

---------

Signed-off-by: glorv <glorvs@163.com>
Co-authored-by: Connor <zbk602423539@gmail.com>
 2023-03-13 21:32:19 +08:00
MyonKeminta f555fdd2c9
txn: support force locking and lock only if exists be used together (#727) 
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
 2023-03-09 18:08:32 +08:00
cfzjywxk bdd41058aa
metrics: seperate metrics with source scope for txn command (#723) 
* seperate metrics with source scope

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

* fix async pessimistic rollback

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

* fix missing definition

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

---------

Signed-off-by: cfzjywxk <lsswxrxr@163.com>
 2023-03-06 22:12:45 +08:00
cfzjywxk 49323956ae
metric: seperate source for txn cmd (#721) 
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
 2023-02-28 15:40:40 +08:00
glorv 596b79404d
set group name in twoPhaseCommitter (#718) 
Signed-off-by: glorv <glorvs@163.com>
 2023-02-27 16:39:23 +08:00
Weizhen Wang 40a82457eb
tikv: configurable pool (#714) 
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
 2023-02-27 11:23:58 +08:00
Yilin Chen a27994e5aa
Support load-based replica read (#675) 
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
 2023-02-22 15:35:43 +08:00
ekexium 2051f572a1
fix: lock_if_exist + aggressive locking (#705) 
* fix: locking commands skipped by lock_only_if_exists are mistakenly calculated as successfully locked for aggressive locking

Signed-off-by: ekexium <ekexium@gmail.com>

* refactor

Signed-off-by: ekexium <ekexium@gmail.com>

---------

Signed-off-by: ekexium <ekexium@gmail.com>
 2023-02-20 09:50:30 +08:00
MyonKeminta 50e86f7d3c
Add metrics and statistics about aggressive locking (#687) 
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
 2023-02-07 12:31:34 +08:00
fengou1 fe5b35c5f5
refactor: range task log with more accurate info (#657) 
Signed-off-by: fengou1 <feng.ou@pingcap.com>
 2023-01-12 14:20:23 +08:00
David f313ddf58d
reformat codec and add keyspace support (#649) 
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
 2023-01-10 15:15:33 +08:00
wjHuang 2d2a3fa00e
*: remove the support of `amend transaction` (#635) 
* done

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

* done

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

* fix test

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

* fix lint

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

* fix test

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

* fix

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>

Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
 2023-01-09 16:30:27 +08:00