Commit Graph

84 Commits

Author SHA1 Message Date
disksing e5f932ae27
txnkv: fix the issue that deleteRange cannot use nil as endkey (#429)
Signed-off-by: disksing <i@disksing.com>
2022-02-24 15:51:38 +08:00
disksing 79b962a84d
integration: update tidb to master branch (#426)
Signed-off-by: disksing <i@disksing.com>
2022-02-22 11:34:36 +08:00
disksing 589cf79b2a
*: update etcd client to v3.5.2 (#423)
Signed-off-by: disksing <i@disksing.com>
2022-02-17 22:01:16 +08:00
MyonKeminta b5eb031edd
Support setting assertions and receiving assertion errors for Prewrite requests (#311)
* support set assertion in 2pc mutation

Signed-off-by: lysu <sulifx@gmail.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Receive assertion fail errors from TiKV

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test log

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove verbose log

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update kvproto

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add metrics counter for assertions

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address some comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try to optimize assertion for pessimistic transactions

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix panic on optimistic transactions

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add InitCheckExistence method for LockCtx

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Support assertion level

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Check assertion level before doing assertion on client side

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Test bitoperations of menBUfferMutations.Push

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test for assertion in tikv

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Support run assertion test with unistore

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix license

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* export DeleteKey

Signed-off-by: ekexium <ekexium@gmail.com>

* Renaming DeleteKey

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* fix build

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Avoid panic when running with old version of TiKV; Add schema check on fast assertion

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test for fast assertion

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test for pessimistic lock check existence

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Test assertion takes no effect if amending is enabled

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add HasAssertUnknown function

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Cleanup locks after assertion fail

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update tidb dependency

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix panic in TestIllegalTSO

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update dependency to tidb

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

Co-authored-by: lysu <sulifx@gmail.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: ekexium <ekexium@gmail.com>
2022-02-10 15:31:32 +08:00
LemonHX cea0c4c6e6
update tidb version (#419)
Signed-off-by: lemonhx <lemonhx@lemonhx.tech>
2022-02-09 09:41:21 +08:00
LemonHX d228150997
implement pd's global config (#405)
* update kvproto and pd, implement global config mock api

Signed-off-by: lemonhx <lemonhx@lemonhx.tech>

* update tidb version

Signed-off-by: lemonhx <lemonhx@lemonhx.tech>

* update pd version to pass tikv integration test

Signed-off-by: lemonhx <lemonhx@lemonhx.tech>
2022-01-06 20:00:26 -08:00
disksing 13298f12fb
rawkv: split BatchPut and BatchPutTTL (#414)
Signed-off-by: disksing <i@disksing.com>
2022-01-06 02:46:37 -08:00
iosmanthus fe2def20bb
fix RawBatchPutRequest forward compatibility (#403)
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
2022-01-05 22:53:06 -08:00
Yilin Chen dd263ab9f1
integration_tests: speed up TestCommitOnTiKVDiskFullOpt (#411)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-01-04 14:58:47 +08:00
you06 62d6b4a2e8
txnkv: set TiKV RC for weak-consistency transactions (#407)
* add tikv-rc support

Signed-off-by: you06 <you1474600@gmail.com>

* add test

Signed-off-by: you06 <you1474600@gmail.com>

* format code

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb && address comment

Signed-off-by: you06 <you1474600@gmail.com>

* update test log

Signed-off-by: you06 <you1474600@gmail.com>
2021-12-29 13:16:14 +08:00
cfzjywxk e9de5625c4
txn: change IsUnnecessaryKeyValue interface to report error for unexpected paths (#390)
* change IsUnnecessaryKeyValue interface to report error for unexpected situations

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

* update integrated tests go mod

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

Co-authored-by: disksing <i@disksing.com>
2021-12-20 20:12:11 -08:00
Yexiang Zhang 6165dbaa95
introduce RPC interceptor mechanism (#389)
* Add interceptor mechanism for tikv RPC

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Rename comment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Modify comments

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix ineffectual assignment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* add AddInterceptor for KVTxn and KVSnapshot

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Separate intercepor package

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add comments

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add order test

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix integration tests

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix integration tests

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Move interceptor call from client to client_collapse

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add comments

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix comment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add client_interceptor

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Modify comment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add ut

Signed-off-by: mornyx <mornyx.z@gmail.com>
2021-12-18 13:03:06 +08:00
Lei Zhao c0e8766154
txnkv: decrease some max and base backoff time (#388)
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-12-06 15:29:23 +08:00
Lei Zhao 3a7675742e
txnkv: read through locks (#380)
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-12-01 16:35:10 +08:00
cfzjywxk e0bf24b24d
add newly inserted flag to optimize delete your write (#378)
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2021-11-15 21:23:32 -08:00
Yexiang Zhang 3b9f591c3f
txnkv: add callback for setting ResourceGroupTag (#368)
* Replace resourceGroupTag with resourceGroupTagFactory(firstKey)

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix npe

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Abstract ResourceGroupFactory

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Optimized as ResourceGroupTagger

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add test for resource group tagger

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix unexpected request failed in mock store

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix integration tests

Signed-off-by: mornyx <mornyx.z@gmail.com>

Co-authored-by: Wenxuan <breezewish@pingcap.com>
2021-11-14 19:11:14 -08:00
Xu Qiaolun 8a4ce38c2a
tikv: remove BeginWithOption method (#370)
* tikv: remove BeginWithOption method

Signed-off-by: Xu Qiaolun <jamesxql@gmail.com>

* reset the tidb version in integration test to latest tidb master

Signed-off-by: Xu Qiaolun <jamesxql@gmail.com>
2021-11-11 00:56:58 -08:00
Yilin Chen 6ac6a8daf2
Abort if the mismatching lock is from the current transaction when resolving locks (#367)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-11-10 10:15:56 +08:00
disksing 9ec50224be
*: replace pingcap/errors with pkg/errors (#357)
Signed-off-by: disksing <i@disksing.com>
2021-11-01 11:59:52 +08:00
JmPotato cef46d13cc
Clean up the temporary changes (#360)
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2021-11-01 10:25:52 +08:00
JmPotato 2fd3841894
Update the PD client to the latest version (#352)
* Update the PD client to the latest version

Signed-off-by: JmPotato <ghzpotato@gmail.com>

* Temporarily change the TiDB test branch

Signed-off-by: JmPotato <ghzpotato@gmail.com>

* Fix the client-go/integration_tests go.mod

Signed-off-by: JmPotato <ghzpotato@gmail.com>

* Fix the client-go/integration_tests go.mod again

Signed-off-by: JmPotato <ghzpotato@gmail.com>

* Fix the integration_tests

Signed-off-by: JmPotato <ghzpotato@gmail.com>
2021-10-29 10:40:11 +00:00
disksing 2d145163d0
txnkv/transaction: remove use of errors.Trace (#351)
Signed-off-by: disksing <i@disksing.com>
2021-10-27 10:13:37 +08:00
disksing 61664cf55c
integration_tests: remove use of errors.Trace (#327)
Signed-off-by: disksing <i@disksing.com>
2021-10-13 06:58:44 +00:00
disksing 9921da23d5
*: remove parser dependency (#324)
Signed-off-by: disksing <i@disksing.com>
2021-10-12 00:54:00 +08:00
disksing 49c8dd23f1
error: remove use of terror.ErrResultUndetermined (#323)
Signed-off-by: disksing <i@disksing.com>
2021-10-11 16:31:57 +08:00
Alkaid 128ccbde43
Add read throughput metric for SLI (#321)
Signed-off-by: jyz0309 <45495947@qq.com>
2021-10-11 11:46:20 +08:00
Yilin Chen ee5063fec3
Fix flaky TestSecondaryListInPrimaryLock (#318) 2021-09-29 13:45:23 +08:00
daimashusheng 4f806a5442
batchput add ttl to every key (#298)
Signed-off-by: tangjk <tangjiankun1226@gmail.com>
2021-09-26 11:50:08 +00:00
disksing 2d250d555d
error: add function to check if error is undetermined (#300)
Signed-off-by: disksing <i@disksing.com>
2021-09-08 13:10:34 +08:00
Lei Zhao 1798c19099
separate rawkv and txnkv tests (#295)
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-09-01 22:05:16 +08:00
Yilin Chen 16d902a3c7
reduce write backoffs but don't count busy errors (#271)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-08-24 09:05:36 +00:00
Yilin Chen b440ea2f58
select replica for forward requests using RegionRequestSender (#264)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-08-16 19:27:02 +08:00
disksing df2119f51b
client: cleanup RPCClient options (#268)
Signed-off-by: disksing <i@disksing.com>
2021-08-16 07:11:08 +00:00
disksing 288c649636
fix license (#267)
Signed-off-by: disksing <i@disksing.com>
2021-08-15 01:48:01 +08:00
tier-cap 14892a598e
*: allow to custom disk_full_opt for transactions (#256)
Signed-off-by: tier-cap <zhengxiaojin@pingcap.com>
2021-08-06 16:56:16 +08:00
Yilin Chen 6f33dd97af
Start heartbeat immediately after locking the primary key (#261)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-08-06 11:14:25 +08:00
disksing d535b62b62
txnkv/transaction: init package (#260)
Signed-off-by: shirly <AndreMouche@126.com>
Co-authored-by: disksing <i@disksing.com>
2021-07-31 16:26:22 +08:00
Shirly 21ea47fe19
tikv: move temporary export types to package txnkv (#258)
Signed-off-by: shirly <AndreMouche@126.com>
2021-07-31 14:50:56 +08:00
Yilin Chen 7329876df3
add flag in Context for retried requests (#244)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-07-27 16:56:30 +08:00
Shirly 070dd85543
txnkv/txnsnapshot: init package for txnsnapshot (#245)
Signed-off-by: shirly <AndreMouche@126.com>
2021-07-27 12:10:35 +08:00
you06 ce977e34b0
test: fix overflow in ttl equal compare (#246)
Signed-off-by: you06 <you1474600@gmail.com>
2021-07-26 05:40:04 +00:00
Shirly ac355144c8
tikv/txn: add new pub function LockKeysWithWaitTime (#230)
* tikv/txn: add new pub function LockKeysWithWaitTime

Signed-off-by: shirly <AndreMouche@126.com>

* address comments

Signed-off-by: shirly <AndreMouche@126.com>

* tikv/pessimistic: update the constant for lock wait time

Signed-off-by: shirly <AndreMouche@126.com>

* kv/LockCtx: make lockWaitTime private

Signed-off-by: shirly <AndreMouche@126.com>

* address comments

Signed-off-by: shirly <AndreMouche@126.com>
2021-07-23 10:31:23 +08:00
iosmanthus f3e7dc3042
add raw key ttl feature (#222)
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
2021-07-21 16:49:17 +08:00
Shirly 654864ded8
txnkv/txnlock: init Lock in txnlock package (#241)
Signed-off-by: shirly <AndreMouche@126.com>
2021-07-21 15:03:08 +08:00
iosmanthus ce04aa0bf1
add raw cas feature (#239)
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
2021-07-21 13:52:18 +08:00
Yilin Chen f99e5a9251
reset resolvedLocks when setting snapshot ts (#233)
* reset resolvedLocks when setting snapshot ts

Signed-off-by: Yilin Chen <sticnarf@gmail.com>

* add test

Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-07-16 14:15:12 +08:00
Shirly 0539869024
txnkv/*: init txnkv/rangetask package (#234) 2021-07-15 10:16:15 +08:00
disksing f4a87d3b4a
clean up test utilities (#223)
Signed-off-by: disksing <i@disksing.com>
2021-07-13 16:11:59 +08:00
Yilin Chen f5a3974312
do not check sessionID for async commit and 1PC (#228)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-07-13 06:53:58 +00:00
disksing 0fdc8e3d6f
rawkv: move rawkv client to rawkv (#227)
Signed-off-by: disksing <i@disksing.com>

Co-authored-by: Shirly <AndreMouche@126.com>
2021-07-13 11:56:49 +08:00