Commit Graph

262 Commits

Author SHA1 Message Date
MyonKeminta 5042c6f2aa
Return assertion failed error less prior to other kinds of errors (#448)
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2022-03-11 12:36:19 +08:00
龙方淞 cd7e7681c2
Update lock ttl when retry to pessimistic lock keys (#417)
Signed-off-by: longfangsong <longfangsong@icloud.com>
2022-03-10 16:10:47 +08:00
Lei Zhao 7e34d88af3
region_cache: support buckets (#439)
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2022-03-09 13:52:39 +08:00
cfzjywxk 079e2dce4e
txn: support read-consistency read with tso checking (#447)
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2022-03-08 10:31:03 +08:00
disksing 1aa72efa9b
mockstore: fix assertion (#446)
Signed-off-by: disksing <i@disksing.com>
2022-03-07 13:10:35 +08:00
disksing 7cbe10ea93
integration: try not to skip tests when run with tikv (#445)
Signed-off-by: disksing <i@disksing.com>
2022-03-07 11:27:20 +08:00
Zak Zhao 8c3f29a55f
Add options parameters in API CompareAndSwap (#433)
Signed-off-by: joccau <zak.zhao@pingcap.com>
2022-03-03 13:09:17 +08:00
Yilin Chen a6db0c9a3e
Fix incorrect LockNoWait value (#435)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-03-02 20:03:17 +08:00
disksing 0214a9efa0
integration: replace some unistore with mockstore (#432)
Signed-off-by: disksing <i@disksing.com>
2022-03-02 16:36:25 +08:00
disksing cb9955362c
integration: use less tidb code (#431)
Signed-off-by: disksing <i@disksing.com>
2022-03-02 15:00:40 +08:00
Zak Zhao d3c2456967
Add a columnFamily feature for client-go when use rawput in tikv (#425)
Signed-off-by: joccau <zak.zhao@pingcap.com>
2022-03-02 11:31:44 +08:00
disksing df187fa79a
*: update readme (#430)
Signed-off-by: disksing <i@disksing.com>
2022-02-24 16:50:07 +08:00
disksing e5f932ae27
txnkv: fix the issue that deleteRange cannot use nil as endkey (#429)
Signed-off-by: disksing <i@disksing.com>
2022-02-24 15:51:38 +08:00
Lei Zhao f43cc3acc0
region_cache: more comments and minor refactor (#427)
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2022-02-23 16:53:24 +08:00
disksing 79b962a84d
integration: update tidb to master branch (#426)
Signed-off-by: disksing <i@disksing.com>
2022-02-22 11:34:36 +08:00
disksing 589cf79b2a
*: update etcd client to v3.5.2 (#423)
Signed-off-by: disksing <i@disksing.com>
2022-02-17 22:01:16 +08:00
disksing b7a3ace8e6
txnkv: add shortcut to get global timestamp (#395)
Signed-off-by: disksing <i@disksing.com>
2022-02-11 12:40:35 +08:00
MyonKeminta b5eb031edd
Support setting assertions and receiving assertion errors for Prewrite requests (#311)
* support set assertion in 2pc mutation

Signed-off-by: lysu <sulifx@gmail.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Receive assertion fail errors from TiKV

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test log

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove verbose log

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update kvproto

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add metrics counter for assertions

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address some comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try to optimize assertion for pessimistic transactions

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix panic on optimistic transactions

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add InitCheckExistence method for LockCtx

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Support assertion level

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Check assertion level before doing assertion on client side

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Test bitoperations of menBUfferMutations.Push

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test for assertion in tikv

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Support run assertion test with unistore

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix license

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* export DeleteKey

Signed-off-by: ekexium <ekexium@gmail.com>

* Renaming DeleteKey

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* fix build

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Avoid panic when running with old version of TiKV; Add schema check on fast assertion

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test for fast assertion

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add test for pessimistic lock check existence

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Test assertion takes no effect if amending is enabled

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add HasAssertUnknown function

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Cleanup locks after assertion fail

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update tidb dependency

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix panic in TestIllegalTSO

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update dependency to tidb

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

Co-authored-by: lysu <sulifx@gmail.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: ekexium <ekexium@gmail.com>
2022-02-10 15:31:32 +08:00
LemonHX cea0c4c6e6
update tidb version (#419)
Signed-off-by: lemonhx <lemonhx@lemonhx.tech>
2022-02-09 09:41:21 +08:00
Yilin Chen 50bafbb1e9
internal/retry: return the longest-sleeping type on backoff timeout (#416)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-01-13 11:19:47 +08:00
LemonHX d228150997
implement pd's global config (#405)
* update kvproto and pd, implement global config mock api

Signed-off-by: lemonhx <lemonhx@lemonhx.tech>

* update tidb version

Signed-off-by: lemonhx <lemonhx@lemonhx.tech>

* update pd version to pass tikv integration test

Signed-off-by: lemonhx <lemonhx@lemonhx.tech>
2022-01-06 20:00:26 -08:00
Matthieu Vachon c24a5ba915
rawkv: Added back `ScanOption` from `v1` but with a more standard Golang pattern (#402)
* rawkv: Added back `ScanOption` from `v1` but with a more standard Golang pattern

This enables specifying:
- KeyOnly
- Cf

#### Notes

This could also be used to remove the `ReverseScan` version and `ScanReverse` option could be used. It could be argue that `ReverseScan` is more discoverable. Happy to remove if it's decided that `ScanReverse` is discoverable enough.

Signed-off-by: Matthieu Vachon <matt@streamingfast.io>

* Fixed linting issues

Signed-off-by: Matthieu Vachon <matt@streamingfast.io>

Co-authored-by: Matthieu Vachon <matt@streamingfast.io>
2022-01-06 18:06:34 -08:00
disksing 13298f12fb
rawkv: split BatchPut and BatchPutTTL (#414)
Signed-off-by: disksing <i@disksing.com>
2022-01-06 02:46:37 -08:00
iosmanthus fe2def20bb
fix RawBatchPutRequest forward compatibility (#403)
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
2022-01-05 22:53:06 -08:00
yiksanchan d2f0290d01
docs: make it explicit that DeleteRange deletes [start, end) (#413)
Signed-off-by: Yik San Chan <evan.chanyiksan@gmail.com>
2022-01-04 15:45:29 +08:00
Yilin Chen dd263ab9f1
integration_tests: speed up TestCommitOnTiKVDiskFullOpt (#411)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-01-04 14:58:47 +08:00
you06 62d6b4a2e8
txnkv: set TiKV RC for weak-consistency transactions (#407)
* add tikv-rc support

Signed-off-by: you06 <you1474600@gmail.com>

* add test

Signed-off-by: you06 <you1474600@gmail.com>

* format code

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb && address comment

Signed-off-by: you06 <you1474600@gmail.com>

* update test log

Signed-off-by: you06 <you1474600@gmail.com>
2021-12-29 13:16:14 +08:00
july2993 3b10001094
Add miss register for TiKVUnsafeDestroyRangeFailuresCounterVec (#406)
Signed-off-by: Jiahao Huang <july2993@gmail.com>
2021-12-28 11:31:51 +08:00
Weizhen Wang 300275dee6
fix data race and panic when exiting (#396)
* fix the data race when exiting

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* remove error and change log level

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* add warn log

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2021-12-23 14:21:59 +08:00
cfzjywxk e9de5625c4
txn: change IsUnnecessaryKeyValue interface to report error for unexpected paths (#390)
* change IsUnnecessaryKeyValue interface to report error for unexpected situations

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

* update integrated tests go mod

Signed-off-by: cfzjywxk <lsswxrxr@163.com>

Co-authored-by: disksing <i@disksing.com>
2021-12-20 20:12:11 -08:00
Jack Yu 02f89e38ec
metrics: make batch client requests more accurate (#397)
Signed-off-by: Yu Shuaipeng <jackysp@gmail.com>

Co-authored-by: disksing <i@disksing.com>
2021-12-20 20:05:26 -08:00
Yexiang Zhang 6165dbaa95
introduce RPC interceptor mechanism (#389)
* Add interceptor mechanism for tikv RPC

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Rename comment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Modify comments

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix ineffectual assignment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* add AddInterceptor for KVTxn and KVSnapshot

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Separate intercepor package

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add comments

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add order test

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix integration tests

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix integration tests

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Move interceptor call from client to client_collapse

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add comments

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix comment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add client_interceptor

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Modify comment

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add ut

Signed-off-by: mornyx <mornyx.z@gmail.com>
2021-12-18 13:03:06 +08:00
tiancaiamao 605f49d3ba
txnkv/transaction: add a kv option for cached table lock lease (#392)
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
2021-12-14 01:37:15 -08:00
Song Gao b147ced35a
client: skip update backward safets (#393)
Signed-off-by: yisaer <disxiaofei@163.com>
2021-12-12 23:51:51 -08:00
disksing 87c1c58064
move tikv.NewTxnClient to txnkv.NewClient (#391)
Signed-off-by: disksing <i@disksing.com>
2021-12-09 21:38:38 -08:00
Lei Zhao c0e8766154
txnkv: decrease some max and base backoff time (#388)
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-12-06 15:29:23 +08:00
Lei Zhao 3a7675742e
txnkv: read through locks (#380)
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-12-01 16:35:10 +08:00
Ziqian Qin 350b6ce9fb
Add an RemoveFromBuffer method for test (#387)
* export DeleteKey

Signed-off-by: ekexium <ekexium@gmail.com>

* rename DeleteKey to UnsafeRemoveRecord

Signed-off-by: ekexium <ekexium@gmail.com>

* rename the old DeleteKey

Signed-off-by: ekexium <ekexium@gmail.com>
2021-11-30 19:43:54 +08:00
Yexiang Zhang b11da6307c
txnkv: add ResourceGroupTagger for PessimisticLockRequest (#381)
* Add ResourceGroupTagger for PessimisticLockRequest

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix circular dependency

Signed-off-by: mornyx <mornyx.z@gmail.com>
2021-11-18 23:41:39 +08:00
Sandy Xu b0d9bb8821
Bump go version to 1.16 (#379)
* Revert "use ioutil.ReadFile to release dependency of Go 1.16 (#217)"

This reverts commit aadf3cf627.

Signed-off-by: Sandy Xu <sandy@juicedata.io>

* use go 1.16 for examples

Signed-off-by: Sandy Xu <sandy@juicedata.io>
2021-11-18 06:06:13 -08:00
cfzjywxk e0bf24b24d
add newly inserted flag to optimize delete your write (#378)
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2021-11-15 21:23:32 -08:00
MyonKeminta a3f1c41ac1
Fix a typo (#376)
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2021-11-15 15:10:40 +08:00
JmPotato 894037ac74
Print the Region key with a more readable format (#375)
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2021-11-14 22:08:58 -08:00
Yexiang Zhang 3b9f591c3f
txnkv: add callback for setting ResourceGroupTag (#368)
* Replace resourceGroupTag with resourceGroupTagFactory(firstKey)

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix npe

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Abstract ResourceGroupFactory

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Optimized as ResourceGroupTagger

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Add test for resource group tagger

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix unexpected request failed in mock store

Signed-off-by: mornyx <mornyx.z@gmail.com>

* Fix integration tests

Signed-off-by: mornyx <mornyx.z@gmail.com>

Co-authored-by: Wenxuan <breezewish@pingcap.com>
2021-11-14 19:11:14 -08:00
disksing dd4bc91453
test: remove ticdc compatibility test (#366)
Signed-off-by: disksing <i@disksing.com>
2021-11-12 00:15:04 -08:00
Xu Qiaolun 8a4ce38c2a
tikv: remove BeginWithOption method (#370)
* tikv: remove BeginWithOption method

Signed-off-by: Xu Qiaolun <jamesxql@gmail.com>

* reset the tidb version in integration test to latest tidb master

Signed-off-by: Xu Qiaolun <jamesxql@gmail.com>
2021-11-11 00:56:58 -08:00
Yilin Chen 6ac6a8daf2
Abort if the mismatching lock is from the current transaction when resolving locks (#367)
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-11-10 10:15:56 +08:00
Xu Qiaolun 5ae005dac3
Use a interface method to update the transaction option (#363)
* txn: Use a interface method to update the transaction option

Signed-off-by: Xu Qiaolun <jamesxql@gmail.com>

* txn: merge Begin and BeginWithOption into one method

Signed-off-by: Xu Qiaolun <jamesxql@gmail.com>
2021-11-03 02:29:33 +00:00
Song Gao b8cc5a319d
fix metrics error (#365)
Signed-off-by: yisaer <disxiaofei@163.com>
2021-11-02 12:05:33 +00:00
disksing c0efee0618
errors: use As to check error type (#361) 2021-11-01 10:32:00 +00:00