88fbc938cd
p-dml: fix unstopped resolve ( #1210 )
...
* fix unstopped resolve when the txn cross regions
Signed-off-by: you06 <you1474600@gmail.com>
* cancel context when flush done or abort
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-03-11 21:20:38 +08:00
af4f9a9b6e
fix data race when collapsing ResolveLockRequest ( #1209 )
...
* fix data race when collapsing ResolveLockRequest
Signed-off-by: you06 <you1474600@gmail.com>
* comment
Signed-off-by: you06 <you1474600@gmail.com>
* sleep to wait goroutine finish
Signed-off-by: you06 <you1474600@gmail.com>
* update comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-03-08 13:24:15 +08:00
83ec5f2f33
add ts information in the pre split path ( #1206 )
...
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-03-06 14:46:42 +08:00
8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML ( #1114 )
...
* Support pipelined memdb.
Signed-off-by: you06 <you1474600@gmail.com>
* fix mutex for pipelined memdb
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
11cb7985f0
txn: enable pessimistic region rollback ( #1149 )
...
* enable pessimistic region rollback
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* add key range parameters
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* change interface to use start and end keys
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-02-05 15:11:26 +08:00
70c148e84e
ErrQueryInterrupted with parameters ( #1124 )
...
* feat: ErrQueryInterrupted with parameters
Signed-off-by: ekexium <eke@fastmail.com>
* Revert "Revert "fix: check kill signal against 0 (#1102 )" (#1129 )"
This reverts commit 3480b5ed7c
2024-02-02 10:26:02 +08:00
d2887d56ab
txn: use region pessimsitic lock rollback to speed up cleanup ( #1125 )
...
* use region pessimsitic lock rollback and clean for write-write conflict processing
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* format
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* debug ci failure patch, disable region pessimistic rollback
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-01-31 09:53:00 +08:00
fd2fc84032
use slices.Sort to eliminate bounds check ( #1128 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-01-23 11:24:15 +08:00
2ea84a6c5a
feat: check kill signal in 2pc committer ( #1084 )
...
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-12-26 14:54:34 +08:00
6ba909c4ad
config: enable user configuration for retry strategy. ( #1055 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 14:54:10 +08:00
18a8414120
*: fix data race at the aggressiveLockingDirty ( #913 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-10-08 14:40:27 +08:00
2f119351bd
*: add `SnapshotIterReverse` and make `iterReverse` supports `lowerBound` ( #883 )
...
Signed-off-by: Jason Mo <mohangjie1995@gmail.com>
2023-07-14 10:36:07 +08:00
178f6fa01a
add explicit request source type to label the external request like lightning/br ( #868 )
...
Signed-off-by: nolouch <nolouch@gmail.com>
2023-07-07 15:02:42 +08:00
c0cf773917
Fix the issue that primary pessimistic lock may be left not cleared after GC ( #866 )
...
* Fix the issue that primary pessimistic lock may be left not cleared after GC
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix mysteriously shown up thing that makes compilation failed
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test effectiveness (forgot to set txn2 to pessimistic txn); add more strict checks
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-07-04 15:17:05 +08:00
5c324b7c1e
lock_resolver: support verifying primary for check_txn_status ( #777 )
...
* support verifying primary for check_txn_status
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* add more failpoint usages
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update depencency and fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Do not skip for unistore; refine logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-05-08 16:18:26 +08:00
aee2f311a5
txn: fix goroutine leak ( #784 )
...
Signed-off-by: disksing <i@disksing.com>
2023-04-27 17:39:07 +08:00
35c1ee47c4
Set resource group penalty ( #772 )
...
* Set resource group penalty
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
2023-04-19 20:39:20 +08:00
a72e61dc64
log: print log if the assertion error is generated from client-go ( #771 )
...
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-04-19 17:42:17 +08:00
92db9f7b15
Support specifying checking forUpdateTS in prewrite ( #756 )
...
* Support specifying checking forUpdateTS in prewrite
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix go mod in tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add successful cases to the test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update dependencies
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Check LockedWithConflictTS greater than requested ForUpdateTS
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-04-17 14:53:28 +08:00
c853ddc68c
metrics: add txn ttl lifetime ( #766 )
...
Signed-off-by: Jack Yu <jackysp@gmail.com>
Co-authored-by: Shirly <AndreMouche@126.com>
2023-04-14 13:59:29 +08:00
67e56a956f
For KeyIsLocked error reported for timeout, if a lock is recently updated, don't try to resolve it. ( #758 )
...
* update client-go; format
Signed-off-by: ekexium <eke@fastmail.com>
* feat: do not resolve lock if duration_to_last_updated is short
Signed-off-by: ekexium <eke@fastmail.com>
* adjust the threshold to 1200ms to allow small deviation
Signed-off-by: ekexium <eke@fastmail.com>
* fix: don't treat it as WriteConflict, simply retry
Signed-off-by: ekexium <eke@fastmail.com>
* update kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* set the threshold to 300ms
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2023-04-13 16:14:28 +08:00
f3e87039d8
Finer `is_retry_request` settings ( #763 )
...
* set is_retry_request only for requests that is possible to have undetermined errors
Signed-off-by: ekexium <eke@fastmail.com>
* rename tryTimes to retryTimes
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2023-04-10 10:39:43 +08:00
ea13e97002
Add exitAggressiveLockingIfInapplicable function and call within mutex of lockKeys ( #749 )
...
* Add exitAggressiveLockingIfInapplicable function and call within mutex of lockKeys
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix fmt
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-03-28 16:41:04 +08:00
9d950905d7
Introduce the RURuntimeStats ( #732 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2023-03-16 10:19:36 +08:00
f555fdd2c9
txn: support force locking and lock only if exists be used together ( #727 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-03-09 18:08:32 +08:00
bdd41058aa
metrics: seperate metrics with source scope for txn command ( #723 )
...
* seperate metrics with source scope
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix async pessimistic rollback
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix missing definition
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
---------
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-03-06 22:12:45 +08:00
49323956ae
metric: seperate source for txn cmd ( #721 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-02-28 15:40:40 +08:00
596b79404d
set group name in twoPhaseCommitter ( #718 )
...
Signed-off-by: glorv <glorvs@163.com>
2023-02-27 16:39:23 +08:00
40a82457eb
tikv: configurable pool ( #714 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2023-02-27 11:23:58 +08:00
2051f572a1
fix: lock_if_exist + aggressive locking ( #705 )
...
* fix: locking commands skipped by lock_only_if_exists are mistakenly calculated as successfully locked for aggressive locking
Signed-off-by: ekexium <ekexium@gmail.com>
* refactor
Signed-off-by: ekexium <ekexium@gmail.com>
---------
Signed-off-by: ekexium <ekexium@gmail.com>
2023-02-20 09:50:30 +08:00
50e86f7d3c
Add metrics and statistics about aggressive locking ( #687 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-02-07 12:31:34 +08:00
2d2a3fa00e
*: remove the support of `amend transaction` ( #635 )
...
* done
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* done
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix test
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix lint
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix test
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
2023-01-09 16:30:27 +08:00
4e1d38d8f8
add resource group name in request context ( #650 )
...
Signed-off-by: glorv <glorvs@163.com>
2023-01-09 15:18:45 +08:00
5a4b9accad
fix data race in the LockKeys ( #655 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2023-01-06 13:31:13 +08:00
0f633e4163
Fix incorrect insertion behavior in aggressive locking mode and add tests ( #651 )
...
* Add more tests and fix the bug of insert in aggressive locking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix insert test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Avoid primary re-selecting in TestAggressiveLockingLoadValueOptionChanges
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-01-04 15:24:09 +08:00
018c59dbd8
bind the goroutine pool with store and close on Close ( #645 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
2022-12-26 16:01:48 +08:00
809eb2d10f
use goroutine pool to avoid the `runtime.newstack` call ( #637 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
2022-12-14 16:16:36 +08:00
5dc09b15e7
Add aggressive-locking mechanism and support locking with conflict ( #528 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2022-12-01 11:03:39 +08:00
f05c6886bb
fix data race in test ( #624 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2022-11-25 10:28:19 +08:00
92f0a82e1a
fix: avoid data race when setting memdb footprint hook ( #621 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2022-11-24 11:10:13 +08:00
e9db9e6a8a
make `txn_source` be u64 type ( #618 )
...
Signed-off-by: xiongjiwei <xiongjiwei1996@outlook.com>
2022-11-21 10:50:13 +08:00
e72337b647
txn: add `txn_source` field in txn ( #617 )
...
Signed-off-by: xiongjiwei <xiongjiwei1996@outlook.com>
2022-11-15 13:18:26 +08:00
6c9c7c7c58
Fix the false positive assertion error caused by loss of pessimistic locks ( #596 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2022-10-26 16:34:54 +08:00
6def8d7b90
Fix the missing unlock in extractKeyExistsErr ( #603 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-10-12 15:48:56 +08:00
8dfd76bf46
Acquire read lock in LockKeys to avoid data race ( #585 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-10-10 23:14:53 +08:00
ffaaf7131a
Don't let pessimistic txn prewrite in the optimistic way ( #584 )
...
* don't let pessimistic txn prewrite in the optimistic way even if it does not lock any key
Signed-off-by: ekexium <eke@fastmail.com>
* add a todo
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
2022-09-13 13:15:14 +08:00
426055b575
Distinguish different types of write conflicts ( #583 )
...
* distinguish different types of write conflicts
Signed-off-by: ekexium <eke@fastmail.com>
* update kvproto; add write conflict reason
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-09-08 19:49:57 +08:00
080fdfe665
extend the scope of using lock_only_if_exists ( #582 )
...
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
2022-09-07 20:09:47 +08:00
fed87c9939
PessimisticLock: Lock the specified key only if the key exists ( #561 )
...
* optimize for lock if exists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* fix bugs for lock if exists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* optimize lock info
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* fix bugs
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* remove the contrl flag for lock stats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* update kvproto
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* change to LockOnlyIfExists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* update kvproto
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* change to LockOnlyIfExists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* make test pass
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* Update txnkv/transaction/txn.go
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* *: add getPDClient to rawKV public api (#570 )
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* Update README.md (#571 )
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* update dependency of integration test (#572 )
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* Support postponed conflict check (#556 )
* replace kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* support NeedConflictCheck
Signed-off-by: ekexium <eke@fastmail.com>
* fix mutation encoding
Signed-off-by: ekexium <eke@fastmail.com>
* support temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* update kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* fix style
Signed-off-by: ekexium <eke@fastmail.com>
* add an option to enable the behavior
Signed-off-by: ekexium <eke@fastmail.com>
* replace AfterCheckPoint with existing canModity
Signed-off-by: ekexium <eke@fastmail.com>
* UpdateFlag do not unset temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* remove unused function
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* fix test
Signed-off-by: ekexium <eke@fastmail.com>
* do no unset flag on read
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* update comment
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
* add testcase
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add test case
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* unset pk if lockifexits failed
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* fix format
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* LockOnlyIfExists only when pk selected
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* remove test function to txn_probe
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add more info to error
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add more info to error
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add protection for input key
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: dongxu <i@huangdx.net>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
Co-authored-by: ekexium <eke@fastmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-09-06 09:50:26 +08:00
0130f76738
Support postponed conflict check ( #556 )
...
* replace kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* support NeedConflictCheck
Signed-off-by: ekexium <eke@fastmail.com>
* fix mutation encoding
Signed-off-by: ekexium <eke@fastmail.com>
* support temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* update kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* fix style
Signed-off-by: ekexium <eke@fastmail.com>
* add an option to enable the behavior
Signed-off-by: ekexium <eke@fastmail.com>
* replace AfterCheckPoint with existing canModity
Signed-off-by: ekexium <eke@fastmail.com>
* UpdateFlag do not unset temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* remove unused function
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* fix test
Signed-off-by: ekexium <eke@fastmail.com>
* do no unset flag on read
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* update comment
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
2022-08-30 15:38:39 +08:00
you06