f555fdd2c9
txn: support force locking and lock only if exists be used together ( #727 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-03-09 18:08:32 +08:00
bdd41058aa
metrics: seperate metrics with source scope for txn command ( #723 )
...
* seperate metrics with source scope
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix async pessimistic rollback
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix missing definition
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
---------
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-03-06 22:12:45 +08:00
49323956ae
metric: seperate source for txn cmd ( #721 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-02-28 15:40:40 +08:00
596b79404d
set group name in twoPhaseCommitter ( #718 )
...
Signed-off-by: glorv <glorvs@163.com>
2023-02-27 16:39:23 +08:00
40a82457eb
tikv: configurable pool ( #714 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2023-02-27 11:23:58 +08:00
a27994e5aa
Support load-based replica read ( #675 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2023-02-22 15:35:43 +08:00
2051f572a1
fix: lock_if_exist + aggressive locking ( #705 )
...
* fix: locking commands skipped by lock_only_if_exists are mistakenly calculated as successfully locked for aggressive locking
Signed-off-by: ekexium <ekexium@gmail.com>
* refactor
Signed-off-by: ekexium <ekexium@gmail.com>
---------
Signed-off-by: ekexium <ekexium@gmail.com>
2023-02-20 09:50:30 +08:00
50e86f7d3c
Add metrics and statistics about aggressive locking ( #687 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-02-07 12:31:34 +08:00
fe5b35c5f5
refactor: range task log with more accurate info ( #657 )
...
Signed-off-by: fengou1 <feng.ou@pingcap.com>
2023-01-12 14:20:23 +08:00
f313ddf58d
reformat codec and add keyspace support ( #649 )
...
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
2023-01-10 15:15:33 +08:00
2d2a3fa00e
*: remove the support of `amend transaction` ( #635 )
...
* done
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* done
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix test
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix lint
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix test
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
* fix
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
Signed-off-by: wjhuang2016 <huangwenjun1997@gmail.com>
2023-01-09 16:30:27 +08:00
4e1d38d8f8
add resource group name in request context ( #650 )
...
Signed-off-by: glorv <glorvs@163.com>
2023-01-09 15:18:45 +08:00
5a4b9accad
fix data race in the LockKeys ( #655 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2023-01-06 13:31:13 +08:00
0f633e4163
Fix incorrect insertion behavior in aggressive locking mode and add tests ( #651 )
...
* Add more tests and fix the bug of insert in aggressive locking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix insert test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Avoid primary re-selecting in TestAggressiveLockingLoadValueOptionChanges
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-01-04 15:24:09 +08:00
018c59dbd8
bind the goroutine pool with store and close on Close ( #645 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
2022-12-26 16:01:48 +08:00
c446b885bd
RuntimeStat: Clone `SnapshotRuntimeStats` Completely ( #641 )
...
* add more info for cloning SnapshotRuntimeStats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add more info for cloning SnapshotRuntimeStats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* shallow copy SnapshotRuntimeStats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-12-16 10:15:14 +08:00
809eb2d10f
use goroutine pool to avoid the `runtime.newstack` call ( #637 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
2022-12-14 16:16:36 +08:00
5dc09b15e7
Add aggressive-locking mechanism and support locking with conflict ( #528 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2022-12-01 11:03:39 +08:00
f05c6886bb
fix data race in test ( #624 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2022-11-25 10:28:19 +08:00
92f0a82e1a
fix: avoid data race when setting memdb footprint hook ( #621 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2022-11-24 11:10:13 +08:00
e9db9e6a8a
make `txn_source` be u64 type ( #618 )
...
Signed-off-by: xiongjiwei <xiongjiwei1996@outlook.com>
2022-11-21 10:50:13 +08:00
e72337b647
txn: add `txn_source` field in txn ( #617 )
...
Signed-off-by: xiongjiwei <xiongjiwei1996@outlook.com>
2022-11-15 13:18:26 +08:00
6c9c7c7c58
Fix the false positive assertion error caused by loss of pessimistic locks ( #596 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2022-10-26 16:34:54 +08:00
6def8d7b90
Fix the missing unlock in extractKeyExistsErr ( #603 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-10-12 15:48:56 +08:00
8dfd76bf46
Acquire read lock in LockKeys to avoid data race ( #585 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-10-10 23:14:53 +08:00
02e2fe1f2a
txnkv: export DefaultScanBatchSize variable ( #588 )
2022-09-21 13:09:30 +08:00
ffaaf7131a
Don't let pessimistic txn prewrite in the optimistic way ( #584 )
...
* don't let pessimistic txn prewrite in the optimistic way even if it does not lock any key
Signed-off-by: ekexium <eke@fastmail.com>
* add a todo
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
2022-09-13 13:15:14 +08:00
426055b575
Distinguish different types of write conflicts ( #583 )
...
* distinguish different types of write conflicts
Signed-off-by: ekexium <eke@fastmail.com>
* update kvproto; add write conflict reason
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-09-08 19:49:57 +08:00
080fdfe665
extend the scope of using lock_only_if_exists ( #582 )
...
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
2022-09-07 20:09:47 +08:00
fed87c9939
PessimisticLock: Lock the specified key only if the key exists ( #561 )
...
* optimize for lock if exists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* fix bugs for lock if exists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* optimize lock info
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* fix bugs
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* remove the contrl flag for lock stats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* update kvproto
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* change to LockOnlyIfExists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* update kvproto
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* change to LockOnlyIfExists
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* make test pass
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* Update txnkv/transaction/txn.go
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* *: add getPDClient to rawKV public api (#570 )
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* Update README.md (#571 )
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* update dependency of integration test (#572 )
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* Support postponed conflict check (#556 )
* replace kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* support NeedConflictCheck
Signed-off-by: ekexium <eke@fastmail.com>
* fix mutation encoding
Signed-off-by: ekexium <eke@fastmail.com>
* support temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* update kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* fix style
Signed-off-by: ekexium <eke@fastmail.com>
* add an option to enable the behavior
Signed-off-by: ekexium <eke@fastmail.com>
* replace AfterCheckPoint with existing canModity
Signed-off-by: ekexium <eke@fastmail.com>
* UpdateFlag do not unset temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* remove unused function
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* fix test
Signed-off-by: ekexium <eke@fastmail.com>
* do no unset flag on read
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* update comment
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
* add testcase
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add test case
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* unset pk if lockifexits failed
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* fix format
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* LockOnlyIfExists only when pk selected
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* remove test function to txn_probe
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add more info to error
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add more info to error
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add protection for input key
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: dongxu <i@huangdx.net>
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: dongxu <i@huangdx.net>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
Co-authored-by: ekexium <eke@fastmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-09-06 09:50:26 +08:00
0130f76738
Support postponed conflict check ( #556 )
...
* replace kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* support NeedConflictCheck
Signed-off-by: ekexium <eke@fastmail.com>
* fix mutation encoding
Signed-off-by: ekexium <eke@fastmail.com>
* support temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* update kvproto
Signed-off-by: ekexium <eke@fastmail.com>
* fix style
Signed-off-by: ekexium <eke@fastmail.com>
* add an option to enable the behavior
Signed-off-by: ekexium <eke@fastmail.com>
* replace AfterCheckPoint with existing canModity
Signed-off-by: ekexium <eke@fastmail.com>
* UpdateFlag do not unset temporary flag
Signed-off-by: ekexium <eke@fastmail.com>
* remove unused function
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* fix test
Signed-off-by: ekexium <eke@fastmail.com>
* do no unset flag on read
Signed-off-by: ekexium <eke@fastmail.com>
* update tidb dependency
Signed-off-by: ekexium <eke@fastmail.com>
* update comment
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: ekexium <eke@fastmail.com>
2022-08-30 15:38:39 +08:00
0d0ae0dcfb
txn: seperate prewrite and commit in detail information and add clone methods ( #567 )
...
* seperate the prewrite and commit primary in runtime info
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix the integration test
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2022-08-18 16:48:34 +08:00
516cfcdecc
txnkv: fix a bug that commitTSUpperBoundCheck err not handled ( #564 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-08-10 16:41:20 +08:00
57c12f7c64
Handle more detailed statistics from TiKV ( #536 )
...
* support more detailed statistics returned from TiKV
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* fix log formatting
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* add to test
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-07-13 16:56:47 +08:00
114ba4082e
add replica read adjuster function in KVSnapshot ( #540 )
...
Signed-off-by: glorv <glorvs@163.com>
2022-07-07 17:12:24 +08:00
97c41742ea
txn: also record get-latest-ts time in commit details ( #538 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2022-07-07 11:11:16 +08:00
79a0fc59e4
Add a callback when MemDB grows ( #520 )
...
* implement the footprint chagne hook
Signed-off-by: ekexium <ekexium@fastmail.com>
* simplify the signature
Signed-off-by: ekexium <ekexium@fastmail.com>
* also call the callback when arena shrinks
Signed-off-by: ekexium <ekexium@fastmail.com>
* also callback on reset
Signed-off-by: ekexium <ekexium@fastmail.com>
Co-authored-by: disksing <i@disksing.com>
2022-06-30 15:19:23 +08:00
bb026bdad8
client: support requests with source label ( #506 )
...
* support request scope and source.
Signed-off-by: you06 <you1474600@gmail.com>
* update
Signed-off-by: you06 <you1474600@gmail.com>
* use counter for requests with too-many source
Signed-off-by: you06 <you1474600@gmail.com>
* refine resolve lock
Signed-off-by: you06 <you1474600@gmail.com>
* remove request source from resolve details
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
2022-06-24 18:17:09 +08:00
98a4e2776e
transaction: update undetermined error log ( #531 )
...
* update undetermined error log
Signed-off-by: Jack Yu <jackysp@gmail.com>
* tiny change
Signed-off-by: Jack Yu <jackysp@gmail.com>
2022-06-20 10:56:30 +08:00
e69cd80e58
Preserve is_retry_request flag for prewrite when retrying with region errors ( #513 )
...
* add a test to verify the incorrect setting of is_retry_request
Signed-off-by: ekexium <ekexium@gmail.com>
* preserve the retry flag for prewrite
Signed-off-by: ekexium <ekexium@gmail.com>
* fix test
Signed-off-by: ekexium <ekexium@gmail.com>
* add more explanations
Signed-off-by: ekexium <ekexium@gmail.com>
2022-05-31 16:04:28 +08:00
de7ca289ac
Fix incorrect clear of resolving array ( #507 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-05-19 00:25:27 +08:00
7ba562bc68
txn: record resolving locks ( #473 )
...
* record resolving locks
Signed-off-by: longfangsong <longfangsong@icloud.com>
* rename to ResolvingLock
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Add test probe for resolving
Signed-off-by: longfangsong <longfangsong@icloud.com>
* refactor according to comments
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Add ResolveLocksDone for other call of ResolveLocks
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Update txnkv/txnlock/lock_resolver.go
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Update to new design
Signed-off-by: longfangsong <longfangsong@icloud.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-17 11:13:16 +08:00
190a4d190c
fix the vulnerability of atomicity in async commit ( #492 )
...
* fix the vulnerability of atomicity in async commit
Signed-off-by: ekexium <ekexium@gmail.com>
Revert "fix the vulnerability of atomicity in async commit"
This reverts commit b0b34c56168ba288e1619e66e626321365cb0921.
tidy up
Signed-off-by: ekexium <ekexium@gmail.com>
* add test
Signed-off-by: ekexium <ekexium@gmail.com>
* shorten the time of the test
Signed-off-by: ekexium <ekexium@gmail.com>
* fix test
Signed-off-by: ekexium <ekexium@gmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-13 17:09:52 +08:00
ff5e35ac28
*: fix data race on the SetResourceGroupTagger ( #491 )
...
* *: fix data race on the SetResourceGroupTagger
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* *: fix data race on the SetResourceGroupTagger
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* *: fix data race on the SetResourceGroupTagger
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2022-05-10 11:22:38 +08:00
038b03552a
atomic PrewriteMaxBackoff ( #488 )
...
* atomic PrewriteMaxBackoff
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* PrewriteMaxBackoff use uint64
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-10 10:24:53 +08:00
cfe691debb
fix data race on the KVSnapshot.interceptor ( #489 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-09 17:23:19 +08:00
e31ba21ae0
add a failpoint for sleep ( #460 )
...
Signed-off-by: ekexium <ekexium@gmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2022-04-11 21:34:25 +08:00
6bf6951325
use global variable instead of config ( #458 )
...
Signed-off-by: Jack Yu <jackysp@gmail.com>
2022-03-29 17:20:50 +08:00
8489c3e8c3
config,txnkv: make txnCommitBatchSize adjustable by config ( #457 )
...
Signed-off-by: Jack Yu <jackysp@gmail.com>
2022-03-28 16:37:38 +08:00
5042c6f2aa
Return assertion failed error less prior to other kinds of errors ( #448 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2022-03-11 12:36:19 +08:00
MyonKeminta