82ff387182
txnkv: fix the issue that batch-get error might be ignored ( #1733 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-08-25 10:24:00 +00:00
9b1e2a6652
snapshot: set `ReplicaRead` to false when `ReplicaReadType` fallbacks to `ReplicaReadLeader` ( #1663 )
...
ref pingcap/tidb#61745
Signed-off-by: you06 <you1474600@gmail.com>
2025-06-18 01:59:54 +00:00
96b6487fac
*: support async batch get ( #1646 )
...
ref tikv/client-go#1586
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-06-16 02:21:36 +00:00
d1ec4b4f67
txn: handle undetermined error in client go ( #1642 )
...
close tikv/client-go#1641
Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2025-05-20 07:00:04 +00:00
157f083989
Fix backoff lose info when forked ( #1627 )
...
ref pingcap/tidb#60271
Signed-off-by: yibin87 <huyibin@pingcap.com>
2025-04-21 02:21:14 +00:00
e53096d518
support redact key in logs ( #1612 )
...
ref pingcap/tidb#59279
Signed-off-by: tangenta <tangenta@126.com>
Co-authored-by: you06 <you1474600@gmail.com>
2025-04-01 07:17:12 +00:00
d0d0c9712f
avoid copy stack in batch-get ( #1562 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2025-01-24 09:10:23 +00:00
57aa917c59
txnkv: optimize batch-get by reducing overhead of backoffer ( #1559 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-01-16 07:07:28 +00:00
a348c17c6b
*: fix two minor bugs ( #1553 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-01-13 03:38:31 +00:00
e0415dcb20
txnkv: introduce ballast object to batch-get worker ( #1535 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-12-25 10:08:54 +00:00
f2266d6bf2
*: Optimize executor runtime stats performance ( #1532 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-12-25 04:06:45 +00:00
06d7f4b923
Try to validate read ts for all RPC requests ( #1513 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-12-09 09:49:30 +00:00
0820519da7
Add GetTimeDetail for SnapshotRuntimeStats ( #1422 )
...
Signed-off-by: yibin <huyibin@pingcap.cn>
2024-08-14 14:30:31 +08:00
31a502b9ba
improve region request log for diagnose ( #1300 )
...
* improve region request log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* rename struct
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine region error metric with store id label and add rpc error metric
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* restrict log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-29 15:56:32 +08:00
0229f4077f
do not cache snapshot with version maxuint64 ( #1320 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-24 13:23:42 +08:00
dcb9b95307
txn: cache result of snapshot.Get ( #1278 )
...
* also cache result of snapshot.Get
Signed-off-by: you06 <you1474600@gmail.com>
* fix test
Signed-off-by: you06 <you1474600@gmail.com>
fix test
Signed-off-by: you06 <you1474600@gmail.com>
* add Get cache test
Signed-off-by: you06 <you1474600@gmail.com>
* move RLock position
Signed-off-by: you06 <you1474600@gmail.com>
* remove comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-13 16:45:31 +08:00
87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB ( #1212 )
...
* feat: introduce GetLocal() for MemBuffer
Signed-off-by: ekexium <eke@fastmail.com>
* downgrade tools to 0.18.0
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set flushingMemDB = nil when an error is returned from a flush
Signed-off-by: ekexium <eke@fastmail.com>
* impl BatchGet for MemBuffer
Signed-off-by: you06 <you1474600@gmail.com>
test membuffer batch get
Signed-off-by: you06 <you1474600@gmail.com>
* add Prefetch & GetPrefetchCache
Signed-off-by: you06 <you1474600@gmail.com>
* cache multi Prefetch call
Signed-off-by: you06 <you1474600@gmail.com>
* add tests
Signed-off-by: you06 <you1474600@gmail.com>
* workaround for golang ci lint failure
Signed-off-by: you06 <you1474600@gmail.com>
* replace assert with require
Signed-off-by: you06 <you1474600@gmail.com>
* workaround golangci lint
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
* remove prefetch interface, pipelined memdb will cache batch get result
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb
Signed-off-by: you06 <you1474600@gmail.com>
* fix batch get cache when membuffer is empty
Signed-off-by: you06 <you1474600@gmail.com>
* fix returned delete value
Signed-off-by: you06 <you1474600@gmail.com>
* fix: handle resourceGroupTag of Flush
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set resource group tag for committer if it's pipelined
Signed-off-by: ekexium <eke@fastmail.com>
* Update internal/unionstore/pipelined_memdb.go
Co-authored-by: ekexium <eke@fastmail.com>
* remove prefetch interface
Signed-off-by: you06 <you1474600@gmail.com>
* release mutex
Signed-off-by: you06 <you1474600@gmail.com>
* flush wait to avoid race
Signed-off-by: you06 <you1474600@gmail.com>
* fix unstopped test
Signed-off-by: you06 <you1474600@gmail.com>
* set resource group tags for committer
Signed-off-by: ekexium <eke@fastmail.com>
* skip test due to tikv image not updated yet
Signed-off-by: you06 <you1474600@gmail.com>
* skip more test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 11:36:05 +08:00
c72236a601
adjust parameter for pipelined memdb by failpoint ( #1205 )
...
* adjust parameter for pipelined memdb by failpoint
Signed-off-by: you06 <you1474600@gmail.com>
* update comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-06 16:44:48 +08:00
8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML ( #1114 )
...
* Support pipelined memdb.
Signed-off-by: you06 <you1474600@gmail.com>
* fix mutex for pipelined memdb
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
abda04c22a
export stats information about RPC count ( #1110 )
...
Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>
2024-01-12 15:16:19 +08:00
6ba909c4ad
config: enable user configuration for retry strategy. ( #1055 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-11-14 14:54:10 +08:00
07432ef6c0
refactor the func name to make it clear ( #1000 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-10-10 14:18:02 +08:00
295094e5b5
add retry info to request source ( #953 )
...
* add retry info to request source
Signed-off-by: you06 <you1474600@gmail.com>
* handle upper layer retry
Signed-off-by: you06 <you1474600@gmail.com>
* stabilize test
Signed-off-by: you06 <you1474600@gmail.com>
* retry in 3 dimension
Signed-off-by: you06 <you1474600@gmail.com>
* record and restore req.ReadType
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2023-08-29 08:28:46 +08:00
8a214402da
use tidb_kv_read_timeout as first kv request timeout ( #919 )
...
* support tidb_kv_read_timeout as first round kv request timeout
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix ci
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix ci
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix ci
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix ci
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix ci
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* update comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2023-08-11 11:37:10 +08:00
2f119351bd
*: add `SnapshotIterReverse` and make `iterReverse` supports `lowerBound` ( #883 )
...
Signed-off-by: Jason Mo <mohangjie1995@gmail.com>
2023-07-14 10:36:07 +08:00
0b4b0ca00e
kvrpc: deduplicate chained intercept by name ( #832 )
...
* dedup chained interceptor
Signed-off-by: glorv <glorvs@163.com>
2023-06-13 20:21:06 -07:00
11121c6913
Remove stale-read flag after resolving lock ( #792 )
...
* remove stale-read when key-is-locked
Signed-off-by: you06 <you1474600@gmail.com>
* disable follower read for batch get
Signed-off-by: you06 <you1474600@gmail.com>
* the stale-read flag may already set to false
Signed-off-by: you06 <you1474600@gmail.com>
* fix batchget
Signed-off-by: you06 <you1474600@gmail.com>
* Reset busy-threshold when stale read fallback to leader read
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* trigger CI
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2023-05-26 13:55:53 +08:00
62cc95123c
Fix wrong resource group name for some requests ( #788 )
...
* fix wrong resource group name
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
2023-04-28 19:03:56 +08:00
35c1ee47c4
Set resource group penalty ( #772 )
...
* Set resource group penalty
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
2023-04-19 20:39:20 +08:00
f3e87039d8
Finer `is_retry_request` settings ( #763 )
...
* set is_retry_request only for requests that is possible to have undetermined errors
Signed-off-by: ekexium <eke@fastmail.com>
* rename tryTimes to retryTimes
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2023-04-10 10:39:43 +08:00
c9119d02ce
support time detail v2 ( #729 )
...
* support time detail v2
Signed-off-by: glorv <glorvs@163.com>
---------
Signed-off-by: glorv <glorvs@163.com>
Co-authored-by: Connor <zbk602423539@gmail.com>
2023-03-13 21:32:19 +08:00
bdd41058aa
metrics: seperate metrics with source scope for txn command ( #723 )
...
* seperate metrics with source scope
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix async pessimistic rollback
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix missing definition
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
---------
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-03-06 22:12:45 +08:00
49323956ae
metric: seperate source for txn cmd ( #721 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-02-28 15:40:40 +08:00
a27994e5aa
Support load-based replica read ( #675 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2023-02-22 15:35:43 +08:00
f313ddf58d
reformat codec and add keyspace support ( #649 )
...
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
2023-01-10 15:15:33 +08:00
4e1d38d8f8
add resource group name in request context ( #650 )
...
Signed-off-by: glorv <glorvs@163.com>
2023-01-09 15:18:45 +08:00
c446b885bd
RuntimeStat: Clone `SnapshotRuntimeStats` Completely ( #641 )
...
* add more info for cloning SnapshotRuntimeStats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* add more info for cloning SnapshotRuntimeStats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
* shallow copy SnapshotRuntimeStats
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Signed-off-by: TonsnakeLin <lpbgytong@163.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-12-16 10:15:14 +08:00
02e2fe1f2a
txnkv: export DefaultScanBatchSize variable ( #588 )
2022-09-21 13:09:30 +08:00
114ba4082e
add replica read adjuster function in KVSnapshot ( #540 )
...
Signed-off-by: glorv <glorvs@163.com>
2022-07-07 17:12:24 +08:00
bb026bdad8
client: support requests with source label ( #506 )
...
* support request scope and source.
Signed-off-by: you06 <you1474600@gmail.com>
* update
Signed-off-by: you06 <you1474600@gmail.com>
* use counter for requests with too-many source
Signed-off-by: you06 <you1474600@gmail.com>
* refine resolve lock
Signed-off-by: you06 <you1474600@gmail.com>
* remove request source from resolve details
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
2022-06-24 18:17:09 +08:00
7ba562bc68
txn: record resolving locks ( #473 )
...
* record resolving locks
Signed-off-by: longfangsong <longfangsong@icloud.com>
* rename to ResolvingLock
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Add test probe for resolving
Signed-off-by: longfangsong <longfangsong@icloud.com>
* refactor according to comments
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Add ResolveLocksDone for other call of ResolveLocks
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Update txnkv/txnlock/lock_resolver.go
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Signed-off-by: longfangsong <longfangsong@icloud.com>
* Update to new design
Signed-off-by: longfangsong <longfangsong@icloud.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-17 11:13:16 +08:00
ff5e35ac28
*: fix data race on the SetResourceGroupTagger ( #491 )
...
* *: fix data race on the SetResourceGroupTagger
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* *: fix data race on the SetResourceGroupTagger
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* *: fix data race on the SetResourceGroupTagger
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2022-05-10 11:22:38 +08:00
cfe691debb
fix data race on the KVSnapshot.interceptor ( #489 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-09 17:23:19 +08:00
079e2dce4e
txn: support read-consistency read with tso checking ( #447 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2022-03-08 10:31:03 +08:00
62d6b4a2e8
txnkv: set TiKV RC for weak-consistency transactions ( #407 )
...
* add tikv-rc support
Signed-off-by: you06 <you1474600@gmail.com>
* add test
Signed-off-by: you06 <you1474600@gmail.com>
* format code
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb && address comment
Signed-off-by: you06 <you1474600@gmail.com>
* update test log
Signed-off-by: you06 <you1474600@gmail.com>
2021-12-29 13:16:14 +08:00
6165dbaa95
introduce RPC interceptor mechanism ( #389 )
...
* Add interceptor mechanism for tikv RPC
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Rename comment
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Modify comments
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Fix ineffectual assignment
Signed-off-by: mornyx <mornyx.z@gmail.com>
* add AddInterceptor for KVTxn and KVSnapshot
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Separate intercepor package
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Add comments
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Add order test
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Fix integration tests
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Fix integration tests
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Move interceptor call from client to client_collapse
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Add comments
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Fix comment
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Add client_interceptor
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Modify comment
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Add ut
Signed-off-by: mornyx <mornyx.z@gmail.com>
2021-12-18 13:03:06 +08:00
c0e8766154
txnkv: decrease some max and base backoff time ( #388 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-12-06 15:29:23 +08:00
3a7675742e
txnkv: read through locks ( #380 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2021-12-01 16:35:10 +08:00
3b9f591c3f
txnkv: add callback for setting ResourceGroupTag ( #368 )
...
* Replace resourceGroupTag with resourceGroupTagFactory(firstKey)
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Fix npe
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Abstract ResourceGroupFactory
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Optimized as ResourceGroupTagger
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Add test for resource group tagger
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Fix unexpected request failed in mock store
Signed-off-by: mornyx <mornyx.z@gmail.com>
* Fix integration tests
Signed-off-by: mornyx <mornyx.z@gmail.com>
Co-authored-by: Wenxuan <breezewish@pingcap.com>
2021-11-14 19:11:14 -08:00
9ec50224be
*: replace pingcap/errors with pkg/errors ( #357 )
...
Signed-off-by: disksing <i@disksing.com>
2021-11-01 11:59:52 +08:00
zyguan