c3e10ae7c8
membuffer: implement cache for ART ( #1470 )
...
ref pingcap/tidb#55287
Signed-off-by: you06 <you1474600@gmail.com>
2024-10-08 08:58:09 +00:00
58f3322fc3
membuffer: implement snapshot get and iterator for ART ( #1467 )
...
ref pingcap/tidb#55287
Signed-off-by: you06 <you1474600@gmail.com>
2024-09-25 07:03:02 +00:00
527f80a186
feat: Broadcast min_commit_ts for pipelined transactions ( #1458 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-09-25 01:34:14 +00:00
7d0f0bc93b
membuffer: support iterator for ART ( #1466 )
...
ref pingcap/tidb#55287
Signed-off-by: you06 <you1474600@gmail.com>
2024-09-24 11:38:48 +00:00
271945f345
membuffer: support staging & checkpoint for ART ( #1465 )
...
ref pingcap/tidb#55287
Signed-off-by: you06 <you1474600@gmail.com>
2024-09-23 03:55:13 +00:00
6beede6998
membuffer: implement ART with basic get/set ( #1451 )
...
ref pingcap/tidb#55287
Signed-off-by: you06 <you1474600@gmail.com>
2024-09-18 09:25:22 +00:00
e7894a7b27
config: change grpc-keepalive-timeout from uint to float64, for support millisecond timeout ( #1461 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
2024-09-11 04:15:06 +00:00
28135fd9d4
OWNERS: Auto Sync OWNERS files from community membership ( #1460 )
...
Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2024-09-09 12:12:20 +00:00
287b364bd6
OWNERS: Auto Sync OWNERS files from community membership ( #1459 )
...
Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2024-09-09 03:18:51 +00:00
f0ea917499
transaction: add hook for async commit to track the life cycle of the async-commit goroutine and secondary lock cleanup goroutine ( #1432 )
...
Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>
2024-09-04 09:21:32 +00:00
6b1453c358
memdb: use atomic u64 addr to reduce allocation ( #1453 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2024-08-28 04:00:17 +00:00
41d133b6b6
membuffer: refactor the memdb to support multi implementations ( #1426 )
...
ref pingcap/tidb#55287
Signed-off-by: you06 <you1474600@gmail.com>
2024-08-23 03:46:43 +00:00
75e3705e58
memdb: fix memdb snapshot get/iter is not actually snapshot ( #1393 )
...
* fix memdb snapshot get/iter is not actually snapshot
Signed-off-by: you06 <you1474600@gmail.com>
* rename test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-08-21 15:35:30 +08:00
77526daca4
OWNERS: Auto Sync OWNERS files from community membership ( #1425 )
...
Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2024-08-19 13:01:42 +00:00
36076b3109
Merge pull request #1421 from tikv/bot/update-owners-1723456948376
...
OWNERS: Auto Sync OWNERS files from community membership
2024-08-19 11:19:09 +08:00
eb941f71c3
Merge branch 'master' into bot/update-owners-1723456948376
2024-08-16 11:38:01 +08:00
c810ed88fb
region_request: remove backoff for stale read ( #1423 )
...
* remove stale read backoff
Signed-off-by: you06 <you1474600@gmail.com>
* sort imports
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-08-15 10:09:19 +08:00
0820519da7
Add GetTimeDetail for SnapshotRuntimeStats ( #1422 )
...
Signed-off-by: yibin <huyibin@pingcap.cn>
2024-08-14 14:30:31 +08:00
4c6b2171b2
improve batch efficiency for high throughput workloads ( #1411 )
...
* improve batch efficiency by multiple attempts
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add support for aggressive batching
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some metrics for batch client
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fetch more requests according to recent wait head durs
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add experimental batch options
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* optimize and refactor
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* some minor updates
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix the metric of head arrival interval
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* a minor update
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* some minor fixes
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* update according to the spec
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix the ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix batch condition
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some doc comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address https://github.com/tikv/client-go/pull/1411#discussion_r1714086744
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* rename some vars according to the comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add more comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-08-13 12:55:44 +08:00
0b3507ccd9
[skip ci] Update OWNERS file
...
skip-checks: true
Signed-off-by: Ti Chi Robot <ti-community-prow-bot@tidb.io>
2024-08-12 18:02:31 +08:00
f0f57f28c4
region cache: batch find regions by key ranges from cache ( #1410 )
...
* try
add benchmark
Signed-off-by: wshwsh12 <793703860@qq.com>
* impl
Signed-off-by: wshwsh12 <793703860@qq.com>
* time
Signed-off-by: wshwsh12 <793703860@qq.com>
* fix ut
Signed-off-by: wshwsh12 <793703860@qq.com>
* address comments
Signed-off-by: wshwsh12 <793703860@qq.com>
* startkey
Signed-off-by: wshwsh12 <793703860@qq.com>
* add comments
Signed-off-by: wshwsh12 <793703860@qq.com>
---------
Signed-off-by: wshwsh12 <793703860@qq.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-08-09 15:36:57 +09:00
aa8b33873f
Let memdb cache last traversed node ( #1389 )
...
* opt: let memdb cache last traversed node
Signed-off-by: ekexium <eke@fastmail.com>
* add a switch
Signed-off-by: ekexium <eke@fastmail.com>
* use atomic pointer
Signed-off-by: ekexium <eke@fastmail.com>
* fix: invalidate cache if the node is deleted
Signed-off-by: ekexium <eke@fastmail.com>
* feat: log flush_wait_duration (#1391 )
Signed-off-by: ekexium <eke@fastmail.com>
* feat: add cache metrics
Signed-off-by: ekexium <eke@fastmail.com>
* opt: parent node tracking in traverse
Signed-off-by: ekexium <eke@fastmail.com>
* opt: isNull
Signed-off-by: ekexium <eke@fastmail.com>
* log: silence verbose failpoint log
Signed-off-by: ekexium <eke@fastmail.com>
* metrics: track traverse duration
Signed-off-by: ekexium <eke@fastmail.com>
* Revert "metrics: track traverse duration"
This reverts commit 05fc82bad51711c5f0fadf7936e118ed2c4990ab.
Signed-off-by: ekexium <eke@fastmail.com>
* fix: flush wait
Signed-off-by: ekexium <eke@fastmail.com>
* fix: duration logs
Signed-off-by: ekexium <eke@fastmail.com>
* refine log message
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-08-08 17:20:52 +08:00
cd64e24de8
fix issue that store's liveness may incorrectly marked as unreachable when the store restarts with label changed ( #1407 )
...
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix ci
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-08-01 19:02:26 +08:00
eec8198343
refine region request debug log ( #1408 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-07-31 17:42:29 +08:00
3ac46e8171
fix: duration logs ( #1405 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-07-31 13:12:27 +08:00
fce0abfd82
Let pipelined memdb ignore bufferLimit ( #1392 )
...
* fix: pipelined memdb ignores bufferLimit
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: introduce unlimitedSize
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: unlimitedSize
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-07-29 12:13:17 +08:00
88ce38492f
feat: log flush_wait_duration ( #1391 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-07-23 09:16:43 +08:00
d73cc1ed65
region cache: check if the pd returned regions covers the ranges ( #1377 )
...
* add reproduce test
Signed-off-by: you06 <you1474600@gmail.com>
* check if the pd returned regions covers the ranges
Signed-off-by: you06 <you1474600@gmail.com>
* handle limit
Signed-off-by: you06 <you1474600@gmail.com>
* fix lint
Signed-off-by: you06 <you1474600@gmail.com>
* add TODO for func
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb for integration test
Signed-off-by: you06 <you1474600@gmail.com>
* rename gap detection function
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* add half bounded cases
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-07-03 17:58:01 +08:00
0206a3c142
region cache: fallback to ScanRegions when BatchScanRegions receive unimplement error ( #1378 )
...
* add fallback
Signed-off-by: you06 <you1474600@gmail.com>
* fallback to ScanRegions when BatchScanRegions receive unimplement error
Signed-off-by: you06 <you1474600@gmail.com>
* add comment & test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-07-03 15:38:53 +08:00
4a72526f6c
add enable-replica-selector-v2 config back but mark it was deprecated ( #1374 )
...
* add enable-replica-selector-v2 config back but mark it was deprecated
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* update comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-06-26 14:42:48 +08:00
14f3d2b5b7
region cache: add `BatchLocateKeyRanges` interface ( #1367 )
...
* add batch scan interface
Signed-off-by: you06 <you1474600@gmail.com>
fix bug
Signed-off-by: you06 <you1474600@gmail.com>
copy encoded ranges
Signed-off-by: you06 <you1474600@gmail.com>
fix panic
Signed-off-by: you06 <you1474600@gmail.com>
add batch scan region test
Signed-off-by: you06 <you1474600@gmail.com>
fix log
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader option
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader optino
Signed-off-by: you06 <you1474600@gmail.com>
add comment
Signed-off-by: you06 <you1474600@gmail.com>
deprecate ScanRegions
Signed-off-by: you06 <you1474600@gmail.com>
update kvproto & pd client
Signed-off-by: you06 <you1474600@gmail.com>
update go.mod
Signed-off-by: you06 <you1474600@gmail.com>
fix need leader
Signed-off-by: you06 <you1474600@gmail.com>
fix integration test
Signed-off-by: you06 <you1474600@gmail.com>
fix lint
Signed-off-by: you06 <you1474600@gmail.com>
export BatchLocateKeyRangesOpt
Signed-off-by: you06 <you1474600@gmail.com>
update integration test's tidb
Signed-off-by: you06 <you1474600@gmail.com>
refactor BatchLocateKeyRanges and add comments
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb of integration test
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-06-24 16:06:06 +08:00
d4f0f4cf12
Revert "upgrade grpc 1.64 ( #1346 )" ( #1369 )
...
This reverts commit 1c06c5cbf5
2024-06-21 17:03:19 +08:00
4f2562f987
region_request: ignore resource group errors that not relative storage layer ( #1354 )
...
Signed-off-by: nolouch <nolouch@gmail.com>
2024-06-20 13:57:02 +08:00
ac8fa1d73a
feat: introduce a new method MemHookSet ( #1362 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-06-14 14:44:55 +08:00
156cebc2de
Update the PD client to adopt the latest fix/optimization ( #1360 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-06-04 12:57:05 +08:00
cb580bc4ea
fix rpc client panic cause by concurrent close ( #1359 )
...
* fix rpc client panic cause by concurrent close
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-31 18:21:21 +08:00
1f814b7a45
add func for new test keyspace tikv store ( #1356 )
2024-05-30 17:52:57 +08:00
a6335a5aaf
Fix the problem that statement being oom-killed within DoneAggressiveLocking causing the transaction still in aggressive locking state ( #1355 )
...
* Quick fix the problem that statement being oom-killed within DoneAggressiveLocking causing the transaction still in aggressive locking state
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments to explain the change
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-30 16:20:09 +08:00
0cc1c5239d
Remove old replica selector ( #1265 )
...
* remove EnableReplicaSelectorV2 config
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* remove replicaSelectorV1
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add replica flag to reduce struct size
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* remove contextPatcher
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* use option.preferLeader
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-27 17:11:02 +08:00
31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV ( #1287 )
...
* Support actively requesting update health feedback information by calling RPC to TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto; follow the protocol change; and add more comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add integration tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* tune the frequency of active updating
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Revert changes to examples
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary empty line
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
1c06c5cbf5
upgrade grpc 1.64 ( #1346 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-05-17 12:20:15 +08:00
38e0dca30c
*: fix panic in get cause error ( #1344 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-16 11:19:22 +08:00
c40432e3ab
feat: record flush_wait_ms in TxnInfo ( #1342 )
...
* feat: record flush_wait_ms in TxnInfo
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: change FlushWaitDuration() to GetFlushMetrics()
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-05-15 11:13:15 +08:00
e1ca512cca
refine error msg ( #1338 )
...
* refine error msg
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-13 21:49:13 +08:00
c82e921992
fix issue of doesn't fast fail request when no available connections ( #1339 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-13 19:07:04 +08:00
6cb0704fce
fail pipelined dml when max ttl exceeded ( #1329 )
...
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-04-30 22:52:41 +08:00
52c232be3d
Update the PD client to adopt the latest optimization ( #1326 )
2024-04-30 16:32:54 +08:00
31a502b9ba
improve region request log for diagnose ( #1300 )
...
* improve region request log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* rename struct
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine region error metric with store id label and add rpc error metric
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* restrict log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-29 15:56:32 +08:00
9c05078660
Add keyspace meta in codec ( #1323 )
...
Signed-off-by: y_static_y@sina.com <y_static_y@sina.com>
2024-04-28 12:20:55 +08:00
3ed6550f3e
build(deps): bump golang.org/x/net in /examples/txnkv/pessimistic_txn ( #1313 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 12:59:22 +08:00
you06