0fdd25559b
metrics: clean up metrics when a store is removed from the cluster ( #1751 )
...
fix pingcap/tidb#63245
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-09-24 03:36:49 +00:00
97cad411eb
metrcis: add replica read traffic metrics ( #1717 )
...
ref pingcap/tidb#62740
Signed-off-by: you06 <you1474600@gmail.com>
2025-08-25 03:37:58 +00:00
58e9bb4b6d
Make the txn safe point polling support falling back to old version of PD ( #1727 )
...
closes tikv/client-go#1728
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2025-07-30 08:44:49 +00:00
fdedc9c488
metrics: observe tail latencies of batch send/recv ( #1651 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-07-07 09:16:21 +00:00
96b6487fac
*: support async batch get ( #1646 )
...
ref tikv/client-go#1586
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-06-16 02:21:36 +00:00
37a2714053
Fix some metrics that miss const labels ( #1652 )
...
Signed-off-by: yibin87 <huyibin@pingcap.com>
2025-06-06 04:03:08 +00:00
a65bf61bbe
Add txn write conflict metrics ( #1551 )
...
close tikv/client-go#1550
Signed-off-by: sujuntao <juntao.su@foxmail.com>
Co-authored-by: sujuntao <juntao.su@foxmail.com>
2025-04-29 09:08:46 +00:00
02eee2e236
metrics: adjust bucket count to reduce metrics data ( #1609 )
...
Signed-off-by: Lynn <zimu_xia@126.com>
2025-03-28 05:42:22 +00:00
1906b778d7
Refine pessimistic lock related metrics and stats ( #1620 )
...
Signed-off-by: yibin87 <huyibin@pingcap.com>
2025-03-27 03:05:28 +00:00
bdaed3eba7
remove useless metric tidb_tikvclient_cop_duration_seconds_bucket ( #1602 )
...
Signed-off-by: XuHuaiyu <391585975@qq.com>
2025-03-20 08:04:15 +00:00
61e09c6539
metrics: optimize rpc latency related metrics ( #1556 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-01-17 03:49:19 +00:00
405d444558
Allow setting concurrency for pipelined flush and resolveLocks ( #1494 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2025-01-16 07:40:52 +00:00
2572ee4a7b
metrics: add StaleRegionFromPDCounter ( #1530 )
...
ref tikv/pd#7431
Signed-off-by: okJiang <819421878@qq.com>
2024-12-19 07:55:51 +00:00
23531ad618
Support adaptive update interval for low resolution ts ( #1484 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-11-11 07:51:03 +00:00
4c6b2171b2
improve batch efficiency for high throughput workloads ( #1411 )
...
* improve batch efficiency by multiple attempts
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add support for aggressive batching
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some metrics for batch client
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fetch more requests according to recent wait head durs
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add experimental batch options
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* optimize and refactor
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* some minor updates
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix the metric of head arrival interval
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* a minor update
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* some minor fixes
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* update according to the spec
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix the ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix batch condition
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some doc comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address https://github.com/tikv/client-go/pull/1411#discussion_r1714086744
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* rename some vars according to the comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add more comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-08-13 12:55:44 +08:00
14f3d2b5b7
region cache: add `BatchLocateKeyRanges` interface ( #1367 )
...
* add batch scan interface
Signed-off-by: you06 <you1474600@gmail.com>
fix bug
Signed-off-by: you06 <you1474600@gmail.com>
copy encoded ranges
Signed-off-by: you06 <you1474600@gmail.com>
fix panic
Signed-off-by: you06 <you1474600@gmail.com>
add batch scan region test
Signed-off-by: you06 <you1474600@gmail.com>
fix log
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader option
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader optino
Signed-off-by: you06 <you1474600@gmail.com>
add comment
Signed-off-by: you06 <you1474600@gmail.com>
deprecate ScanRegions
Signed-off-by: you06 <you1474600@gmail.com>
update kvproto & pd client
Signed-off-by: you06 <you1474600@gmail.com>
update go.mod
Signed-off-by: you06 <you1474600@gmail.com>
fix need leader
Signed-off-by: you06 <you1474600@gmail.com>
fix integration test
Signed-off-by: you06 <you1474600@gmail.com>
fix lint
Signed-off-by: you06 <you1474600@gmail.com>
export BatchLocateKeyRangesOpt
Signed-off-by: you06 <you1474600@gmail.com>
update integration test's tidb
Signed-off-by: you06 <you1474600@gmail.com>
refactor BatchLocateKeyRanges and add comments
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb of integration test
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-06-24 16:06:06 +08:00
31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV ( #1287 )
...
* Support actively requesting update health feedback information by calling RPC to TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto; follow the protocol change; and add more comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add integration tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* tune the frequency of active updating
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Revert changes to examples
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary empty line
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
31a502b9ba
improve region request log for diagnose ( #1300 )
...
* improve region request log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* rename struct
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine region error metric with store id label and add rpc error metric
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* restrict log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-29 15:56:32 +08:00
03bbadb277
Support receving health feedback ( #1153 )
...
* Register health feedback handler in batch client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update protocol
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Collect the health status and record to the stores
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove the filter which looks not necessary
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix checkAndUpdateStoreHealthStatus panicking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Merge SetHealthFeedbackHandler to Client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try another pattern: separated callback registry
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try region cache eventListener pattern
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test receiving froim callback
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests to region cache
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary debug log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove replace of tidb repo
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update comments; fix lint
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add the isSlow method of Store back
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove unused method for now to make lint happy
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML ( #1114 )
...
* Support pipelined memdb.
Signed-off-by: you06 <you1474600@gmail.com>
* fix mutex for pipelined memdb
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
8b3b01e8d7
introduce a random jitter to region cache ttl ( #1148 )
...
* introduce a random jitter to region cache ttl
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* refactor searching cached region
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* observe load region by reason
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-02-02 20:13:23 +08:00
1362f1e875
Revert "*: add metrics for batch client ( #996 )" ( #1046 )
...
This reverts commit 1a44252779
2023-10-30 20:08:15 +08:00
1a44252779
*: add metrics for batch client ( #996 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-10-07 11:46:05 +08:00
daed9fdaba
metrics: make upper limit of RPC latency larger ( #997 )
...
Signed-off-by: Jack Yu <jackysp@gmail.com>
2023-10-07 11:20:06 +08:00
be2b4c78a9
Revert "*: fix batch-client wait too long and add some metrics ( #973 )" ( #984 )
...
This reverts commit adb7db13c3
2023-09-19 11:15:11 +08:00
adb7db13c3
*: fix batch-client wait too long and add some metrics ( #973 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-09-18 11:39:42 +08:00
4ec212d5f2
*: fix stale read ops metric ( #878 ) ( #889 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-07-14 11:03:10 +08:00
cf07be2ba0
add const label for metrics ( #802 )
...
Signed-off-by: zeminzhou <zhouzemin@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2023-06-07 13:48:17 +08:00
c619a50474
add metrics for stale-read traffic ( #776 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2023-04-27 14:55:46 +08:00
c853ddc68c
metrics: add txn ttl lifetime ( #766 )
...
Signed-off-by: Jack Yu <jackysp@gmail.com>
Co-authored-by: Shirly <AndreMouche@126.com>
2023-04-14 13:59:29 +08:00
4f59f92a24
fix the incorrect txn size metric usages ( #746 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-03-27 13:58:40 +08:00
a7e3df4ab1
[Metrics] Supply extra metrics to monitor the flows under `prefer-leader` mode. ( #716 )
...
Signed-off-by: Lucasliang <nkcs_lykx@hotmail.com>
2023-03-13 18:08:39 +08:00
bdd41058aa
metrics: seperate metrics with source scope for txn command ( #723 )
...
* seperate metrics with source scope
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix async pessimistic rollback
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix missing definition
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
---------
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-03-06 22:12:45 +08:00
49323956ae
metric: seperate source for txn cmd ( #721 )
...
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-02-28 15:40:40 +08:00
bce56a5831
ReadMode: introduce AutoFollowerRead mechanism by adding a new `ReadMode == PreferLeader` ( #671 )
...
Signed-off-by: lucasliang <nkcs_lykx@hotmail.com>
2023-02-07 16:42:52 +08:00
50e86f7d3c
Add metrics and statistics about aggressive locking ( #687 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2023-02-07 12:31:34 +08:00
35416c4c43
Add gRPC connection monitor and fix related bug ( #665 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-01-31 16:28:48 +08:00
c20405f345
support witness peers ( #633 )
...
Signed-off-by: Wenbo Zhang <ethercflow@gmail.com>
2023-01-09 20:53:19 +08:00
57c12f7c64
Handle more detailed statistics from TiKV ( #536 )
...
* support more detailed statistics returned from TiKV
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* fix log formatting
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* add to test
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-07-13 16:56:47 +08:00
bb026bdad8
client: support requests with source label ( #506 )
...
* support request scope and source.
Signed-off-by: you06 <you1474600@gmail.com>
* update
Signed-off-by: you06 <you1474600@gmail.com>
* use counter for requests with too-many source
Signed-off-by: you06 <you1474600@gmail.com>
* refine resolve lock
Signed-off-by: you06 <you1474600@gmail.com>
* remove request source from resolve details
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
2022-06-24 18:17:09 +08:00
681fb6e537
[RawKV] support checksum ( #519 )
...
Signed-off-by: haojinming <jinming.hao@pingcap.com>
Co-authored-by: iosmanthus <dengliming@pingcap.com>
2022-06-22 16:08:35 +08:00
79f010d806
metrics: add loading region cache duration ( #502 )
...
* add latency for loading region region
Signed-off-by: you06 <you1474600@gmail.com>
* rename "*ReginoMiss" to "*CacaheMiss"
Signed-off-by: you06 <you1474600@gmail.com>
* rename metrics
Signed-off-by: you06 <you1474600@gmail.com>
2022-05-16 15:21:22 +08:00
a38ac96984
Add backoff for error recovery in progress ( #485 )
...
* add backoff for error recovery in progress
Signed-off-by: Connor1996 <zbk602423539@gmail.com>
2022-05-09 11:50:19 +08:00
b5eb031edd
Support setting assertions and receiving assertion errors for Prewrite requests ( #311 )
...
* support set assertion in 2pc mutation
Signed-off-by: lysu <sulifx@gmail.com>
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Receive assertion fail errors from TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add test log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove verbose log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics counter for assertions
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address some comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try to optimize assertion for pessimistic transactions
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix panic on optimistic transactions
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add InitCheckExistence method for LockCtx
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Support assertion level
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Check assertion level before doing assertion on client side
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test bitoperations of menBUfferMutations.Push
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add test for assertion in tikv
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Support run assertion test with unistore
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix license
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* export DeleteKey
Signed-off-by: ekexium <ekexium@gmail.com>
* Renaming DeleteKey
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Avoid panic when running with old version of TiKV; Add schema check on fast assertion
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add test for fast assertion
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add test for pessimistic lock check existence
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test assertion takes no effect if amending is enabled
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add HasAssertUnknown function
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Cleanup locks after assertion fail
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update tidb dependency
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix panic in TestIllegalTSO
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update dependency to tidb
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: lysu <sulifx@gmail.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: ekexium <ekexium@gmail.com>
2022-02-10 15:31:32 +08:00
3b10001094
Add miss register for TiKVUnsafeDestroyRangeFailuresCounterVec ( #406 )
...
Signed-off-by: Jiahao Huang <july2993@gmail.com>
2021-12-28 11:31:51 +08:00
02f89e38ec
metrics: make batch client requests more accurate ( #397 )
...
Signed-off-by: Yu Shuaipeng <jackysp@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2021-12-20 20:05:26 -08:00
128ccbde43
Add read throughput metric for SLI ( #321 )
...
Signed-off-by: jyz0309 <45495947@qq.com>
2021-10-11 11:46:20 +08:00
3cc2459779
add label ( #314 )
...
Signed-off-by: yisaer <disxiaofei@163.com>
2021-09-26 18:06:28 +08:00
f4a640e8e0
rechange to retry when tikv disk full ( #302 )
...
Signed-off-by: tier-cap <zhengxiaojin@pingcap.com>
2021-09-15 13:01:35 +08:00
feeb9d1ed1
metrics,internal/client: add metrics for batch client stream receiving duration ( #279 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2021-08-23 12:13:37 +00:00
zyguan