e85edd4206
client: refactor batch client send loop ( #1564 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-01-24 06:43:32 +00:00
61e09c6539
metrics: optimize rpc latency related metrics ( #1556 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-01-17 03:49:19 +00:00
20764920b4
*: reduce overhead of codec client ( #1555 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-01-17 03:47:01 +00:00
0e4728cfaa
*: add the traffic statistic ( #1505 )
...
ref pingcap/tidb#57543
Signed-off-by: nolouch <nolouch@gmail.com>
2024-12-16 06:58:54 +00:00
e7894a7b27
config: change grpc-keepalive-timeout from uint to float64, for support millisecond timeout ( #1461 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: MyonKeminta <9948422+MyonKeminta@users.noreply.github.com>
2024-09-11 04:15:06 +00:00
4c6b2171b2
improve batch efficiency for high throughput workloads ( #1411 )
...
* improve batch efficiency by multiple attempts
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add support for aggressive batching
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some metrics for batch client
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fetch more requests according to recent wait head durs
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add experimental batch options
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* optimize and refactor
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* some minor updates
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix the metric of head arrival interval
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* a minor update
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* some minor fixes
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* update according to the spec
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix the ut
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix batch condition
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add some doc comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address https://github.com/tikv/client-go/pull/1411#discussion_r1714086744
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* rename some vars according to the comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add more comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-08-13 12:55:44 +08:00
d4f0f4cf12
Revert "upgrade grpc 1.64 ( #1346 )" ( #1369 )
...
This reverts commit 1c06c5cbf5
2024-06-21 17:03:19 +08:00
cb580bc4ea
fix rpc client panic cause by concurrent close ( #1359 )
...
* fix rpc client panic cause by concurrent close
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-31 18:21:21 +08:00
31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV ( #1287 )
...
* Support actively requesting update health feedback information by calling RPC to TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto; follow the protocol change; and add more comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add integration tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* tune the frequency of active updating
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Revert changes to examples
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary empty line
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
1c06c5cbf5
upgrade grpc 1.64 ( #1346 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-05-17 12:20:15 +08:00
44b2944b64
Add grpc window size config and change the default value
...
Signed-off-by: gengliqi <gengliqiii@gmail.com>
2024-03-19 23:01:50 +08:00
98a7df8f41
fix issue read req timeout bug cause by pr#1223 ( #1232 )
...
* fix issue read req timeout bug cause by pr#1223
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* implement Cause
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix race test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 19:58:42 +09:00
7bce6ea4df
Pick connection handling improvements from tidb-cse-7.5 ( #1223 )
...
* close on canceled (#1121 )
Signed-off-by: Ping Yu <yuping@pingcap.com>
* Fix wait recvloop timeout (#1134 )
* wip
Signed-off-by: Ping Yu <yuping@pingcap.com>
* CloseAddr with ver
Signed-off-by: Ping Yu <yuping@pingcap.com>
* fix ErrConn
Signed-off-by: Ping Yu <yuping@pingcap.com>
* fix ut
Signed-off-by: Ping Yu <yuping@pingcap.com>
* cleanup
Signed-off-by: Ping Yu <yuping@pingcap.com>
---------
Signed-off-by: Ping Yu <yuping@pingcap.com>
* Use correct address for CloseAddr (#1140 )
* wip
Signed-off-by: Ping Yu <yuping@pingcap.com>
* CloseAddr with ver
Signed-off-by: Ping Yu <yuping@pingcap.com>
* fix ErrConn
Signed-off-by: Ping Yu <yuping@pingcap.com>
* fix ut
Signed-off-by: Ping Yu <yuping@pingcap.com>
* polish
Signed-off-by: Ping Yu <yuping@pingcap.com>
* polish
Signed-off-by: Ping Yu <yuping@pingcap.com>
* polish
Signed-off-by: Ping Yu <yuping@pingcap.com>
---------
Signed-off-by: Ping Yu <yuping@pingcap.com>
* rebase
Signed-off-by: Ping Yu <yuping@pingcap.com>
---------
Signed-off-by: Ping Yu <yuping@pingcap.com>
Co-authored-by: iosmanthus <dengliming@pingcap.com>
2024-03-14 16:30:36 +08:00
03bbadb277
Support receving health feedback ( #1153 )
...
* Register health feedback handler in batch client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update protocol
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Collect the health status and record to the stores
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove the filter which looks not necessary
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix checkAndUpdateStoreHealthStatus panicking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Merge SetHealthFeedbackHandler to Client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try another pattern: separated callback registry
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try region cache eventListener pattern
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test receiving froim callback
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests to region cache
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary debug log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove replace of tidb repo
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update comments; fix lint
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add the isSlow method of Store back
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove unused method for now to make lint happy
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
824302acd0
client: Cache tikv request in tidb client side ( #1098 )
...
* impl priority queue
Signed-off-by: bufferflies <1045931706@qq.com>
* replace priority queue
Signed-off-by: bufferflies <1045931706@qq.com>
* cache request in tidb side
Signed-off-by: bufferflies <1045931706@qq.com>
* fix gosimple
Signed-off-by: bufferflies <1045931706@qq.com>
* impl priority
Signed-off-by: bufferflies <1045931706@qq.com>
* pass ut
Signed-off-by: bufferflies <1045931706@qq.com>
* add
Signed-off-by: bufferflies <1045931706@qq.com>
* remove request if the request has been canceled
Signed-off-by: bufferflies <1045931706@qq.com>
* remove request if it has been canceled
Signed-off-by: bufferflies <1045931706@qq.com>
* add comment for cancel
Signed-off-by: bufferflies <1045931706@qq.com>
* not make the loop is busy
Signed-off-by: bufferflies <1045931706@qq.com>
* lint
Signed-off-by: bufferflies <1045931706@qq.com>
* revert busy loop
Signed-off-by: bufferflies <1045931706@qq.com>
* add unit test
Signed-off-by: bufferflies <1045931706@qq.com>
* not limit ehigh prioirty test
Signed-off-by: bufferflies <1045931706@qq.com>
* pass lint
Signed-off-by: bufferflies <1045931706@qq.com>
* support all
Signed-off-by: bufferflies <1045931706@qq.com>
* add comment
Signed-off-by: bufferflies <1045931706@qq.com>
* squash
Signed-off-by: bufferflies <1045931706@qq.com>
* revert all to All
Signed-off-by: bufferflies <1045931706@qq.com>
* remove index from entry
Signed-off-by: bufferflies <1045931706@qq.com>
* make fail reasons more clear
Signed-off-by: bufferflies <1045931706@qq.com>
---------
Signed-off-by: bufferflies <1045931706@qq.com>
2024-02-21 14:54:33 +08:00
f90605363e
support grpc's shard buffer pool ( #1132 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-01-24 17:57:47 +08:00
be2b4c78a9
Revert "*: fix batch-client wait too long and add some metrics ( #973 )" ( #984 )
...
This reverts commit adb7db13c3
2023-09-19 11:15:11 +08:00
adb7db13c3
*: fix batch-client wait too long and add some metrics ( #973 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
Co-authored-by: disksing <i@disksing.com>
2023-09-18 11:39:42 +08:00
c9119d02ce
support time detail v2 ( #729 )
...
* support time detail v2
Signed-off-by: glorv <glorvs@163.com>
---------
Signed-off-by: glorv <glorvs@163.com>
Co-authored-by: Connor <zbk602423539@gmail.com>
2023-03-13 21:32:19 +08:00
bdd41058aa
metrics: seperate metrics with source scope for txn command ( #723 )
...
* seperate metrics with source scope
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix async pessimistic rollback
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
* fix missing definition
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
---------
Signed-off-by: cfzjywxk <lsswxrxr@163.com>
2023-03-06 22:12:45 +08:00
e0e6019f8a
attach keyspace_id to tikv/tiflash requests ( #682 )
2023-02-20 13:10:03 +08:00
35416c4c43
Add gRPC connection monitor and fix related bug ( #665 )
...
Signed-off-by: Smityz <smityz@qq.com>
Co-authored-by: disksing <i@disksing.com>
2023-01-31 16:28:48 +08:00
f313ddf58d
reformat codec and add keyspace support ( #649 )
...
Signed-off-by: David <8039876+AmoebaProtozoa@users.noreply.github.com>
2023-01-10 15:15:33 +08:00
ce9203ef66
*: disable PermitWithoutStream to avoid too many pings ( #574 )
...
Signed-off-by: Jack Yu <jackysp@gmail.com>
2022-09-21 18:16:51 +08:00
2ed2113d10
Fix that histogram is not inserted into rpcNetLatencyHistCache ( #563 )
...
* Fix that histogram is not inserted into rpcNetLatencyHistCache
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* downgrade golangci-lint to avoid gofmt complaints temporarily
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-08-09 11:48:08 +08:00
e10841f2d1
support adding `ExecDetailsV2` to tracing ( #559 )
...
* support adding `ExecDetailsV2` to tracing
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* rename `TraceExecEnabled` to `TraceExecDetailsEnabled`
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* revert changes for #558
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address comments
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* add unit test
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2022-07-29 11:44:04 +08:00
e38d2b07de
do not format error when response does not include ExecDetailsV2 ( #553 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-07-21 11:16:57 +08:00
57c12f7c64
Handle more detailed statistics from TiKV ( #536 )
...
* support more detailed statistics returned from TiKV
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* fix log formatting
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
* add to test
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2022-07-13 16:56:47 +08:00
68cd1bc6c1
Introduce API v2 ( #494 )
...
* introduce raw.Client with api v2
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* migrate rawkv test to api v2
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* git fire: refactoring the interface SplitRegion of CodecClientV2
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* go mod tidy
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* fix split region issue in CodecClientV2
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* rearrange imports
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* add version info for RegionCache
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* delay request encoding to RPCClient
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* prevent multiple encoding
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* wip: debugging retry logic
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* fix EpochNotMatch region decoding
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* remove txn request branch
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* copy request before encoding it
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* add gRPCDialOptions for rawkv.Client
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* fix api breaking change of raw.NewClient
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* address menglong's comments
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* go mod tidy
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* address comments and fix linter
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* fix api_test
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* fix TestScan
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* fix TestScan
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
* fix compatibility
Signed-off-by: iosmanthus <myosmanthustree@gmail.com>
Co-authored-by: Xiaoguang Sun <sunxiaoguang@users.noreply.github.com>
2022-07-09 14:49:46 +08:00
bb026bdad8
client: support requests with source label ( #506 )
...
* support request scope and source.
Signed-off-by: you06 <you1474600@gmail.com>
* update
Signed-off-by: you06 <you1474600@gmail.com>
* use counter for requests with too-many source
Signed-off-by: you06 <you1474600@gmail.com>
* refine resolve lock
Signed-off-by: you06 <you1474600@gmail.com>
* remove request source from resolve details
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
2022-06-24 18:17:09 +08:00
2807409d49
client: fix nil dereference when closing conns ( #514 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-31 16:17:49 +08:00
700cbe60d2
support find stores with tiflash_mpp label ( #481 )
...
Signed-off-by: guo-shaoge <shaoge1994@163.com>
Co-authored-by: Yilin Chen <sticnarf@gmail.com>
2022-05-23 11:22:53 +08:00
ec236b6435
fix linter warning ( #470 )
...
Signed-off-by: disksing <i@disksing.com>
2022-04-11 21:29:30 +08:00
f73ec0e675
client: fix the race between connArray.Close() and connArray.Get() ( #465 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2022-04-06 17:12:03 +08:00
5b4040a95c
reconnect when meets store-not-match errors ( #456 )
...
Signed-off-by: youjiali1995 <zlwgx1023@gmail.com>
2022-03-24 17:30:51 +08:00
589cf79b2a
*: update etcd client to v3.5.2 ( #423 )
...
Signed-off-by: disksing <i@disksing.com>
2022-02-17 22:01:16 +08:00
b8cc5a319d
fix metrics error ( #365 )
...
Signed-off-by: yisaer <disxiaofei@163.com>
2021-11-02 12:05:33 +00:00
9ec50224be
*: replace pingcap/errors with pkg/errors ( #357 )
...
Signed-off-by: disksing <i@disksing.com>
2021-11-01 11:59:52 +08:00
b074cb9ad3
internal/client: remove use of errors.Trace ( #329 )
...
Signed-off-by: disksing <i@disksing.com>
2021-10-11 16:59:49 +00:00
97fac134af
error: replace usages of terror.Log ( #322 )
...
Signed-off-by: disksing <i@disksing.com>
2021-10-11 06:08:21 +00:00
3cc2459779
add label ( #314 )
...
Signed-off-by: yisaer <disxiaofei@163.com>
2021-09-26 18:06:28 +08:00
16d902a3c7
reduce write backoffs but don't count busy errors ( #271 )
...
Signed-off-by: Yilin Chen <sticnarf@gmail.com>
2021-08-24 09:05:36 +00:00
9d2ad5209e
internal/client: remove the lock for idle connection recycle ( #278 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
2021-08-23 15:23:11 +08:00
31906d4105
internal/client: fix recycle idle connection block sending request ( #270 )
...
Signed-off-by: tiancaiamao <tiancaiamao@gmail.com>
2021-08-19 23:40:46 +08:00
df2119f51b
client: cleanup RPCClient options ( #268 )
...
Signed-off-by: disksing <i@disksing.com>
2021-08-16 07:11:08 +00:00
288c649636
fix license ( #267 )
...
Signed-off-by: disksing <i@disksing.com>
2021-08-15 01:48:01 +08:00
55155ad2e5
*: make grpc request duration more accurate ( #253 )
...
Signed-off-by: Shuaipeng Yu <jackysp@gmail.com>
2021-07-27 12:09:05 +00:00
d0a98864a6
*: move logutil to internal directory ( #207 )
...
Signed-off-by: disksing <i@disksing.com>
2021-07-05 15:15:41 +08:00
4e8c0a34c8
move client to internal package ( #194 )
...
Signed-off-by: disksing <i@disksing.com>
Co-authored-by: iosmanthus <MyOsmanthusTree@gmail.com>
2021-06-30 16:18:10 +08:00
zyguan
ShuNing
crazycs
yibin
MyonKeminta
Weizhen Wang
Liqi Geng
Ping Yu
tongjian
glorv
cfzjywxk
iosmanthus
Smilencer
David
Jack Yu
Yilin Chen
you06
Lei Zhao
guo-shaoge
disksing
Song Gao
tiancaiamao