Commit Graph

162 Commits

Author SHA1 Message Date
ekexium dcb62bb121
Trace events (#1776)
ref pingcap/tidb#64008

Signed-off-by: ekexium <eke@fastmail.com>
2025-10-23 14:33:37 +00:00
Daniël van Eeden bea72b1d3b
Update the location of mathutil (#1750)
ref pingcap/tidb#63312

Signed-off-by: Daniël van Eeden <git@myname.nl>
2025-09-12 08:52:21 +00:00
山岚 1a0daf3ee7
txn: Set "CommitRequest.UseAsyncCommit" for async commit (#1730)
Signed-off-by: Ping Yu <yuping@pingcap.com>
Signed-off-by: Juncen Yu <yujuncen@pingcap.com>

Co-authored-by: Ping Yu <yuping@pingcap.com>
2025-08-01 15:12:33 +00:00
glorv 1430158810
resource_control: support collecting cross AZ traffic in ru consumption (#1669)
Signed-off-by: glorv <glorvs@163.com>
2025-06-25 21:28:54 +00:00
yibin f75488ba72
Update kvproto dependancy and set keyspace name for rpc context (#1667)
close tikv/client-go#1668

Signed-off-by: yibin87 <huyibin@pingcap.com>
2025-06-17 10:47:51 +00:00
MyonKeminta 74c0a81150
Replace etcd safe point with txn safe point for read safety check (#1634)
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2025-06-09 07:39:40 +00:00
zyguan af9bd3c5d2
tests: bump tidb to fix integration tests (#1650)
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-06-05 01:27:38 +00:00
lhy1024 e84f1a780f
update pd client for resource group and keyspace (#1645)
Signed-off-by: lhy1024 <admin@liudos.us>
2025-05-28 09:09:49 +00:00
王超 d1ec4b4f67
txn: handle undetermined error in client go (#1642)
close tikv/client-go#1641

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2025-05-20 07:00:04 +00:00
王超 2058fbc062
*: update kvproto version (#1636)
ref tikv/client-go#1631

Signed-off-by: Chao Wang <cclcwangchao@hotmail.com>
2025-05-06 05:25:04 +00:00
tangenta 2b8c6a7761
update integration_test/go.mod (#1624)
Signed-off-by: tangenta <tangenta@126.com>
2025-04-02 12:26:58 +00:00
disksing e3b8af59f4
update tidb for integration tests (#1621)
Signed-off-by: disksing <i@disksing.com>
2025-03-28 06:19:30 +00:00
disksing 599e671b3a
Update pd client (#1615)
Signed-off-by: disksing <i@disksing.com>
2025-03-19 14:08:00 +08:00
Ryan Leung 34130b733a
*: update pd client (#1605)
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2025-03-11 02:15:13 +00:00
Ryan Leung a805ea84f3
*: bump pd client (#1575)
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2025-02-14 06:40:15 +00:00
Ryan Leung ec9ad0fd70
*: bump pd client (#1567)
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2025-02-07 06:53:27 +00:00
zyguan e85edd4206
client: refactor batch client send loop (#1564)
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2025-01-24 06:43:32 +00:00
Ryan Leung 23abf0a9c8
*: bump pd client (#1542)
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2025-01-07 05:34:22 +00:00
crazycs 15c9cdb119
change ci reference to original tidb (#1540)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-12-30 03:58:15 +00:00
crazycs f2266d6bf2
*: Optimize executor runtime stats performance (#1532)
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-12-25 04:06:45 +00:00
JmPotato fd950fcf9f
Update the PD client to adopt tikv/pd#8943 (#1536)
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-12-23 07:08:48 +00:00
JmPotato ff126df68a
Upgrade the PD client to adopt the latest refactor (#1525)
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-12-20 05:23:27 +00:00
MyonKeminta 06d7f4b923
Try to validate read ts for all RPC requests (#1513)
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-12-09 09:49:30 +00:00
Neil Shen 245d2c39cc
go.mod: update kvproto (#1492)
Signed-off-by: Neil Shen <overvenus@gmail.com>
2024-11-15 06:51:08 +00:00
ShuNing 70049ae310
mod: update pd client (#1490)
Signed-off-by: nolouch <nolouch@gmail.com>
2024-11-11 09:02:27 +00:00
lance6716 8dfa86b5d1
dep: update github.com/dgryski/go-farm (#1476)
ref pingcap/tidb#56635

Signed-off-by: lance6716 <lance6716@gmail.com>
2024-10-15 06:50:14 +00:00
ekexium 527f80a186
feat: Broadcast min_commit_ts for pipelined transactions (#1458)
Signed-off-by: ekexium <eke@fastmail.com>
2024-09-25 01:34:14 +00:00
you06 d73cc1ed65
region cache: check if the pd returned regions covers the ranges (#1377)
* add reproduce test

Signed-off-by: you06 <you1474600@gmail.com>

* check if the pd returned regions covers the ranges

Signed-off-by: you06 <you1474600@gmail.com>

* handle limit

Signed-off-by: you06 <you1474600@gmail.com>

* fix lint

Signed-off-by: you06 <you1474600@gmail.com>

* add TODO for  func

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb for integration test

Signed-off-by: you06 <you1474600@gmail.com>

* rename gap detection function

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

* add half bounded cases

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-07-03 17:58:01 +08:00
you06 14f3d2b5b7
region cache: add `BatchLocateKeyRanges` interface (#1367)
* add batch scan interface

Signed-off-by: you06 <you1474600@gmail.com>

fix bug

Signed-off-by: you06 <you1474600@gmail.com>

copy encoded ranges

Signed-off-by: you06 <you1474600@gmail.com>

fix panic

Signed-off-by: you06 <you1474600@gmail.com>

add batch scan region test

Signed-off-by: you06 <you1474600@gmail.com>

fix log

Signed-off-by: you06 <you1474600@gmail.com>

remove need-leader option

Signed-off-by: you06 <you1474600@gmail.com>

remove need-leader optino

Signed-off-by: you06 <you1474600@gmail.com>

add comment

Signed-off-by: you06 <you1474600@gmail.com>

deprecate ScanRegions

Signed-off-by: you06 <you1474600@gmail.com>

update kvproto & pd client

Signed-off-by: you06 <you1474600@gmail.com>

update go.mod

Signed-off-by: you06 <you1474600@gmail.com>

fix need leader

Signed-off-by: you06 <you1474600@gmail.com>

fix integration test

Signed-off-by: you06 <you1474600@gmail.com>

fix lint

Signed-off-by: you06 <you1474600@gmail.com>

export BatchLocateKeyRangesOpt

Signed-off-by: you06 <you1474600@gmail.com>

update integration test's tidb

Signed-off-by: you06 <you1474600@gmail.com>

refactor BatchLocateKeyRanges and add comments

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb of integration test

Signed-off-by: you06 <you1474600@gmail.com>

* remove key in error since redact is not supported

Signed-off-by: you06 <you1474600@gmail.com>

* remove key in error since redact is not supported

Signed-off-by: you06 <you1474600@gmail.com>

* address comment

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-06-24 16:06:06 +08:00
yibin d4f0f4cf12
Revert "upgrade grpc 1.64 (#1346)" (#1369)
This reverts commit 1c06c5cbf5.

Signed-off-by: yibin <huyibin@pingcap.com>
2024-06-21 17:03:19 +08:00
JmPotato 156cebc2de
Update the PD client to adopt the latest fix/optimization (#1360)
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-06-04 12:57:05 +08:00
MyonKeminta 31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV (#1287)
* Support actively requesting update health feedback information by calling RPC to TiKV

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update kvproto; follow the protocol change; and add more comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix build

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add integration tests

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* tune the frequency of active updating

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update kvproto

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix test

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Revert changes to examples

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove unnecessary empty line

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
Weizhen Wang 1c06c5cbf5
upgrade grpc 1.64 (#1346)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-05-17 12:20:15 +08:00
JmPotato 52c232be3d
Update the PD client to adopt the latest optimization (#1326) 2024-04-30 16:32:54 +08:00
Weizhen Wang 1e1fe07b05
*: upgrade grpc 1.63.2 (#1289)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-04-26 10:56:09 +08:00
JmPotato ab337d6c51
Upgrade PD client to fix the TSO hang bug (#1241) 2024-03-20 17:03:30 +08:00
JmPotato c9767e5558
Upgrade PD client to fix the TSO hang bug (#1237) 2024-03-19 16:08:47 +08:00
ekexium 87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB (#1212)
* feat: introduce GetLocal() for MemBuffer

Signed-off-by: ekexium <eke@fastmail.com>

* downgrade tools to 0.18.0

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set flushingMemDB = nil when an error is returned from a flush

Signed-off-by: ekexium <eke@fastmail.com>

* impl BatchGet for MemBuffer

Signed-off-by: you06 <you1474600@gmail.com>

test membuffer batch get

Signed-off-by: you06 <you1474600@gmail.com>

* add Prefetch & GetPrefetchCache

Signed-off-by: you06 <you1474600@gmail.com>

* cache multi Prefetch call

Signed-off-by: you06 <you1474600@gmail.com>

* add tests

Signed-off-by: you06 <you1474600@gmail.com>

* workaround for golang ci lint failure

Signed-off-by: you06 <you1474600@gmail.com>

* replace assert with require

Signed-off-by: you06 <you1474600@gmail.com>

* workaround golangci lint

Signed-off-by: you06 <you1474600@gmail.com>

* lint

Signed-off-by: you06 <you1474600@gmail.com>

* remove prefetch interface, pipelined memdb will cache batch get result

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb

Signed-off-by: you06 <you1474600@gmail.com>

* fix batch get cache when membuffer is empty

Signed-off-by: you06 <you1474600@gmail.com>

* fix returned delete value

Signed-off-by: you06 <you1474600@gmail.com>

* fix: handle resourceGroupTag of Flush

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set resource group tag for committer if it's pipelined

Signed-off-by: ekexium <eke@fastmail.com>

* Update internal/unionstore/pipelined_memdb.go

Co-authored-by: ekexium <eke@fastmail.com>

* remove prefetch interface

Signed-off-by: you06 <you1474600@gmail.com>

* release mutex

Signed-off-by: you06 <you1474600@gmail.com>

* flush wait to avoid race

Signed-off-by: you06 <you1474600@gmail.com>

* fix unstopped test

Signed-off-by: you06 <you1474600@gmail.com>

* set resource group tags for committer

Signed-off-by: ekexium <eke@fastmail.com>

* skip test due to tikv image not updated yet

Signed-off-by: you06 <you1474600@gmail.com>

* skip more test

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 11:36:05 +08:00
Weizhen Wang b81a5ee7c4
*: upgrade grpc 1.62.1 (#1213)
* *: upgrade grpc 1.62.1

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

* *: upgrade grpc 1.62.1

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>

---------

Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-03-11 16:37:22 +08:00
cfzjywxk 9a37a0a77f
txn: attach cluster id in kv request (#1208)
* attach cluster id in kv requests

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* update go mod in integration test

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

---------

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-03-08 18:59:12 +08:00
Ryan Leung bfab41501b
update pd client to the latest master (#1195)
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2024-02-29 15:58:03 +08:00
MyonKeminta 03bbadb277
Support receving health feedback (#1153)
* Register health feedback handler in batch client

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* update protocol

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Collect the health status and record to the stores

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove the filter which looks not necessary

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add metrics

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Fix checkAndUpdateStoreHealthStatus panicking

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add logs

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Merge SetHealthFeedbackHandler to Client

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try another pattern: separated callback registry

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Try region cache eventListener pattern

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add tests

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Test receiving froim callback

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add tests to region cache

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Remove unnecessary debug log

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* remove replace of tidb repo

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* fix build

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Update comments; fix lint

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Add the isSlow method of Store back

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* remove unused method for now to make lint happy

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

* Address comments

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>

---------

Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
you06 8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML (#1114)
* Support pipelined memdb.

Signed-off-by: you06 <you1474600@gmail.com>

* fix mutex for pipelined memdb

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
Weizhen Wang 8c13f6b5a6
*: upgrade grpc1.62 (#1168)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-22 13:15:55 +08:00
Weizhen Wang 4bb62b6db7
upgrade grpc (#1156)
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-18 11:41:12 +08:00
cfzjywxk 11cb7985f0
txn: enable pessimistic region rollback (#1149)
* enable pessimistic region rollback

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* add key range parameters

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

* change interface to use start and end keys

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>

---------

Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-02-05 15:11:26 +08:00
lance6716 d1b611cbe2 integration_tests: upgrade tidb to remove replace
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-26 13:48:55 +08:00
lance6716 20a4aa99a3 fix replace is missing
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 16:03:34 +08:00
lance6716 cab3a43e0e fix replace is missing
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:56:27 +08:00
lance6716 abcd9b5e50 fix go mod tidy under integration_tests
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:50:13 +08:00