fd950fcf9f
Update the PD client to adopt tikv/pd#8943 ( #1536 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-12-23 07:08:48 +00:00
ff126df68a
Upgrade the PD client to adopt the latest refactor ( #1525 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-12-20 05:23:27 +00:00
8e0275cc8d
Fix the problem that ttlManager may stop working if no need to lock after retry aggressive locking ( #1522 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-12-17 09:14:33 +00:00
06d7f4b923
Try to validate read ts for all RPC requests ( #1513 )
...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-12-09 09:49:30 +00:00
89643b0e8c
Export commitTS of KVTxn ( #1489 )
...
Signed-off-by: Bin Zhang <b6g2021@gmail.com>
2024-11-27 07:15:26 +00:00
245d2c39cc
go.mod: update kvproto ( #1492 )
...
Signed-off-by: Neil Shen <overvenus@gmail.com>
2024-11-15 06:51:08 +00:00
70049ae310
mod: update pd client ( #1490 )
...
Signed-off-by: nolouch <nolouch@gmail.com>
2024-11-11 09:02:27 +00:00
8dfa86b5d1
dep: update github.com/dgryski/go-farm ( #1476 )
...
ref pingcap/tidb#56635
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-10-15 06:50:14 +00:00
e30a3e6e3a
*: upgrade go1.23 ( #1473 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-10-11 06:35:57 +00:00
527f80a186
feat: Broadcast min_commit_ts for pipelined transactions ( #1458 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-09-25 01:34:14 +00:00
f0ea917499
transaction: add hook for async commit to track the life cycle of the async-commit goroutine and secondary lock cleanup goroutine ( #1432 )
...
Signed-off-by: Yang Keao <yangkeao@chunibyo.icu>
2024-09-04 09:21:32 +00:00
d73cc1ed65
region cache: check if the pd returned regions covers the ranges ( #1377 )
...
* add reproduce test
Signed-off-by: you06 <you1474600@gmail.com>
* check if the pd returned regions covers the ranges
Signed-off-by: you06 <you1474600@gmail.com>
* handle limit
Signed-off-by: you06 <you1474600@gmail.com>
* fix lint
Signed-off-by: you06 <you1474600@gmail.com>
* add TODO for func
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb for integration test
Signed-off-by: you06 <you1474600@gmail.com>
* rename gap detection function
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* add half bounded cases
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-07-03 17:58:01 +08:00
14f3d2b5b7
region cache: add `BatchLocateKeyRanges` interface ( #1367 )
...
* add batch scan interface
Signed-off-by: you06 <you1474600@gmail.com>
fix bug
Signed-off-by: you06 <you1474600@gmail.com>
copy encoded ranges
Signed-off-by: you06 <you1474600@gmail.com>
fix panic
Signed-off-by: you06 <you1474600@gmail.com>
add batch scan region test
Signed-off-by: you06 <you1474600@gmail.com>
fix log
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader option
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader optino
Signed-off-by: you06 <you1474600@gmail.com>
add comment
Signed-off-by: you06 <you1474600@gmail.com>
deprecate ScanRegions
Signed-off-by: you06 <you1474600@gmail.com>
update kvproto & pd client
Signed-off-by: you06 <you1474600@gmail.com>
update go.mod
Signed-off-by: you06 <you1474600@gmail.com>
fix need leader
Signed-off-by: you06 <you1474600@gmail.com>
fix integration test
Signed-off-by: you06 <you1474600@gmail.com>
fix lint
Signed-off-by: you06 <you1474600@gmail.com>
export BatchLocateKeyRangesOpt
Signed-off-by: you06 <you1474600@gmail.com>
update integration test's tidb
Signed-off-by: you06 <you1474600@gmail.com>
refactor BatchLocateKeyRanges and add comments
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb of integration test
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-06-24 16:06:06 +08:00
d4f0f4cf12
Revert "upgrade grpc 1.64 ( #1346 )" ( #1369 )
...
This reverts commit 1c06c5cbf5
2024-06-21 17:03:19 +08:00
156cebc2de
Update the PD client to adopt the latest fix/optimization ( #1360 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-06-04 12:57:05 +08:00
31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV ( #1287 )
...
* Support actively requesting update health feedback information by calling RPC to TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto; follow the protocol change; and add more comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add integration tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* tune the frequency of active updating
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Revert changes to examples
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary empty line
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
1c06c5cbf5
upgrade grpc 1.64 ( #1346 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-05-17 12:20:15 +08:00
6cb0704fce
fail pipelined dml when max ttl exceeded ( #1329 )
...
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-04-30 22:52:41 +08:00
52c232be3d
Update the PD client to adopt the latest optimization ( #1326 )
2024-04-30 16:32:54 +08:00
31a502b9ba
improve region request log for diagnose ( #1300 )
...
* improve region request log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* rename struct
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine region error metric with store id label and add rpc error metric
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* restrict log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-29 15:56:32 +08:00
1e1fe07b05
*: upgrade grpc 1.63.2 ( #1289 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-04-26 10:56:09 +08:00
0229f4077f
do not cache snapshot with version maxuint64 ( #1320 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-24 13:23:42 +08:00
edba2544d5
txn: abort pipelined dml when pk rollback ( #1317 )
...
* fail p-dml when ttl manager is closed
Signed-off-by: you06 <you1474600@gmail.com>
* close ttl manager when pk is rollbcked
Signed-off-by: you06 <you1474600@gmail.com>
* stablize test
Signed-off-by: you06 <you1474600@gmail.com>
* add test
Signed-off-by: you06 <you1474600@gmail.com>
* fix race
Signed-off-by: you06 <you1474600@gmail.com>
* fix leak in test
Signed-off-by: you06 <you1474600@gmail.com>
* remote testlog
Signed-off-by: you06 <you1474600@gmail.com>
* check error message
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-23 17:36:44 +08:00
dcb9b95307
txn: cache result of snapshot.Get ( #1278 )
...
* also cache result of snapshot.Get
Signed-off-by: you06 <you1474600@gmail.com>
* fix test
Signed-off-by: you06 <you1474600@gmail.com>
fix test
Signed-off-by: you06 <you1474600@gmail.com>
* add Get cache test
Signed-off-by: you06 <you1474600@gmail.com>
* move RLock position
Signed-off-by: you06 <you1474600@gmail.com>
* remove comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-13 16:45:31 +08:00
714958ccd4
implement rollback for pipelined dml ( #1235 )
...
* impl rollback for p-dml
Signed-off-by: you06 <you1474600@gmail.com>
* add test to ensure the rollback locks cannot be read
Signed-off-by: you06 <you1474600@gmail.com>
* remove TODO comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
address comment
Signed-off-by: you06 <you1474600@gmail.com>
* resolve conflict
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-04-09 10:27:18 +08:00
f800b05581
Distinguish identifier and name in task runner ( #1271 )
...
* refactor: distinguish identifier and name in task runner, to avoid messy metrics
Signed-off-by: ekexium <eke@fastmail.com>
* keep the original interface to avoid breaking compatibility
Signed-off-by: ekexium <eke@fastmail.com>
* log: use ctx logger
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-08 19:35:52 +08:00
c2927c0ec6
cop: fix time detail merge ( #1258 )
...
* time detail merge
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* fix test
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-04-03 12:05:38 +08:00
81d8dea0eb
tikv: ensure safe-ts won't be max uint64 ( #1250 )
...
* tikv: ensure safe-ts won't be max uint64
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix a typo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix lint issue
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment and fix test
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-03-28 14:00:22 +08:00
ab337d6c51
Upgrade PD client to fix the TSO hang bug ( #1241 )
2024-03-20 17:03:30 +08:00
58ef395164
test: restore skipped test for pipelined dml ( #1238 )
...
* test: restore skipped test
Signed-off-by: ekexium <eke@fastmail.com>
* test: FlushWait() to avoid race
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-03-19 20:37:44 +08:00
c9767e5558
Upgrade PD client to fix the TSO hang bug ( #1237 )
2024-03-19 16:08:47 +08:00
87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB ( #1212 )
...
* feat: introduce GetLocal() for MemBuffer
Signed-off-by: ekexium <eke@fastmail.com>
* downgrade tools to 0.18.0
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set flushingMemDB = nil when an error is returned from a flush
Signed-off-by: ekexium <eke@fastmail.com>
* impl BatchGet for MemBuffer
Signed-off-by: you06 <you1474600@gmail.com>
test membuffer batch get
Signed-off-by: you06 <you1474600@gmail.com>
* add Prefetch & GetPrefetchCache
Signed-off-by: you06 <you1474600@gmail.com>
* cache multi Prefetch call
Signed-off-by: you06 <you1474600@gmail.com>
* add tests
Signed-off-by: you06 <you1474600@gmail.com>
* workaround for golang ci lint failure
Signed-off-by: you06 <you1474600@gmail.com>
* replace assert with require
Signed-off-by: you06 <you1474600@gmail.com>
* workaround golangci lint
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
* remove prefetch interface, pipelined memdb will cache batch get result
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb
Signed-off-by: you06 <you1474600@gmail.com>
* fix batch get cache when membuffer is empty
Signed-off-by: you06 <you1474600@gmail.com>
* fix returned delete value
Signed-off-by: you06 <you1474600@gmail.com>
* fix: handle resourceGroupTag of Flush
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set resource group tag for committer if it's pipelined
Signed-off-by: ekexium <eke@fastmail.com>
* Update internal/unionstore/pipelined_memdb.go
Co-authored-by: ekexium <eke@fastmail.com>
* remove prefetch interface
Signed-off-by: you06 <you1474600@gmail.com>
* release mutex
Signed-off-by: you06 <you1474600@gmail.com>
* flush wait to avoid race
Signed-off-by: you06 <you1474600@gmail.com>
* fix unstopped test
Signed-off-by: you06 <you1474600@gmail.com>
* set resource group tags for committer
Signed-off-by: ekexium <eke@fastmail.com>
* skip test due to tikv image not updated yet
Signed-off-by: you06 <you1474600@gmail.com>
* skip more test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 11:36:05 +08:00
88fbc938cd
p-dml: fix unstopped resolve ( #1210 )
...
* fix unstopped resolve when the txn cross regions
Signed-off-by: you06 <you1474600@gmail.com>
* cancel context when flush done or abort
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-03-11 21:20:38 +08:00
b81a5ee7c4
*: upgrade grpc 1.62.1 ( #1213 )
...
* *: upgrade grpc 1.62.1
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* *: upgrade grpc 1.62.1
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
---------
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-03-11 16:37:22 +08:00
9a37a0a77f
txn: attach cluster id in kv request ( #1208 )
...
* attach cluster id in kv requests
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* update go mod in integration test
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-03-08 18:59:12 +08:00
af4f9a9b6e
fix data race when collapsing ResolveLockRequest ( #1209 )
...
* fix data race when collapsing ResolveLockRequest
Signed-off-by: you06 <you1474600@gmail.com>
* comment
Signed-off-by: you06 <you1474600@gmail.com>
* sleep to wait goroutine finish
Signed-off-by: you06 <you1474600@gmail.com>
* update comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-03-08 13:24:15 +08:00
bfab41501b
update pd client to the latest master ( #1195 )
...
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2024-02-29 15:58:03 +08:00
03bbadb277
Support receving health feedback ( #1153 )
...
* Register health feedback handler in batch client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update protocol
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Collect the health status and record to the stores
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove the filter which looks not necessary
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix checkAndUpdateStoreHealthStatus panicking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Merge SetHealthFeedbackHandler to Client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try another pattern: separated callback registry
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try region cache eventListener pattern
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test receiving froim callback
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests to region cache
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary debug log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove replace of tidb repo
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update comments; fix lint
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add the isSlow method of Store back
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove unused method for now to make lint happy
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML ( #1114 )
...
* Support pipelined memdb.
Signed-off-by: you06 <you1474600@gmail.com>
* fix mutex for pipelined memdb
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
8c13f6b5a6
*: upgrade grpc1.62 ( #1168 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-22 13:15:55 +08:00
4bb62b6db7
upgrade grpc ( #1156 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-18 11:41:12 +08:00
11cb7985f0
txn: enable pessimistic region rollback ( #1149 )
...
* enable pessimistic region rollback
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* add key range parameters
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* change interface to use start and end keys
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-02-05 15:11:26 +08:00
70c148e84e
ErrQueryInterrupted with parameters ( #1124 )
...
* feat: ErrQueryInterrupted with parameters
Signed-off-by: ekexium <eke@fastmail.com>
* Revert "Revert "fix: check kill signal against 0 (#1102 )" (#1129 )"
This reverts commit 3480b5ed7c
2024-02-02 10:26:02 +08:00
d2887d56ab
txn: use region pessimsitic lock rollback to speed up cleanup ( #1125 )
...
* use region pessimsitic lock rollback and clean for write-write conflict processing
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* format
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* debug ci failure patch, disable region pessimistic rollback
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-01-31 09:53:00 +08:00
30684f7d3c
use new interface
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-26 13:53:07 +08:00
d1b611cbe2
integration_tests: upgrade tidb to remove replace
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-26 13:48:55 +08:00
20a4aa99a3
fix replace is missing
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 16:03:34 +08:00
cab3a43e0e
fix replace is missing
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:56:27 +08:00
abcd9b5e50
fix go mod tidy under integration_tests
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:50:13 +08:00
db1a348d5d
update pd client
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:47:36 +08:00
JmPotato