d73cc1ed65
region cache: check if the pd returned regions covers the ranges ( #1377 )
...
* add reproduce test
Signed-off-by: you06 <you1474600@gmail.com>
* check if the pd returned regions covers the ranges
Signed-off-by: you06 <you1474600@gmail.com>
* handle limit
Signed-off-by: you06 <you1474600@gmail.com>
* fix lint
Signed-off-by: you06 <you1474600@gmail.com>
* add TODO for func
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb for integration test
Signed-off-by: you06 <you1474600@gmail.com>
* rename gap detection function
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
* add half bounded cases
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-07-03 17:58:01 +08:00
14f3d2b5b7
region cache: add `BatchLocateKeyRanges` interface ( #1367 )
...
* add batch scan interface
Signed-off-by: you06 <you1474600@gmail.com>
fix bug
Signed-off-by: you06 <you1474600@gmail.com>
copy encoded ranges
Signed-off-by: you06 <you1474600@gmail.com>
fix panic
Signed-off-by: you06 <you1474600@gmail.com>
add batch scan region test
Signed-off-by: you06 <you1474600@gmail.com>
fix log
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader option
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader optino
Signed-off-by: you06 <you1474600@gmail.com>
add comment
Signed-off-by: you06 <you1474600@gmail.com>
deprecate ScanRegions
Signed-off-by: you06 <you1474600@gmail.com>
update kvproto & pd client
Signed-off-by: you06 <you1474600@gmail.com>
update go.mod
Signed-off-by: you06 <you1474600@gmail.com>
fix need leader
Signed-off-by: you06 <you1474600@gmail.com>
fix integration test
Signed-off-by: you06 <you1474600@gmail.com>
fix lint
Signed-off-by: you06 <you1474600@gmail.com>
export BatchLocateKeyRangesOpt
Signed-off-by: you06 <you1474600@gmail.com>
update integration test's tidb
Signed-off-by: you06 <you1474600@gmail.com>
refactor BatchLocateKeyRanges and add comments
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb of integration test
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-06-24 16:06:06 +08:00
d4f0f4cf12
Revert "upgrade grpc 1.64 ( #1346 )" ( #1369 )
...
This reverts commit 1c06c5cbf5
2024-06-21 17:03:19 +08:00
156cebc2de
Update the PD client to adopt the latest fix/optimization ( #1360 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-06-04 12:57:05 +08:00
31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV ( #1287 )
...
* Support actively requesting update health feedback information by calling RPC to TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto; follow the protocol change; and add more comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add integration tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* tune the frequency of active updating
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Revert changes to examples
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary empty line
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
1c06c5cbf5
upgrade grpc 1.64 ( #1346 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-05-17 12:20:15 +08:00
6cb0704fce
fail pipelined dml when max ttl exceeded ( #1329 )
...
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-04-30 22:52:41 +08:00
52c232be3d
Update the PD client to adopt the latest optimization ( #1326 )
2024-04-30 16:32:54 +08:00
31a502b9ba
improve region request log for diagnose ( #1300 )
...
* improve region request log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* rename struct
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine region error metric with store id label and add rpc error metric
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* restrict log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-29 15:56:32 +08:00
1e1fe07b05
*: upgrade grpc 1.63.2 ( #1289 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-04-26 10:56:09 +08:00
0229f4077f
do not cache snapshot with version maxuint64 ( #1320 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-24 13:23:42 +08:00
edba2544d5
txn: abort pipelined dml when pk rollback ( #1317 )
...
* fail p-dml when ttl manager is closed
Signed-off-by: you06 <you1474600@gmail.com>
* close ttl manager when pk is rollbcked
Signed-off-by: you06 <you1474600@gmail.com>
* stablize test
Signed-off-by: you06 <you1474600@gmail.com>
* add test
Signed-off-by: you06 <you1474600@gmail.com>
* fix race
Signed-off-by: you06 <you1474600@gmail.com>
* fix leak in test
Signed-off-by: you06 <you1474600@gmail.com>
* remote testlog
Signed-off-by: you06 <you1474600@gmail.com>
* check error message
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-23 17:36:44 +08:00
dcb9b95307
txn: cache result of snapshot.Get ( #1278 )
...
* also cache result of snapshot.Get
Signed-off-by: you06 <you1474600@gmail.com>
* fix test
Signed-off-by: you06 <you1474600@gmail.com>
fix test
Signed-off-by: you06 <you1474600@gmail.com>
* add Get cache test
Signed-off-by: you06 <you1474600@gmail.com>
* move RLock position
Signed-off-by: you06 <you1474600@gmail.com>
* remove comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-13 16:45:31 +08:00
714958ccd4
implement rollback for pipelined dml ( #1235 )
...
* impl rollback for p-dml
Signed-off-by: you06 <you1474600@gmail.com>
* add test to ensure the rollback locks cannot be read
Signed-off-by: you06 <you1474600@gmail.com>
* remove TODO comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
address comment
Signed-off-by: you06 <you1474600@gmail.com>
* resolve conflict
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-04-09 10:27:18 +08:00
f800b05581
Distinguish identifier and name in task runner ( #1271 )
...
* refactor: distinguish identifier and name in task runner, to avoid messy metrics
Signed-off-by: ekexium <eke@fastmail.com>
* keep the original interface to avoid breaking compatibility
Signed-off-by: ekexium <eke@fastmail.com>
* log: use ctx logger
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-08 19:35:52 +08:00
c2927c0ec6
cop: fix time detail merge ( #1258 )
...
* time detail merge
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* fix test
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-04-03 12:05:38 +08:00
81d8dea0eb
tikv: ensure safe-ts won't be max uint64 ( #1250 )
...
* tikv: ensure safe-ts won't be max uint64
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix a typo
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix lint issue
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment and fix test
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-03-28 14:00:22 +08:00
ab337d6c51
Upgrade PD client to fix the TSO hang bug ( #1241 )
2024-03-20 17:03:30 +08:00
58ef395164
test: restore skipped test for pipelined dml ( #1238 )
...
* test: restore skipped test
Signed-off-by: ekexium <eke@fastmail.com>
* test: FlushWait() to avoid race
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-03-19 20:37:44 +08:00
c9767e5558
Upgrade PD client to fix the TSO hang bug ( #1237 )
2024-03-19 16:08:47 +08:00
87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB ( #1212 )
...
* feat: introduce GetLocal() for MemBuffer
Signed-off-by: ekexium <eke@fastmail.com>
* downgrade tools to 0.18.0
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set flushingMemDB = nil when an error is returned from a flush
Signed-off-by: ekexium <eke@fastmail.com>
* impl BatchGet for MemBuffer
Signed-off-by: you06 <you1474600@gmail.com>
test membuffer batch get
Signed-off-by: you06 <you1474600@gmail.com>
* add Prefetch & GetPrefetchCache
Signed-off-by: you06 <you1474600@gmail.com>
* cache multi Prefetch call
Signed-off-by: you06 <you1474600@gmail.com>
* add tests
Signed-off-by: you06 <you1474600@gmail.com>
* workaround for golang ci lint failure
Signed-off-by: you06 <you1474600@gmail.com>
* replace assert with require
Signed-off-by: you06 <you1474600@gmail.com>
* workaround golangci lint
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
* remove prefetch interface, pipelined memdb will cache batch get result
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb
Signed-off-by: you06 <you1474600@gmail.com>
* fix batch get cache when membuffer is empty
Signed-off-by: you06 <you1474600@gmail.com>
* fix returned delete value
Signed-off-by: you06 <you1474600@gmail.com>
* fix: handle resourceGroupTag of Flush
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set resource group tag for committer if it's pipelined
Signed-off-by: ekexium <eke@fastmail.com>
* Update internal/unionstore/pipelined_memdb.go
Co-authored-by: ekexium <eke@fastmail.com>
* remove prefetch interface
Signed-off-by: you06 <you1474600@gmail.com>
* release mutex
Signed-off-by: you06 <you1474600@gmail.com>
* flush wait to avoid race
Signed-off-by: you06 <you1474600@gmail.com>
* fix unstopped test
Signed-off-by: you06 <you1474600@gmail.com>
* set resource group tags for committer
Signed-off-by: ekexium <eke@fastmail.com>
* skip test due to tikv image not updated yet
Signed-off-by: you06 <you1474600@gmail.com>
* skip more test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 11:36:05 +08:00
88fbc938cd
p-dml: fix unstopped resolve ( #1210 )
...
* fix unstopped resolve when the txn cross regions
Signed-off-by: you06 <you1474600@gmail.com>
* cancel context when flush done or abort
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-03-11 21:20:38 +08:00
b81a5ee7c4
*: upgrade grpc 1.62.1 ( #1213 )
...
* *: upgrade grpc 1.62.1
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
* *: upgrade grpc 1.62.1
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
---------
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-03-11 16:37:22 +08:00
9a37a0a77f
txn: attach cluster id in kv request ( #1208 )
...
* attach cluster id in kv requests
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* update go mod in integration test
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-03-08 18:59:12 +08:00
af4f9a9b6e
fix data race when collapsing ResolveLockRequest ( #1209 )
...
* fix data race when collapsing ResolveLockRequest
Signed-off-by: you06 <you1474600@gmail.com>
* comment
Signed-off-by: you06 <you1474600@gmail.com>
* sleep to wait goroutine finish
Signed-off-by: you06 <you1474600@gmail.com>
* update comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-03-08 13:24:15 +08:00
bfab41501b
update pd client to the latest master ( #1195 )
...
Signed-off-by: Ryan Leung <rleungx@gmail.com>
2024-02-29 15:58:03 +08:00
03bbadb277
Support receving health feedback ( #1153 )
...
* Register health feedback handler in batch client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update protocol
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Collect the health status and record to the stores
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove the filter which looks not necessary
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add metrics
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix checkAndUpdateStoreHealthStatus panicking
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Merge SetHealthFeedbackHandler to Client
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try another pattern: separated callback registry
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Try region cache eventListener pattern
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Test receiving froim callback
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add tests to region cache
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary debug log
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove replace of tidb repo
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update comments; fix lint
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add the isSlow method of Store back
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* remove unused method for now to make lint happy
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Address comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-02-28 19:10:32 +08:00
8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML ( #1114 )
...
* Support pipelined memdb.
Signed-off-by: you06 <you1474600@gmail.com>
* fix mutex for pipelined memdb
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
8c13f6b5a6
*: upgrade grpc1.62 ( #1168 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-22 13:15:55 +08:00
4bb62b6db7
upgrade grpc ( #1156 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-02-18 11:41:12 +08:00
11cb7985f0
txn: enable pessimistic region rollback ( #1149 )
...
* enable pessimistic region rollback
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* add key range parameters
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* change interface to use start and end keys
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-02-05 15:11:26 +08:00
70c148e84e
ErrQueryInterrupted with parameters ( #1124 )
...
* feat: ErrQueryInterrupted with parameters
Signed-off-by: ekexium <eke@fastmail.com>
* Revert "Revert "fix: check kill signal against 0 (#1102 )" (#1129 )"
This reverts commit 3480b5ed7c
2024-02-02 10:26:02 +08:00
d2887d56ab
txn: use region pessimsitic lock rollback to speed up cleanup ( #1125 )
...
* use region pessimsitic lock rollback and clean for write-write conflict processing
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* format
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
* debug ci failure patch, disable region pessimistic rollback
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
---------
Signed-off-by: cfzjywxk <cfzjywxk@gmail.com>
2024-01-31 09:53:00 +08:00
30684f7d3c
use new interface
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-26 13:53:07 +08:00
d1b611cbe2
integration_tests: upgrade tidb to remove replace
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-26 13:48:55 +08:00
20a4aa99a3
fix replace is missing
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 16:03:34 +08:00
cab3a43e0e
fix replace is missing
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:56:27 +08:00
abcd9b5e50
fix go mod tidy under integration_tests
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:50:13 +08:00
db1a348d5d
update pd client
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:47:36 +08:00
89e999557b
Merge branch 'master' of github.com:tikv/client-go into update-dep
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-24 15:41:40 +08:00
1af8d0b0ca
*: upgrade grpc1.61 ( #1130 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
Co-authored-by: disksing <i@disksing.com>
2024-01-24 11:39:43 +08:00
9d6071477a
Merge branch 'update-dep' of github.com:lance6716/client-go into update-dep
2024-01-23 16:06:13 +08:00
cfee5d73dd
fix goroutine leak
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-23 16:05:55 +08:00
3d7ff4c3c8
Merge branch 'master' into update-dep
2024-01-23 15:23:15 +08:00
29355d50d1
update pd
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-23 15:07:19 +08:00
3480b5ed7c
Revert "fix: check kill signal against 0 ( #1102 )" ( #1129 )
...
This reverts commit 057c479dd8
2024-01-23 13:54:05 +08:00
77d23f7f5c
update
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-22 15:16:42 +08:00
7d4676e27a
update
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-22 15:12:23 +08:00
c3ce73e405
update replace
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-22 15:12:23 +08:00
5d925263d7
replace tidb package
...
Signed-off-by: lance6716 <lance6716@gmail.com>
2024-01-22 15:12:23 +08:00
you06