4a72526f6c
add enable-replica-selector-v2 config back but mark it was deprecated ( #1374 )
...
* add enable-replica-selector-v2 config back but mark it was deprecated
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* update comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-06-26 14:42:48 +08:00
14f3d2b5b7
region cache: add `BatchLocateKeyRanges` interface ( #1367 )
...
* add batch scan interface
Signed-off-by: you06 <you1474600@gmail.com>
fix bug
Signed-off-by: you06 <you1474600@gmail.com>
copy encoded ranges
Signed-off-by: you06 <you1474600@gmail.com>
fix panic
Signed-off-by: you06 <you1474600@gmail.com>
add batch scan region test
Signed-off-by: you06 <you1474600@gmail.com>
fix log
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader option
Signed-off-by: you06 <you1474600@gmail.com>
remove need-leader optino
Signed-off-by: you06 <you1474600@gmail.com>
add comment
Signed-off-by: you06 <you1474600@gmail.com>
deprecate ScanRegions
Signed-off-by: you06 <you1474600@gmail.com>
update kvproto & pd client
Signed-off-by: you06 <you1474600@gmail.com>
update go.mod
Signed-off-by: you06 <you1474600@gmail.com>
fix need leader
Signed-off-by: you06 <you1474600@gmail.com>
fix integration test
Signed-off-by: you06 <you1474600@gmail.com>
fix lint
Signed-off-by: you06 <you1474600@gmail.com>
export BatchLocateKeyRangesOpt
Signed-off-by: you06 <you1474600@gmail.com>
update integration test's tidb
Signed-off-by: you06 <you1474600@gmail.com>
refactor BatchLocateKeyRanges and add comments
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb of integration test
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* remove key in error since redact is not supported
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-06-24 16:06:06 +08:00
d4f0f4cf12
Revert "upgrade grpc 1.64 ( #1346 )" ( #1369 )
...
This reverts commit 1c06c5cbf5
2024-06-21 17:03:19 +08:00
4f2562f987
region_request: ignore resource group errors that not relative storage layer ( #1354 )
...
Signed-off-by: nolouch <nolouch@gmail.com>
2024-06-20 13:57:02 +08:00
ac8fa1d73a
feat: introduce a new method MemHookSet ( #1362 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-06-14 14:44:55 +08:00
156cebc2de
Update the PD client to adopt the latest fix/optimization ( #1360 )
...
Signed-off-by: JmPotato <ghzpotato@gmail.com>
2024-06-04 12:57:05 +08:00
cb580bc4ea
fix rpc client panic cause by concurrent close ( #1359 )
...
* fix rpc client panic cause by concurrent close
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-31 18:21:21 +08:00
1f814b7a45
add func for new test keyspace tikv store ( #1356 )
2024-05-30 17:52:57 +08:00
a6335a5aaf
Fix the problem that statement being oom-killed within DoneAggressiveLocking causing the transaction still in aggressive locking state ( #1355 )
...
* Quick fix the problem that statement being oom-killed within DoneAggressiveLocking causing the transaction still in aggressive locking state
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add comments to explain the change
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-30 16:20:09 +08:00
0cc1c5239d
Remove old replica selector ( #1265 )
...
* remove EnableReplicaSelectorV2 config
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* remove replicaSelectorV1
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add replica flag to reduce struct size
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* remove contextPatcher
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* use option.preferLeader
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-27 17:11:02 +08:00
31a8ddab19
Support actively requesting update health feedback information by calling RPC to TiKV ( #1287 )
...
* Support actively requesting update health feedback information by calling RPC to TiKV
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto; follow the protocol change; and add more comments
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix build
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add integration tests
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* tune the frequency of active updating
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Update kvproto
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Fix test
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Revert changes to examples
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove unnecessary empty line
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-05-27 14:36:27 +08:00
1c06c5cbf5
upgrade grpc 1.64 ( #1346 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-05-17 12:20:15 +08:00
38e0dca30c
*: fix panic in get cause error ( #1344 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-16 11:19:22 +08:00
c40432e3ab
feat: record flush_wait_ms in TxnInfo ( #1342 )
...
* feat: record flush_wait_ms in TxnInfo
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: change FlushWaitDuration() to GetFlushMetrics()
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-05-15 11:13:15 +08:00
e1ca512cca
refine error msg ( #1338 )
...
* refine error msg
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-13 21:49:13 +08:00
c82e921992
fix issue of doesn't fast fail request when no available connections ( #1339 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-05-13 19:07:04 +08:00
6cb0704fce
fail pipelined dml when max ttl exceeded ( #1329 )
...
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-04-30 22:52:41 +08:00
52c232be3d
Update the PD client to adopt the latest optimization ( #1326 )
2024-04-30 16:32:54 +08:00
31a502b9ba
improve region request log for diagnose ( #1300 )
...
* improve region request log for diagnose
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* rename struct
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine region error metric with store id label and add rpc error metric
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* restrict log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine code
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* fix test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* address comment
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* refine log
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-29 15:56:32 +08:00
9c05078660
Add keyspace meta in codec ( #1323 )
...
Signed-off-by: y_static_y@sina.com <y_static_y@sina.com>
2024-04-28 12:20:55 +08:00
3ed6550f3e
build(deps): bump golang.org/x/net in /examples/txnkv/pessimistic_txn ( #1313 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 12:59:22 +08:00
d2b97619cd
build(deps): bump golang.org/x/net in /examples/txnkv/unsafedestoryrange ( #1312 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 12:59:17 +08:00
7779094ec9
build(deps): bump golang.org/x/net in /examples/rawkv ( #1311 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 12:58:59 +08:00
ced64f9652
build(deps): bump golang.org/x/net in /examples/txnkv/1pc_txn ( #1310 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: disksing <i@disksing.com>
2024-04-26 11:59:22 +08:00
9037f3a612
build(deps): bump golang.org/x/net in /examples/gcworker ( #1309 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 11:42:31 +08:00
86d0a00441
build(deps): bump golang.org/x/net in /examples/txnkv/delete_range ( #1308 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2024-04-26 11:42:17 +08:00
5fbbbe4a77
build(deps): bump golang.org/x/net in /examples/txnkv/async_commit ( #1307 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: disksing <i@disksing.com>
2024-04-26 11:41:57 +08:00
8a03f8d6b1
build(deps): bump golang.org/x/net in /examples/txnkv ( #1306 )
...
Bumps [golang.org/x/net](https://github.com/golang/net ) from 0.22.0 to 0.23.0.
- [Commits](https://github.com/golang/net/compare/v0.22.0...v0.23.0 )
---
updated-dependencies:
- dependency-name: golang.org/x/net
dependency-type: indirect
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: disksing <i@disksing.com>
2024-04-26 11:41:28 +08:00
1e1fe07b05
*: upgrade grpc 1.63.2 ( #1289 )
...
Signed-off-by: Weizhen Wang <wangweizhen@pingcap.com>
2024-04-26 10:56:09 +08:00
8635acc840
feat: return error if commit ts expired is returned for a non-primary key ( #1316 )
...
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-04-24 19:16:26 +08:00
0229f4077f
do not cache snapshot with version maxuint64 ( #1320 )
...
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-24 13:23:42 +08:00
edba2544d5
txn: abort pipelined dml when pk rollback ( #1317 )
...
* fail p-dml when ttl manager is closed
Signed-off-by: you06 <you1474600@gmail.com>
* close ttl manager when pk is rollbcked
Signed-off-by: you06 <you1474600@gmail.com>
* stablize test
Signed-off-by: you06 <you1474600@gmail.com>
* add test
Signed-off-by: you06 <you1474600@gmail.com>
* fix race
Signed-off-by: you06 <you1474600@gmail.com>
* fix leak in test
Signed-off-by: you06 <you1474600@gmail.com>
* remote testlog
Signed-off-by: you06 <you1474600@gmail.com>
* check error message
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-23 17:36:44 +08:00
059938f7eb
reduce log ( #1304 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-19 17:26:33 +08:00
cf37e5e03e
skip TestReplicaReadAccessPathByGenError because it's unstable and slow ( #1302 )
...
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-19 15:24:53 +08:00
36c8d2c668
region_cache: extract store related fields to store cache ( #1279 )
...
Signed-off-by: zyguan <zhongyangguan@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-04-18 21:37:35 +08:00
09b120cdf7
region_cache: fix issue that LocateKey may returns a wrong region ( #1299 )
...
* region_cache: fix issue that LocateKey may returns a wrong region
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* address the comment
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-04-18 20:14:18 +08:00
e02b025475
log: only log the first lock in every flush or prewrite response ( #1295 )
...
* log: only log the first lock in every flush or prewrite response
Signed-off-by: ekexium <eke@fastmail.com>
* log each conflicting txn at most once
Signed-off-by: ekexium <eke@fastmail.com>
* log: ttlManager
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-18 16:47:21 +08:00
5b3e266723
log: log the resolve progress of pipelined-dml every 30s ( #1291 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-18 15:51:40 +08:00
47891a931f
feat: stop flush when first error is met, like for prewrite ( #1296 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-18 15:40:37 +08:00
713ceee265
only set MinSafeTS to 0 when all stores' are 0 ( #1284 )
...
* only set MinSafeTS to 0 when all stores' are 0
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* try to make ut stable
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-04-18 14:45:56 +08:00
f9591044f1
reverts PR #988 to fix transaction is undetermined cause by region invalid error ( #1298 )
...
* Revert "dco (#988 )"
This reverts commit 4c2ae43454
2024-04-16 21:44:58 +08:00
4183ab10fa
Use Mem() instead of Size() to estimate pipelined-memdb size ( #1286 )
...
* fix: use Mem() instead of Size() to evaluate pipelined-memdb size, for better memory control
Signed-off-by: ekexium <eke@fastmail.com>
* test: fix TestPipelinedFlushTrigger
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-15 16:53:35 +08:00
a23f6cac0c
log: silence schema check log ( #1290 )
...
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-04-15 15:08:02 +08:00
dcb9b95307
txn: cache result of snapshot.Get ( #1278 )
...
* also cache result of snapshot.Get
Signed-off-by: you06 <you1474600@gmail.com>
* fix test
Signed-off-by: you06 <you1474600@gmail.com>
fix test
Signed-off-by: you06 <you1474600@gmail.com>
* add Get cache test
Signed-off-by: you06 <you1474600@gmail.com>
* move RLock position
Signed-off-by: you06 <you1474600@gmail.com>
* remove comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-04-13 16:45:31 +08:00
714958ccd4
implement rollback for pipelined dml ( #1235 )
...
* impl rollback for p-dml
Signed-off-by: you06 <you1474600@gmail.com>
* add test to ensure the rollback locks cannot be read
Signed-off-by: you06 <you1474600@gmail.com>
* remove TODO comment
Signed-off-by: you06 <you1474600@gmail.com>
* address comment
Signed-off-by: you06 <you1474600@gmail.com>
address comment
Signed-off-by: you06 <you1474600@gmail.com>
* resolve conflict
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
Co-authored-by: ekexium <eke@fastmail.com>
2024-04-09 10:27:18 +08:00
6906de0612
avoid calling setMinSafeTS with max u64 ( #1277 )
...
* avoid calling setMinSafeTS with max u64
Signed-off-by: zyguan <zhongyangguan@gmail.com>
* fix lint issues
Signed-off-by: zyguan <zhongyangguan@gmail.com>
---------
Signed-off-by: zyguan <zhongyangguan@gmail.com>
2024-04-09 09:27:09 +08:00
f800b05581
Distinguish identifier and name in task runner ( #1271 )
...
* refactor: distinguish identifier and name in task runner, to avoid messy metrics
Signed-off-by: ekexium <eke@fastmail.com>
* keep the original interface to avoid breaking compatibility
Signed-off-by: ekexium <eke@fastmail.com>
* log: use ctx logger
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-08 19:35:52 +08:00
7c70c54016
Allow setting session id of KVTxn ( #1270 )
...
* allow setting session id
Signed-off-by: ekexium <eke@fastmail.com>
* comment: explain usage
Signed-off-by: ekexium <eke@fastmail.com>
* try using ctx logger as much as possible
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-04-08 16:03:59 +08:00
2bd95773ce
fix bug of not invalidate store when tikv store is down in replica-selector-v2 with enable forwarding ( #1273 )
...
* fix bug of enable-forwarding
Signed-off-by: crazycs520 <crazycs520@gmail.com>
* add test
Signed-off-by: crazycs520 <crazycs520@gmail.com>
---------
Signed-off-by: crazycs520 <crazycs520@gmail.com>
2024-04-08 14:51:17 +08:00
642a09bef1
Make slow store filtering the highest priority in replica selector v2 ( #1267 )
...
* Add some logs
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Make slow store filtering the highest priority in replica selector v2
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Add non stale read case to TestMultiReplicaInOneAZ
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Enrich the multi replcia in one AZ case but it failed...
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* update test to adapt the fix on master branch
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
* Remove TestMultiReplicaInOneAZ
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
---------
Signed-off-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
Co-authored-by: MyonKeminta <MyonKeminta@users.noreply.github.com>
2024-04-08 14:23:38 +08:00
crazycs