Commit Graph

7 Commits

Author SHA1 Message Date
you06 41d133b6b6
membuffer: refactor the memdb to support multi implementations (#1426)
ref pingcap/tidb#55287

Signed-off-by: you06 <you1474600@gmail.com>
2024-08-23 03:46:43 +00:00
you06 edba2544d5
txn: abort pipelined dml when pk rollback (#1317)
* fail p-dml when ttl manager is closed

Signed-off-by: you06 <you1474600@gmail.com>

* close ttl manager when pk is rollbcked

Signed-off-by: you06 <you1474600@gmail.com>

* stablize test

Signed-off-by: you06 <you1474600@gmail.com>

* add test

Signed-off-by: you06 <you1474600@gmail.com>

* fix race

Signed-off-by: you06 <you1474600@gmail.com>

* fix leak in test

Signed-off-by: you06 <you1474600@gmail.com>

* remote testlog

Signed-off-by: you06 <you1474600@gmail.com>

* check error message

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-04-23 17:36:44 +08:00
ekexium 4183ab10fa
Use Mem() instead of Size() to estimate pipelined-memdb size (#1286)
* fix: use Mem() instead of Size() to evaluate pipelined-memdb size, for better memory control

Signed-off-by: ekexium <eke@fastmail.com>

* test: fix TestPipelinedFlushTrigger

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
2024-04-15 16:53:35 +08:00
ekexium 58ef395164
test: restore skipped test for pipelined dml (#1238)
* test: restore skipped test

Signed-off-by: ekexium <eke@fastmail.com>

* test: FlushWait() to avoid race

Signed-off-by: ekexium <eke@fastmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
2024-03-19 20:37:44 +08:00
ekexium 87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB (#1212)
* feat: introduce GetLocal() for MemBuffer

Signed-off-by: ekexium <eke@fastmail.com>

* downgrade tools to 0.18.0

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set flushingMemDB = nil when an error is returned from a flush

Signed-off-by: ekexium <eke@fastmail.com>

* impl BatchGet for MemBuffer

Signed-off-by: you06 <you1474600@gmail.com>

test membuffer batch get

Signed-off-by: you06 <you1474600@gmail.com>

* add Prefetch & GetPrefetchCache

Signed-off-by: you06 <you1474600@gmail.com>

* cache multi Prefetch call

Signed-off-by: you06 <you1474600@gmail.com>

* add tests

Signed-off-by: you06 <you1474600@gmail.com>

* workaround for golang ci lint failure

Signed-off-by: you06 <you1474600@gmail.com>

* replace assert with require

Signed-off-by: you06 <you1474600@gmail.com>

* workaround golangci lint

Signed-off-by: you06 <you1474600@gmail.com>

* lint

Signed-off-by: you06 <you1474600@gmail.com>

* remove prefetch interface, pipelined memdb will cache batch get result

Signed-off-by: you06 <you1474600@gmail.com>

* update tidb

Signed-off-by: you06 <you1474600@gmail.com>

* fix batch get cache when membuffer is empty

Signed-off-by: you06 <you1474600@gmail.com>

* fix returned delete value

Signed-off-by: you06 <you1474600@gmail.com>

* fix: handle resourceGroupTag of Flush

Signed-off-by: ekexium <eke@fastmail.com>

* fix: set resource group tag for committer if it's pipelined

Signed-off-by: ekexium <eke@fastmail.com>

* Update internal/unionstore/pipelined_memdb.go

Co-authored-by: ekexium <eke@fastmail.com>

* remove prefetch interface

Signed-off-by: you06 <you1474600@gmail.com>

* release mutex

Signed-off-by: you06 <you1474600@gmail.com>

* flush wait to avoid race

Signed-off-by: you06 <you1474600@gmail.com>

* fix unstopped test

Signed-off-by: you06 <you1474600@gmail.com>

* set resource group tags for committer

Signed-off-by: ekexium <eke@fastmail.com>

* skip test due to tikv image not updated yet

Signed-off-by: you06 <you1474600@gmail.com>

* skip more test

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 11:36:05 +08:00
you06 c72236a601
adjust parameter for pipelined memdb by failpoint (#1205)
* adjust parameter for pipelined memdb by failpoint

Signed-off-by: you06 <you1474600@gmail.com>

* update comment

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-06 16:44:48 +08:00
you06 8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML (#1114)
* Support pipelined memdb.

Signed-off-by: you06 <you1474600@gmail.com>

* fix mutex for pipelined memdb

Signed-off-by: you06 <you1474600@gmail.com>

---------

Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00