183817ac81
memdb: introduce snapshot interface ( #1623 )
...
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: ekexium <eke@fastmail.com>
2025-04-15 14:17:37 +00:00
ddec8237fd
memdb: prevent iterator invalidation ( #1563 )
...
ref pingcap/tidb#59153
Signed-off-by: ekexium <eke@fastmail.com>
2025-02-19 08:02:06 +00:00
41d133b6b6
membuffer: refactor the memdb to support multi implementations ( #1426 )
...
ref pingcap/tidb#55287
Signed-off-by: you06 <you1474600@gmail.com>
2024-08-23 03:46:43 +00:00
aa8b33873f
Let memdb cache last traversed node ( #1389 )
...
* opt: let memdb cache last traversed node
Signed-off-by: ekexium <eke@fastmail.com>
* add a switch
Signed-off-by: ekexium <eke@fastmail.com>
* use atomic pointer
Signed-off-by: ekexium <eke@fastmail.com>
* fix: invalidate cache if the node is deleted
Signed-off-by: ekexium <eke@fastmail.com>
* feat: log flush_wait_duration (#1391 )
Signed-off-by: ekexium <eke@fastmail.com>
* feat: add cache metrics
Signed-off-by: ekexium <eke@fastmail.com>
* opt: parent node tracking in traverse
Signed-off-by: ekexium <eke@fastmail.com>
* opt: isNull
Signed-off-by: ekexium <eke@fastmail.com>
* log: silence verbose failpoint log
Signed-off-by: ekexium <eke@fastmail.com>
* metrics: track traverse duration
Signed-off-by: ekexium <eke@fastmail.com>
* Revert "metrics: track traverse duration"
This reverts commit 05fc82bad51711c5f0fadf7936e118ed2c4990ab.
Signed-off-by: ekexium <eke@fastmail.com>
* fix: flush wait
Signed-off-by: ekexium <eke@fastmail.com>
* fix: duration logs
Signed-off-by: ekexium <eke@fastmail.com>
* refine log message
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-08-08 17:20:52 +08:00
3ac46e8171
fix: duration logs ( #1405 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-07-31 13:12:27 +08:00
fce0abfd82
Let pipelined memdb ignore bufferLimit ( #1392 )
...
* fix: pipelined memdb ignores bufferLimit
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: introduce unlimitedSize
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: unlimitedSize
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-07-29 12:13:17 +08:00
ac8fa1d73a
feat: introduce a new method MemHookSet ( #1362 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-06-14 14:44:55 +08:00
c40432e3ab
feat: record flush_wait_ms in TxnInfo ( #1342 )
...
* feat: record flush_wait_ms in TxnInfo
Signed-off-by: ekexium <eke@fastmail.com>
* refactor: change FlushWaitDuration() to GetFlushMetrics()
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-05-15 11:13:15 +08:00
4183ab10fa
Use Mem() instead of Size() to estimate pipelined-memdb size ( #1286 )
...
* fix: use Mem() instead of Size() to evaluate pipelined-memdb size, for better memory control
Signed-off-by: ekexium <eke@fastmail.com>
* test: fix TestPipelinedFlushTrigger
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-04-15 16:53:35 +08:00
a9128e8200
fix: check mem != nil before calling Mem() ( #1236 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-03-18 14:55:17 +08:00
73c0712c01
txn: implement staging for PipelinedMemDB ( #1230 )
...
* feat: implement staging for PipelinedMemDB
Signed-off-by: ekexium <eke@fastmail.com>
* comment: explain staging methods
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
2024-03-18 14:25:05 +08:00
56418bf28f
feat: implement Mem() and SetMemoryFootprintChangeHook ( #1233 )
...
Signed-off-by: ekexium <eke@fastmail.com>
2024-03-18 11:02:04 +08:00
87a984a72d
feat: introduce GetLocal() and BatchGet cache for PipelinedMemDB ( #1212 )
...
* feat: introduce GetLocal() for MemBuffer
Signed-off-by: ekexium <eke@fastmail.com>
* downgrade tools to 0.18.0
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set flushingMemDB = nil when an error is returned from a flush
Signed-off-by: ekexium <eke@fastmail.com>
* impl BatchGet for MemBuffer
Signed-off-by: you06 <you1474600@gmail.com>
test membuffer batch get
Signed-off-by: you06 <you1474600@gmail.com>
* add Prefetch & GetPrefetchCache
Signed-off-by: you06 <you1474600@gmail.com>
* cache multi Prefetch call
Signed-off-by: you06 <you1474600@gmail.com>
* add tests
Signed-off-by: you06 <you1474600@gmail.com>
* workaround for golang ci lint failure
Signed-off-by: you06 <you1474600@gmail.com>
* replace assert with require
Signed-off-by: you06 <you1474600@gmail.com>
* workaround golangci lint
Signed-off-by: you06 <you1474600@gmail.com>
* lint
Signed-off-by: you06 <you1474600@gmail.com>
* remove prefetch interface, pipelined memdb will cache batch get result
Signed-off-by: you06 <you1474600@gmail.com>
* update tidb
Signed-off-by: you06 <you1474600@gmail.com>
* fix batch get cache when membuffer is empty
Signed-off-by: you06 <you1474600@gmail.com>
* fix returned delete value
Signed-off-by: you06 <you1474600@gmail.com>
* fix: handle resourceGroupTag of Flush
Signed-off-by: ekexium <eke@fastmail.com>
* fix: set resource group tag for committer if it's pipelined
Signed-off-by: ekexium <eke@fastmail.com>
* Update internal/unionstore/pipelined_memdb.go
Co-authored-by: ekexium <eke@fastmail.com>
* remove prefetch interface
Signed-off-by: you06 <you1474600@gmail.com>
* release mutex
Signed-off-by: you06 <you1474600@gmail.com>
* flush wait to avoid race
Signed-off-by: you06 <you1474600@gmail.com>
* fix unstopped test
Signed-off-by: you06 <you1474600@gmail.com>
* set resource group tags for committer
Signed-off-by: ekexium <eke@fastmail.com>
* skip test due to tikv image not updated yet
Signed-off-by: you06 <you1474600@gmail.com>
* skip more test
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: you06 <you1474600@gmail.com>
2024-03-16 11:36:05 +08:00
d59fea5757
P-DML: for AlreadyExist error, attach the value from flushingMemDB ( #1218 )
...
* feat: for AlreadyExist error during Flush, attach the value from flushingMemDB to it
Signed-off-by: ekexium <eke@fastmail.com>
* style: apply comment suggestion
Signed-off-by: ekexium <eke@fastmail.com>
---------
Signed-off-by: ekexium <eke@fastmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-13 10:23:20 +08:00
c72236a601
adjust parameter for pipelined memdb by failpoint ( #1205 )
...
* adjust parameter for pipelined memdb by failpoint
Signed-off-by: you06 <you1474600@gmail.com>
* update comment
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
Co-authored-by: cfzjywxk <lsswxrxr@163.com>
2024-03-06 16:44:48 +08:00
8d28d3cd3a
txn: introduce pipelined memdb, Flush and BufferBatchGet for pipelined DML ( #1114 )
...
* Support pipelined memdb.
Signed-off-by: you06 <you1474600@gmail.com>
* fix mutex for pipelined memdb
Signed-off-by: you06 <you1474600@gmail.com>
---------
Signed-off-by: you06 <you1474600@gmail.com>
2024-02-23 10:24:44 +08:00
you06