mirror of https://github.com/tikv/website.git
Merge pull request #329 from haojinming/tikv-checksum-doc
Add rawkv checksum doc
This commit is contained in:
Ping Yu
GitHub
commit
b0833b33c4
|
|
@ -154,9 +154,10 @@ Explanations for the above message are as follows:
|
|||
|
||||
### Data Verification of Backup & Restore
|
||||
|
||||
TiKV-BR can do checksum between TiKV cluster and backup files after backup or restoration finishes with the config `--checksum=true`. Checksum is using the [checksum](https://github.com/tikv/client-go/blob/ffaaf7131a8df6ab4e858bf27e39cd7445cf7929/rawkv/rawkv.go#L584) interface in TiKV [client-go](https://github.com/tikv/client-go), which send checksum request to all TiKV regions to calculate the checksum of all **VALID** data. Then compare the checksum value of backup files which is calculated during the backup process.
|
||||
|
||||
Please note that if data is stored in TiKV with [TTL](../ttl), and expiration happens during backup or restore, the persisted checksum in backup files must be different from the checksum of TiKV cluster. So checksum should **NOT** be enabled in this scenario. To verify the correctness of backup and restoration in this scenario, you can perform a full comparison for all existing non-expired data between backup cluster and restore cluster by using [scan](../../../develop/rawkv/scan) interface..
|
||||
TiKV-BR can do checksum between TiKV cluster and backup files after backup or restoration finishes with the config `--checksum=true`. Checksum is using the [checksum](../../../develop/rawkv/checksum) interface in TiKV [client-go](https://github.com/tikv/client-go), which send checksum request to all TiKV regions to calculate the checksum of all **VALID** data. Then compare the checksum value of backup files which is calculated during the backup process.
|
||||
|
||||
Please note that if data is stored in TiKV with [TTL](../ttl), and expiration happens during backup or restore, the persisted checksum in backup files must be different from the checksum of TiKV cluster. So checksum should **NOT** be enabled in this scenario. To verify the correctness of backup and restoration in this scenario, you can perform a full comparison for all existing non-expired data between backup cluster and restore cluster by using [scan](../../../develop/rawkv/scan) interface.
|
||||
|
||||
### Security During Backup & Restoration
|
||||
|
||||
|
|
|
|||
|
|
@ -0,0 +1,74 @@
|
|||
---
|
||||
title: Checksum
|
||||
description: Learn how to use RawKV's Checksum API.
|
||||
menu:
|
||||
"dev":
|
||||
parent: RawKV-dev
|
||||
weight: 5
|
||||
identifier: Checksum-dev
|
||||
---
|
||||
|
||||
This document walks you through how to use RawKV's `Checksum` API.
|
||||
|
||||
`Checksum` API returns `Crc64Xor`, `TotalKvs` and `TotalBytes` from TiKV cluster.
|
||||
- `Crc64Xor`: The [XOR](https://en.wikipedia.org/wiki/Exclusive_or) of every key-value pair's [crc64](https://en.wikipedia.org/wiki/Cyclic_redundancy_check) value.
|
||||
- `TotalKVs`: The number of key-value pairs.
|
||||
- `TotalBytes`: The size of key-value pairs in bytes.
|
||||
|
||||
*Note: If [API V2](../../../concepts/explore-tikv-features/api-v2) is enabled, a `4` bytes prefix is encoded with keys, and also calculated by `Checksum` API*.
|
||||
|
||||
## Go
|
||||
|
||||
### Checksum with range
|
||||
|
||||
Using the `Checksum` API, you can get `{Crc64Xor, TotalKvs, TotalBytes}` of a range from `startKey` (inclusive) to `endKey` (exclusive).
|
||||
|
||||
{{< info >}}
|
||||
To calculate checksum of all keys, specify `startKey` and `endKey` as `[]byte("")`.
|
||||
|
||||
{{< /info >}}
|
||||
|
||||
```go
|
||||
package main
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
|
||||
"github.com/pingcap/kvproto/pkg/kvrpcpb"
|
||||
"github.com/tikv/client-go/v2/rawkv"
|
||||
)
|
||||
|
||||
func main() {
|
||||
ctx := context.TODO()
|
||||
cli, err := rawkv.NewClientWithOpts(ctx, []string{"127.0.0.1:2379"},
|
||||
rawkv.WithAPIVersion(kvrpcpb.APIVersion_V2))
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
defer cli.Close()
|
||||
|
||||
fmt.Printf("Cluster ID: %d\n", cli.ClusterID())
|
||||
|
||||
// put key into tikv
|
||||
cli.Put(ctx, []byte("k1"), []byte("v1"))
|
||||
cli.Put(ctx, []byte("k2"), []byte("v2"))
|
||||
cli.Put(ctx, []byte("k3"), []byte("v3"))
|
||||
cli.Put(ctx, []byte("k4"), []byte("v4"))
|
||||
cli.Put(ctx, []byte("k5"), []byte("v5"))
|
||||
|
||||
checksum, err := cli.Checksum(ctx, []byte("k1"), []byte("k6"))
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
fmt.Printf("Get checksum, Crc64Xor:%d, TotalKvs:%d, TotalBytes:%d.\n",
|
||||
checksum.Crc64Xor, checksum.TotalKvs, checksum.TotalBytes)
|
||||
}
|
||||
```
|
||||
You will get the result as following:
|
||||
|
||||
```bash
|
||||
Cluster ID: 7166545317297238572
|
||||
Get checksum, Crc64Xor:7402990595130313958, TotalKvs:5, TotalBytes:40.
|
||||
```
|
||||
|
|
@ -14,5 +14,6 @@ TiKV supports both transactional (TxnKV) API and non-transactional (RawKV) API.
|
|||
- [Scan](../scan)
|
||||
- [Time to Live (TTL)](../ttl)
|
||||
- [Compare And Swap (CAS)](../cas)
|
||||
- [Checksum](../checksum)
|
||||
|
||||
To get the example code in this chapter, click [here](https://github.com/marsishandsome/tikv-client-examples).
|
||||
|
|
|
|||
Loading…
Reference in New Issue