tikv
/
website
mirror of https://github.com/tikv/website.git
1
0
Fork 0
Code Issues Packages Projects Releases Wiki Activity

address review comments 

Signed-off-by: haojinming <jinming.hao@pingcap.com>
This commit is contained in:
haojinming 2022-12-02 11:04:20 +08:00
parent 700d859146
commit 3bb9359e18
3 changed files with 65 additions and 37 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
content/docs/dev/concepts/explore-tikv-features/br/backup-restore-cn.md → content/docs/dev/concepts/explore-tikv-features/backup-restore-cn.md
View File

@ -20,6 +20,19 @@ TiKV-BR 将备份或恢复操作命令下发到各个 TiKV 节点。TiKV 收到
caption="TiKV-BR 工作原理"
number="1" >}}
### 部署
#### 使用 TiUP 部署
`tikv-br` 是 [TiUP] 的一个组件，你可以使用 [TiUP] 来部署 `tikv-br`:
```bash
tiup tikv-br:v1.1.0 <命令> <子命令>
```
如果是第一次使用，TiUP 会自动下载和安装 `tikv-br` 组件。
#### 手工部署
你可以从 [GitHub] 上下载 `tikv-br` 的最新发行版。
### 推荐部署配置
- 生产环境中，推荐 TiKV-BR 运行在（4 核+/8 GB+）的节点上。操作系统版本要求可参考 [操作系统及平台的要求]。
@ -37,7 +50,7 @@ TiKV-BR 将备份或恢复操作命令下发到各个 TiKV 节点。TiKV 收到
- TiKV-BR 备份最好串行执行。不同备份任务并行会导致备份性能降低，同时也会影响在线业务。
- TiKV-BR 恢复最好串行执行。不同恢复任务并行会导致 Region 冲突增多，恢复的性能降低。
- 可以通过指定 `--checksum=true`，在备份、恢复完成后进行一轮数据校验。数据校验将分别计算备份数据与 TiKV 集群中数据的 checksum，并对比二者是否相同。请注意，如果需要进行数据校验，请确保在备份或恢复的全过程，TiKV 集群没有数据变更和 TTL 过期。
- TiKV-BR 可用于实现 [`api-version`] 从 V1 到 V2 的集群数据迁移。通过指定 `--dst-api-version V2` 将 `api-version=1` 的 TiKV 集群备份为 V2 格式，然后将备份文件恢复到新的 `api-version=2` TiKV 集群中。
- TiKV-BR 可用于实现 [api-version] 从 V1 到 V2 的集群数据迁移。通过指定 `--dst-api-version V2` 将 `api-version=1` 的 TiKV 集群备份为 V2 格式，然后将备份文件恢复到新的 `api-version=2` TiKV 集群中。
### TiKV-BR 命令行描述
一条 `tikv-br` 命令是由子命令、选项和参数组成的。子命令即不带 `-` 或者 `--` 的字符。选项即以 `-` 或者 `--` 开头的字符。参数即子命令或选项字符后紧跟的、并传递给命令和选项的字符。
@ -61,15 +74,15 @@ tikv-br backup raw \
- `backup`：`tikv-br` 的子命令。
- `raw`：`backup` 的子命令。
- `-s` 或 `--storage`：备份保存的路径。
- `"s3://backup-data/2022-09-16/"`：`--storage` 的参数，保存的路径为各个 TiKV 节点本地磁盘的 s3 的 `/backup-data/2022-09-16/` 目录。
- `"s3://backup-data/2022-09-16/"`：`--storage` 的参数，保存的路径为 S3 的 `/backup-data/2022-09-16/` 目录。
- `--pd`：`PD` 服务地址。
- `"${PDIP}:2379"`：`--pd` 的参数。
- `--dst-api-version`: 指定备份文件的 `api-version`，请见 [tikv-server 配置文件]。
- `--dst-api-version`: 指定备份文件的 `api-version`，请见 [TiKV API v2]。
- `v2`: `--dst-api-version` 的参数，可选参数为 `v1`，`v1ttl`，`v2`(不区分大小写)，如果不指定 `dst-api-version` 参数，则备份文件的 `api-version` 与指定 `--pd` 所属的 TiKV 集群 `api-version` 相同。
- `gcttl`: GC 暂停时长。可用于确保从存量数据备份到 [创建 TiKV-CDC 同步任务] 的这段时间内，增量数据不会被 GC 清除。默认为 5 分钟。
- `5m`: `gcttl` 的参数，数据格式为`数字 + 时间单位`, 例如 `24h` 表示 24 小时，`60m` 表示 60 分钟。
- `start`, `end`: 用于指定需要备份的数据区间，为左闭右开区间 `[start, end)`。默认为`["", "")`， 即全部数据。
- `format`：`start` 和 `end` 的格式，支持 `raw`、[`hex`] 和 [`escaped`] 三种格式。
- `format`：`start` 和 `end` 的格式，支持 `raw`、[hex] 和 [escaped] 三种格式。
备份期间会有进度条在终端中显示，当进度条前进到 100% 时，说明备份已完成。
@ -124,23 +137,23 @@ tikv-br restore raw \
TiKV-BR 可以在 TiKV 集群备份和恢复操作完成后执行 `checksum` 来确保备份文件的完整性和正确性。 checksum 可以通过 `--checksum` 来开启。
checksum 开启时，备份或恢复操作完成后，会使用 [client-go] 的 [checksum] 接口来计算 TiKV 集群中有效数据的 checksum 结果，并与备份文件保存的 checksum 结果进行对比。
Checksum 开启时，备份或恢复操作完成后，会使用 [client-go] 的 [checksum] 接口来计算 TiKV 集群中有效数据的 checksum 结果，并与备份文件保存的 checksum 结果进行对比。
在某些场景中，TiKV 集群中的数据具有 [TTL] 属性，如果在备份和恢复过程中，数据的 TTL 过期，此时 TiKV 集群的 checksum 结果跟备份文件的 checksum 会不相同，因此不建议在此场景中开启 `checksum`。客户可以选择使用 [client-go] 的 [scan] 接口，在恢复操作完成后扫描出需要校验的数据，来确保备份文件的正确性。
注意，当 TiKV 集群启用了 [TTL]，如果在备份或恢复过程中出现数据 TTL 过期，此时 TiKV 集群的 checksum 结果跟备份文件的 checksum 会不相同，因此无法在此场景中使用 `checksum`。
### 备份恢复操作的安全性
TiKV-BR 支持在开启了 [TLS 配置] 的 TiKV 集群中执行备份和恢复操作，用户可以通过设置 `--ca`， `--cert` 和 `--key` 参数来指定客户端证书。
TiKV-BR 支持在开启了 [TLS 配置] 的 TiKV 集群中执行备份和恢复操作，你可以通过设置 `--ca`， `--cert` 和 `--key` 参数来指定客户端证书。
### 性能
TiKV-BR 的备份和恢复都是分布式的，因此在存储和网络没有达到瓶颈的时候，性能可以随着 TiKV 节点的增长而增长。下面提供了 TiKV-BR 的关键性能指标供参考。
TiKV-BR 的备份和恢复都是分布式的，在存储和网络没有达到瓶颈的时候，性能可以随着 TiKV 节点数量线性提升。以下是 TiKV-BR 的性能测试指标，以供参考。
- TiKV 节点：4 核 CPU， 8G 内存，v6.4.0
- PD 节点：4 核 CPU， 8G 内存，v6.4.0
- TiKV-BR 节点：4 核 CPU， 8G 内存，v1.1.0
- 存储容量： 50TB
|指标|TiKV API V1|TiKV API V2|
|指标|TiKV API V1|TiKV API v2|
|:-:|:-:|:-:|
|备份速度|每 TiKV 节点 40MB/s|每 TiKV 节点 40MB/s|
|恢复速度|每 TiKV 节点 70MB/s|每 TiKV 节点 70MB/s|
@ -148,20 +161,22 @@ TiKV-BR 的备份和恢复都是分布式的，因此在存储和网络没有达
#### 性能调优
如果你希望减少备份对集群的影响，你可以开启[自动调节]功能。开启该功能后，备份功能会在不过度影响集群的前提下，以最快的速度进行数据备份。
或者，你也可以使用参数 `--ratelimit` 进行备份限速。
如果你希望减少备份对集群的影响，你可以开启 [auto-tune] 功能。开启该功能后，备份功能会在不过度影响集群正常业务的前提下，以最快的速度进行数据备份。详见[自动调节]。
或者你也可以使用参数 `--ratelimit` 进行备份限速。
[TiKV Backup & Restore (TiKV-BR)]: https://github.com/tikv/migration/tree/main/br
[TiUP]: https://tiup.io
[GitHub]: https://github.com/tikv/migration/releases
[操作系统及平台的要求]: https://docs.pingcap.com/zh/tidb/dev/hardware-and-software-requirements
[`api-version`]: https://docs.pingcap.com/zh/tidb/stable/tikv-configuration-file#api-version-%E4%BB%8E-v610-%E7%89%88%E6%9C%AC%E5%BC%80%E5%A7%8B%E5%BC%95%E5%85%A5
[tikv-server 配置文件]: https://docs.pingcap.com/zh/tidb/stable/tikv-configuration-file#api-version-%E4%BB%8E-v610-%E7%89%88%E6%9C%AC%E5%BC%80%E5%A7%8B%E5%BC%95%E5%85%A5
[创建 TiKV-CDC 同步任务]: ../../cdc/cdc-cn/#%E7%AE%A1%E7%90%86%E5%90%8C%E6%AD%A5%E4%BB%BB%E5%8A%A1-changefeed
[`hex`]: https://zh.wikipedia.org/wiki/%E5%8D%81%E5%85%AD%E8%BF%9B%E5%88%B6
[`escaped`]: https://zh.wikipedia.org/wiki/%E8%BD%AC%E4%B9%89%E5%AD%97%E7%AC%A6
[checksum]: ../../../../develop/rawkv/checksum
[api-version]: ../api-v2
[TiKV API v2]: ../api-v2
[创建 TiKV-CDC 同步任务]: ../cdc/cdc-cn/#%E7%AE%A1%E7%90%86%E5%90%8C%E6%AD%A5%E4%BB%BB%E5%8A%A1-changefeed
[hex]: https://zh.wikipedia.org/wiki/%E5%8D%81%E5%85%AD%E8%BF%9B%E5%88%B6
[escaped]: https://zh.wikipedia.org/wiki/%E8%BD%AC%E4%B9%89%E5%AD%97%E7%AC%A6
[checksum]: ../../../develop/rawkv/checksum
[client-go]: https://github.com/tikv/client-go
[TTL]: ../../ttl
[scan]: ../../../../develop/rawkv/scan
[TTL]: ../ttl
[TLS 配置]: https://docs.pingcap.com/zh/tidb/dev/enable-tls-between-components
[auto-tune]: https://docs.pingcap.com/zh/tidb/stable/tikv-configuration-file#enable-auto-tune-%E4%BB%8E-v54-%E7%89%88%E6%9C%AC%E5%BC%80%E5%A7%8B%E5%BC%95%E5%85%A5
[自动调节]: https://docs.pingcap.com/zh/tidb/dev/br-auto-tune

47
content/docs/dev/concepts/explore-tikv-features/br/backup-restore.md → content/docs/dev/concepts/explore-tikv-features/backup-restore.md
View File

@ -23,6 +23,18 @@ This page introduces what's RawKV BR and how to use it.
caption="TiKV-BR architecture"
number="1" >}}
### Deployment
#### Deploy by TiUP
`tikv-br` is a component of [TiUP], so you can easily use `tikv-br` with [TiUP] as following:
```bash
tiup tikv-br:v1.1.0 <command> <subcommand>
```
If `tikv-br` is not installed before, TiUP will download it automatically.
#### Deploy manually
Please found the latest release and download the binary from [GitHub].
### Recommended Deployment Configuration
- In production environments, deploy `TiKV-BR` on a node with at least 4 cores CPU and 8 GB memory. Select an appropriate OS version by following [Linux OS version requirements].
@ -44,7 +56,7 @@ The following are some recommended operations for using `TiKV-BR` for backup and
- It is recommended that you execute multiple backup operations serially. Running different backup operations in parallel reduces backup performance and also affects the online application.
- It is recommended that you execute multiple restoration operations serially. Running different restoration operations in parallel increases Region conflicts and also reduces restoration performance.
- `TiKV-BR` supports checksum between `TiKV` cluster and backup files after backup or restoration with the config `--checksum=true`. Note that, if checksum is enabled, please make sure no data is changed or `TTL` expired in `TiKV` cluster during backup or restoration.
- TiKV-BR supports [`api-version`] conversion from V1 to V2 with config `--dst-api-version=V2`. Then restore the backup files to API V2 `TiKV` cluster. This is mainly used to upgrade from API V1 cluster to API V2 cluster.
- TiKV-BR supports [api-version] conversion from V1 to V2 with config `--dst-api-version=V2`. Then restore the backup files to API v2 `TiKV` cluster. This is mainly used to upgrade from API V1 cluster to API v2 cluster.
### TiKV-BR Command Line Description
A tikv-br command consists of sub-commands, options, and parameters.
@ -79,12 +91,12 @@ Explanations for some options in the above command are as follows:
- `128`: The value of `ratelimit`, unit is MiB/s.
- `--pd`: Service address of `PD`.
- `"${PDIP}:2379"`: Parameter of `--pd`.
- `--dst-api-version`: The `api-version`, please see [API V2].
- `--dst-api-version`: The `api-version`, please see [API v2].
- `v2`: Parameter of `--dst-api-version`, the optionals are `v1`, `v1ttl`, `v2`(Case insensitive). If no `dst-api-version` is specified, the `api-version` is the same with TiKV cluster of `--pd`.
- `gcttl`: The pause duration of GC. This can be used to make sure that the incremental data from the beginning of backup to [Create a replication task] will NOT be deleted by GC. 5 minutes by default.
- `gcttl`: The pause duration of GC. This can be used to make sure that the incremental data from the beginning of backup to [Creating a replication task] will NOT be deleted by GC. 5 minutes by default.
- `5m`: Parameter of `gcttl`. Its format is `number + unit`, e.g. `24h` means 24 hours, `60m` means 60 minutes.
- `start`, `end`: The backup key range. It's closed left and open right `[start, end)`.
- `format`: Format of `start` and `end`. Supported formats are `raw`、[`hex`] and [`escaped`].
- `format`: Format of `start` and `end`. Supported formats are `raw`、[hex] and [escaped].
A progress bar is displayed in the terminal during the backup. When the progress bar advances to 100%, the backup is complete. The progress bar is displayed as follows:
```bash
@ -105,7 +117,7 @@ Explanations for the above message are as follows:
- `ranges-failed`: Number of failed ranges.
- `backup-total-regions`: The tikv regions that backup takes.
- `total-take`: The backup duration.
- `backup-ts`: The backup start timestamp, only takes effect for API V2 TiKV cluster, which can be used as `start-ts` of `TiKV-CDC` when creating replication tasks. Refer to [Create a replication task].
- `backup-ts`: The backup start timestamp, only takes effect for API v2 TiKV cluster, which can be used as `start-ts` of `TiKV-CDC` when creating replication tasks. Refer to [Creating a replication task].
- `total-kv`: Total number of key-value pairs in backup files.
- `total-kv-size`: Total size of key-value pairs in backup files. Note that this is the original size before compression.
- `average-speed`: The backup speed, which approximately equals to `total-kv-size` / `total-take`.
@ -161,7 +173,7 @@ Explanations for the above message are as follows:
TiKV-BR can do checksum between TiKV cluster and backup files after backup or restoration finishes with the config `--checksum=true`. Checksum is using the [checksum] interface in TiKV [client-go], which send checksum request to all TiKV regions to calculate the checksum of all **VALID** data. Then compare the checksum value of backup files which is calculated during the backup process.
Please note that if data is stored in TiKV with [TTL], and expiration happens during backup or restore, the persisted checksum in backup files must be different from the checksum of TiKV cluster. So checksum should **NOT** be enabled in this scenario. To verify the correctness of backup and restoration in this scenario, you can perform a full comparison for all existing non-expired data between backup cluster and restore cluster by using [scan] interface.
Please note that if data is stored in TiKV with [TTL], and expiration happens during backup or restore, the persisted checksum in backup files must be different from the checksum of TiKV cluster. So checksum should **NOT** be enabled in this scenario.
### Security During Backup & Restoration
@ -177,7 +189,7 @@ The backup and restoration are both distributed, so performance can increase lin
- TiKV-BR node: 4 cores CPU, 8 GB memory, v1.1.0
- Data volume: 50 TB
|Metric|TiKV API V1|TiKV API V2|
|Metric|TiKV API V1|TiKV API v2|
|:-:|:-:|:-:|
|Backup speed|40MB/s per TiKV node|40MB/s per TiKV node|
|Restoration speed|70MB/s per TiKV node|70MB/s per TiKV node|
@ -185,7 +197,7 @@ The backup and restoration are both distributed, so performance can increase lin
#### Performance tuning
If you want to reduce the business impact of backup tasks, you can enable the [`auto-tune`] feature. With this feature enabled, TiKV-BR performs backup tasks as fast as possible without excessively affecting the cluster. See [BR Auto-Tune] for details.
If you want to reduce the business impact of backup tasks, you can enable the [auto-tune] feature. With this feature enabled, TiKV-BR performs backup tasks as fast as possible without excessively affecting the cluster. See [BR Auto-Tune] for details.
Alternatively, you can limit the backup speed by using the parameter `--ratelimit` of tikv-br.
@ -193,15 +205,16 @@ Alternatively, you can limit the backup speed by using the parameter `--ratelimi
[中文使用手册]: ../backup-restore-cn
[TiKV Backup & Restore (TiKV-BR)]: https://github.com/tikv/migration/tree/main/br
[Linux OS version requirements]: https://docs.pingcap.com/tidb/dev/hardware-and-software-requirements#linux-os-version-requirements
[`api-version`]: https://docs.pingcap.com/tidb/stable/tikv-configuration-file#api-version-new-in-v610
[API V2]: ../../api-v2
[`hex`]: https://en.wikipedia.org/wiki/Hexadecimal
[`escaped`]: https://en.wikipedia.org/wiki/Escape_character
[Create a replication task]: ../../cdc/cdc#manage-replication-tasks-changefeed
[checksum]: ../../../../develop/rawkv/checksum
[TiUP]: https://tiup.io
[GitHub]: https://github.com/tikv/migration/releases
[api-version]: ../api-v2
[API v2]: ../api-v2
[hex]: https://en.wikipedia.org/wiki/Hexadecimal
[escaped]: https://en.wikipedia.org/wiki/Escape_character
[Creating a replication task]: ../cdc/cdc#manage-replication-tasks-changefeed
[checksum]: ../../../develop/rawkv/checksum
[client-go]: https://github.com/tikv/client-go
[TTL]: ../../ttl
[scan]: ../../../../develop/rawkv/scan
[TTL]: ../ttl
[TLS config]: https://docs.pingcap.com/tidb/dev/enable-tls-between-components
[`auto-tune`]: https://docs.pingcap.com/tidb/stable/tikv-configuration-file#enable-auto-tune-new-in-v540
[auto-tune]: https://docs.pingcap.com/tidb/stable/tikv-configuration-file#enable-auto-tune-new-in-v540
[BR Auto-Tune]: https://docs.pingcap.com/tidb/dev/br-auto-tune

2
content/docs/dev/concepts/explore-tikv-features/overview.md
View File

@ -18,5 +18,5 @@ TiKV offers the following key features:
| [CAS (Compare-And-Swap) on RawKV](../cas) | RawKV supports the compare-and-swap operation to achieve synchronization in multi-threading. |
| [Distributed Transaction](../distributed-transaction) | Similar to [Google Spanner](https://ai.google/research/pubs/pub39966), TiKV supports externally consistent distributed transactions. |
| [TiKV API V2](../api-v2) | TiKV API v2 provides new storage format to support more features, such as **Changed Data Capture** and **Key Space**. |
| [RawKV BR](../br/backup-restore) | RawKV supports backup and restoration. |
| [RawKV BR](../backup-restore) | RawKV supports backup and restoration. |
| [RawKV CDC](../cdc/cdc) | RawKV supports Changed Data Capture. |