From 0b723affea9b86eff21a92fac203ab28612e3516 Mon Sep 17 00:00:00 2001 From: Dongpo Liu Date: Tue, 14 Jul 2026 12:21:14 +0200 Subject: [PATCH 1/8] This is an automated cherry-pick of #21938 Signed-off-by: ti-chi-bot --- best-practices/tidb-best-practices.md | 2 +- statistics.md | 72 ++++++++++ system-variables.md | 65 +++++++-- ...v8.1-performance-benchmarking-with-tpcc.md | 4 +- ...v8.5-performance-benchmarking-with-tpcc.md | 123 ++++++++++++++++++ troubleshoot-tidb-oom.md | 3 +- 6 files changed, 256 insertions(+), 13 deletions(-) create mode 100644 tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index a941c4e56d88b..4a0c37ffba861 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. + You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). * Reverse index scan diff --git a/statistics.md b/statistics.md index 6794499756850..dbe6116a8de2d 100644 --- a/statistics.md +++ b/statistics.md @@ -962,6 +962,78 @@ The following table describes the behaviors of locking statistics: | A partitioned table and the whole table is locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The old partition lock information is invalid, and the new partition is automatically locked | The new partition is automatically locked | The lock information of the deleted partition is cleared, and the lock of the whole table continues to take effect | The lock information of the deleted partition is cleared, and the new partition is automatically locked | The lock information is transferred to the exchanged table, and the new partition is automatically locked | | A partitioned table and only some partitions are locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | / | The deleted partition lock information is cleared | The deleted partition lock information is cleared | The lock information is transferred to the exchanged table | +<<<<<<< HEAD +======= +## Manage `ANALYZE` tasks and concurrency + +This section describes how to terminate background `ANALYZE` tasks and control the `ANALYZE` concurrency. + +### Terminate background `ANALYZE` tasks + +Since TiDB v6.0, TiDB supports using the `KILL` statement to terminate an `ANALYZE` task running in the background. If you find that an `ANALYZE` task running in the background consumes a lot of resources and affects your application, you can terminate the `ANALYZE` task by taking the following steps: + +1. Execute the following SQL statement: + + ```sql + SHOW ANALYZE STATUS + ``` + + By checking the `instance` column and the `process_id` column in the result, you can get the TiDB instance address and the task `ID` of the background `ANALYZE` task. + +2. Terminate the `ANALYZE` task that is running in the background. + + + + - If [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is `true` (`true` by default), you can execute the `KILL TIDB ${id};` statement directly, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. + - If `enable-global-kill` is `false`, you need to use a client to connect to the TiDB instance that is executing the backend `ANALYZE` task, and then execute the `KILL TIDB ${id};` statement. If you use a client to connect to another TiDB instance, or if there is a proxy between the client and the TiDB cluster, the `KILL` statement cannot terminate the background `ANALYZE` task. + + + + + + To terminate the `ANALYZE` task, you can execute the `KILL TIDB ${id};` statement, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. + + + +For more information on the `KILL` statement, see [`KILL`](/sql-statements/sql-statement-kill.md). + +### Control `ANALYZE` concurrency + +When you run the `ANALYZE` statement, you can adjust the concurrency using system variables, to control its effect on the system. + +The relationships of the relevant system variables are shown below: + +![analyze_concurrency](/media/analyze_concurrency.png) + +`tidb_build_stats_concurrency`, `tidb_build_sampling_stats_concurrency`, and `tidb_analyze_partition_concurrency` are in an upstream-downstream relationship, as shown in the preceding diagram. The actual total concurrency is: `tidb_build_stats_concurrency` * (`tidb_build_sampling_stats_concurrency` + `tidb_analyze_partition_concurrency`). When modifying these variables, you need to consider their respective values at the same time. It is recommended to adjust them one by one in the order of `tidb_analyze_partition_concurrency`, `tidb_build_sampling_stats_concurrency`, `tidb_build_stats_concurrency`, and observe the impact on the system. The larger the values of these three variables, the greater the resource overhead on the system. + +#### `tidb_build_stats_concurrency` + +This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. + +#### `tidb_build_sampling_stats_concurrency` + +This variable controls the following aspects of `ANALYZE` concurrency: + +- The concurrency for merging samples collected from different Regions. +- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. + +The default value is `2`. + +#### `tidb_analyze_partition_concurrency` + +This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. + +#### `tidb_analyze_distsql_scan_concurrency` + +This variable controls the following aspects of `ANALYZE` concurrency: + +- The concurrency of scanning TiKV Regions. +- The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). + +The default value is `4`. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ## See also diff --git a/system-variables.md b/system-variables.md index 5212aca69fc55..1510ac4787143 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1086,13 +1086,54 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `OFF` - This variable is used to set whether the `AUTO_INCREMENT` property of a column is allowed to be removed by executing `ALTER TABLE MODIFY` or `ALTER TABLE CHANGE` statements. It is not allowed by default. +<<<<<<< HEAD +======= +### tidb_analyze_column_options New in v8.3.0 + +> **Note:** +> +> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. +> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. +> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. +> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. + +- Scope: GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Enumeration +- Default value: `ALL` +- Value options:`ALL`, `PREDICATE` +- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. + +### tidb_analyze_distsql_scan_concurrency New in v7.6.0 + +> **Note:** +> +> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. + +- Scope: SESSION | GLOBAL +- Persists to cluster: Yes +- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No +- Type: Integer +- Default value: `4` +- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. +- This variable controls the following aspects of `ANALYZE` concurrency: + - The concurrency of scanning TiKV Regions. + - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. + +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_partition_concurrency - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. +<<<<<<< HEAD - This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. +======= +- Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. +- For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_version New in v5.1.0 @@ -1136,7 +1177,7 @@ mysql> SELECT @@tidb_analyze_skip_column_types; +----------------------------------+ | @@tidb_analyze_skip_column_types | +----------------------------------+ -| json,blob,mediumblob,longblob | +| json,blob,mediumblob,longblob,mediumtext,longtext | +----------------------------------+ 1 row in set (0.00 sec) @@ -1214,7 +1255,12 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Type: Integer - Default value: `1` - Range: `[1, 256]` +<<<<<<< HEAD - This variable is used to set the concurrency of executing the automatic update of statistics. +======= +- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously +- Starting from v8.5.7 and v9.0.0, the default value of this variable changes from `1` to `2`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. +>>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_backoff_lock_fast @@ -1319,8 +1365,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Default value: `2`. The default value is `4` for v7.4.0 and earlier versions. - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of executing the `ANALYZE` statement. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. ### tidb_build_sampling_stats_concurrency New in v7.5.0 @@ -1331,8 +1376,9 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Unit: Threads - Default value：`2` - Range: `[1, 256]` -- This variable is used to set the sampling concurrency in the `ANALYZE` process. -- When the variable is set to a larger value, the execution performance of other queries is affected. +- This variable controls the following aspects of `ANALYZE` concurrency: + - The concurrency for merging samples collected from different Regions. + - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. ### tidb_capture_plan_baselines New in v4.0 @@ -3201,6 +3247,10 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency +> **Warning:** +> +> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. + - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): Yes @@ -3208,8 +3258,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable is used to set the concurrency of the `serial scan` operation. -- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. +- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. ### tidb_init_chunk_size @@ -3663,7 +3712,7 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Default value: `1` -- This variable specifies the concurrency of merging statistics for a partitioned table when TiDB analyzes the partitioned table. +- This variable controls the concurrency for merging TopN results of partitioned tables. ### tidb_enable_async_merge_global_stats New in v7.5.0 diff --git a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md index 8aad399883823..79e74aec086dd 100644 --- a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md +++ b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md @@ -96,8 +96,8 @@ This section introduces how to perform the TPC-C performance test step by step. ```sql SET tidb_build_stats_concurrency=16; - SET tidb_distsql_scan_concurrency=16; - SET tidb_index_serial_scan_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; ``` 5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. diff --git a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md new file mode 100644 index 0000000000000..3bd5bbd2a576a --- /dev/null +++ b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md @@ -0,0 +1,123 @@ +--- +title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 +summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. +--- + +# TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 + +This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.5.0 clusters. + +## Test overview + +This test aims at showing the TPC-C performance of TiDB v8.5.0 in the Online Transactional Processing (OLTP) scenario. + +## Test environment + +### TiDB cluster + +The test is conducted on a TiDB cluster with the following settings: + +- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) +- Cluster version: v8.5.0 +- Cloud provider: AWS (us-west-2) +- Cluster configuration: + + | Node type | Node size | Node quantity | Node storage | + |:----------|:----------|:----------|:----------| + | TiDB | 16 vCPU, 32 GiB | 2 | N/A | + | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | + +### Parameter configuration + +> **Note:** +> +> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. + +The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: + +```yaml +raft-engine.prefill-for-recycle = true +``` + +### Benchmark executor + +The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: + +- Machine type: Amazon EC2 (us-west-2) +- Instance type: c6a.2xlarge + +## Test steps + +This section introduces how to perform the TPC-C performance test step by step. + +1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. + + For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). + +2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. + + To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). + + To create the `tpcc` database, execute the following SQL statement: + + ```sql + CREATE DATABASE tpcc; + ``` + +3. Load TPC-C data to the `tpcc` database. + + 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: + + ```shell + curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh + ``` + + 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. + + ```shell + go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: + + ```sql + ANALYZE TABLE customer; + ANALYZE TABLE district; + ANALYZE TABLE history; + ANALYZE TABLE item; + ANALYZE TABLE new_order; + ANALYZE TABLE order_line; + ANALYZE TABLE orders; + ANALYZE TABLE stock; + ANALYZE TABLE warehouse; + ``` + + To accelerate the collection of statistics, execute the following SQL statements before collecting: + + ```sql + SET tidb_build_stats_concurrency=16; + SET tidb_build_sampling_stats_concurrency = 16; + SET tidb_analyze_distsql_scan_concurrency=16; + ``` + +5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. + + ```shell + go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error + ``` + +6. Extract the tpmC data of `NEW_ORDER` from the result. + + TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. + +## Test results + +The TPC-C performance of v8.5.0 in the [test environment](#test-environment) is as follows: + +| Threads | v8.5.0 tpmC | +|:--------|:----------| +| 50 | 43,146 | +| 100 | 73,875 | +| 200 | 103,395 | + +![TPC-C](/media/tidb-cloud/v8.5.0_tpcc.png) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 1c2f102cd0681..c888f278f5e63 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) - - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) From 02a82647fa6f80245489d593f6b47122c8223ebe Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 11:21:49 +0800 Subject: [PATCH 2/8] docs: adapt cherry-pick for release-7.5 --- best-practices/tidb-best-practices.md | 2 +- statistics.md | 72 ---------- system-variables.md | 58 +-------- ...v8.1-performance-benchmarking-with-tpcc.md | 4 +- ...v8.5-performance-benchmarking-with-tpcc.md | 123 ------------------ troubleshoot-tidb-oom.md | 3 +- 6 files changed, 10 insertions(+), 252 deletions(-) delete mode 100644 tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md diff --git a/best-practices/tidb-best-practices.md b/best-practices/tidb-best-practices.md index 4a0c37ffba861..a941c4e56d88b 100644 --- a/best-practices/tidb-best-practices.md +++ b/best-practices/tidb-best-practices.md @@ -129,7 +129,7 @@ Lots of MySQL experience is also applicable to TiDB. It is noted that TiDB has i * Ensure the order of results through indexes - You can use indexes to filter or sort data. TiDB first gets row IDs according to the index order, and then returns the row content according to the order of those row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but you can increase it by adjusting [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50). + You can use indexes to filter or sort data. Firstly, get row IDs according to the index order. Then, return the row content according to the return order of row IDs. In this way, the returned results are ordered according to the index column. It has been mentioned earlier that the model of scanning index and getting row is parallel + pipeline. If the row is returned according to the index order, a high concurrency between two queries does not reduce latency. Thus, the concurrency is low by default, but it can be modified through the [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) variable. * Reverse index scan diff --git a/statistics.md b/statistics.md index dbe6116a8de2d..6794499756850 100644 --- a/statistics.md +++ b/statistics.md @@ -962,78 +962,6 @@ The following table describes the behaviors of locking statistics: | A partitioned table and the whole table is locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The old partition lock information is invalid, and the new partition is automatically locked | The new partition is automatically locked | The lock information of the deleted partition is cleared, and the lock of the whole table continues to take effect | The lock information of the deleted partition is cleared, and the new partition is automatically locked | The lock information is transferred to the exchanged table, and the new partition is automatically locked | | A partitioned table and only some partitions are locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | / | The deleted partition lock information is cleared | The deleted partition lock information is cleared | The lock information is transferred to the exchanged table | -<<<<<<< HEAD -======= -## Manage `ANALYZE` tasks and concurrency - -This section describes how to terminate background `ANALYZE` tasks and control the `ANALYZE` concurrency. - -### Terminate background `ANALYZE` tasks - -Since TiDB v6.0, TiDB supports using the `KILL` statement to terminate an `ANALYZE` task running in the background. If you find that an `ANALYZE` task running in the background consumes a lot of resources and affects your application, you can terminate the `ANALYZE` task by taking the following steps: - -1. Execute the following SQL statement: - - ```sql - SHOW ANALYZE STATUS - ``` - - By checking the `instance` column and the `process_id` column in the result, you can get the TiDB instance address and the task `ID` of the background `ANALYZE` task. - -2. Terminate the `ANALYZE` task that is running in the background. - - - - - If [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is `true` (`true` by default), you can execute the `KILL TIDB ${id};` statement directly, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. - - If `enable-global-kill` is `false`, you need to use a client to connect to the TiDB instance that is executing the backend `ANALYZE` task, and then execute the `KILL TIDB ${id};` statement. If you use a client to connect to another TiDB instance, or if there is a proxy between the client and the TiDB cluster, the `KILL` statement cannot terminate the background `ANALYZE` task. - - - - - - To terminate the `ANALYZE` task, you can execute the `KILL TIDB ${id};` statement, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. - - - -For more information on the `KILL` statement, see [`KILL`](/sql-statements/sql-statement-kill.md). - -### Control `ANALYZE` concurrency - -When you run the `ANALYZE` statement, you can adjust the concurrency using system variables, to control its effect on the system. - -The relationships of the relevant system variables are shown below: - -![analyze_concurrency](/media/analyze_concurrency.png) - -`tidb_build_stats_concurrency`, `tidb_build_sampling_stats_concurrency`, and `tidb_analyze_partition_concurrency` are in an upstream-downstream relationship, as shown in the preceding diagram. The actual total concurrency is: `tidb_build_stats_concurrency` * (`tidb_build_sampling_stats_concurrency` + `tidb_analyze_partition_concurrency`). When modifying these variables, you need to consider their respective values at the same time. It is recommended to adjust them one by one in the order of `tidb_analyze_partition_concurrency`, `tidb_build_sampling_stats_concurrency`, `tidb_build_stats_concurrency`, and observe the impact on the system. The larger the values of these three variables, the greater the resource overhead on the system. - -#### `tidb_build_stats_concurrency` - -This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. - -#### `tidb_build_sampling_stats_concurrency` - -This variable controls the following aspects of `ANALYZE` concurrency: - -- The concurrency for merging samples collected from different Regions. -- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. - -The default value is `2`. - -#### `tidb_analyze_partition_concurrency` - -This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. - -#### `tidb_analyze_distsql_scan_concurrency` - -This variable controls the following aspects of `ANALYZE` concurrency: - -- The concurrency of scanning TiKV Regions. -- The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). - -The default value is `4`. - ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ## See also diff --git a/system-variables.md b/system-variables.md index 1510ac4787143..e53c2818d086e 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1086,54 +1086,14 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Default value: `OFF` - This variable is used to set whether the `AUTO_INCREMENT` property of a column is allowed to be removed by executing `ALTER TABLE MODIFY` or `ALTER TABLE CHANGE` statements. It is not allowed by default. -<<<<<<< HEAD -======= -### tidb_analyze_column_options New in v8.3.0 - -> **Note:** -> -> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. -> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. -> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. -> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. - -- Scope: GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Enumeration -- Default value: `ALL` -- Value options:`ALL`, `PREDICATE` -- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. - -### tidb_analyze_distsql_scan_concurrency New in v7.6.0 - -> **Note:** -> -> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. - -- Scope: SESSION | GLOBAL -- Persists to cluster: Yes -- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No -- Type: Integer -- Default value: `4` -- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. -- This variable controls the following aspects of `ANALYZE` concurrency: - - The concurrency of scanning TiKV Regions. - - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. - ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_partition_concurrency - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. -<<<<<<< HEAD -- This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. -======= -- Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. +- Range: `[1, 18446744073709551615]` - For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) ### tidb_analyze_version New in v5.1.0 @@ -1177,7 +1137,7 @@ mysql> SELECT @@tidb_analyze_skip_column_types; +----------------------------------+ | @@tidb_analyze_skip_column_types | +----------------------------------+ -| json,blob,mediumblob,longblob,mediumtext,longtext | +| json,blob,mediumblob,longblob | +----------------------------------+ 1 row in set (0.00 sec) @@ -1255,12 +1215,7 @@ mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; - Type: Integer - Default value: `1` - Range: `[1, 256]` -<<<<<<< HEAD -- This variable is used to set the concurrency of executing the automatic update of statistics. -======= -- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously -- Starting from v8.5.7 and v9.0.0, the default value of this variable changes from `1` to `2`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. ->>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) +- This variable controls the concurrency for building statistics during auto `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. ### tidb_backoff_lock_fast @@ -3247,10 +3202,6 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified ### tidb_index_serial_scan_concurrency -> **Warning:** -> -> This variable is deprecated and no longer controls execution behavior. The concurrency of sequential index scans is now controlled by [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50), and [`ANALYZE TABLE`](/sql-statements/sql-statement-analyze-table.md) uses [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. - - Scope: SESSION | GLOBAL - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): Yes @@ -3258,7 +3209,8 @@ For a system upgraded to v5.0 from an earlier version, if you have not modified - Default value: `1` - Range: `[1, 256]` - Unit: Threads -- This variable remains only for backward compatibility. Use [`tidb_executor_concurrency`](#tidb_executor_concurrency-new-in-v50) to control the concurrency of sequential index scans, or [`tidb_analyze_distsql_scan_concurrency`](#tidb_analyze_distsql_scan_concurrency-new-in-v760) to control the concurrency of index statistics scans. +- This variable is used to set the concurrency of the `serial scan` operation. +- Use a bigger value in OLAP scenarios, and a smaller value in OLTP scenarios. ### tidb_init_chunk_size diff --git a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md index 79e74aec086dd..8aad399883823 100644 --- a/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md +++ b/tidb-cloud/v8.1-performance-benchmarking-with-tpcc.md @@ -96,8 +96,8 @@ This section introduces how to perform the TPC-C performance test step by step. ```sql SET tidb_build_stats_concurrency=16; - SET tidb_build_sampling_stats_concurrency = 16; - SET tidb_analyze_distsql_scan_concurrency=16; + SET tidb_distsql_scan_concurrency=16; + SET tidb_index_serial_scan_concurrency=16; ``` 5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. diff --git a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md b/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md deleted file mode 100644 index 3bd5bbd2a576a..0000000000000 --- a/tidb-cloud/v8.5-performance-benchmarking-with-tpcc.md +++ /dev/null @@ -1,123 +0,0 @@ ---- -title: TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 -summary: Introduce the TPC-C performance test results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. ---- - -# TiDB Cloud TPC-C Performance Test Report for TiDB v8.5.0 - -This document provides the TPC-C performance test steps and results for a TiDB Cloud Dedicated cluster with the TiDB version of v8.5.0. This report can also be used as a reference for the performance of TiDB Self-Managed v8.5.0 clusters. - -## Test overview - -This test aims at showing the TPC-C performance of TiDB v8.5.0 in the Online Transactional Processing (OLTP) scenario. - -## Test environment - -### TiDB cluster - -The test is conducted on a TiDB cluster with the following settings: - -- Cluster type: [TiDB Cloud Dedicated](/tidb-cloud/select-cluster-tier.md#tidb-cloud-dedicated) -- Cluster version: v8.5.0 -- Cloud provider: AWS (us-west-2) -- Cluster configuration: - - | Node type | Node size | Node quantity | Node storage | - |:----------|:----------|:----------|:----------| - | TiDB | 16 vCPU, 32 GiB | 2 | N/A | - | TiKV | 16 vCPU, 64 GiB | 3 | 1000 GiB | - -### Parameter configuration - -> **Note:** -> -> For TiDB Cloud, to modify the TiKV parameters of your cluster, you can contact [PingCAP Support](/tidb-cloud/tidb-cloud-support.md) for help. - -The TiKV parameter [`prefill-for-recycle`](https://docs.pingcap.com/tidb/stable/tikv-configuration-file#prefill-for-recycle-new-in-v700) can make log recycling effective immediately after initialization. This document conducts tests with `prefill-for-recycle` enabled: - -```yaml -raft-engine.prefill-for-recycle = true -``` - -### Benchmark executor - -The benchmark executor sends SQL queries to the TiDB cluster. In this test, its hardware configuration is as follows: - -- Machine type: Amazon EC2 (us-west-2) -- Instance type: c6a.2xlarge - -## Test steps - -This section introduces how to perform the TPC-C performance test step by step. - -1. In the [TiDB Cloud console](https://tidbcloud.com/), create a TiDB Cloud Dedicated cluster that meets the [test environment](#tidb-cluster) requirements. - - For more information, see [Create a TiDB Cloud Dedicated cluster](/tidb-cloud/create-tidb-cluster.md). - -2. On the benchmark executor, connect to the newly created cluster and create a database named `tpcc`. - - To connect to the cluster, see [Connect to TiDB Cloud Dedicated via Private Endpoint](/tidb-cloud/set-up-private-endpoint-connections.md). - - To create the `tpcc` database, execute the following SQL statement: - - ```sql - CREATE DATABASE tpcc; - ``` - -3. Load TPC-C data to the `tpcc` database. - - 1. The test in this document is implemented based on [go-tpc](https://github.com/pingcap/go-tpc). You can download the test program using the following command: - - ```shell - curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/pingcap/go-tpc/master/install.sh | sh - ``` - - 2. Run the following `go-tpc tpcc` command to import 1,000 warehouses to the `tpcc` database. Replace `${HOST}`, `${THREAD}`, and `${PASSWORD}` with your actual values. This document conducts three tests with the `${THREAD}` value of `50`, `100`, and `200`. - - ```shell - go-tpc tpcc --host ${HOST} --warehouses 1000 prepare -P 4000 -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -4. To ensure that the TiDB optimizer can generate the optimal execution plan, execute the following SQL statements to collect statistics before conducting the TPC-C test: - - ```sql - ANALYZE TABLE customer; - ANALYZE TABLE district; - ANALYZE TABLE history; - ANALYZE TABLE item; - ANALYZE TABLE new_order; - ANALYZE TABLE order_line; - ANALYZE TABLE orders; - ANALYZE TABLE stock; - ANALYZE TABLE warehouse; - ``` - - To accelerate the collection of statistics, execute the following SQL statements before collecting: - - ```sql - SET tidb_build_stats_concurrency=16; - SET tidb_build_sampling_stats_concurrency = 16; - SET tidb_analyze_distsql_scan_concurrency=16; - ``` - -5. Run the following `go-tpc tpcc` command to conduct stress tests on the TiDB Cloud Dedicated cluster. For each concurrency, the test takes two hours. - - ```shell - go-tpc tpcc --host ${HOST} -P 4000 --warehouses 1000 run -D tpcc -T ${THREAD} --time 2h0m0s -p ${PASSWORD} --ignore-error - ``` - -6. Extract the tpmC data of `NEW_ORDER` from the result. - - TPC-C uses tpmC (transactions per minute) to measure the maximum qualified throughput (MQTh, Max Qualified Throughput). The transactions are the NewOrder transactions and the final unit of measure is the number of new orders processed per minute. - -## Test results - -The TPC-C performance of v8.5.0 in the [test environment](#test-environment) is as follows: - -| Threads | v8.5.0 tpmC | -|:--------|:----------| -| 50 | 43,146 | -| 100 | 73,875 | -| 200 | 103,395 | - -![TPC-C](/media/tidb-cloud/v8.5.0_tpcc.png) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index c888f278f5e63..1c2f102cd0681 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,6 +99,7 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) + - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -194,4 +195,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file From 554e6eb44e93c27133b16b900c3678d20a67bd0d Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 11:39:13 +0800 Subject: [PATCH 3/8] docs: add release-7.5 analyze concurrency guidance --- statistics.md | 60 +++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 60 insertions(+) diff --git a/statistics.md b/statistics.md index 6794499756850..142a79fe2b884 100644 --- a/statistics.md +++ b/statistics.md @@ -962,6 +962,66 @@ The following table describes the behaviors of locking statistics: | A partitioned table and the whole table is locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The old partition lock information is invalid, and the new partition is automatically locked | The new partition is automatically locked | The lock information of the deleted partition is cleared, and the lock of the whole table continues to take effect | The lock information of the deleted partition is cleared, and the new partition is automatically locked | The lock information is transferred to the exchanged table, and the new partition is automatically locked | | A partitioned table and only some partitions are locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | / | The deleted partition lock information is cleared | The deleted partition lock information is cleared | The lock information is transferred to the exchanged table | +## Manage `ANALYZE` tasks and concurrency + +This section describes how to terminate background `ANALYZE` tasks and control the `ANALYZE` concurrency. + +### Terminate background `ANALYZE` tasks + +Since TiDB v6.0, TiDB supports using the `KILL` statement to terminate an `ANALYZE` task running in the background. If you find that an `ANALYZE` task running in the background consumes a lot of resources and affects your application, you can terminate the `ANALYZE` task by taking the following steps: + +1. Execute the following SQL statement: + + ```sql + SHOW ANALYZE STATUS + ``` + + By checking the `instance` column and the `process_id` column in the result, you can get the TiDB instance address and the task `ID` of the background `ANALYZE` task. + +2. Terminate the `ANALYZE` task that is running in the background. + + + + - If [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is `true` (`true` by default), you can execute the `KILL TIDB ${id};` statement directly, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. + - If `enable-global-kill` is `false`, you need to use a client to connect to the TiDB instance that is executing the backend `ANALYZE` task, and then execute the `KILL TIDB ${id};` statement. If you use a client to connect to another TiDB instance, or if there is a proxy between the client and the TiDB cluster, the `KILL` statement cannot terminate the background `ANALYZE` task. + + + + + + To terminate the `ANALYZE` task, you can execute the `KILL TIDB ${id};` statement, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. + + + +For more information on the `KILL` statement, see [`KILL`](/sql-statements/sql-statement-kill.md). + +### Control `ANALYZE` concurrency + +When you run the `ANALYZE` statement, you can adjust the concurrency using system variables, to control its effect on the system. + +The relationships of the relevant system variables are shown below: + +![analyze_concurrency](/media/analyze_concurrency.png) + +`tidb_build_stats_concurrency`, `tidb_build_sampling_stats_concurrency`, and `tidb_analyze_partition_concurrency` are in an upstream-downstream relationship, as shown in the preceding diagram. The actual total concurrency is: `tidb_build_stats_concurrency` * (`tidb_build_sampling_stats_concurrency` + `tidb_analyze_partition_concurrency`). When modifying these variables, you need to consider their respective values at the same time. It is recommended to adjust them one by one in the order of `tidb_analyze_partition_concurrency`, `tidb_build_sampling_stats_concurrency`, `tidb_build_stats_concurrency`, and observe the impact on the system. The larger the values of these three variables, the greater the resource overhead on the system. + +#### `tidb_build_stats_concurrency` + +This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. + +#### `tidb_build_sampling_stats_concurrency` + +This variable controls the following aspects of `ANALYZE` concurrency: + +- The concurrency for merging samples collected from different Regions. +- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. + +The default value is `2`. + +#### `tidb_analyze_partition_concurrency` + +This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. + ## See also From 5778a1dfc89994116f04c47104770b11b81b2849 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 11:40:14 +0800 Subject: [PATCH 4/8] docs: update release-7.5 OOM guidance --- troubleshoot-tidb-oom.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 1c2f102cd0681..c888f278f5e63 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,7 +99,6 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) - - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. @@ -195,4 +194,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) From 8f54cbce28d31b3f121d05e064fd2c66edb364f1 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 11:45:14 +0800 Subject: [PATCH 5/8] docs: update release-7.5 statistics descriptions in place --- statistics.md | 71 ++++++--------------------------------------------- 1 file changed, 8 insertions(+), 63 deletions(-) diff --git a/statistics.md b/statistics.md index 142a79fe2b884..d8c5aa3169e4e 100644 --- a/statistics.md +++ b/statistics.md @@ -407,15 +407,20 @@ The relationships of the relevant system variables are shown below: #### `tidb_build_stats_concurrency` -When you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on statistics of one column or index. You can use the [`tidb_build_stats_concurrency`](/system-variables.md#tidb_build_stats_concurrency) variable to control the number of simultaneous small tasks. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. +This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. #### `tidb_build_sampling_stats_concurrency` -When analyzing ordinary columns, you can use [`tidb_build_sampling_stats_concurrency`](/system-variables.md#tidb_build_sampling_stats_concurrency-new-in-v750) to control the concurrency of executing sampling tasks. The default value is `2`. +This variable controls the following aspects of `ANALYZE` concurrency: + +- The concurrency for merging samples collected from different Regions. +- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. + +The default value is `2`. #### `tidb_analyze_partition_concurrency` -When running the `ANALYZE` statement, you can use [`tidb_analyze_partition_concurrency`](/system-variables.md#tidb_analyze_partition_concurrency) to control the concurrency of reading and writing statistics for a partitioned table. The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. +This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. #### `tidb_distsql_scan_concurrency` @@ -962,66 +967,6 @@ The following table describes the behaviors of locking statistics: | A partitioned table and the whole table is locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The old partition lock information is invalid, and the new partition is automatically locked | The new partition is automatically locked | The lock information of the deleted partition is cleared, and the lock of the whole table continues to take effect | The lock information of the deleted partition is cleared, and the new partition is automatically locked | The lock information is transferred to the exchanged table, and the new partition is automatically locked | | A partitioned table and only some partitions are locked | The lock is invalid | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | The lock is invalid because TiDB deletes the old table, so the lock information is also deleted | / | The deleted partition lock information is cleared | The deleted partition lock information is cleared | The lock information is transferred to the exchanged table | -## Manage `ANALYZE` tasks and concurrency - -This section describes how to terminate background `ANALYZE` tasks and control the `ANALYZE` concurrency. - -### Terminate background `ANALYZE` tasks - -Since TiDB v6.0, TiDB supports using the `KILL` statement to terminate an `ANALYZE` task running in the background. If you find that an `ANALYZE` task running in the background consumes a lot of resources and affects your application, you can terminate the `ANALYZE` task by taking the following steps: - -1. Execute the following SQL statement: - - ```sql - SHOW ANALYZE STATUS - ``` - - By checking the `instance` column and the `process_id` column in the result, you can get the TiDB instance address and the task `ID` of the background `ANALYZE` task. - -2. Terminate the `ANALYZE` task that is running in the background. - - - - - If [`enable-global-kill`](/tidb-configuration-file.md#enable-global-kill-new-in-v610) is `true` (`true` by default), you can execute the `KILL TIDB ${id};` statement directly, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. - - If `enable-global-kill` is `false`, you need to use a client to connect to the TiDB instance that is executing the backend `ANALYZE` task, and then execute the `KILL TIDB ${id};` statement. If you use a client to connect to another TiDB instance, or if there is a proxy between the client and the TiDB cluster, the `KILL` statement cannot terminate the background `ANALYZE` task. - - - - - - To terminate the `ANALYZE` task, you can execute the `KILL TIDB ${id};` statement, where `${id}` is the `ID` of the background `ANALYZE` task obtained from the previous step. - - - -For more information on the `KILL` statement, see [`KILL`](/sql-statements/sql-statement-kill.md). - -### Control `ANALYZE` concurrency - -When you run the `ANALYZE` statement, you can adjust the concurrency using system variables, to control its effect on the system. - -The relationships of the relevant system variables are shown below: - -![analyze_concurrency](/media/analyze_concurrency.png) - -`tidb_build_stats_concurrency`, `tidb_build_sampling_stats_concurrency`, and `tidb_analyze_partition_concurrency` are in an upstream-downstream relationship, as shown in the preceding diagram. The actual total concurrency is: `tidb_build_stats_concurrency` * (`tidb_build_sampling_stats_concurrency` + `tidb_analyze_partition_concurrency`). When modifying these variables, you need to consider their respective values at the same time. It is recommended to adjust them one by one in the order of `tidb_analyze_partition_concurrency`, `tidb_build_sampling_stats_concurrency`, `tidb_build_stats_concurrency`, and observe the impact on the system. The larger the values of these three variables, the greater the resource overhead on the system. - -#### `tidb_build_stats_concurrency` - -This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. - -#### `tidb_build_sampling_stats_concurrency` - -This variable controls the following aspects of `ANALYZE` concurrency: - -- The concurrency for merging samples collected from different Regions. -- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. - -The default value is `2`. - -#### `tidb_analyze_partition_concurrency` - -This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. - ## See also From de04b4bdf9636b983faea4f1ee04d7e5f6b33d68 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:41:25 +0800 Subject: [PATCH 6/8] docs: retain release-7.5 OOM concurrency guidance --- troubleshoot-tidb-oom.md | 1 + 1 file changed, 1 insertion(+) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index c888f278f5e63..545ec8c1d0c16 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -99,6 +99,7 @@ You can take the following measures to reduce the memory usage of SQL statements - Reduce the number of Regions to be read simultaneously or reduce the concurrency of operators to avoid memory problems caused by high concurrency. The corresponding system variables include: - [`tidb_distsql_scan_concurrency`](/system-variables.md#tidb_distsql_scan_concurrency) + - [`tidb_index_serial_scan_concurrency`](/system-variables.md#tidb_index_serial_scan_concurrency) - [`tidb_executor_concurrency`](/system-variables.md#tidb_executor_concurrency-new-in-v50) - The concurrency of sessions is too high near the time point when the problem occurs. In this case, consider scaling out the TiDB cluster by adding more TiDB nodes. From 65330fc1093fea6d031dcea6a42e7e52a7d1e1b8 Mon Sep 17 00:00:00 2001 From: qiancai Date: Wed, 15 Jul 2026 12:43:47 +0800 Subject: [PATCH 7/8] docs: exclude no-op OOM guidance from release-7.5 --- troubleshoot-tidb-oom.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/troubleshoot-tidb-oom.md b/troubleshoot-tidb-oom.md index 545ec8c1d0c16..1c2f102cd0681 100644 --- a/troubleshoot-tidb-oom.md +++ b/troubleshoot-tidb-oom.md @@ -195,4 +195,4 @@ To locate the root cause of an OOM issue, you need to collect the following info ## See also - [TiDB Memory Control](/configure-memory-usage.md) -- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) +- [Tune TiKV Memory Parameter Performance](/tune-tikv-memory-performance.md) \ No newline at end of file From 990af96826072e2461367f925c43644816c7a979 Mon Sep 17 00:00:00 2001 From: Grace Cai Date: Thu, 16 Jul 2026 15:36:29 +0800 Subject: [PATCH 8/8] Update system-variables.md --- system-variables.md | 1 - 1 file changed, 1 deletion(-) diff --git a/system-variables.md b/system-variables.md index e53c2818d086e..a9724405332f4 100644 --- a/system-variables.md +++ b/system-variables.md @@ -1092,7 +1092,6 @@ MPP is a distributed computing framework provided by the TiFlash engine, which a - Persists to cluster: Yes - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No - Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. -- Range: `[1, 18446744073709551615]` - For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. ### tidb_analyze_version New in v5.1.0