tidb: update tidb_index_serial_scan_concurrency docs (#21938)#23268
tidb: update tidb_index_serial_scan_concurrency docs (#21938)#23268ti-chi-bot wants to merge 1 commit into
Conversation
Signed-off-by: ti-chi-bot <ti-community-prow-bot@tidb.io>
ti-chi-bot
added
do-not-merge/hold
|
@0xPoe This PR has conflicts, I have hold it. |
|
@ti-chi-bot: ## If you want to know how to resolve it, please read the guide in TiDB Dev Guide. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the ti-community-infra/tichi repository. |
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
ti-chi-bot
Bot
added
size/XL
|
@ti-chi-bot: The following test failed, say
Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request updates the TiDB documentation, including best practices, statistics, system variables, and performance benchmarking reports for TiDB Cloud v8.1.0 and v8.5.0. However, there are multiple critical issues with unresolved git merge conflicts and formatting inconsistencies in statistics.md and system-variables.md that must be addressed before merging.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| <<<<<<< HEAD | ||
| Currently, when you run the `ANALYZE` statement, the task is divided into multiple small tasks. Each task only works on one column or index. You can use the `tidb_build_stats_concurrency` parameter to control the number of simultaneous tasks. The default value is `4`. | ||
| ======= | ||
| This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions. | ||
|
|
||
| #### `tidb_distsql_scan_concurrency` | ||
| #### `tidb_build_sampling_stats_concurrency` | ||
|
|
||
| This variable controls the following aspects of `ANALYZE` concurrency: | ||
|
|
||
| - The concurrency for merging samples collected from different Regions. | ||
| - The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for. | ||
|
|
||
| The default value is `2`. | ||
|
|
||
| #### `tidb_analyze_partition_concurrency` | ||
|
|
||
| This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions. | ||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) | ||
|
|
||
| #### `tidb_analyze_distsql_scan_concurrency` | ||
|
|
||
| <<<<<<< HEAD | ||
| When you analyze regular columns, you can use the `tidb_distsql_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `15`. | ||
| ======= | ||
| This variable controls the following aspects of `ANALYZE` concurrency: | ||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) | ||
|
|
||
| #### `tidb_index_serial_scan_concurrency` | ||
| - The concurrency of scanning TiKV Regions. | ||
| - The concurrency of scanning Regions for special indexes (indexes generated from virtual columns). | ||
|
|
||
| <<<<<<< HEAD | ||
| When you analyze index columns, you can use the `tidb_index_serial_scan_concurrency` parameter to control the number of Region to be read at one time. The default value is `1`. |
There was a problem hiding this comment.
Unresolved git merge conflicts are present in this section. Please resolve the conflicts by keeping the updated descriptions and removing the conflict markers.
This variable controls the concurrency for building statistics during manual `ANALYZE`, such as the number of table or partition analysis tasks that can be processed simultaneously. The default value is `2`. The default value is `4` for v7.4.0 and earlier versions.
#### `tidb_build_sampling_stats_concurrency`
This variable controls the following aspects of `ANALYZE` concurrency:
- The concurrency for merging samples collected from different Regions.
- The concurrency for collecting statistics on special indexes (such as indexes on generated virtual columns), for example, the number of indexes that TiDB can concurrently collect statistics for.
The default value is `2`.
#### `tidb_analyze_partition_concurrency`
This variable controls the concurrency for saving `ANALYZE` results (writing TopN and histograms to system tables). The default value is `2`. The default value is `1` for v7.4.0 and earlier versions.
#### `tidb_analyze_distsql_scan_concurrency`
This variable controls the following aspects of `ANALYZE` concurrency:
- The concurrency of scanning TiKV Regions.
- The concurrency of scanning Regions for special indexes (indexes generated from virtual columns).
The default value is `4`.| ======= | ||
| The default value is `4`. | ||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
There was a problem hiding this comment.
This misplaced conflict block is a result of a git merge misalignment and should be removed entirely.
| <<<<<<< HEAD | ||
| ======= | ||
| ### tidb_analyze_column_options <span class="version-mark">New in v8.3.0</span> | ||
|
|
||
| > **Note:** | ||
| > | ||
| > - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`. | ||
| > - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior. | ||
| > - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default. | ||
| > - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default. | ||
|
|
||
| - Scope: GLOBAL | ||
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Type: Enumeration | ||
| - Default value: `ALL` | ||
| - Value options:`ALL`, `PREDICATE` | ||
| - This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance. | ||
|
|
||
| ### tidb_analyze_distsql_scan_concurrency <span class="version-mark">New in v7.6.0</span> | ||
|
|
||
| > **Note:** | ||
| > | ||
| > In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`. | ||
|
|
||
| - Scope: SESSION | GLOBAL | ||
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Type: Integer | ||
| - Default value: `4` | ||
| - Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size. | ||
| - This variable controls the following aspects of `ANALYZE` concurrency: | ||
| - The concurrency of scanning TiKV Regions. | ||
| - The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns. | ||
|
|
||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
There was a problem hiding this comment.
Unresolved git merge conflicts are present in this section. Please resolve the conflicts by keeping the new variables and removing the conflict markers. Additionally, ensure there is a space after the colon in the Value options list item to adhere to standard Markdown formatting.
### tidb_analyze_column_options <span class="version-mark">New in v8.3.0</span>
> **Note:**
>
> - This variable only works when [`tidb_analyze_version`](#tidb_analyze_version-new-in-v510) is set to `2`.
> - If you upgrade your TiDB cluster from a version earlier than v8.3.0 to v8.3.0 or later, this variable is set to `ALL` by default to keep the original behavior.
> - For a newly deployed TiDB cluster from v8.3.0 to v8.5.4, this variable is set to `PREDICATE` by default.
> - For a newly deployed TiDB cluster from v8.5.5 and v9.0.0 onwards, this variable is set to `ALL` by default.
- Scope: GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: Enumeration
- Default value: `ALL`
- Value options: `ALL`, `PREDICATE`
- This variable controls the behavior of the `ANALYZE TABLE` statement. Setting it to `PREDICATE` means only collecting statistics for [predicate columns](/statistics.md#collect-statistics-on-some-columns); setting it to `ALL` means collecting statistics for all columns. In scenarios where OLAP queries are used, it is recommended to set it to `ALL`, otherwise collecting statistics can result in a significant drop in query performance.
### tidb_analyze_distsql_scan_concurrency <span class="version-mark">New in v7.6.0</span>
> **Note:**
>
> In versions earlier than v7.6.0, regular `ANALYZE` region scans are controlled by `tidb_distsql_scan_concurrency`, and index statistics scans are controlled by `tidb_index_serial_scan_concurrency`. Therefore, to adjust the concurrency of scanning TiKV Regions for these versions, consider changing the value of `tidb_distsql_scan_concurrency`.
- Scope: SESSION | GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Type: Integer
- Default value: `4`
- Range: `[0, 4294967295]`. In versions earlier than v8.2.0, the minimum value is `1`. When you set it to `0`, it adaptively adjusts the concurrency based on the cluster size.
- This variable controls the following aspects of `ANALYZE` concurrency:
- The concurrency of scanning TiKV Regions.
- The concurrency of scanning Regions for special indexes, such as indexes on generated virtual columns.References
- Ensure consistent and correct Markdown formatting, including proper spacing after colons in list items. (link)
| <<<<<<< HEAD | ||
| - Default value: `1` | ||
| - This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. | ||
| ======= | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. | ||
| - Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. | ||
| - For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. | ||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
There was a problem hiding this comment.
Unresolved git merge conflicts are present in this section. Please resolve the conflicts by keeping the updated default value and range for newer TiDB versions and removing the conflict markers.
| <<<<<<< HEAD | |
| - Default value: `1` | |
| - This variable specifies the concurrency of reading and writing statistics for a partitioned table when TiDB analyzes the partitioned table. | |
| ======= | |
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | |
| - Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. | |
| - Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. | |
| - For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. | |
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) | |
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | |
| - Default value: `2`. The default value is `1` for v7.4.0 and earlier versions. | |
| - Range: `[1, 128]`. Before v8.4.0, the value range is `[1, 18446744073709551615]`. | |
| - For manual and auto `ANALYZE`, this variable controls the concurrency for saving `ANALYZE` results, including writing TopN and histograms to system tables. |
| <<<<<<< HEAD | ||
| ======= | ||
| ### tidb_analyze_skip_column_types <span class="version-mark">New in v7.2.0</span> | ||
|
|
||
| - Scope: SESSION | GLOBAL | ||
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Default value: "json,blob,mediumblob,longblob,mediumtext,longtext". Before v8.2.0, the default value is "json,blob,mediumblob,longblob". | ||
| - Possible values: "json,blob,mediumblob,longblob,text,mediumtext,longtext" | ||
| - This variable controls which types of columns are skipped for statistics collection when executing the `ANALYZE` command to collect statistics. The variable is only applicable for `tidb_analyze_version = 2`. Even if you specify a column using `ANALYZE TABLE t COLUMNS c1, ... , cn`, no statistics will be collected for the specified column if its type is in `tidb_analyze_skip_column_types`. | ||
|
|
||
| ``` | ||
| mysql> SHOW CREATE TABLE t; | ||
| +-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| | Table | Create Table | | ||
| +-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| | t | CREATE TABLE `t` ( | ||
| `a` int DEFAULT NULL, | ||
| `b` varchar(10) DEFAULT NULL, | ||
| `c` json DEFAULT NULL, | ||
| `d` blob DEFAULT NULL, | ||
| `e` longblob DEFAULT NULL | ||
| ) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin | | ||
| +-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+ | ||
| 1 row in set (0.00 sec) | ||
|
|
||
| mysql> SELECT @@tidb_analyze_skip_column_types; | ||
| +----------------------------------+ | ||
| | @@tidb_analyze_skip_column_types | | ||
| +----------------------------------+ | ||
| | json,blob,mediumblob,longblob,mediumtext,longtext | | ||
| +----------------------------------+ | ||
| 1 row in set (0.00 sec) | ||
|
|
||
| mysql> ANALYZE TABLE t; | ||
| Query OK, 0 rows affected, 1 warning (0.05 sec) | ||
|
|
||
| mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; | ||
| +---------------------------------------------------------------------+ | ||
| | job_info | | ||
| +---------------------------------------------------------------------+ | ||
| | analyze table columns a, b with 256 buckets, 500 topn, 1 samplerate | | ||
| +---------------------------------------------------------------------+ | ||
| 1 row in set (0.00 sec) | ||
|
|
||
| mysql> ANALYZE TABLE t COLUMNS a, c; | ||
| Query OK, 0 rows affected, 1 warning (0.04 sec) | ||
|
|
||
| mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1; | ||
| +------------------------------------------------------------------+ | ||
| | job_info | | ||
| +------------------------------------------------------------------+ | ||
| | analyze table columns a with 256 buckets, 500 topn, 1 samplerate | | ||
| +------------------------------------------------------------------+ | ||
| 1 row in set (0.00 sec) | ||
| ``` | ||
|
|
||
| ### tidb_auto_analyze_concurrency <span class="version-mark">New in v8.4.0</span> | ||
|
|
||
| - Scope: GLOBAL | ||
| - Persists to cluster: Yes | ||
| - Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No | ||
| - Type: Integer | ||
| - Default value: `3` | ||
| - Range: `[1, 2147483647]` | ||
| - This variable controls the number of concurrent auto-analyze operations that can run in a TiDB cluster. To accelerate statistics collection tasks, you can increase this concurrency based on the available resources in your cluster. | ||
| - Before v8.4.0, this concurrency is fixed at `1`. | ||
| - Starting from v8.5.7 and v9.0.0, the default value changes from `1` to `3`. If your cluster is upgraded from an earlier version, the value of this variable remains unchanged after the upgrade. | ||
|
|
||
| >>>>>>> 3d8180e56a (tidb: update tidb_index_serial_scan_concurrency docs (#21938)) |
There was a problem hiding this comment.
Unresolved git merge conflicts are present in this section. Please resolve the conflicts by keeping the new variables and removing the conflict markers. Additionally, use backticks instead of double quotes for default and possible values to adhere to the style guide.
### tidb_analyze_skip_column_types <span class="version-mark">New in v7.2.0</span>
- Scope: SESSION | GLOBAL
- Persists to cluster: Yes
- Applies to hint [SET_VAR](/optimizer-hints.md#set_varvar_namevar_value): No
- Default value: `json,blob,mediumblob,longblob,mediumtext,longtext`. Before v8.2.0, the default value is `json,blob,mediumblob,longblob`.
- Possible values: `json,blob,mediumblob,longblob,text,mediumtext,longtext`
- This variable controls which types of columns are skipped for statistics collection when executing the `ANALYZE` command to collect statistics. The variable is only applicable for `tidb_analyze_version = 2`. Even if you specify a column using `ANALYZE TABLE t COLUMNS c1, ... , cn`, no statistics will be collected for the specified column if its type is in `tidb_analyze_skip_column_types`.
mysql> SHOW CREATE TABLE t;
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Table | Create Table |
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| t | CREATE TABLE t (
a int DEFAULT NULL,
b varchar(10) DEFAULT NULL,
c json DEFAULT NULL,
d blob DEFAULT NULL,
e longblob DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin |
+-------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> SELECT @@tidb_analyze_skip_column_types;
+----------------------------------+
| @@tidb_analyze_skip_column_types |
+----------------------------------+
| json,blob,mediumblob,longblob,mediumtext,longtext |
+----------------------------------+
1 row in set (0.00 sec)
mysql> ANALYZE TABLE t;
Query OK, 0 rows affected, 1 warning (0.05 sec)
mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1;
+---------------------------------------------------------------------+
| job_info |
+---------------------------------------------------------------------+
| analyze table columns a, b with 256 buckets, 500 topn, 1 samplerate |
+---------------------------------------------------------------------+
1 row in set (0.00 sec)
mysql> ANALYZE TABLE t COLUMNS a, c;
Query OK, 0 rows affected, 1 warning (0.04 sec)
mysql> SELECT job_info FROM mysql.analyze_jobs ORDER BY end_time DESC LIMIT 1;
+------------------------------------------------------------------+
| job_info |
+------------------------------------------------------------------+
| analyze table columns a with 256 buckets, 500 topn, 1 samplerate |
+------------------------------------------------------------------+
1 row in set (0.00 sec)
### tidb_auto_analyze_concurrency <span class=
2 participants
This is an automated cherry-pick of #21938
First-time contributors' checklist
What is changed, added or deleted? (Required)
See: pingcap/tidb#64023
Which TiDB version(s) do your changes apply to? (Required)
Tips for choosing the affected version(s):
By default, CHOOSE MASTER ONLY so your changes will be applied to the next TiDB major or minor releases. If your PR involves a product feature behavior change or a compatibility change, CHOOSE THE AFFECTED RELEASE BRANCH(ES) AND MASTER.
For details, see tips for choosing the affected versions.
What is the related PR or file link(s)?
Do your changes match any of the following descriptions?