cherry pick 60989：[opt](query cache) Support multiple tablets in cache key building#61527
Merged
yiguolei merged 2 commits intoapache:branch-4.1from Mar 20, 2026
Merged
cherry pick 60989：[opt](query cache) Support multiple tablets in cache key building#61527yiguolei merged 2 commits intoapache:branch-4.1from
yiguolei merged 2 commits intoapache:branch-4.1from
Conversation
Contributor
|
Thank you for your contribution to Apache Doris. Please clearly describe your PR:
|
Contributor
Author
|
run buildall |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR cherry-picks changes to extend BE query-cache key construction to support fragment instances that scan multiple tablets, and updates BE unit tests accordingly.
Changes:
- Update
QueryCache::build_cache_key()to accept multiple scan ranges and incorporate multiple tablet IDs into the cache key. - Remove the “only one scan range” restriction in
CacheSourceLocalState::init()and record multiple tablet IDs in the runtime profile. - Adjust/extend BE unit tests to set
digestand validate multi-tablet cache-key behavior.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| be/test/pipeline/operator/query_cache_operator_test.cpp | Updates operator tests to use Status::ok() and set digest for cache params. |
| be/test/pipeline/exec/query_cache_test.cpp | Expands cache-key tests to cover multiple tablets and additional error paths. |
| be/src/pipeline/query_cache/query_cache.h | Implements multi-tablet cache-key building logic (sorting tablet IDs, validating version/range assumptions). |
| be/src/pipeline/exec/cache_source_operator.cpp | Removes single-scan-range restriction and records multiple tablet IDs in the profile. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
You can also share your feedback on Copilot code review. Take the survey.
Comment on lines
+163
to
+171
| if (current_version != first_version) { | ||
| return Status::InternalError( | ||
| "All tablets in one instance must have the same version, plan error"); | ||
| } | ||
| if (find_tablet->second != first_tablet_range) { | ||
| return Status::InternalError( | ||
| "All tablets in one instance must have the same tablet_to_range, plan " | ||
| "error"); | ||
| } |
Comment on lines
+116
to
+121
| std::string digest; | ||
| try { | ||
| digest = cache_param.digest; | ||
| } catch (const std::exception&) { | ||
| return Status::InternalError("digest is invalid, plan error"); | ||
| } |
Comment on lines
+130
to
+135
| std::vector<int64_t> tablet_ids; | ||
| tablet_ids.reserve(scan_ranges.size()); | ||
| for (const auto& scan_range : scan_ranges) { | ||
| auto tablet_id = scan_range.scan_range.palo_scan_range.tablet_id; | ||
| tablet_ids.push_back(tablet_id); | ||
| } |
Comment on lines
+153
to
+158
| int64_t current_version = -1; | ||
| std::from_chars(scan_range_iter->scan_range.palo_scan_range.version.data(), | ||
| scan_range_iter->scan_range.palo_scan_range.version.data() + | ||
| scan_range_iter->scan_range.palo_scan_range.version.size(), | ||
| current_version); | ||
|
|
Comment on lines
+130
to
+152
| std::vector<int64_t> tablet_ids; | ||
| tablet_ids.reserve(scan_ranges.size()); | ||
| for (const auto& scan_range : scan_ranges) { | ||
| auto tablet_id = scan_range.scan_range.palo_scan_range.tablet_id; | ||
| tablet_ids.push_back(tablet_id); | ||
| } | ||
| std::sort(tablet_ids.begin(), tablet_ids.end()); | ||
|
|
||
| int64_t first_version = -1; | ||
| std::string first_tablet_range; | ||
| for (size_t i = 0; i < tablet_ids.size(); ++i) { | ||
| auto tablet_id = tablet_ids[i]; | ||
|
|
||
| auto find_tablet = cache_param.tablet_to_range.find(tablet_id); | ||
| if (find_tablet == cache_param.tablet_to_range.end()) { | ||
| return Status::InternalError("Not find tablet in partition_to_tablets, plan error"); | ||
| } | ||
|
|
||
| auto scan_range_iter = | ||
| std::find_if(scan_ranges.begin(), scan_ranges.end(), | ||
| [&tablet_id](const TScanRangeParams& range) { | ||
| return range.scan_range.palo_scan_range.tablet_id == tablet_id; | ||
| }); |
|
|
||
| auto find_tablet = cache_param.tablet_to_range.find(tablet_id); | ||
| if (find_tablet == cache_param.tablet_to_range.end()) { | ||
| return Status::InternalError("Not find tablet in partition_to_tablets, plan error"); |
Comment on lines
+143
to
146
| cache_param.digest = "test_digest"; | ||
| cache_param.output_slot_mapping[0] = 0; | ||
| cache_param.tablet_to_range.insert({42, "test"}); | ||
| cache_param.force_refresh_query_cache = false; |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
|
run buildall |
Contributor
BE UT Coverage ReportIncrement line coverage Increment coverage report
|
Contributor
BE Regression && UT Coverage ReportIncrement line coverage Increment coverage report
|
yiguolei
approved these changes
Mar 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Release note
None
Check List (For Author)
Test
Behavior changed:
Does this need documentation?
Check List (For Reviewer who merge this PR)