[SPARK-56140][UI] Add server-side pagination for SQL tab query listing#55073
Open
yaooqinn wants to merge 1 commit intoapache:masterfrom
Open
[SPARK-56140][UI] Add server-side pagination for SQL tab query listing#55073yaooqinn wants to merge 1 commit intoapache:masterfrom
yaooqinn wants to merge 1 commit intoapache:masterfrom
Conversation
Member
Author
Screenshots — Server-Side PaginationLive Spark UI (9 executions, "Showing 1 to 9 of 9 entries"): History Server (32 executions from TPC-DS run, paginated 20 per page, "Showing 1 to 20 of 32 entries"): Both show: server-side pagination with length menu (20/50/100/All), status filter dropdown, search box, sortable columns, status badges, job links, error messages. |
Switch the SQL tab from client-side DataTables (fetching all executions
at once) to server-side pagination following the stagepage.js pattern.
Changes:
- Add SqlResource.sqlTable endpoint returning {aaData, recordsTotal,
recordsFiltered} with DataTables server-side parameters (start,
length, order, search)
- Rewrite allexecutionspage.js to use serverSide:true with ajax
config pointing to the new endpoint
- Server handles sorting (id, status, description, submissionTime,
duration), search (filters by description, status, id), and
pagination
- Removes client-side sub-execution grouping (can be re-added as
server-side feature later)
This scales to 50K+ SQL executions (ThriftServer/SparkConnect) without
straining the browser.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
dongjoon-hyun
approved these changes
Mar 30, 2026
Member
dongjoon-hyun
left a comment
dongjoon-hyun
left a comment
There was a problem hiding this comment.
+1, LGTM. Thank you, @yaooqinn .
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changes were proposed in this pull request?
Switch the SQL tab from client-side DataTables (fetching all executions at once) to server-side pagination, following the
stagepage.jspattern.Server-side (
SqlResource.sqlTable):/sql/sqlTableGET endpoint returning{aaData, recordsTotal, recordsFiltered}start,length,order[0][column],order[0][dir],search[value]Client-side (
allexecutionspage.js):serverSide: truewithajaxconfigWhy are the changes needed?
Long-running applications (ThriftServer, Spark Connect Server) can accumulate 50K+ SQL executions. The previous client-side approach fetched all executions in one JSON response, causing browser strain. Server-side pagination fetches only the visible page.
Does this PR introduce any user-facing change?
Yes — SQL tab now paginates server-side with length menu options.
How was this patch tested?
Compilation verified. Manual testing with TPC-DS SF100.
Was this patch authored or co-authored using generative AI tooling?
Generated-by: GitHub Copilot (Claude Opus 4.6)