Skip to content

perf: trim zctl listview instead of full rebuild#8164

Open
mhk197 wants to merge 6 commits into
developfrom
mk/listview-export-try-trim
Open

perf: trim zctl listview instead of full rebuild#8164
mhk197 wants to merge 6 commits into
developfrom
mk/listview-export-try-trim

Conversation

@mhk197
Copy link
Copy Markdown
Contributor

@mhk197 mhk197 commented May 29, 2026

When exporting a ListViewArray to an Arrow list-view, the rebuild gate now branches on whether the array is already zero-copy-to-list:

  • Zero-copy-to-list arrays have no overlaps and no interior gaps, so their only waste is leading/trailing unreferenced elements. These are reclaimed with a cheap trim_elements rebuild (a lazy elements slice plus an O(num_lists) offset adjustment, no element data copy), gated on the unreferenced fraction exceeding DEFAULT_TRIM_ELEMENTS_THRESHOLD.
  • Non-zero-copy arrays keep the existing density-based MakeZeroCopyToList full rebuild, since they may contain overlaps, gaps, or garbage that only a full rebuild can reclaim.

@mhk197 mhk197 changed the title trim zctl listview instead of full rebuild fix: trim zctl listview instead of full rebuild May 29, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 29, 2026

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 1 improved benchmark
❌ 1 regressed benchmark
✅ 1273 untouched benchmarks

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation chunked_varbinview_opt_canonical_into[(1000, 10)] 188.1 µs 225 µs -16.41%
Simulation chunked_varbinview_canonical_into[(100, 100)] 307.9 µs 273.1 µs +12.74%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing mk/listview-export-try-trim (e24a014) with develop (23ebab1)

Open in CodSpeed

mhk197 added 6 commits May 29, 2026 15:28
Signed-off-by: Matt Katz <katz@spiraldb.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <katz@spiraldb.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <katz@spiraldb.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <katz@spiraldb.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
Signed-off-by: Matt Katz <mhkatz97@gmail.com>
@mhk197 mhk197 force-pushed the mk/listview-export-try-trim branch from e389e24 to e24a014 Compare May 29, 2026 19:29
@mhk197 mhk197 changed the title fix: trim zctl listview instead of full rebuild perf: trim zctl listview instead of full rebuild May 29, 2026
@mhk197 mhk197 added the changelog/performance A performance improvement label May 29, 2026
@mhk197 mhk197 marked this pull request as ready for review May 29, 2026 19:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant