Skip to content

Storage policy: remove DROP NATIVE, add reserved TO REMOTE stage#458

Merged
RaphDal merged 3 commits into
mainfrom
rd_sp_drop_native
May 29, 2026
Merged

Storage policy: remove DROP NATIVE, add reserved TO REMOTE stage#458
RaphDal merged 3 commits into
mainfrom
rd_sp_drop_native

Conversation

@RaphDal
Copy link
Copy Markdown
Contributor

@RaphDal RaphDal commented May 29, 2026

Summary

Updates the storage policy documentation to match the new lifecycle model.

DROP NATIVE is removed entirely. Converting a partition with TO PARQUET now removes the native files in place and serves reads directly from the Parquet file, so a separate native-drop stage is no longer needed. The drop_native column is dropped from the storage_policies system view.

A reserved TO REMOTE stage is added, mirroring the existing reserved DROP REMOTE clause. This makes the lifecycle coherent for the future object storage integration: convert locally, upload to remote, drop the local copy, then drop the remote copy. Both remote stages are rejected at SQL parse time and surface as always-blank columns in the system view.

The stages now form a partial order rather than a single chain. A drop stage may not fire before the write it depends on:

TO PARQUET <= DROP LOCAL
TO REMOTE  <= DROP LOCAL <= DROP REMOTE

TO PARQUET and TO REMOTE are independent. If TO REMOTE fires first, both native and Parquet copies are written locally and reads are served from native until TO PARQUET removes the native files. TO PARQUET and DROP LOCAL are active today; TO REMOTE and DROP REMOTE are reserved for a future release.

Changes by area:

  • Concept (concepts/storage-policy.md): lifecycle diagram, stage table, TTL evaluation, ordering constraint, storage_policies example, and guidelines.
  • SQL reference (alter-table-set-storage-policy.md, alter-mat-view-set-storage-policy.md, create-table.md, create-mat-view.md, show.md): syntax blocks, stage tables, constraints, examples, and SHOW CREATE TABLE output.
  • Meta (query/functions/meta.md): storage_policies column reference and example tables (removed drop_native, added reserved to_remote).
  • Guides and ops (getting-started/enterprise-quick-start.md, operations/backup.md, high-availability/overview.md): migration guidance, backup interaction, and replicated-policy description.
  • Changelog: added a May 2026 entry.

Dependencies

This documents an engine behavior change and should merge alongside the corresponding QuestDB Enterprise PR. The reserved TO REMOTE parse error is documented as 'TO REMOTE' is not supported yet, mirroring DROP REMOTE; confirm the exact string and the to_remote view column name against the engine implementation before merging.

@RaphDal RaphDal added the enhancement New feature or request label May 29, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 29, 2026

🚀 Build success!

Latest successful preview: https://preview-458--questdb-documentation.netlify.app/docs/

Commit SHA: 79dba3f

📦 Build generates a preview & updates link on each commit.

Comment thread documentation/concepts/storage-policy.md
Comment thread documentation/query/sql/alter-mat-view-set-storage-policy.md Outdated
Comment thread documentation/query/sql/alter-table-set-storage-policy.md
Comment thread documentation/query/sql/create-mat-view.md
@RaphDal RaphDal merged commit b873c36 into main May 29, 2026
3 checks passed
@RaphDal RaphDal deleted the rd_sp_drop_native branch May 29, 2026 17:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants