Skip to content

refactor(sqlite-storage): extract DBHead to vbare-versioned protocol crate#4815

Closed
NathanFlurry wants to merge 1 commit intoengine-stabilize/sqlite-origin-renamefrom
engine-stabilize/sqlite-storage-vbare-protocol
Closed

refactor(sqlite-storage): extract DBHead to vbare-versioned protocol crate#4815
NathanFlurry wants to merge 1 commit intoengine-stabilize/sqlite-origin-renamefrom
engine-stabilize/sqlite-storage-vbare-protocol

Conversation

@NathanFlurry
Copy link
Copy Markdown
Member

Description

Please include a summary of the changes and the related issue. Please also include relevant motivation and context.

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • This change requires a documentation update

How Has This Been Tested?

Please describe the tests that you ran to verify your changes.

Checklist:

  • My code follows the style guidelines of this project
  • I have performed a self-review of my code
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective or that my feature works
  • New and existing unit tests pass locally with my changes

Copy link
Copy Markdown
Member Author

NathanFlurry commented Apr 27, 2026

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@claude
Copy link
Copy Markdown

claude Bot commented Apr 27, 2026

PR Review: refactor(sqlite-storage): extract DBHead to vbare-versioned protocol crate

Overview

This PR extracts DBHead and SqliteOrigin from engine/packages/sqlite-storage/src/types.rs into a new dedicated protocol crate rivet-sqlite-storage-protocol. It introduces a versioned BARE-schema encoding layer (via vbare) with a 2-byte version prefix, replacing the raw serde_bare calls throughout the codebase.


Critical: Breaking On-Disk Format Change

The old encoding stored DBHead as a raw serde_bare blob with no version prefix. The new encoding adds a 2-byte LE version prefix via vbare::OwnedVersionedData::serialize_with_embedded_version.

Any existing actor SQLite meta records written by the old code will fail to decode:

  • deserialize_with_embedded_version reads the first 2 bytes as the version number.
  • Old data starts with the BARE varint for schema_version: 2 (0x02) followed by generation (0x01), so the two bytes are read as version 0x0102 = 258.
  • There is no 258 => branch in deserialize_version, so bail! fires.

The old decode_db_head handled two legacy formats (with and without the origin field). Both fallbacks are removed. The test decode_db_head_defaults_legacy_rows_to_created_on_v2_origin that explicitly covered this path is also deleted.

Action required: Either confirm that existing stored data is always wiped on upgrade (pre-release only), or add a migration bridge in versioned::decode_db_head that first tries the versioned prefix parse and, on failure, falls back to the old raw serde_bare decode (matching the two-step logic that was in types.rs).


Convention: Free Function vs. Extension Trait

DBHead::new() is replaced by a free function new_db_head(). CLAUDE.md says: when rivetkit needs ergonomic helpers on a re-exported type, prefer an extension trait plus prelude re-export instead of a free function. A small DBHeadExt trait with a new(creation_ts_ms: i64) -> DBHead method would keep the call-site ergonomics of the previous DBHead::new(...) while staying within the project convention.


Minor: Unnecessary Clone in encode_db_head

The types.rs wrapper takes &DBHead and clones it to pass to versioned::encode_db_head which takes DBHead by value. Since serialize_version only needs a reference (serde_bare::to_vec takes &T), changing versioned::encode_db_head to accept &v1::DBHead would eliminate the clone.


Positive Points

  • Good architectural fit. Follows the established envoy-protocol / epoxy-protocol / ups-protocol crate pattern under engine/sdks/rust/.
  • Complete migration. Every serde_bare::to_vec(&head) / serde_bare::from_slice call for DBHead is replaced uniformly across production code and all test fixtures.
  • Forward-versioning foundation. The BARE schema + vbare 2-byte prefix makes future schema changes non-breaking by design (add v2.bare, new match arm, converter).
  • Build script path. Traversal from engine/sdks/rust/sqlite-storage-protocol/ up three levels to engine/, then joining sdks/schemas/sqlite-storage, is correct and consistent with sibling protocol crates.
  • Test name updated to db_head_round_trips_through_versioned_encoding, accurately reflecting the new encoding path.

Summary

Severity Issue
Critical On-disk format break: old serde_bare blobs fail to decode with the new versioned encoder; backward-compat fallback removed
Moderate Free function new_db_head should be an extension trait per project convention
Minor Unnecessary clone() in types::encode_db_head wrapper

The core refactoring direction is sound. The format-compatibility concern needs to be resolved (or explicitly acknowledged as intentional for a pre-release wipe) before merging.

@NathanFlurry NathanFlurry marked this pull request as ready for review April 27, 2026 10:03
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-origin-rename branch from 2d28410 to d55b8cf Compare April 27, 2026 17:35
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-storage-vbare-protocol branch from 6a0ae51 to 5c43398 Compare April 27, 2026 17:35
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-origin-rename branch from d55b8cf to 40f9e0b Compare April 27, 2026 19:30
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-storage-vbare-protocol branch from 5c43398 to 57aa81f Compare April 27, 2026 19:30
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-origin-rename branch from 40f9e0b to 76265f2 Compare April 27, 2026 19:40
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-storage-vbare-protocol branch from 57aa81f to 3bb9717 Compare April 27, 2026 19:40
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-origin-rename branch from 76265f2 to 9293029 Compare April 27, 2026 20:07
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-storage-vbare-protocol branch 2 times, most recently from 9b501ed to 58ff56b Compare April 27, 2026 20:30
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-origin-rename branch 2 times, most recently from 2c9727e to cd5775d Compare April 27, 2026 20:48
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-storage-vbare-protocol branch 2 times, most recently from 3d61f72 to 6d1d4e3 Compare April 27, 2026 21:29
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-storage-vbare-protocol branch from 6d1d4e3 to ee84fb7 Compare April 27, 2026 21:38
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-origin-rename branch from 20a8181 to cfbcbfd Compare April 27, 2026 21:38
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-origin-rename branch from cfbcbfd to a36c978 Compare April 27, 2026 22:03
@NathanFlurry NathanFlurry force-pushed the engine-stabilize/sqlite-storage-vbare-protocol branch from ee84fb7 to 0c85098 Compare April 27, 2026 22:03
@NathanFlurry
Copy link
Copy Markdown
Member Author

Landed in main via stack-merge fast-forward push. Commits are in main; closing to match.

@github-actions
Copy link
Copy Markdown
Contributor

Preview packages published to npm

Install with:

npm install rivetkit@pr-4815

All packages published as 0.0.0-pr.4815.3b70228 with tag pr-4815.

Engine binary is shipped via @rivetkit/engine-cli on linux-x64-musl, linux-arm64-musl, darwin-x64, and darwin-arm64. Windows users should use the release installer or set RIVET_ENGINE_BINARY.

Docker images:

docker pull rivetdev/engine:slim-3b70228
docker pull rivetdev/engine:full-3b70228
Individual packages
npm install rivetkit@pr-4815
npm install @rivetkit/react@pr-4815
npm install @rivetkit/rivetkit-napi@pr-4815
npm install @rivetkit/workflow-engine@pr-4815

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant