Skip to content

feat: upgrade protect-ffi to 0.23.0#473

Open
calvinbrewer wants to merge 6 commits into
mainfrom
protect-ffi-0-22-0
Open

feat: upgrade protect-ffi to 0.23.0#473
calvinbrewer wants to merge 6 commits into
mainfrom
protect-ffi-0-22-0

Conversation

@calvinbrewer
Copy link
Copy Markdown
Contributor

@calvinbrewer calvinbrewer commented May 20, 2026

Summary by CodeRabbit

  • Chores

    • Upgraded encryption library to version 0.23.0 with bundled EQL extension to 2.3.1
    • Updated PostgreSQL service image and Docker configuration for EQL 2.3.1 compatibility
    • Updated all related documentation and migration scripts to reflect new EQL version
  • Breaking Changes

    • Encryption configuration schema now requires version validation
    • PostgreSQL databases require reinstallation of encryption extension
    • Configuration validation error messages updated

Review Change Stack

@calvinbrewer calvinbrewer requested a review from a team as a code owner May 20, 2026 22:34
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 20, 2026

🦋 Changeset detected

Latest commit: 1bed6f7

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 10 packages
Name Type
@cipherstash/protect Minor
@cipherstash/stack Minor
@cipherstash/schema Minor
@cipherstash/protect-dynamodb Major
@cipherstash/prisma-next Minor
stash Minor
@cipherstash/e2e Patch
@cipherstash/bench Patch
@cipherstash/basic-example Patch
@cipherstash/prisma-next-example Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 20, 2026

📝 Walkthrough

Walkthrough

PR upgrades protect-ffi to 0.23.0 and EQL to 2.3.1. Introduces k-field discriminated encrypted payload types, separates storage (Encrypted) from query (EncryptedQuery) shapes, migrates encrypt-config to v:1, updates DynamoDB and search validation to use the discriminator, and adapts test assertions across all packages.

Changes

EQL 2.3.1 Upgrade & Discriminated Payload Adoption

Layer / File(s) Summary
Dependency, docs, and infrastructure updates
packages/protect/package.json, packages/stack/package.json, .changeset/protect-ffi-0-23-0.md, .claude/skills/prisma-next-extension-upgrade, skills-lock.json, skills/prisma-next-extension-upgrade, packages/prisma-next/src/migration/eql-bundle.ts, examples/prisma/migrations/*/migration.json, packages/prisma-next/migrations/*/migration.json, .github/workflows/tests.yml, local/docker-compose.yml, packages/cli/src/installer/index.ts, packages/drizzle/**/GENERATE_EQL_MIGRATION_CLI.md, packages/drizzle/README.md, packages/drizzle/src/bin/generate-eql-migration.ts, packages/protect/README.md, packages/protect/src/ffi/operations/batch-encrypt-query.ts, packages/protect/src/types.ts
Bump Protect-FFI to 0.23.0 and EQL version to 2.3.1 across docs, installers, and Docker/workflow configs; document breaking changes in changeset; add skills metadata; update workspace release exclusions.
Encrypt-config schema version: v:2 → v:1
packages/schema/src/index.ts, packages/schema/__tests__/schema.test.ts, packages/stack/__tests__/schema-builders.test.ts, packages/stack/src/schema/index.ts
Change buildEncryptConfig to generate v:1 instead of v:2; update all schema-builder test assertions to expect v:1 and table structures.
Encrypted payload types: storage vs query separation
packages/protect/src/types.ts, packages/stack/src/types.ts
Introduce EncryptedQuery type for query-only payloads; expand EncryptedSearchTerm and EncryptedQueryResult to include both Encrypted (stored) and EncryptedQuery shapes.
Query-term union widening in helper functions
packages/protect/src/helpers/index.ts, packages/stack/src/encryption/helpers/index.ts, packages/stack/src/encryption/operations/batch-encrypt-query.ts
Widen encryptedToCompositeLiteral, encryptedToEscapedCompositeLiteral, formatEncryptedResult, and assembleResults to accept EncryptedQueryTerm union (both stored and query-term results) instead of ciphertext-only payloads.
DynamoDB adapter: k-field discriminator for payload routing
packages/protect-dynamodb/src/helpers.ts, packages/stack/src/dynamodb/helpers.ts
Update toEncryptedDynamoItem to branch on encryptPayload.k ('ct' or 'sv') and only emit ciphertext/searchable-value when discriminator matches; update toItemWithEqlPayloads cast_as check from jsonb to json.
Search-term validation: require k discriminator and HMAC
packages/protect-dynamodb/src/operations/search-terms.ts
Tighten SearchTermsOperation.execute() to require k === 'ct' and string hm field on encrypted search terms; raise HMAC validation error on failure.
Test helper: encrypted payload discriminator validation
packages/drizzle/__tests__/integration-test-helpers.ts
Update assertEncryptedPayload to validate k field matches `/^(ct
Backward-compat tests: k-field discriminator & round-trip validation
packages/protect/__tests__/backward-compat.test.ts, packages/stack/__tests__/backward-compat.test.ts
Rename test suites to emphasize k-field discriminator; assert new encrypted payloads include k: 'ct' alongside c, v, i; add encrypt→decrypt round-trip checks; remove legacy payload injection steps.
Searchable-JSON equality tests: sv-iteration & ste_vec_entry casting
packages/protect/__tests__/searchable-json-pg.test.ts
Refactor array selector/equality tests to iterate sv via LATERAL jsonb_array_elements, use LATERAL eql_v2.jsonb_path_query(...) for array operations, cast equality operands to eql_v2.ste_vec_entry, and update cross-document equality from missing-hash errors to join-based matching.
Widespread JSON/number test assertions: k-field presence
packages/protect/__tests__/json-protect.test.ts, packages/protect/__tests__/number-protect.test.ts, packages/stack/__tests__/json-protect.test.ts, packages/stack/__tests__/number-protect.test.ts
Update ~80 assertion points across all JSON encryption scenarios (simple, model, bulk, lock-context, nested, edge-case) and number bulk tests to expect k field presence instead of absence.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related issues

  • cipherstash/stack#471: Directly related; both track EQL 2.3.x upgrade with protected-ffi version bump, payload discriminator handling, and test adaptation across the same code paths.

Possibly related PRs

  • cipherstash/stack#476: Modifies the same EQL version pins and Docker/docs in opposite direction (reverting from 2.3.1 to 2.2.1).
  • cipherstash/stack#386: Also upgrades @cipherstash/protect-ffi dependency version in the same package.json files, forming part of the same FFI migration path.
  • cipherstash/stack#477: Pins postgres-eql 2.2.1 and skips =-equality tests; this PR upgrades to 2.3.1 and reworks those same searchable-json equality test patterns.

Suggested reviewers

  • coderdan

Poem

🐰 A discriminator hops through the code,
k-fields now mark each encrypted load,
Payloads split: stored and queries fly,
Schema v1 beneath the EQL sky,
Tests all bloom with new assertions bright!

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The PR title 'feat: upgrade protect-ffi to 0.23.0' accurately summarizes the main change, matching the actual upgrade from 0.21.4 to 0.23.0 across multiple package.json files and related updates.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch protect-ffi-0-22-0

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 ESLint

If the error stems from missing dependencies, add them to the package.json file. For unrecoverable errors (e.g., due to private dependencies), disable the tool in the CodeRabbit configuration.

ESLint skipped: no ESLint configuration detected in root package.json. To enable, add eslint to devDependencies.


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🧹 Nitpick comments (2)
packages/stack/__tests__/json-protect.test.ts (1)

61-61: ⚡ Quick win

Strengthen discriminator assertions to k: 'ct'.

toHaveProperty('k') validates presence only. Using 'ct' here better locks the ciphertext contract.

Suggested diff
- expect(ciphertext.data).toHaveProperty('k')
+ expect(ciphertext.data).toHaveProperty('k', 'ct')

- expect(encryptedModel.data.json).toHaveProperty('k')
+ expect(encryptedModel.data.json).toHaveProperty('k', 'ct')

Also applies to: 113-113, 200-203, 312-318, 446-446

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/__tests__/json-protect.test.ts` at line 61, The tests
currently assert only the presence of the discriminator key with
expect(ciphertext.data).toHaveProperty('k'); update these assertions to assert
the exact discriminator value 'ct' (e.g., use toHaveProperty('k', 'ct') or
expect(ciphertext.data.k).toBe('ct')) so the ciphertext contract is locked;
apply the same change for the other similar assertions in this file that
reference ciphertext.data (and any ciphertext2/ciphertext3 usages).
packages/protect/__tests__/json-protect.test.ts (1)

58-58: ⚡ Quick win

Assert the discriminator value, not just key presence.

These checks pass even if k has the wrong value. For ciphertext payloads, assert k: 'ct' explicitly.

Suggested diff
- expect(ciphertext.data).toHaveProperty('k')
+ expect(ciphertext.data).toHaveProperty('k', 'ct')

- expect(encryptedModel.data.email).toHaveProperty('k')
+ expect(encryptedModel.data.email).toHaveProperty('k', 'ct')

Also applies to: 110-110, 217-219, 329-335, 514-514

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/protect/__tests__/json-protect.test.ts` at line 58, The test
currently only checks presence of the discriminator key 'k'
(expect(ciphertext.data).toHaveProperty('k')) but should assert its value equals
'ct'; update the assertions that inspect ciphertext payloads (e.g., where you
reference ciphertext.data) to check expect(ciphertext.data.k).toBe('ct') (and
similarly replace loose has-property checks at the other occurrences) so the
tests verify the discriminator value, not just key presence.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@packages/protect/__tests__/number-protect.test.ts`:
- Around line 307-310: The comment is contradictory with the assertions: update
the comment to state new encryptions SHOULD have the discriminator 'k' (or
adjust to the intended expectation), and change the three expect lines on
encryptedData.data[i].data to assert the explicit discriminator value (e.g.,
expect(encryptedData.data[0].data.k).toBe('k') or
expect(encryptedData.data[0].data).toHaveProperty('k',
<expectedDiscriminator>)), doing the same for indices 1 and 2; ensure the test
checks the discriminator value explicitly rather than only presence, and make
the comment reflect that explicit check.

In `@packages/schema/src/index.ts`:
- Line 353: The schema currently allows any numeric version while
buildEncryptConfig emits only 1; update the encryptConfigSchema and the
TypeScript type in schema/type to require v: 1 (not number) so validation fails
for other versions, and adjust any dependent inference so buildEncryptConfig's
output type matches; then ensure EncryptionClient.init uses encryptConfigSchema
validation (or its parsed type) to reject invalid configs at schema-validation
time rather than later in the FFI by invoking the validator and handling
failures before continuing.

In `@packages/stack/__tests__/number-protect.test.ts`:
- Around line 288-291: The test comment is stale and the assertions contradict
it: update the comment to state that encrypted items should include a
discriminator 'k' with value 'ct', and change the assertions on
encryptedData.data[0].data, encryptedData.data[1].data, and
encryptedData.data[2].data to assert that the property 'k' === 'ct' (i.e.,
validate the discriminator value rather than only checking existence).

---

Nitpick comments:
In `@packages/protect/__tests__/json-protect.test.ts`:
- Line 58: The test currently only checks presence of the discriminator key 'k'
(expect(ciphertext.data).toHaveProperty('k')) but should assert its value equals
'ct'; update the assertions that inspect ciphertext payloads (e.g., where you
reference ciphertext.data) to check expect(ciphertext.data.k).toBe('ct') (and
similarly replace loose has-property checks at the other occurrences) so the
tests verify the discriminator value, not just key presence.

In `@packages/stack/__tests__/json-protect.test.ts`:
- Line 61: The tests currently assert only the presence of the discriminator key
with expect(ciphertext.data).toHaveProperty('k'); update these assertions to
assert the exact discriminator value 'ct' (e.g., use toHaveProperty('k', 'ct')
or expect(ciphertext.data.k).toBe('ct')) so the ciphertext contract is locked;
apply the same change for the other similar assertions in this file that
reference ciphertext.data (and any ciphertext2/ciphertext3 usages).
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 969eecc3-fae8-4b80-9c9f-1a6c068ea869

📥 Commits

Reviewing files that changed from the base of the PR and between c9b2831 and b455678.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (27)
  • .changeset/protect-ffi-0-22-0.md
  • .claude/skills/prisma-next-extension-upgrade
  • packages/cli/src/sql/cipherstash-encrypt-no-operator-family.sql
  • packages/cli/src/sql/cipherstash-encrypt-supabase.sql
  • packages/cli/src/sql/cipherstash-encrypt.sql
  • packages/drizzle/__tests__/integration-test-helpers.ts
  • packages/prisma-next/migrations/20260601T0000_install_eql_bundle/ops.json
  • packages/prisma-next/src/migration/eql-install.generated.ts
  • packages/protect-dynamodb/src/helpers.ts
  • packages/protect-dynamodb/src/operations/search-terms.ts
  • packages/protect/__tests__/backward-compat.test.ts
  • packages/protect/__tests__/json-protect.test.ts
  • packages/protect/__tests__/number-protect.test.ts
  • packages/protect/__tests__/searchable-json-pg.test.ts
  • packages/protect/package.json
  • packages/schema/__tests__/schema.test.ts
  • packages/schema/src/index.ts
  • packages/stack/__tests__/backward-compat.test.ts
  • packages/stack/__tests__/json-protect.test.ts
  • packages/stack/__tests__/number-protect.test.ts
  • packages/stack/__tests__/schema-builders.test.ts
  • packages/stack/package.json
  • packages/stack/src/dynamodb/helpers.ts
  • packages/stack/src/schema/index.ts
  • pnpm-workspace.yaml
  • skills-lock.json
  • skills/prisma-next-extension-upgrade

Comment on lines 307 to +310
// Forward compatibility: new encryptions should NOT have k field
expect(encryptedData.data[0].data).not.toHaveProperty('k')
expect(encryptedData.data[1].data).not.toHaveProperty('k')
expect(encryptedData.data[2].data).not.toHaveProperty('k')
expect(encryptedData.data[0].data).toHaveProperty('k')
expect(encryptedData.data[1].data).toHaveProperty('k')
expect(encryptedData.data[2].data).toHaveProperty('k')
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Fix contradictory forward-compat note and make the check explicit.

Line 307 says new payloads should not have k, but Lines 308-310 assert the opposite. Update the comment and assert the expected discriminator value.

Suggested diff
- // Forward compatibility: new encryptions should NOT have k field
- expect(encryptedData.data[0].data).toHaveProperty('k')
- expect(encryptedData.data[1].data).toHaveProperty('k')
- expect(encryptedData.data[2].data).toHaveProperty('k')
+ // Forward compatibility: new encryptions should include k: 'ct'
+ expect(encryptedData.data[0].data).toHaveProperty('k', 'ct')
+ expect(encryptedData.data[1].data).toHaveProperty('k', 'ct')
+ expect(encryptedData.data[2].data).toHaveProperty('k', 'ct')
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Forward compatibility: new encryptions should NOT have k field
expect(encryptedData.data[0].data).not.toHaveProperty('k')
expect(encryptedData.data[1].data).not.toHaveProperty('k')
expect(encryptedData.data[2].data).not.toHaveProperty('k')
expect(encryptedData.data[0].data).toHaveProperty('k')
expect(encryptedData.data[1].data).toHaveProperty('k')
expect(encryptedData.data[2].data).toHaveProperty('k')
// Forward compatibility: new encryptions should include k: 'ct'
expect(encryptedData.data[0].data).toHaveProperty('k', 'ct')
expect(encryptedData.data[1].data).toHaveProperty('k', 'ct')
expect(encryptedData.data[2].data).toHaveProperty('k', 'ct')
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/protect/__tests__/number-protect.test.ts` around lines 307 - 310,
The comment is contradictory with the assertions: update the comment to state
new encryptions SHOULD have the discriminator 'k' (or adjust to the intended
expectation), and change the three expect lines on encryptedData.data[i].data to
assert the explicit discriminator value (e.g.,
expect(encryptedData.data[0].data.k).toBe('k') or
expect(encryptedData.data[0].data).toHaveProperty('k',
<expectedDiscriminator>)), doing the same for indices 1 and 2; ensure the test
checks the discriminator value explicitly rather than only presence, and make
the comment reflect that explicit check.

): EncryptConfig {
const config: EncryptConfig = {
v: 2,
v: 1,
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot May 20, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major | ⚡ Quick win

Tighten the version contract in schema/type to v: 1 as well.

buildEncryptConfig now emits only 1, but encryptConfigSchema still accepts any number. Please align validation/type inference so invalid config versions fail at schema-validation time, not later in FFI.

Suggested patch
 export const encryptConfigSchema = z.object({
-  v: z.number(),
+  v: z.literal(1),
   tables: tablesSchema,
 })
As per coding guidelines, "When changing schema behavior (packages/schema), update type definitions and ensure validation works in EncryptionClient.init".
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
v: 1,
export const encryptConfigSchema = z.object({
v: z.literal(1),
tables: tablesSchema,
})
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/schema/src/index.ts` at line 353, The schema currently allows any
numeric version while buildEncryptConfig emits only 1; update the
encryptConfigSchema and the TypeScript type in schema/type to require v: 1 (not
number) so validation fails for other versions, and adjust any dependent
inference so buildEncryptConfig's output type matches; then ensure
EncryptionClient.init uses encryptConfigSchema validation (or its parsed type)
to reject invalid configs at schema-validation time rather than later in the FFI
by invoking the validator and handling failures before continuing.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@coderdan I think the version contract regression from the config is my fault. It should be 2. The fix needs to be made in cipherstash client too.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

False alarm.

Comment on lines 288 to +291
// Forward compatibility: new encryptions should NOT have k field
expect(encryptedData.data[0].data).not.toHaveProperty('k')
expect(encryptedData.data[1].data).not.toHaveProperty('k')
expect(encryptedData.data[2].data).not.toHaveProperty('k')
expect(encryptedData.data[0].data).toHaveProperty('k')
expect(encryptedData.data[1].data).toHaveProperty('k')
expect(encryptedData.data[2].data).toHaveProperty('k')
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Update stale comment and assert the expected discriminator value.

The comment on Line 288 contradicts the assertions on Lines 289-291. Align the wording and validate k as 'ct'.

Suggested diff
- // Forward compatibility: new encryptions should NOT have k field
- expect(encryptedData.data[0].data).toHaveProperty('k')
- expect(encryptedData.data[1].data).toHaveProperty('k')
- expect(encryptedData.data[2].data).toHaveProperty('k')
+ // Forward compatibility: new encryptions should include k: 'ct'
+ expect(encryptedData.data[0].data).toHaveProperty('k', 'ct')
+ expect(encryptedData.data[1].data).toHaveProperty('k', 'ct')
+ expect(encryptedData.data[2].data).toHaveProperty('k', 'ct')
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Forward compatibility: new encryptions should NOT have k field
expect(encryptedData.data[0].data).not.toHaveProperty('k')
expect(encryptedData.data[1].data).not.toHaveProperty('k')
expect(encryptedData.data[2].data).not.toHaveProperty('k')
expect(encryptedData.data[0].data).toHaveProperty('k')
expect(encryptedData.data[1].data).toHaveProperty('k')
expect(encryptedData.data[2].data).toHaveProperty('k')
// Forward compatibility: new encryptions should include k: 'ct'
expect(encryptedData.data[0].data).toHaveProperty('k', 'ct')
expect(encryptedData.data[1].data).toHaveProperty('k', 'ct')
expect(encryptedData.data[2].data).toHaveProperty('k', 'ct')
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/__tests__/number-protect.test.ts` around lines 288 - 291, The
test comment is stale and the assertions contradict it: update the comment to
state that encrypted items should include a discriminator 'k' with value 'ct',
and change the assertions on encryptedData.data[0].data,
encryptedData.data[1].data, and encryptedData.data[2].data to assert that the
property 'k' === 'ct' (i.e., validate the discriminator value rather than only
checking existence).

}

if (!term?.hm) {
if (term?.k !== 'ct' || !term.hm) {
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This isn't correct but its in part because the types from protectjs-ffi aren't being exposed or used. Might need a follow-up there.

I'll investigate.

coderdan added a commit that referenced this pull request May 21, 2026
…build

The 4 'WHERE comparison: = equality' tests assert STE-vec entry equality behaviour and error wording that match no published EQL release (verified 2.1.x-2.3.x) — they passed only against an unreleased EQL build on the old shared CI database. With CI now pinned to a published EQL image, they fail. Skip them; re-calibrate as part of the EQL 2.3 upgrade (#473).
calvinbrewer and others added 6 commits May 21, 2026 10:52
protect-ffi 0.23.0 is a types-only release that splits the previously-
conflated `Encrypted` type into:

- `Encrypted` — storage-only payload returned by encrypt/encryptBulk,
  the only shape decrypt accepts. `c` is now required (was `c?`).
- `EncryptedQuery` (new) — query payload returned by encryptQuery /
  encryptQueryBulk for scalar unique/match/ore lookups and ste_vec
  selector path queries. Carries no ciphertext (`c?: never`) so the
  union discriminates cleanly via `'c' in payload`. JSON containment
  queries (ste_vec_term) still come back as a storage-shaped
  `Encrypted`, hence the union return.
- `isEncrypted` arg widened to `unknown`.
- `EncryptedSteVecStorage` collapsed into `EncryptedSteVec`.

No runtime change; consumers that lied about the loose union now have
to narrow properly.

Stack-side updates:

- `EncryptedSearchTerm` and `EncryptedQueryResult` (in protect and
  stack) widen to `Encrypted | EncryptedQuery | string [| null]`.
- `formatEncryptedResult`, `encryptedToCompositeLiteral`, and
  `encryptedToEscapedCompositeLiteral` accept the union via a new
  internal `EncryptedQueryTerm = CipherStashEncrypted |
  CipherStashEncryptedQuery` alias.
- `BatchEncryptQueryOperation.assembleResults` takes the union for the
  FFI's bulk return shape.
- `protect-dynamodb`'s `SearchTermsOperation` narrows via
  `'hm' in term && typeof term.hm === 'string'` so the new
  `EncryptedScalarQuery & { hm }` / `& { bf }` / `& { ob }` discriminator
  unions narrow cleanly without reading off the loose union.

Verified: pnpm test (1876 passed | 20 skipped, 14/14 packages) and
prisma-next e2e (36/36 across the seven shape suites). No DB-side
changes needed since the runtime is unchanged from 0.22.0 → 0.23.0.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
The branch reconstructing the SteVec storage payload from DynamoDB
attributes was gated on `cast_as === 'jsonb'`, but protect-ffi's
`CastAs` union has no `'jsonb'` member — the value the schema emits
for JSON columns is `'json'`. The branch was dead, so JSON / ste_vec
columns were silently rebuilt as a scalar `{k:'ct', c}` payload,
losing the `sv` array on the way back out.

The sibling `packages/stack/src/dynamodb/helpers.ts` already uses
`'json'` correctly — the two files had drifted.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PR #476 (merged before this branch's rebase) pinned every EQL install
script and the postgres-eql CI container at eql-2.2.1. This branch
upgrades the code to emit the EQL v2.3 payload format, so every pin
must move in lockstep — otherwise CI runs the new bytes against the
old EQL extension and JSON inserts hit `_encrypted_check_c`.

Flipped:

- `.github/workflows/tests.yml` — container image
  `ghcr.io/cipherstash/postgres-eql:17-2.2.1` → `17-2.3.1`.
- `local/docker-compose.yml` — same tag; the bench workflow
  (`tests-bench.yml`) consumes this compose file via
  `docker compose up --wait`.
- `packages/cli/src/installer/index.ts` and
  `packages/drizzle/src/bin/generate-eql-migration.ts` — `EQL_VERSION`
  constant.
- The three docs the `EQL_VERSION` pin appears in
  (`packages/protect/README.md`, `packages/drizzle/README.md`,
  `packages/drizzle/GENERATE_EQL_MIGRATION_CLI.md`).

The vendored SQL in `packages/cli/src/sql/*.sql` and the prisma-next
install bundle were already eql-2.3.1 from earlier in this PR; the
example app's `examples/prisma/migrations/cipherstash/install_eql_bundle/`
copy was also synced. `@cipherstash/schema` no longer bundles any
`.sql` (verified — no `*.sql` under `packages/schema/`).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@calvinbrewer calvinbrewer changed the title feat: upgrade protect-ffi to 0.22.0 feat: upgrade protect-ffi to 0.23.0 May 21, 2026
Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
packages/stack/__tests__/number-protect.test.ts (1)

288-291: ⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Update stale comment and validate discriminator value.

The comment contradicts the assertions and should reflect EQL v2.3 discriminator behavior. The assertions should also validate the discriminator value 'ct' rather than just checking for presence.

📝 Proposed fix
-    // Forward compatibility: new encryptions should NOT have k field
-    expect(encryptedData.data[0].data).toHaveProperty('k')
-    expect(encryptedData.data[1].data).toHaveProperty('k')
-    expect(encryptedData.data[2].data).toHaveProperty('k')
+    // Forward compatibility: new encryptions should include k: 'ct' discriminator
+    expect(encryptedData.data[0].data).toHaveProperty('k', 'ct')
+    expect(encryptedData.data[1].data).toHaveProperty('k', 'ct')
+    expect(encryptedData.data[2].data).toHaveProperty('k', 'ct')
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/__tests__/number-protect.test.ts` around lines 288 - 291,
Update the stale comment to state that EQL v2.3 uses a discriminator and then
change the three assertions that currently check for presence of 'k' to
explicitly assert the discriminator equals 'ct' for each entry (i.e., check
encryptedData.data[0].data.discriminator === 'ct',
encryptedData.data[1].data.discriminator === 'ct',
encryptedData.data[2].data.discriminator === 'ct'). Ensure you reference the
encryptedData variable and its .data array entries when making the replacement
so the test validates the discriminator value rather than merely the presence of
'k'.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/tests.yml:
- Around line 16-19: Update the stale justification comment that mentions
"protect-ffi 0.21.x": change the reference to reflect the current bump to
`@cipherstash/protect-ffi` 0.23.0 (or remove the specific minor-version phrasing
and say "protect-ffi 0.23.x") so it matches the EQL image pin "eql-2.3.1"
referenced in the same comment; edit the comment block that discusses the
Postgres + EQL test image and its lockstep relationship with protect-ffi to use
the correct package version string (`@cipherstash/protect-ffi` 0.23.0) or a
generalized "0.23.x" wording for future-proofing.

In `@packages/stack/package.json`:
- Line 206: The PR title and possibly the source branch name are out of sync
with the package.json change: package "`@cipherstash/protect-ffi`" was bumped to
0.23.0 but the PR title (and likely branch protect-ffi-0-22-0) still reference
0.22.0; update the PR title to reflect "upgrade protect-ffi to 0.23.0" and
rename or update the source branch/description if autogenerated so
changelog/release notes match the package.json change for
`@cipherstash/protect-ffi` v0.23.0.

---

Duplicate comments:
In `@packages/stack/__tests__/number-protect.test.ts`:
- Around line 288-291: Update the stale comment to state that EQL v2.3 uses a
discriminator and then change the three assertions that currently check for
presence of 'k' to explicitly assert the discriminator equals 'ct' for each
entry (i.e., check encryptedData.data[0].data.discriminator === 'ct',
encryptedData.data[1].data.discriminator === 'ct',
encryptedData.data[2].data.discriminator === 'ct'). Ensure you reference the
encryptedData variable and its .data array entries when making the replacement
so the test validates the discriminator value rather than merely the presence of
'k'.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: e682f838-39d6-4b64-8f19-297f6bf9b5cf

📥 Commits

Reviewing files that changed from the base of the PR and between 18875b5 and 1bed6f7.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (44)
  • .changeset/protect-ffi-0-23-0.md
  • .claude/skills/prisma-next-extension-upgrade
  • .github/workflows/tests.yml
  • examples/prisma/migrations/cipherstash/20260601T0000_install_eql_bundle/migration.json
  • examples/prisma/migrations/cipherstash/20260601T0000_install_eql_bundle/ops.json
  • local/docker-compose.yml
  • packages/cli/src/installer/index.ts
  • packages/cli/src/sql/cipherstash-encrypt-no-operator-family.sql
  • packages/cli/src/sql/cipherstash-encrypt-supabase.sql
  • packages/cli/src/sql/cipherstash-encrypt.sql
  • packages/drizzle/GENERATE_EQL_MIGRATION_CLI.md
  • packages/drizzle/README.md
  • packages/drizzle/__tests__/integration-test-helpers.ts
  • packages/drizzle/src/bin/generate-eql-migration.ts
  • packages/prisma-next/migrations/20260601T0000_install_eql_bundle/migration.json
  • packages/prisma-next/migrations/20260601T0000_install_eql_bundle/ops.json
  • packages/prisma-next/src/migration/eql-bundle.ts
  • packages/prisma-next/src/migration/eql-install.generated.ts
  • packages/protect-dynamodb/src/helpers.ts
  • packages/protect-dynamodb/src/operations/search-terms.ts
  • packages/protect/README.md
  • packages/protect/__tests__/backward-compat.test.ts
  • packages/protect/__tests__/json-protect.test.ts
  • packages/protect/__tests__/number-protect.test.ts
  • packages/protect/__tests__/searchable-json-pg.test.ts
  • packages/protect/package.json
  • packages/protect/src/ffi/operations/batch-encrypt-query.ts
  • packages/protect/src/helpers/index.ts
  • packages/protect/src/types.ts
  • packages/schema/__tests__/schema.test.ts
  • packages/schema/src/index.ts
  • packages/stack/__tests__/backward-compat.test.ts
  • packages/stack/__tests__/json-protect.test.ts
  • packages/stack/__tests__/number-protect.test.ts
  • packages/stack/__tests__/schema-builders.test.ts
  • packages/stack/package.json
  • packages/stack/src/dynamodb/helpers.ts
  • packages/stack/src/encryption/helpers/index.ts
  • packages/stack/src/encryption/operations/batch-encrypt-query.ts
  • packages/stack/src/schema/index.ts
  • packages/stack/src/types.ts
  • pnpm-workspace.yaml
  • skills-lock.json
  • skills/prisma-next-extension-upgrade
✅ Files skipped from review due to trivial changes (16)
  • local/docker-compose.yml
  • packages/drizzle/src/bin/generate-eql-migration.ts
  • pnpm-workspace.yaml
  • packages/drizzle/GENERATE_EQL_MIGRATION_CLI.md
  • skills-lock.json
  • .changeset/protect-ffi-0-23-0.md
  • examples/prisma/migrations/cipherstash/20260601T0000_install_eql_bundle/migration.json
  • packages/prisma-next/migrations/20260601T0000_install_eql_bundle/migration.json
  • packages/protect/package.json
  • packages/protect/README.md
  • packages/prisma-next/src/migration/eql-bundle.ts
  • .claude/skills/prisma-next-extension-upgrade
  • packages/protect/tests/number-protect.test.ts
  • skills/prisma-next-extension-upgrade
  • packages/drizzle/README.md
  • packages/protect/tests/json-protect.test.ts

Comment on lines 16 to 19
# Postgres + EQL for the integration tests. Official EQL image —
# PostgreSQL 17 with EQL pre-installed via /docker-entrypoint-initdb.d.
# Pinned to eql-2.2.1 to match the EQL payload format the code emits
# Pinned to eql-2.3.1 to match the EQL payload format the code emits
# (protect-ffi 0.21.x); bump in lockstep with the protect-ffi upgrade.
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

Stale comment: references protect-ffi 0.21.x after the 0.23.0 bump.

The justification block says the EQL image is pinned to match protect-ffi 0.21.x, but this PR moves @cipherstash/protect-ffi to 0.23.0 and the EQL image to 17-2.3.1. Refresh the comment so future maintainers don't read a misleading lockstep reference.

📝 Suggested fix
-    # Postgres + EQL for the integration tests. Official EQL image —
-    # PostgreSQL 17 with EQL pre-installed via /docker-entrypoint-initdb.d.
-    # Pinned to eql-2.3.1 to match the EQL payload format the code emits
-    # (protect-ffi 0.21.x); bump in lockstep with the protect-ffi upgrade.
+    # Postgres + EQL for the integration tests. Official EQL image —
+    # PostgreSQL 17 with EQL pre-installed via /docker-entrypoint-initdb.d.
+    # Pinned to eql-2.3.1 to match the EQL payload format the code emits
+    # (protect-ffi 0.23.x); bump in lockstep with the protect-ffi upgrade.
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
# Postgres + EQL for the integration tests. Official EQL image —
# PostgreSQL 17 with EQL pre-installed via /docker-entrypoint-initdb.d.
# Pinned to eql-2.2.1 to match the EQL payload format the code emits
# Pinned to eql-2.3.1 to match the EQL payload format the code emits
# (protect-ffi 0.21.x); bump in lockstep with the protect-ffi upgrade.
# Postgres + EQL for the integration tests. Official EQL image —
# PostgreSQL 17 with EQL pre-installed via /docker-entrypoint-initdb.d.
# Pinned to eql-2.3.1 to match the EQL payload format the code emits
# (protect-ffi 0.23.x); bump in lockstep with the protect-ffi upgrade.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/tests.yml around lines 16 - 19, Update the stale
justification comment that mentions "protect-ffi 0.21.x": change the reference
to reflect the current bump to `@cipherstash/protect-ffi` 0.23.0 (or remove the
specific minor-version phrasing and say "protect-ffi 0.23.x") so it matches the
EQL image pin "eql-2.3.1" referenced in the same comment; edit the comment block
that discusses the Postgres + EQL test image and its lockstep relationship with
protect-ffi to use the correct package version string (`@cipherstash/protect-ffi`
0.23.0) or a generalized "0.23.x" wording for future-proofing.

"dependencies": {
"@byteslice/result": "0.2.0",
"@cipherstash/protect-ffi": "0.21.4",
"@cipherstash/protect-ffi": "0.23.0",
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

PR title is out of sync with the actual version bump.

The PR title says feat: upgrade protect-ffi to 0.22.0 but this diff bumps @cipherstash/protect-ffi to 0.23.0 (and the stack context/changeset describe a 0.23.0 upgrade). Update the PR title (and source branch description if it's autogenerated from protect-ffi-0-22-0) so changelog/release notes are accurate.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/stack/package.json` at line 206, The PR title and possibly the
source branch name are out of sync with the package.json change: package
"`@cipherstash/protect-ffi`" was bumped to 0.23.0 but the PR title (and likely
branch protect-ffi-0-22-0) still reference 0.22.0; update the PR title to
reflect "upgrade protect-ffi to 0.23.0" and rename or update the source
branch/description if autogenerated so changelog/release notes match the
package.json change for `@cipherstash/protect-ffi` v0.23.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants