Upgrade to EQL 2.3.0 — query, migration & index changes

[coderdan]

Context

EQL 2.3.0 is released: https://github.com/cipherstash/encrypt-query-language/releases/tag/eql-2.3.0

It is a breaking release. Authoritative detail: the upgrade guide (numbered notes U-001…U-008) and the CHANGELOG. Three themes matter for cipherstash/stack:

1. Breaking payload format. The on-the-wire CipherCell changed — root b3 removed; ste_vec elements carry hm (not b3); the ocf/ocv ORE-CLLW split collapsed to a single oc; OPE removed. Existing encrypted data must be re-encrypted.
2. Functional indexes are now canonical (U-001) and the comparison operators (=, <>, <, <=, >, >=, LIKE, ILIKE) are inlinable — bare-form queries (WHERE col > $1) structurally match the functional index. The eql_v2.gt()/lt()/gte()/lte() helper functions are deprecated (removal in EQL 3.0) and do not engage the functional index.
3. -> now returns eql_v2.ste_vec_entry (U-007); new eql_v2.stevec_query DOMAIN for typed containment (U-008).

Good news: every eql_v2.* function the stack packages currently call (gt, lt, gte, lte, eq, neq, like, ilike, order_by, ste_vec, hmac_256, bloom_filter, jsonb_path_query_first, jsonb_path_exists) still exists in 2.3 — verified against the release SQL — so nothing throws "function does not exist". The work is (a) the data-plane bump, and (b) migrating query/index code off the deprecated, non-index-engaging forms onto the inlinable operator forms.

File paths below are relative to this repo; line numbers are from a snapshot — treat as approximate.

---

1. Data plane — bundled EQL SQL + protect-ffi (breaking, must-do)

1a. Bump the bundled EQL installer SQL to eql-2.3.0

Three locations hold a pinned EQL bundle, currently eql-2.2.1:

• packages/prisma-next/src/migration/eql-install.generated.ts — regenerate from the 2.3.0 release:
  • the // Bundle pinned version: comment (line ~2)
  • EQL_INSTALL_VERSION = 'eql-2.2.1' → 'eql-2.3.0' (line ~5)
  • the embedded version-check assertion SELECT 'eql-2.2.1'; → SELECT 'eql-2.3.0'; (line ~12)
  • the embedded schema bundle (~5,750-line body) — replace with the 2.3.0 release SQL
• packages/cli/src/sql/ — re-copy from the 2.3.0 release assets:
  • cipherstash-encrypt.sql ← https://github.com/cipherstash/encrypt-query-language/releases/download/eql-2.3.0/cipherstash-encrypt.sql
  • cipherstash-encrypt-supabase.sql ← https://github.com/cipherstash/encrypt-query-language/releases/download/eql-2.3.0/cipherstash-encrypt-supabase.sql
  • cipherstash-encrypt-no-operator-family.sql — ⚠️ verify: the 2.3.0 release ships only cipherstash-encrypt.sql and cipherstash-encrypt-supabase.sql. Confirm which variant this file tracks (the Supabase build is the operator-class-free variant).
• packages/drizzle/src/bin/generate-eql-migration.ts (line ~8) downloads releases/latest/.../cipherstash-encrypt.sql — latest now resolves to 2.3.0 automatically. Consider pinning an explicit tag for reproducible migrations.

1b. Bump @cipherstash/protect-ffi

Ciphertext is produced by the @cipherstash/protect-ffi native binding wrapped by packages/protect/. EQL 2.3's payload format is breaking, so protect-ffi must be bumped to the version that emits the 2.3 ste_vec shape (hm, single oc, no OPE). If protect-ffi keeps emitting 2.2 payloads against a 2.3-installed schema, equality / range / containment silently mismatch (no error — zero rows). This is the critical coordination point: the SQL bundle bump (1a) and the protect-ffi bump must ship together.

1c. Re-encryption

Existing encrypted columns must be re-encrypted (b3→hm, ocf/ocv→oc). Smoke test from the EQL upgrade guide: a GROUP BY / DISTINCT on an encrypted column raises loudly via eql_v2.hash_encrypted, naming the missing hm term, if a column has not been re-encrypted. (WHERE col = $1 fails silently with zero rows — see U-002.)

---

2. Query construction — range/sort/match operators (correctness + index engagement)

Drizzle adapter — packages/drizzle/src/pg/operators.ts

• gt / gte / lt / lte (~lines 656–714) emit eql_v2.gt(col, $1) etc. These helpers are deprecated in 2.3 (removal in EQL 3.0, U-005) and — being plpgsql wrappers around eql_v2.compare()'s priority list — do not structurally match the functional ORE index. Switch to Drizzle's native gt/gte/lt/lte operators (exactly as eq/ne already use native =/!= at ~lines 716–751). Native operators emit bare col > $1, which EQL 2.3 made inlinable so the planner matches a functional btree on eql_v2.ore_block_u64_8_256(col).
• between / notBetween (~lines 757–812) emit eql_v2.gte(...) AND eql_v2.lte(...) — rewrite to bare col >= $1 AND col <= $2.
• asc / desc (~lines 1628–1657) emit ORDER BY eql_v2.order_by(col). eql_v2.order_by still exists but is plpgsql (returns eql_v2.ore_block_u64_8_256(col), non-inlined) so it will not engage the functional index. Emit the extractor form: ORDER BY eql_v2.ore_block_u64_8_256(col) — the documented 2.3 recipe that matches a functional btree. (A bare ORDER BY col does not match the index expression; the extractor form is required.)
• like / ilike / notIlike (~lines 817–871) emit eql_v2.like(col, $1) / eql_v2.ilike(...). Not deprecated, but the bare LIKE/ILIKE operators are the forms EQL 2.3 made inlinable + index-matching against GIN (eql_v2.bloom_filter(col)). Prefer native like/ilike operators for reliable index engagement.
• eq / ne (~lines 716–751) already use native =/!= — ✅ correct 2.3 pattern, no change. Note 2.3 made = strict-hmac (U-002): equality requires both sides carry hm; on a non-re-encrypted column it returns zero rows.

Prisma-Next lowering — packages/prisma-next/src/execution/operators.ts

The operator-lowering templates (~lines 533–591) lower everything to eql_v2.* function calls: eql_v2.eq(...), eql_v2.gt/gte/lt/lte(...), eql_v2.gte(...) AND eql_v2.lte(...), eql_v2.ilike(...). Same rationale as the drizzle adapter — these should emit PostgreSQL operators (=, <>, <, <=, >, >=, LIKE, ILIKE), not eql_v2.* function calls, so the planner inlines and engages the functional indexes. eql_v2.gt/gte/lt/lte are additionally deprecated.

---

3. JSONB / -> changes (U-007, U-008)

-> return type changed

packages/drizzle/src/pg/operators.ts jsonbGet() (~line 929) emits col -> selector. In EQL 2.3 the -> operator returns eql_v2.ste_vec_entry (a DOMAIN over jsonb), not eql_v2_encrypted. The returned value still carries {i, v, s, c, hm-or-oc}, so decryption of the projected c still works. Verify any code path consuming a jsonbGet result as eql_v2_encrypted (casts, type annotations, decrypt calls). If you build WHERE col -> 'sel' = $1 predicates, the bound parameter must now be cast $1::eql_v2.ste_vec_entry (not ::eql_v2_encrypted); equality on a selected entry is XOR-aware (eql_v2.eq_term, covering both hm- and oc-indexed selectors).

jsonbPathExists / jsonbPathQueryFirst

eql_v2.jsonb_path_exists / eql_v2.jsonb_path_query_first still exist and are now inlinable SQL — ✅ no change.

New: typed containment (additive)

EQL 2.3 adds eql_v2.stevec_query (a DOMAIN for containment needles — {"sv":[...]} with no c field) and eql_v2.to_stevec_query(eql_v2_encrypted). If/when the adapters expose @> containment, the typed recipe is WHERE col @> $1::eql_v2.stevec_query. Additive — nothing breaks — but the recommended path if containment is on the roadmap.

---

4. Index creation recipes

The only CREATE INDEX emission found is packages/bench/sql/schema.sql (a bench fixture). Verify whether the product migration path auto-creates per-column functional indexes: if it does, those recipes must match 2.3; if indexes are left to the user, the docs must carry the 2.3 recipes.

Canonical EQL 2.3 functional-index recipes:

Query	Index
Equality (=, <>, joins, GROUP BY)	USING hash (eql_v2.hmac_256(col))
LIKE / ILIKE	USING gin (eql_v2.bloom_filter(col))
Range + ORDER BY	USING btree (eql_v2.ore_block_u64_8_256(col)) — ORDER BY additionally needs a btree opclass; not available on Supabase
JSONB field equality	USING hash (eql_v2.eq_term(col -> '<selector>'))
JSONB containment (@>)	USING gin (eql_v2.to_stevec_query(col)::jsonb jsonb_path_ops)

bench/sql/schema.sql currently uses USING gin (eql_v2.ste_vec(col)) for JSONB — update to the to_stevec_query(...)::jsonb jsonb_path_ops recipe (what the typed @> inlines into in 2.3). The hmac_256 hash and bloom_filter GIN indexes in that file are unchanged in 2.3.

Operator-class indexes (col eql_v2.encrypted_operator_class) are deprecated for the equality / LIKE path (U-001) — functional indexes work on Supabase / managed PG without superuser. The narrow exception is ORDER BY over ORE columns, which still needs a btree opclass.

---

5. Test fixtures / payload-shape assumptions

packages/stack/__tests__/encryption-helpers.test.ts hard-codes payload fixtures and isEncryptedPayload() validation with fields v, c, sv, i, k, ob, bf, hm — all valid in 2.3. Verify:

• no fixture carries a root-level b3 (removed), an ocf/ocv split, or op/OPE fields;
• sv-element fixtures use the 2.3 element shape — s + c + exactly one of hm or oc (no b3, no ocf/ocv);
• isEncryptedPayload() requires no removed field.

Once protect-ffi is bumped (1b), regenerate any fixtures captured from real encrypt output.

---

Verification checklist

• All three bundled EQL SQL artefacts updated to eql-2.3.0; version string + version-check assertion updated.
• @cipherstash/protect-ffi bumped to the EQL-2.3-aligned release.
• Drizzle gt/gte/lt/lte/between emit bare operators; asc/desc emit ORDER BY eql_v2.ore_block_u64_8_256(col).
• Prisma-next lowering templates emit bare operators.
• like/ilike emit bare operators.
• -> return-type change audited (jsonbGet consumers, predicate param casts).
• Index recipes match the 2.3 table above.
• Test fixtures free of removed payload fields; regenerated post protect-ffi bump.
• Integration test against a DB with eql-2.3.0 installed + 2.3-encrypted data: equality, range, ORDER BY, LIKE, JSONB field access/containment, GROUP BY.

References

• EQL 2.3.0 release: https://github.com/cipherstash/encrypt-query-language/releases/tag/eql-2.3.0
• Upgrade guide (U-001…U-008): https://github.com/cipherstash/encrypt-query-language/blob/main/docs/upgrading/v2.3.md
• CHANGELOG: https://github.com/cipherstash/encrypt-query-language/blob/main/CHANGELOG.md