Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
92 changes: 80 additions & 12 deletions content/stack/cipherstash/encryption/queries.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ description: Equality, match, and range query patterns for encrypted PostgreSQL

# Searchable encryption queries

This page covers the three query families available for encrypted columns: equality, match (free-text), and range/order. Each section shows the SDK predicate, the raw SQL form, the underlying EQL index, and links to the corresponding index setup.
This page covers the query patterns available for encrypted columns: equality, match (free-text), range/order, JSONB, grouping, and joins. Each section shows the SDK predicate, the raw SQL form, the underlying EQL index, and links to the corresponding index setup.

For index creation (the `CREATE INDEX` statements your database needs), see [Setting up indexes](/stack/cipherstash/encryption/indexes).

Expand Down Expand Up @@ -215,7 +215,7 @@ const { data } = await eSupabase

## JSONB queries

Query encrypted JSON columns using path existence or containment. Uses the `ste_vec` index.
Encrypted JSON columns — configured with `.searchableJson()` — support containment, field access, field-level filtering, and JSON path queries, all without decrypting the document. The operators and functions mirror PostgreSQL's native JSONB surface; the [searchable JSON reference](/stack/cipherstash/proxy/searchable-json) lists the complete set.

**Schema:**

Expand All @@ -225,10 +225,17 @@ const documents = encryptedTable("documents", {
})
```

**SDK (path existence):**
### Containment

Test whether an encrypted document contains a given field, value, or nested structure with `@>`:

```sql
SELECT * FROM documents
WHERE eql_v2.jsonb_array(metadata) @> eql_v2.jsonb_array($1::eql_v2_encrypted);
```

```typescript filename="src/queries.ts"
const term = await client.encryptQuery("$.user.role", {
const term = await client.encryptQuery({ role: "admin" }, {
column: documents.metadata,
table: documents,
})
Expand All @@ -239,16 +246,39 @@ const result = await pgClient.query(
)
```

**SDK (containment):**
Engages the `GIN (eql_v2.jsonb_array(metadata))` index — see [JSONB index setup](/stack/cipherstash/encryption/indexes#jsonb).

```typescript filename="src/queries.ts"
const term = await client.encryptQuery({ role: "admin" }, {
column: documents.metadata,
table: documents,
})
### Field access

Extract a field from an encrypted JSON document with `->`. The result is a typed encrypted value you can filter on, order by, or decrypt:

```sql
SELECT metadata -> 'role' FROM documents;
```

**Drizzle:**
### Field equality and ordering

An extracted field behaves like any encrypted column — filter it by equality, or order by it:

```sql
-- Equality on a JSON field
SELECT * FROM documents WHERE metadata -> 'role' = $1::eql_v2.ste_vec_entry;

-- Order by a JSON field
SELECT * FROM documents ORDER BY eql_v2.ore_cllw(metadata -> 'created_at');
```

### Path queries

Check whether a JSON path resolves, or read the value at a path:

```sql
-- Does the path exist?
SELECT * FROM documents WHERE jsonb_path_exists(metadata, '$.user.role');

-- Read the value at a path
SELECT jsonb_path_query(metadata, '$.user.role') FROM documents;
```

```typescript filename="src/queries.ts"
const results = await db
Expand All @@ -257,4 +287,42 @@ const results = await db
.where(await encryptionOps.jsonbPathExists(documentsTable.metadata, "$.user.role"))
```

**Underlying index:** [JSONB index setup](/stack/cipherstash/encryption/indexes#jsonb)
For the full JSONB operator and function reference — `->>`, `<@`, `jsonb_path_query_first`, `jsonb_array_elements`, `jsonb_array_length`, and JSONPath syntax — see [searchable JSON](/stack/cipherstash/proxy/searchable-json).

---

## GROUP BY and DISTINCT

Encrypted columns can be grouped and deduplicated. Equality on encrypted data is by HMAC term — two encryptions of the same plaintext share the same HMAC term — so PostgreSQL groups rows by plaintext value without decrypting them:

```sql
-- Count rows per distinct (encrypted) category
SELECT category, count(*) FROM products GROUP BY category;

-- Distinct encrypted values
SELECT DISTINCT category FROM products;
```

To group by a field inside an encrypted JSON column, group by the field's equality term:

```sql
SELECT count(*) FROM users GROUP BY eql_v2.eq_term(profile -> 'department');
```

`GROUP BY` and `DISTINCT` require the column to carry an HMAC term — set the `unique` index in the column's encryption config. No functional index is needed for grouping itself: a `GROUP BY` reads every row regardless.

---

## Joins

Encrypted columns can be joined on equality. The `=` operator compares HMAC terms, so PostgreSQL satisfies the join with a hash join, exactly as it would for a plaintext column:

```sql
SELECT o.id, c.name
FROM orders o
JOIN customers c ON o.customer_email = c.email;
```

The two joined columns must be encrypted under the same configuration, so that equal plaintexts produce equal HMAC terms. Standard ORM joins (Drizzle's `.innerJoin()`, and so on) work unchanged — there is no special API and no query value to encrypt.

**Underlying index:** a `hash` functional index on each joined column — see [Equality index setup](/stack/cipherstash/encryption/indexes#equality).
91 changes: 43 additions & 48 deletions content/stack/reference/cipher-cell.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -47,11 +47,11 @@ A CipherCell is the **unit of encrypted storage** in CipherStash: a portable, se
## Structure

<Callout type="warn">
**Required fields**: Only `i` (identifier) and `v` (version) are required.
**Required fields**: `v` (version) and `i` (identifier) are always present. A scalar payload also requires `c` (ciphertext); an STE-vector payload requires `k` and `sv`.

**Payload requirement**: Either `c` (ciphertext) or `sv` (structured encryption vector) must be present, but never both.
**Payload shape**: a CipherCell is either a scalar value (`c`) or an STE vector (`sv`) — never both. The `k` field names the shape.

**Optional fields**: All searchable encrypted metadata (SEM) fields are optional and only included when the corresponding index types are configured.
**Optional fields**: all searchable encrypted metadata (SEM) fields are optional, included only when the corresponding index type is configured.
</Callout>

A CipherCell is stored as a JSON object with the following top-level structure:
Expand All @@ -63,6 +63,7 @@ A CipherCell is stored as a JSON object with the following top-level structure:
"c": "column_name"
},
"v": 2,
"k": "ct",
"c": "encrypted_data_in_messagepack_base85",
"hm": "2e182f0c444d1d51f5f70f32d778b2eaa854f5921a4a2acaa4446c44055cb777",
"ob": ["ore_block_1", "ore_block_2"],
Expand Down Expand Up @@ -107,49 +108,51 @@ The encryption version used for this CipherCell.

The version field allows for cryptographic algorithm upgrades over time while maintaining backward compatibility.

### `c` - Ciphertext (required unless `sv` is present)
### `k` - Kind

The encrypted record containing the actual plaintext data.
Discriminates the two CipherCell shapes.

**Type**: String (MessagePack encoded and Base85 encoded)
**Type**: String — `"ct"` (scalar ciphertext) or `"sv"` (STE vector)

**Required**: Yes (unless `sv` field is present)
**Required**: Yes for STE-vector payloads; optional for scalar payloads

```json
{
"c": "Xk}0>Z*pVbW@%*8a%F0@"
"k": "ct"
}
```

<Callout type="warn">
Either `c` or `sv` must be present in every CipherCell, but never both. Use `c` for standard encrypted values and `sv` for structured encryption vectors (arrays or JSON structures).
</Callout>
`k` tells EQL — and any other consumer — which shape to expect: `"ct"` for a single encrypted value (carries a `c` field), `"sv"` for a structured-encryption vector (carries an `sv` array).

### `a` - Array item flag
### `c` - Ciphertext (required unless `sv` is present)

Indicates whether this CipherCell represents an item within an array.
The encrypted record containing the actual plaintext data.

**Type**: Boolean
**Type**: String (MessagePack encoded and Base85 encoded)

**Required**: No
**Required**: Yes (unless `sv` field is present)

```json
{
"a": true
"c": "Xk}0>Z*pVbW@%*8a%F0@"
}
```

<Callout type="warn">
Either `c` or `sv` must be present in every CipherCell, but never both. Use `c` for standard encrypted values and `sv` for structured encryption vectors (arrays or JSON structures).
</Callout>

## Searchable Encrypted Metadata (SEM)

The CipherCell can contain various types of searchable encrypted metadata, each enabling different query capabilities. All SEM fields are optional and only included when the corresponding index type is configured.

### `hm` - HMAC-SHA256

Enables exact match queries using HMAC-SHA256.
Enables exact match queries using HMAC-SHA256. Used both at the root (scalar equality) and on STE-vector elements — where in v2.3 it replaced the former `b3` term.

**Type**: Hex-encoded string (64 characters)

**Index Type**: [Exact](/stack/cipherstash/encryption/searchable-encryption#exact-match)
**Index Type**: [Exact](/stack/cipherstash/encryption/searchable-encryption#exact-matching)

```json
{
Expand All @@ -163,7 +166,7 @@ Enables range queries and ordering using Order Revealing Encryption.

**Type**: Array of strings

**Index Type**: [Order / Range](/stack/cipherstash/encryption/searchable-encryption#range--order)
**Index Type**: [Order / Range](/stack/cipherstash/encryption/searchable-encryption#sorting-and-range-queries)

```json
{
Expand All @@ -180,22 +183,14 @@ Enables substring and pattern matching queries using encrypted Bloom filters wit

**Type**: Array of integers

**Index Type**: [Match](/stack/cipherstash/encryption/searchable-encryption#match-pattern)
**Index Type**: [Match](/stack/cipherstash/encryption/searchable-encryption#free-text-search)

```json
{
"bf": [1234, 5678, 9012, 3456, 7890]
}
```

### `b3` - Blake3

Blake3 hash for exact matches in structured encryption vectors.

**Type**: Hex-encoded string

**Used in**: SteVec (Structured Encryption Vector) subfield

### `s` - Selector

Selector value for field selection in structured encryption vectors.
Expand All @@ -204,48 +199,47 @@ Selector value for field selection in structured encryption vectors.

**Used in**: SteVec (Structured Encryption Vector) subfield

### `ocf` - ORE CLWW Fixed-Width

ORE CLWW (Chenette-Lewi-Weis-Wu) fixed-width scheme for 64-bit integer values in structured encryption vectors.
### `oc` - ORE CLLW

**Type**: String

**Used in**: SteVec (Structured Encryption Vector) subfield

### `ocv` - ORE CLWW Variable-Width
CLLW Order-Revealing Encryption term for STE-vector elements — enables range and ordering queries on a field inside an encrypted JSON document. In v2.3 this single field replaced the former `ocf` / `ocv` split: width is now carried on the ciphertext itself, via a leading domain-tag byte (`0x00` numeric, `0x01` string), so one column can hold mixed numeric and string values with a consistent total order.

ORE CLWW variable-width scheme for string comparison in structured encryption vectors.
**Type**: Hex-encoded string

**Type**: String
**Used in**: STE-vector element

**Used in**: SteVec (Structured Encryption Vector) subfield
**Index Type**: [Order / Range](/stack/cipherstash/encryption/searchable-encryption#sorting-and-range-queries)

### `sv` - Structured Encryption Vector (SteVec) (required unless `c` is present)

Nested array of CipherCells for supporting containment queries and JSON-style operations.
Array of per-selector encrypted terms for an encrypted JSON document — supports containment (`@>`, `<@`) and path queries.

**Type**: Array of CipherCell objects
**Type**: Array of STE-vector elements

**Required**: Yes (unless `c` attribute is present)
**Required**: Yes (unless `c` is present)

```json
{
"sv": [
{
"s": "selector1",
"c": "Xk}0>Z*pVbW@%*8a%F0@",
"hm": "hash1...",
"s": "selector1"
"hm": "hash1..."
},
{
"s": "selector2",
"c": "Yl~1?A+qWcX#&+9b&G1#",
"hm": "hash2...",
"s": "selector2"
"oc": "01a3f9..."
}
]
}
```

SteVec enables queries on array elements and JSON document structures while maintaining encryption. Each element in the `sv` array is itself a CipherCell that can contain SEM fields like `b3`, `s`, `ocf`, and `ocv`.
An STE-vector element is **not** a full CipherCell — it has no `i`, `v`, or nested `sv`. Each element carries:

- `s` — the selector, deterministic per path/key within the document. Required.
- `c` — the element's ciphertext. Required.
- `a` — array marker; `true` when the selector points at a JSON array context. Optional.
- exactly one of `hm` (equality — boolean leaves, array / object roots) **or** `oc` (ordering — string and number leaves).

## Complete example

Expand All @@ -258,6 +252,7 @@ Here's a complete CipherCell with multiple index types enabled:
"c": "price"
},
"v": 2,
"k": "ct",
"c": "Xk}0>Z*pVbW@%*8a%F0@Yl~1?A+qWcX#&+9b&G1#",
"hm": "2e182f0c444d1d51f5f70f32d778b2eaa854f5921a4a2acaa4446c44055cb777",
"ob": [
Expand Down Expand Up @@ -298,4 +293,4 @@ The CipherCell format is consistent across all CipherStash SDKs and tools, ensur
## Database storage

CipherCells can be stored as JSON in any database that supports JSON data types.
However, for search to be supported using the [Encryption SDK](/stack/cipherstash/encryption) or [CipherStash Proxy](/stack/cipherstash/proxy), the `eql_v2.encrypted` database type must be used which is available when the Encrypt Query Language (EQL) helpers have been installed.
However, for search to be supported using the [Encryption SDK](/stack/cipherstash/encryption) or [CipherStash Proxy](/stack/cipherstash/proxy), the `eql_v2_encrypted` database type must be used, which is available once the Encrypt Query Language (EQL) helpers have been installed.
2 changes: 1 addition & 1 deletion content/stack/reference/drizzle.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ description: Encrypted query operators, schema extraction, EQL migration generat
Non-encrypted columns fall back to the standard Drizzle operator automatically.

<Callout type="warn">
Sorting encrypted columns with `asc()` or `desc()` requires operator family support in the database. On managed databases (Supabase, RDS) or when EQL is installed with `--exclude-operator-family`, sort application-side after decrypting instead.
Sorting encrypted columns with `asc()` or `desc()` needs the range index, which relies on a custom operator class — creatable on most providers (self-hosted PostgreSQL, AWS RDS, …) but not Supabase. Where it is unavailable, sort application-side after decrypting instead.
</Callout>

```typescript filename="queries.ts"
Expand Down
Loading