|
| 1 | +--- |
| 2 | +name: drizzle |
| 3 | +description: Use this skill when writing or modifying Drizzle ORM schemas, queries, or migrations in this repo — specifically the `@internal/dashboard-agent-db` package (the dashboard agent's conversation datastore). Covers pg-core schema definition, the postgres-js driver, drizzle-kit migrations, and this repo's conventions: a dedicated Postgres schema, foreign-key-free cross-database design, pooler-safe connections, and the access-pattern query layer. Drizzle is NOT the main database — that's Prisma. |
| 4 | +allowed-tools: Read, Write, Edit, Glob, Grep, Bash |
| 5 | +--- |
| 6 | + |
| 7 | +# Drizzle ORM (this repo) |
| 8 | + |
| 9 | +Drizzle is used in exactly one place: **`internal-packages/dashboard-agent-db`** (`@internal/dashboard-agent-db`), the in-dashboard agent's conversation store. Everything else in the monorepo is **Prisma** (`@trigger.dev/database`). Keep them separate. |
| 10 | + |
| 11 | +Pinned versions: **`drizzle-orm` ^0.45**, **`drizzle-kit` ^0.31** (dev), **`postgres` ^3.4** (postgres.js driver). drizzle-orm and drizzle-kit are intentionally on different version lines — 0.31.x is the correct companion for 0.45.x, there is no peer dependency between them. |
| 12 | + |
| 13 | +## Critical rules |
| 14 | + |
| 15 | +1. **Drizzle is only the agent's own datastore.** The agent (and its task bundle) must have **no access to the main Prisma database or ClickHouse**. Never import the Prisma client into the agent task or into `@internal/dashboard-agent-db`. Main data is reached via the API, not Drizzle. |
| 16 | +2. **Foreign-key-free.** In cloud this DB is a *separate* PlanetScale database, so it can't FK into the main DB. Reference main entities (`organizationId`, `userId`, …) **by id only — never `.references()`**. Joins happen in app code; tenant scoping is enforced in the query layer. |
| 17 | +3. **One dedicated Postgres schema.** All tables live under `pgSchema("trigger_dashboard_agent")` so they're schema-qualified and isolated from Prisma's `public` schema (this is what makes the OSS single-database fallback safe). |
| 18 | +4. **Pooler-safe connections.** Connections go through a transaction-mode pooler (PlanetScale / PgBouncer-style), so postgres.js must run with **`prepare: false`** — prepared statements don't survive a connection being handed to another client between checkouts. |
| 19 | +5. **Node16 module resolution.** Relative imports need explicit **`.js`** extensions (`import { chats } from "./schema.js"`), even though the source is `.ts`. |
| 20 | +6. **Scope every user query.** All queries that touch user data go through `src/queries.ts` and are scoped by `organizationId` / `userId`, so callers can't forget the `where`. Don't write ad-hoc cross-tenant queries elsewhere. |
| 21 | + |
| 22 | +## Package layout |
| 23 | + |
| 24 | +```text |
| 25 | +internal-packages/dashboard-agent-db/ |
| 26 | + drizzle.config.ts # drizzle-kit config (schema path, out dir, schemaFilter) |
| 27 | + drizzle/ # generated migrations (committed) |
| 28 | + src/ |
| 29 | + schema.ts # pgSchema + table definitions |
| 30 | + client.ts # createDashboardAgentDb() — postgres.js + drizzle |
| 31 | + queries.ts # the access-pattern layer (org/user-scoped) |
| 32 | + index.ts # barrel: re-exports schema, client, queries |
| 33 | +``` |
| 34 | + |
| 35 | +`package.json` points `main`/`types` at `./src/index.ts` (consumed as source, no build step) — same as other simple internal packages. |
| 36 | + |
| 37 | +## Schema (pg-core) |
| 38 | + |
| 39 | +Use `pgSchema(...).table(...)`, not the bare `pgTable`, so tables land in the dedicated schema. ([schemas](https://orm.drizzle.team/docs/schemas), [pg column types](https://orm.drizzle.team/docs/column-types/pg), [indexes](https://orm.drizzle.team/docs/indexes-constraints)) |
| 40 | + |
| 41 | +```ts |
| 42 | +import { sql } from "drizzle-orm"; |
| 43 | +import { index, jsonb, pgSchema, text, timestamp } from "drizzle-orm/pg-core"; |
| 44 | + |
| 45 | +export const dashboardAgentSchema = pgSchema("trigger_dashboard_agent"); |
| 46 | + |
| 47 | +export const chats = dashboardAgentSchema.table( |
| 48 | + "chats", |
| 49 | + { |
| 50 | + id: text("id").primaryKey(), |
| 51 | + organizationId: text("organization_id").notNull(), // FK-free: id only, no .references() |
| 52 | + userId: text("user_id").notNull(), |
| 53 | + title: text("title").notNull().default("New chat"), |
| 54 | + // JSONB with a typed view; .default([]) / .default({}) emit '[]'::jsonb / '{}'::jsonb |
| 55 | + messages: jsonb("messages").$type<unknown[]>().notNull().default([]), |
| 56 | + metadata: jsonb("metadata").$type<Record<string, unknown>>().notNull().default({}), |
| 57 | + deletedAt: timestamp("deleted_at", { withTimezone: true }), // soft delete |
| 58 | + lastMessageAt: timestamp("last_message_at", { withTimezone: true }), |
| 59 | + createdAt: timestamp("created_at", { withTimezone: true }).notNull().defaultNow(), |
| 60 | + }, |
| 61 | + // Extra config returns an ARRAY in drizzle-orm 0.36+ (not an object). |
| 62 | + (t) => [ |
| 63 | + // Partial + ordered composite index. `.desc()` on the column, `.where(sql`...`)` for partial. |
| 64 | + index("chats_org_user_last_msg_idx") |
| 65 | + .on(t.organizationId, t.userId, t.lastMessageAt.desc()) |
| 66 | + .where(sql`${t.deletedAt} is null`), |
| 67 | + ] |
| 68 | +); |
| 69 | + |
| 70 | +// Inferred row types for the query layer + consumers. |
| 71 | +export type Chat = typeof chats.$inferSelect; |
| 72 | +export type NewChat = typeof chats.$inferInsert; |
| 73 | +``` |
| 74 | + |
| 75 | +Notes: |
| 76 | +- `timestamp(..., { withTimezone: true })` → `timestamp with time zone`. Use `.defaultNow()` for `DEFAULT now()`. |
| 77 | +- For a "newest first, nulls last" sort the partial index uses `.desc()`; the *query* uses raw `sql` for `NULLS LAST` (see below). |
| 78 | +- Don't add `.references()` — see critical rule 2. |
| 79 | + |
| 80 | +## Client (postgres.js + drizzle) |
| 81 | + |
| 82 | +([connect overview](https://orm.drizzle.team/docs/connect-overview)) One small pool, `prepare: false`. In the agent task create it once in `onBoot` (per-process); in the webapp wrap it in the `singleton(...)` helper. |
| 83 | + |
| 84 | +```ts |
| 85 | +import { drizzle, type PostgresJsDatabase } from "drizzle-orm/postgres-js"; |
| 86 | +import postgres, { type Sql } from "postgres"; |
| 87 | +import * as schema from "./schema.js"; |
| 88 | + |
| 89 | +export type DashboardAgentDb = PostgresJsDatabase<typeof schema>; |
| 90 | + |
| 91 | +export function createDashboardAgentDb(connectionString: string, opts: { max?: number } = {}) { |
| 92 | + const sql: Sql = postgres(connectionString, { |
| 93 | + max: opts.max ?? 5, // small — the pooler does the real pooling |
| 94 | + idle_timeout: 20, // release conns when an agent run suspends |
| 95 | + prepare: false, // REQUIRED for transaction-mode poolers |
| 96 | + }); |
| 97 | + return { db: drizzle(sql, { schema }), sql, close: () => sql.end() }; |
| 98 | +} |
| 99 | +``` |
| 100 | + |
| 101 | +## Queries (the access-pattern layer) |
| 102 | + |
| 103 | +([select](https://orm.drizzle.team/docs/select), [insert](https://orm.drizzle.team/docs/insert), [operators](https://orm.drizzle.team/docs/operators), [transactions](https://orm.drizzle.team/docs/transactions), [joins](https://orm.drizzle.team/docs/joins)) |
| 104 | + |
| 105 | +```ts |
| 106 | +import { and, desc, eq, isNull, sql } from "drizzle-orm"; |
| 107 | + |
| 108 | +// Select EXPLICIT columns for list views — never select a large blob (messages) |
| 109 | +// or a secret (tokens) you don't need. `NULLS LAST` needs raw sql in orderBy. |
| 110 | +await db |
| 111 | + .select({ id: chats.id, title: chats.title, lastMessageAt: chats.lastMessageAt }) |
| 112 | + .from(chats) |
| 113 | + .where(and(eq(chats.organizationId, orgId), eq(chats.userId, userId), isNull(chats.deletedAt))) |
| 114 | + .orderBy(sql`${chats.pinnedAt} desc nulls last`, desc(chats.lastMessageAt)) |
| 115 | + .limit(50); |
| 116 | + |
| 117 | +// Idempotent create (avoids a duplicate-key race between two writers). |
| 118 | +await db.insert(chats).values({ id, organizationId: orgId, userId }).onConflictDoNothing(); |
| 119 | + |
| 120 | +// Upsert. |
| 121 | +await db |
| 122 | + .insert(chatSessions) |
| 123 | + .values({ chatId, publicAccessToken }) |
| 124 | + .onConflictDoUpdate({ target: chatSessions.chatId, set: { publicAccessToken, updatedAt: sql`now()` } }); |
| 125 | + |
| 126 | +// Owner-scope a join (this DB is FK-free, so enforce ownership in the query). |
| 127 | +await db |
| 128 | + .select({ /* session cols */ }) |
| 129 | + .from(chatSessions) |
| 130 | + .innerJoin(chats, eq(chats.id, chatSessions.chatId)) |
| 131 | + .where(and(eq(chatSessions.chatId, chatId), eq(chats.userId, userId))); |
| 132 | + |
| 133 | +// Multi-write that must be consistent on the next read → one transaction. |
| 134 | +await db.transaction(async (tx) => { |
| 135 | + await tx.update(chats).set({ messages, updatedAt: sql`now()` }).where(eq(chats.id, chatId)); |
| 136 | + await tx.insert(chatSessions).values({ /* ... */ }).onConflictDoUpdate({ /* ... */ }); |
| 137 | +}); |
| 138 | +``` |
| 139 | + |
| 140 | +Use `sql\`now()\`` for DB-side timestamps in updates. |
| 141 | + |
| 142 | +## Migrations (drizzle-kit) |
| 143 | + |
| 144 | +([kit overview](https://orm.drizzle.team/docs/kit-overview), [generate](https://orm.drizzle.team/docs/drizzle-kit-generate), [migrate](https://orm.drizzle.team/docs/drizzle-kit-migrate)) |
| 145 | + |
| 146 | +`drizzle.config.ts` must set **`schemaFilter`** so drizzle-kit only ever manages our schema — never Prisma's `public` (critical in the OSS single-DB fallback): |
| 147 | + |
| 148 | +```ts |
| 149 | +import { defineConfig } from "drizzle-kit"; |
| 150 | +export default defineConfig({ |
| 151 | + schema: "./src/schema.ts", |
| 152 | + out: "./drizzle", |
| 153 | + dialect: "postgresql", |
| 154 | + schemaFilter: ["trigger_dashboard_agent"], |
| 155 | + dbCredentials: { url: process.env.DASHBOARD_AGENT_DATABASE_URL ?? process.env.DATABASE_URL ?? "postgres://placeholder" }, |
| 156 | +}); |
| 157 | +``` |
| 158 | + |
| 159 | +Workflow: |
| 160 | + |
| 161 | +```bash |
| 162 | +cd internal-packages/dashboard-agent-db |
| 163 | +pnpm run db:generate # diff schema.ts → emit SQL into drizzle/. OFFLINE (no DB needed). |
| 164 | +# review the generated drizzle/000N_*.sql before committing |
| 165 | +pnpm run db:migrate # apply pending migrations. Needs a real DATABASE URL. |
| 166 | +``` |
| 167 | + |
| 168 | +- `db:generate` is **offline** — it only reads `schema.ts`, so you can verify a schema change compiles to valid DDL with no database. Use it as a fast check. |
| 169 | +- drizzle-kit names migration files with a **random suffix** (`0000_magenta_lilandra.sql`). Don't regenerate a committed migration just to "refresh" it — that churns the filename. After the first migration is committed, schema changes produce a **new** `000N_*.sql`; commit that. |
| 170 | +- Generated DDL for a new schema is one `CREATE SCHEMA` + schema-qualified `CREATE TABLE`s + indexes, **no foreign keys** (by design here). |
| 171 | + |
| 172 | +## Common gotchas |
| 173 | + |
| 174 | +- **`prepare: false`** is not optional with a pooler — without it you'll get prepared-statement errors under load. |
| 175 | +- **Missing `.js` extension** on a relative import → TS2835 under Node16 resolution. |
| 176 | +- **Extra-config callback returns an array** `(t) => [ ... ]` in drizzle-orm 0.36+. The old object form `(t) => ({ ... })` is deprecated. |
| 177 | +- **`NULLS LAST` / `NULLS FIRST`** aren't on the `desc()` helper — use raw `sql\`col desc nulls last\`` in `orderBy`. |
| 178 | +- **Don't `SELECT *` into list views** — explicitly pick columns so you never ship a megabyte `messages` blob or a session token to a list query. |
| 179 | +- **Adding a dependency**: edit `package.json`, then `pnpm i` from the repo root (never `pnpm add`). Mind the repo's `minimumReleaseAge` (3 days) — pin with a caret range and let pnpm resolve an old-enough version. |
| 180 | + |
| 181 | +## Reference (official docs) |
| 182 | + |
| 183 | +- Schema declaration — https://orm.drizzle.team/docs/sql-schema-declaration |
| 184 | +- PostgreSQL column types — https://orm.drizzle.team/docs/column-types/pg |
| 185 | +- Schemas (`pgSchema`) — https://orm.drizzle.team/docs/schemas |
| 186 | +- Indexes & constraints — https://orm.drizzle.team/docs/indexes-constraints |
| 187 | +- Connect (postgres-js) — https://orm.drizzle.team/docs/connect-overview |
| 188 | +- Select / Insert / Update / Delete — https://orm.drizzle.team/docs/select · /insert · /update · /delete |
| 189 | +- Joins / Operators — https://orm.drizzle.team/docs/joins · /operators |
| 190 | +- Transactions — https://orm.drizzle.team/docs/transactions |
| 191 | +- drizzle-kit (generate / migrate / push) — https://orm.drizzle.team/docs/kit-overview |
0 commit comments