Skip to content

Commit 392855d

Browse files
committed
feat: push --dry-run preview mode
## ELI5 **Problem.** `npm run push -- <env>` immediately starts hitting the live dashboard. There was no way to ask "what would this push do?" before firing it. So a fat-fingered command — wrong org, missing file path, wide-scope push when you meant scoped — hit production immediately, and recovery meant `pull` + manual revert. The only existing dry-run concept gated *deletions*, not creates or updates. **What this fix does.** Adds a `--dry-run` flag to `push`. Instead of firing POST/PATCH/DELETE, the engine counts the intent and prints `[dry-run] would <METHOD> <endpoint> <body-preview>` per resource. The state file is never written (so synthetic IDs don't pollute it), and the end-of-run summary shows `Would create N, would update M, would delete K`. GETs still run because drift detection (Stack G) and operator preview both need to see current platform state. **Outcome you'll notice.** Run `npm run push -- <env> --dry-run` to preview any push. Especially useful for "did I scope this right?" and "is the pre-push lint reporting drift I should address first?" before the real push. Cheapest individual operator-safety win in the stack — no schema changes, no engine architecture moves. --- Operators today can't validate "is this push doing what I think it's doing" before it lands on prod. push.ts has a dry-run concept only for deletions; updates and creates fire immediately. Cheapest individual operator-safety win (improvements.md #5). - src/config.ts: parseFlags now accepts --dry-run alongside --force / --bootstrap. Exports DRY_RUN. - src/api.ts: vapiRequest gates POST/PATCH on DRY_RUN — counts the intent, prints `[dry-run] would <METHOD> <endpoint>` with a 120-char body preview, and returns a synthetic id so caller code threads through. vapiDelete gets the same treatment. GETs always run (drift preview needs them). - src/push.ts: banner ("🧪 DRY-RUN") at start, summary at end ("Would create N, would update M, would delete K"), saveState entirely skipped in dry-run so synthetic ids never leak into the state file. - AGENTS.md: document --dry-run in Available Commands. - tests/push-dry-run.test.ts: --dry-run is parse-accepted, banner prints, state file is NEVER created (verified end-to-end via spawn). - improvements.md: #5 → RESOLVED. Closes improvements.md #5. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
1 parent 898200a commit 392855d

6 files changed

Lines changed: 210 additions & 14 deletions

File tree

AGENTS.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -747,6 +747,7 @@ npm run push -- <org> # Push all local changes to V
747747
npm run push -- <org> assistants # Push only assistants
748748
npm run push -- <org> resources/<org>/assistants/my-agent.md # Push single file
749749
npm run push -- <org> <path1> <path2> # Push multiple specific files (one state write)
750+
npm run push -- <org> --dry-run # Preview without applying any platform changes
750751
npm run apply -- <org> # Pull then push (full sync)
751752
752753
# Testing

improvements.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -56,7 +56,7 @@ you which stack PR closes the row.**
5656
| 2 | `apply` same-file conflict | `apply` drops concurrent same-file dashboard edits | #4 | Open (Stack G planned) |
5757
| 3 | Rollback | Current undo can clobber newer live changes | #4, #5 | Open (Stack H planned) |
5858
| 4 | State schema content hashes | Architectural unlock for #1, #2, #3, #6, #7 | None | Open (Stack F planned) |
59-
| 5 | `push --dry-run` | Cheapest operator-safety win | None | Open (Stack C planned) |
59+
| 5 | `push --dry-run` | Cheapest operator-safety win | None | RESOLVED 2026-04-30 (Stack C) |
6060
| 6 | API-level optimistic concurrency | Server-side conflict rejection | Platform | Deferred (Stack I, gated) |
6161
| 7 | Cartesia voice swap drops `pronunciationDictId` | Silent regression on dashboard voice picker | #4 | Open (Stack G planned) |
6262
| 8 | Dashboard prompt edits can in-place duplicate the prompt | Two stacked prompt versions = stitched output | None | Open (Stack D planned) |

src/api.ts

Lines changed: 52 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,37 @@
1-
import { VAPI_BASE_URL, VAPI_TOKEN } from "./config.ts";
1+
import { DRY_RUN, VAPI_BASE_URL, VAPI_TOKEN } from "./config.ts";
22
import type { VapiResponse } from "./types.ts";
33

4+
// ─────────────────────────────────────────────────────────────────────────────
5+
// Dry-run accounting
6+
//
7+
// In `--dry-run` mode, mutating requests (POST/PATCH/DELETE) are gated and
8+
// counted instead of executed. The end-of-run summary in push.ts reads
9+
// `getDryRunCounts()` to print "would create N, would update M, would delete K."
10+
//
11+
// GETs always run — drift detection (Stack G) and dry-run preview both need
12+
// to fetch current platform state.
13+
// ─────────────────────────────────────────────────────────────────────────────
14+
15+
const DRY_RUN_COUNTS = { POST: 0, PATCH: 0, DELETE: 0 };
16+
17+
export function getDryRunCounts(): { POST: number; PATCH: number; DELETE: number } {
18+
return { ...DRY_RUN_COUNTS };
19+
}
20+
21+
function formatBodyPreview(body: Record<string, unknown>): string {
22+
// One-line preview: the first ~120 chars of the canonicalized JSON,
23+
// truncated with an ellipsis. Helps the operator see *what* is being
24+
// requested without dumping a multi-page payload per resource.
25+
let preview: string;
26+
try {
27+
preview = JSON.stringify(body);
28+
} catch {
29+
preview = String(body);
30+
}
31+
if (preview.length > 120) preview = `${preview.slice(0, 117)}...`;
32+
return preview;
33+
}
34+
435
// ─────────────────────────────────────────────────────────────────────────────
536
// HTTP Client for Vapi API
637
// ─────────────────────────────────────────────────────────────────────────────
@@ -60,6 +91,20 @@ export async function vapiRequest<T = VapiResponse>(
6091
): Promise<T> {
6192
const url = `${VAPI_BASE_URL}${endpoint}`;
6293

94+
if (DRY_RUN) {
95+
DRY_RUN_COUNTS[method]++;
96+
console.log(
97+
` 🧪 [dry-run] would ${method} ${endpoint} ${formatBodyPreview(body)}`,
98+
);
99+
// Returning a stable fake response shaped like a typical create response.
100+
// For PATCH the engine ignores the return (other than for `.id`); for
101+
// POST the engine writes the returned id into state. In dry-run the
102+
// state file is never persisted, so the synthetic id is local-only.
103+
return {
104+
id: `dry-run-${method.toLowerCase()}-${Date.now()}`,
105+
} as unknown as T;
106+
}
107+
63108
for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
64109
await throttle();
65110
const response = await fetch(url, {
@@ -93,6 +138,12 @@ export async function vapiRequest<T = VapiResponse>(
93138
export async function vapiDelete(endpoint: string): Promise<void> {
94139
const url = `${VAPI_BASE_URL}${endpoint}`;
95140

141+
if (DRY_RUN) {
142+
DRY_RUN_COUNTS.DELETE++;
143+
console.log(` 🧪 [dry-run] would DELETE ${endpoint}`);
144+
return;
145+
}
146+
96147
for (let attempt = 0; attempt <= MAX_RETRIES; attempt++) {
97148
await throttle();
98149
const response = await fetch(url, {

src/config.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -85,16 +85,19 @@ const VALID_TYPE_ARGS = [
8585
function parseFlags(): {
8686
forceDelete: boolean;
8787
bootstrapSync: boolean;
88+
dryRun: boolean;
8889
applyFilter: ApplyFilter;
8990
} {
9091
const args = process.argv.slice(3);
9192
const result: {
9293
forceDelete: boolean;
9394
bootstrapSync: boolean;
95+
dryRun: boolean;
9496
applyFilter: ApplyFilter;
9597
} = {
9698
forceDelete: args.includes("--force"),
9799
bootstrapSync: args.includes("--bootstrap"),
100+
dryRun: args.includes("--dry-run"),
98101
applyFilter: {},
99102
};
100103

@@ -105,7 +108,8 @@ function parseFlags(): {
105108
const arg = args[i];
106109
if (!arg) continue;
107110

108-
if (arg === "--force" || arg === "--bootstrap") continue;
111+
if (arg === "--force" || arg === "--bootstrap" || arg === "--dry-run")
112+
continue;
109113

110114
// --confirm <slug>: consumed by cleanup.ts directly. Eat the value here so
111115
// parseFlags' strict-arg check below doesn't trip on the slug.
@@ -238,6 +242,7 @@ export const VAPI_ENV = parseEnvironment();
238242
export const {
239243
forceDelete: FORCE_DELETE,
240244
bootstrapSync: BOOTSTRAP_SYNC,
245+
dryRun: DRY_RUN,
241246
applyFilter: APPLY_FILTER,
242247
} = parseFlags();
243248

src/push.ts

Lines changed: 32 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,11 @@
11
import { resolve } from "path";
22
import { fileURLToPath } from "url";
3-
import { vapiRequest, VapiApiError } from "./api.ts";
3+
import { vapiRequest, VapiApiError, getDryRunCounts } from "./api.ts";
44
import {
55
VAPI_ENV,
66
VAPI_BASE_URL,
77
FORCE_DELETE,
8+
DRY_RUN,
89
APPLY_FILTER,
910
BASE_DIR,
1011
removeExcludedKeys,
@@ -841,6 +842,9 @@ async function main(): Promise<void> {
841842
console.log(
842843
` Deletions: ${FORCE_DELETE ? "⚠️ ENABLED (--force)" : "🔒 Disabled (dry-run)"}`,
843844
);
845+
if (DRY_RUN) {
846+
console.log(" Mode: 🧪 DRY-RUN (no API mutations, no state file write)");
847+
}
844848
if (APPLY_FILTER.resourceTypes?.length) {
845849
console.log(` Filter: ${APPLY_FILTER.resourceTypes.join(", ")}`);
846850
}
@@ -1230,11 +1234,18 @@ async function main(): Promise<void> {
12301234
console.log(
12311235
"\n═══════════════════════════════════════════════════════════════",
12321236
);
1233-
console.log("✅ Apply complete!");
1237+
console.log(DRY_RUN ? "🧪 Dry-run complete (no changes applied)!" : "✅ Apply complete!");
12341238
console.log(
12351239
"═══════════════════════════════════════════════════════════════\n",
12361240
);
12371241

1242+
if (DRY_RUN) {
1243+
const counts = getDryRunCounts();
1244+
console.log(
1245+
`🧪 Would create ${counts.POST}, would update ${counts.PATCH}, would delete ${counts.DELETE} (no API calls fired)`,
1246+
);
1247+
}
1248+
12381249
// Summary - show what was applied vs total in state
12391250
const totalApplied = Object.values(applied).reduce((a, b) => a + b, 0);
12401251

@@ -1275,16 +1286,26 @@ async function main(): Promise<void> {
12751286
// Always flush state, even on partial failure — resources that already
12761287
// received UUIDs from the API must be recorded so the next run does not
12771288
// re-create them.
1278-
try {
1279-
await saveState(state);
1280-
} catch (saveError) {
1281-
console.error(
1282-
"\n⚠️ Failed to persist state file after apply:",
1283-
saveError instanceof Error ? saveError.message : saveError,
1284-
);
1285-
console.error(
1286-
` Local state may be out of sync with platform. Run \`npm run pull -- ${VAPI_ENV} --bootstrap\` to recover.`,
1289+
//
1290+
// EXCEPT in dry-run mode: no real API calls fired, so the state file
1291+
// would be polluted with synthetic dry-run UUIDs. Skip the save entirely.
1292+
if (DRY_RUN) {
1293+
console.log(
1294+
"\n🧪 [dry-run] Skipping state file write (would have written to "
1295+
+ `.vapi-state.${VAPI_ENV}.json)`,
12871296
);
1297+
} else {
1298+
try {
1299+
await saveState(state);
1300+
} catch (saveError) {
1301+
console.error(
1302+
"\n⚠️ Failed to persist state file after apply:",
1303+
saveError instanceof Error ? saveError.message : saveError,
1304+
);
1305+
console.error(
1306+
` Local state may be out of sync with platform. Run \`npm run pull -- ${VAPI_ENV} --bootstrap\` to recover.`,
1307+
);
1308+
}
12881309
}
12891310
}
12901311
}

tests/push-dry-run.test.ts

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
import test from "node:test";
2+
import assert from "node:assert/strict";
3+
import { spawnSync } from "node:child_process";
4+
import {
5+
mkdtempSync,
6+
writeFileSync,
7+
rmSync,
8+
cpSync,
9+
symlinkSync,
10+
existsSync,
11+
mkdirSync,
12+
} from "node:fs";
13+
import { join, dirname } from "node:path";
14+
import { tmpdir } from "node:os";
15+
import { fileURLToPath } from "node:url";
16+
17+
// Stack C — push --dry-run regression coverage.
18+
//
19+
// `--dry-run` MUST:
20+
// 1. Be accepted at parse time (no "Unrecognized argument" error)
21+
// 2. Print the dry-run mode banner so the operator can't miss it
22+
// 3. NOT write the state file (a real run would; dry-run never does)
23+
// 4. NOT fire any actual API calls (verified indirectly by lack of API
24+
// error output and by no state-file mutation, plus the "would PATCH/
25+
// POST/DELETE" log lines)
26+
27+
const __dirname = dirname(fileURLToPath(import.meta.url));
28+
const REPO_ROOT = join(__dirname, "..");
29+
30+
interface Fixture {
31+
dir: string;
32+
cleanup: () => void;
33+
}
34+
35+
function setupFixture(): Fixture {
36+
const dir = mkdtempSync(join(tmpdir(), "vapi-dry-run-test-"));
37+
cpSync(join(REPO_ROOT, "src"), join(dir, "src"), { recursive: true });
38+
cpSync(join(REPO_ROOT, "package.json"), join(dir, "package.json"));
39+
symlinkSync(join(REPO_ROOT, "node_modules"), join(dir, "node_modules"), "dir");
40+
// Empty resource tree — push has nothing real to do, but parsing and the
41+
// dry-run banner must still fire correctly.
42+
mkdirSync(join(dir, "resources", "test-dry-run"), { recursive: true });
43+
writeFileSync(
44+
join(dir, ".env.test-dry-run"),
45+
"VAPI_TOKEN=fake-token-not-used\n",
46+
);
47+
return {
48+
dir,
49+
cleanup: () => rmSync(dir, { recursive: true, force: true }),
50+
};
51+
}
52+
53+
function runPush(
54+
cwd: string,
55+
extraArgs: string[],
56+
): { code: number | null; stdout: string; stderr: string } {
57+
const result = spawnSync(
58+
"node",
59+
["--import", "tsx", "src/push.ts", "test-dry-run", ...extraArgs],
60+
{
61+
cwd,
62+
env: { ...process.env, VAPI_TOKEN: "fake-token-not-used" },
63+
encoding: "utf-8",
64+
timeout: 20_000,
65+
},
66+
);
67+
return {
68+
code: result.status,
69+
stdout: result.stdout || "",
70+
stderr: result.stderr || "",
71+
};
72+
}
73+
74+
test("--dry-run is accepted at parse time without unrecognized-arg error", () => {
75+
const fx = setupFixture();
76+
try {
77+
const res = runPush(fx.dir, ["--dry-run", "--bootstrap"]);
78+
assert.doesNotMatch(res.stderr, /Unrecognized argument/);
79+
} finally {
80+
fx.cleanup();
81+
}
82+
});
83+
84+
test("--dry-run prints the dry-run banner so the operator sees it", () => {
85+
const fx = setupFixture();
86+
try {
87+
const res = runPush(fx.dir, ["--dry-run", "--bootstrap"]);
88+
// The banner mentions DRY-RUN explicitly so it's noisy enough not to be
89+
// missed in a CI log scroll.
90+
assert.match(res.stdout, /DRY-RUN/);
91+
} finally {
92+
fx.cleanup();
93+
}
94+
});
95+
96+
test("--dry-run does NOT write the state file", () => {
97+
const fx = setupFixture();
98+
try {
99+
const stateFilePath = join(fx.dir, ".vapi-state.test-dry-run.json");
100+
assert.equal(
101+
existsSync(stateFilePath),
102+
false,
103+
"precondition: state file should not exist before run",
104+
);
105+
106+
const res = runPush(fx.dir, ["--dry-run", "--bootstrap"]);
107+
// Even with --bootstrap, dry-run must skip the state save. Bootstrap
108+
// would normally write the state file with refreshed credentials/UUIDs;
109+
// in dry-run we want zero filesystem mutation.
110+
assert.equal(
111+
existsSync(stateFilePath),
112+
false,
113+
`state file must not be created in dry-run; stdout=${res.stdout}`,
114+
);
115+
} finally {
116+
fx.cleanup();
117+
}
118+
});

0 commit comments

Comments
 (0)