codemod iterations

.changeset/codemod-preserve-shebang.md

@@ -0,0 +1,5 @@
+---
+'@modelcontextprotocol/codemod': patch
+---
+
+Preserve a leading `#!` shebang (and the blank lines after it) when migrating a file. Some transforms drop the shebang because it is leading trivia of the first import they rewrite; the codemod now captures it before transforms and restores it before saving, so CLI packages whose `bin` points at the migrated entry keep working.

packages/codemod/batch-test/README.md

@@ -9,8 +9,8 @@ For each repo in `repos.json`, the batch test:
 1. Clones the repo (or resets an existing clone)
 2. Installs dependencies
 3. Runs baseline checks (typecheck, build, test, lint) to confirm the repo is healthy
-4. Runs the codemod using the programmatic API
-5. Packs local SDK packages as tarballs and rewrites `package.json` deps to use them (so the test runs against the current SDK branch, not published npm versions)
+4. Runs the codemod — in-process via the programmatic API (`--codemod=local`, the default), or by shelling out to the published CLI via `npx` (`--codemod=published`)
+5. Packs local SDK packages as tarballs and rewrites `package.json` deps to use them, so the test runs against the current SDK branch — only when `--sdk=local` (the default); with `--sdk=published` the v2 deps are installed from npm instead (see [Modes](#modes))
 6. Re-installs dependencies
 7. Re-runs the same checks
 8. Writes structured JSON reports
@@ -30,12 +30,48 @@ pnpm --filter @modelcontextprotocol/codemod batch-test
 pnpm --filter @modelcontextprotocol/codemod batch-test:clean
 ```
 
+## Modes
+
+Each run independently chooses where the **SDK packages** and the **codemod** come from:
+
+| Flag                       | Values                 | Default   | Effect                                                                                                              |
+| -------------------------- | ---------------------- | --------- | ------------------------------------------------------------------------------------------------------------------- |
+| `--sdk`                    | `local` \| `published` | `local`   | `local`: pack local packages and rewrite deps to `file:` tarballs. `published`: install the v2 deps from npm.       |
+| `--codemod`                | `local` \| `published` | `local`   | `local`: run the working-copy codemod in-process. `published`: `npx` the published CLI.                             |
+| `--codemod-version <spec>` | npm version/tag/range  | `latest`  | Only with `--codemod=published`.                                                                                    |
+| `--sdk-version <spec>`     | npm version/tag/range  | _(unset)_ | Only with `--sdk=published`. Pins each v2 dep to its own resolved version. Unset = use whatever the codemod writes. |
+
+Both `--flag value` and `--flag=value` forms work.
+
+```bash
+# default (today's behavior)
+pnpm --filter @modelcontextprotocol/codemod batch-test
+
+# both from npm @latest
+pnpm --filter @modelcontextprotocol/codemod batch-test -- --sdk=published --codemod=published
+```
+
+Published specs are resolved to concrete versions via `npm view` at startup. A failure resolving the codemod version or the representative SDK label aborts the run; a per-package `--sdk-version` miss only warns and leaves that package on the codemod-written range. Each
+`@modelcontextprotocol/*` package is resolved **independently** — they are not released in lockstep. Results are written to a per-run directory keyed on the resolved versions; the SDK segment is the resolved `@modelcontextprotocol/server` version used as a representative label,
+while the full per-package set is recorded in `summary.json` → `sdkVersions`:
+
+```
+results/codemod-local__sdk-local/                  # default (--sdk=local --codemod=local)
+results/codemod-2.0.0-alpha.2__sdk-2.0.0-alpha.2/  # --codemod=published --sdk=published --sdk-version=2.0.0-alpha.2
+results/codemod-2.0.0-alpha.2__sdk-from-codemod/   # --codemod=published --sdk=published  (no --sdk-version)
+```
+
+The segment `sdk-from-codemod` appears when `--sdk=published` with no `--sdk-version` and `--codemod=published` (the SDK version is baked into the published codemod and only known after install; the installed versions are recorded in `summary.json` → `sdkVersions`).
+
+**Limitation:** in published-codemod mode the CLI emits text, not structured diagnostics, so `codemod.diagnostics` is empty and the raw CLI output is captured under `codemod.cli` instead. Diagnostics categorization applies only to local-codemod runs.
+
 ## Output
 
-Results are written to `batch-test/results/`:
+Results are written to a per-run directory keyed on the resolved versions, `batch-test/results/<config>/`, where `<config>` is the `codemod-…__sdk-…` leaf (distinct `--codemod-version`/`--sdk-version` values produce distinct directories even within the same `--sdk`/`--codemod`
+mode; see [Modes](#modes)):
 
-- `summary.json` — overview across all repos: which passed, which failed, error counts
-- `<repo-slug>/report.json` — per-repo detail: baseline vs post-codemod check results, codemod diagnostics, change counts
+- `results/<config>/summary.json` — overview across all repos: which passed, which failed, error counts, plus the run `config` and the resolved per-package `sdkVersions`
+- `results/<config>/<repo-slug>/report.json` — per-repo detail: baseline vs post-codemod check results, codemod diagnostics, change counts
 
 ## Repo manifest (`repos.json`)
 
@@ -86,7 +122,7 @@ When `checks` is omitted, the runner auto-detects commands from the package's `p
 
 1. Edit `repos.json` and add an entry
 2. Run `pnpm --filter @modelcontextprotocol/codemod batch-test`
-3. Check `results/<repo-slug>/report.json` for new findings
+3. Check `results/<config>/<repo-slug>/report.json` for new findings
 
 For monorepos, list each package that uses `@modelcontextprotocol/sdk` as a separate entry in `packages`.
 

packages/codemod/batch-test/analyze-prompt.md

@@ -32,11 +32,13 @@ The goal is to find issues in the codemod itself — incorrect transforms, missi
     pnpm --filter @modelcontextprotocol/codemod batch-test
     ```
 
-3. Read `packages/codemod/batch-test/results/summary.json` for the overview. Note which repos have `postCodemodClean: false` and which check types have new errors.
+3. Read `packages/codemod/batch-test/results/*/summary.json` for the overview (there may be several config directories — one per `--sdk`/`--codemod` mode; read the run you're analyzing). Note which repos have `postCodemodClean: false` and which check types have new errors.
 
-4. For each repo with new errors, read its `packages/codemod/batch-test/results/<repo-slug>/report.json`. Compare `baseline` vs `postCodemod` for each check — only errors that appear in `postCodemod` but not in `baseline` are codemod-introduced.
+4. For each repo with new errors, read its `packages/codemod/batch-test/results/<config>/<repo-slug>/report.json` (the same `<config>` directory as the summary). Compare `baseline` vs `postCodemod` for each check — only errors that appear in `postCodemod` but not in `baseline`
+   are codemod-introduced.
 
-5. Also review the `codemod.diagnostics` array in each report — these are warnings the codemod itself emitted about patterns it couldn't fully handle.
+5. Also review the `codemod.diagnostics` array in each report — these are warnings the codemod itself emitted about patterns it couldn't fully handle. This applies only to local-codemod runs; when `config.codemodSource` is `published`, `codemod.diagnostics` is empty and
+   `codemod.cli` holds the raw CLI output instead.
 
 6. For each codemod-introduced error, look at the actual source file in the cloned repo (`packages/codemod/batch-test/repos/<repo-slug>/...`) to understand what the codemod produced and what it should have produced.