Skip to content

Prototype LLRT native executor and package#14

Draft
robinbraemer wants to merge 1 commit into
mainfrom
codex/llrt-executor-poc
Draft

Prototype LLRT native executor and package#14
robinbraemer wants to merge 1 commit into
mainfrom
codex/llrt-executor-poc

Conversation

@robinbraemer

Copy link
Copy Markdown
Member

Summary

This PR prototypes an LLRT-first execution path for codemode and prepares a standalone TypeScript-friendly @robinbraemer/llrt package.

It includes:

  • A new packages/llrt package with a napi-rs Rust binding around LLRT's Rust VM API.
  • A JSON-safe LlrtRuntime.callJson<TInput, TOutput>() API with typed success/failure results, memory/wall-time/stack options, host callbacks, fresh VM per call, and tests for errors, isolation, callbacks, memory, timeout, and stress behavior.
  • Optional native package layout for darwin/linux arm64/x64, native artifact verification scripts, packed-install smoke testing, and a GitHub Actions native matrix workflow.
  • Codemode integration via LlrtNativeExecutor, createExecutor() preference for LLRT when installed, and fallback executors left in place.
  • An LLRT process executor proof of concept for comparison.
  • Benchmark tooling and a saved report comparing llrt-native, isolated-vm, and QuickJS WASM.
  • Design and implementation notes under internal/superpowers/.

Verification run locally

  • mise exec -- task ci
  • mise exec actionlint -- actionlint .github/workflows/*.yml
  • mise exec -- pnpm --filter @robinbraemer/llrt run verify:native-artifacts
  • mise exec -- npm pack --dry-run from packages/llrt
  • mise exec -- pnpm --filter @robinbraemer/codemode run benchmark:executors -- --report internal/superpowers/reports/2026-06-10-llrt-executor-benchmark.md --json internal/superpowers/reports/2026-06-10-llrt-executor-benchmark.json

Current verdict

LLRT is a strong default candidate, but this PR intentionally does not claim the production migration is complete. The local prototype proves the core feasibility: importable TypeScript API, napi-rs bridge, fresh VM isolation, JSON-safe host callbacks, resource controls, codemode executor compatibility, and competitive local benchmark behavior.

The remaining hard gate is a real GitHub Actions run of .github/workflows/llrt-native.yml across the four native runners, followed by artifact inspection and packed install smoke results from CI.

Handoff prompt for the next AI agent

You are continuing the LLRT-first runtime adoption work in the cnap-tech/codemode repository. Start from this PR branch: codex/llrt-executor-poc.

Goal: finish proving whether LLRT can become codemode's default executor, then prepare the follow-up path to consume the package from CNAP. Preserve the current direction unless evidence contradicts it: standalone @robinbraemer/llrt package first, codemode consumes it through a thin adapter, fallback engines remain available for rollback.

Context already completed in this PR:

  • packages/llrt exists as a standalone Node/TypeScript package backed by a Rust napi-rs addon.
  • Public API is LlrtRuntime.callJson<TInput, TOutput>() with typed LlrtResult<TOutput>.
  • Host callbacks are explicit and JSON-safe; no arbitrary live object injection.
  • Each execution creates a fresh LLRT VM.
  • Runtime/call options include memoryMB, wallTimeMs, cpuTimeMs placeholder, and maxStackBytes.
  • Native binding uses LLRT/rquickjs APIs and reports typed errors such as TIMEOUT, MEMORY_LIMIT, SERIALIZATION_ERROR, EVALUATION_ERROR, NATIVE_LOAD_ERROR, and RUNTIME_DISPOSED.
  • Codemode has LlrtNativeExecutor; createExecutor() prefers LLRT when @robinbraemer/llrt is installed, falls back only when the LLRT package itself is missing, and fails loudly when installed LLRT is broken.
  • isolated-vm, QuickJS WASM, and LlrtProcessExecutor remain available.
  • Native packages are laid out under packages/llrt/npm/* for darwin/linux arm64/x64.
  • .github/workflows/llrt-native.yml defines the native prebuild/package/publish workflow.
  • verify:native-artifacts validates optional native package manifests; strict mode also requires .node files after artifacts are downloaded.
  • smoke:packed-install packs main + current native package and tests import/execution from a temporary consumer project.
  • Benchmark tooling exists at packages/codemode/scripts/benchmark-executors.ts, with a saved report at internal/superpowers/reports/2026-06-10-llrt-executor-benchmark.md.
  • Design/spec notes live at internal/superpowers/specs/2026-06-10-standalone-llrt-typescript-runtime-design.md and internal/superpowers/plans/2026-06-10-standalone-llrt-calljson.md.

Important local caveats:

  • Use mise exec -- ... for verification so Node stays at the pinned Node 24 ABI. Running raw task ci under Node 26 can fail because isolated-vm was built for Node 24.
  • .codex/ is unrelated and should not be committed.
  • Generated native binaries, dist/, node_modules/, packages/llrt/vendor/, packages/llrt/native/target/, *.node, and *.tgz are intentionally ignored.
  • The root package currently emits a pnpm warning about pnpm.onlyBuiltDependencies; do not chase it unless you choose to clean package-manager config as a separate scoped change.

Recommended next steps:

  1. Trigger the GitHub Actions workflow .github/workflows/llrt-native.yml on this branch.
  2. Verify the four build jobs run on:
    • macos-15 for aarch64-apple-darwin
    • macos-15-intel for x86_64-apple-darwin
    • ubuntu-24.04 for x86_64-unknown-linux-gnu
    • ubuntu-24.04-arm for aarch64-unknown-linux-gnu
  3. Confirm the package job downloads all four llrt-native-* artifacts, runs verify:native-artifacts:strict, dry-run-packs every optional package, and passes smoke:packed-install.
  4. If workflow failures occur, fix them with TDD/guard tests where possible. Do not weaken the artifact or smoke checks just to make CI green.
  5. Add a small post-run audit note to internal/superpowers/specs/2026-06-10-standalone-llrt-typescript-runtime-design.md with the actual run URL, job conclusions, and artifact names.
  6. Decide whether to publish @robinbraemer/llrt or first consume it via workspace/git dependency for one CNAP integration test.
  7. In the CNAP repo, update the codemode dependency path only after the LLRT package has reproducible native artifacts or an explicit temporary consumption strategy.
  8. Only make LLRT the unqualified production default after: native CI passes, codemode executor contract passes, memory/timeout/stalled-callback/concurrency tests pass, representative benchmarks remain acceptable, and an emergency fallback engine selection remains available.

Suggested verification commands before changing this PR from draft to ready:

mise exec -- task ci
mise exec actionlint -- actionlint .github/workflows/*.yml
mise exec -- pnpm --filter @robinbraemer/llrt run verify:native-artifacts
mise exec -- pnpm --filter @robinbraemer/llrt run smoke:packed-install
mise exec -- pnpm --filter @robinbraemer/codemode run benchmark:executors -- \
  --report internal/superpowers/reports/2026-06-10-llrt-executor-benchmark.md \
  --json internal/superpowers/reports/2026-06-10-llrt-executor-benchmark.json

Do not mark the larger adoption goal complete until the actual native matrix run and downstream consumption plan are verified with current evidence.

Rationale: Explore LLRT as the preferred lightweight execution runtime by adding a standalone TypeScript-friendly @robinbraemer/llrt package, napi-rs native bridge, codemode executor adapter, native packaging workflow, and benchmark/stress evidence.

Rejected: Do not replace the fallback executors yet; LLRT still needs a real GitHub Actions native matrix run and artifact inspection before production-default adoption. Do not embed LLRT directly into codemode; the standalone package keeps native packaging and runtime API reusable.

Risk: Introduces a new Rust/native package and release workflow. The prototype is intentionally JSON-safe and fresh-VM-per-call, which is safer but may leave reusable-VM performance for later.

Tested: mise exec -- task ci; mise exec actionlint -- actionlint .github/workflows/*.yml; mise exec -- pnpm --filter @robinbraemer/llrt run verify:native-artifacts; mise exec -- npm pack --dry-run from packages/llrt; benchmark:executors report generation.

Not-tested: Real GitHub Actions native matrix run across macOS and Linux runners; publishing to npm; CNAP repository consumption of the published package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant