chore: Scriptorium as a Python test framework (+ Fable 5.8.0 bump)

[dbrattli]

What this is

A spike to evaluate Scriptorium (Maxime Mangel's F#/Fable testing stack — Nib assertions + Quill runner) as a test framework for Fable.Python, compiled to Python. Opened so we can decide the direction later — it is intentionally not wired into CI or the main suite.

Contents

1. chore(deps): bump Fable to 5.8.0

• fable tool 5.6.0 → 5.8.0, fable-library 5.6.0 → 5.8.0 (lockstep), uv.lock re-locked.
• 5.8.0's Python backend fixes [<EntryPoint>] exit-code propagation — it now emits sys.exit(int(main(...))), so a failing runner is visible to CI. This bug was found by the spike; the fix was contributed upstream and shipped in 5.8.0 (fable-compiler/Fable 7848eae46).
• Verified: full pytest suite still 563/563 green under 5.8.0.

2. spike: evaluate Scriptorium (Nib + Quill)

• spike/scriptorium/ — Spike.fsproj, Main.fs, README.md, plus a just spike-scriptorium target.
• Small Quill/Nib suite that exercises plain F# assertions and real Fable.Python.Math bindings, compiled --lang python and run on CPython.

Findings (full write-up in spike/scriptorium/README.md)

• ✅ Scriptorium compiles cleanly to Python — it already ships explicit #if FABLE_COMPILER_PYTHON support. The runner works on CPython (colored output, diffs, source links), and exit codes are correct (pass→0, fail→1) with no shim on 5.8.0.
• Nib assertions are separable from the Quill runner — assertThat throws on failure, which pytest reports fine. So we could adopt Nib's richer assertions while keeping [<Fact>] + pytest, without a runner swap.
• Out of scope / blocked: Nib.Browser (Playwright/JS-only); Nib.Snapshot and Hedgehog hit fable-library-python runtime gaps.
• Key caveat for a full migration: Scriptorium's headline cross-target benefit doesn't apply here — these tests exercise Python-only bindings. A full move to the Quill runner trades away pytest's ecosystem (discovery, -k/-x, parametrize, plugins, JUnit XML, coverage).

Direction options (for discussion, not decided here)

1. Nib under pytest — richest assertions, lowest risk, keep pytest.
2. Full Quill+Nib (phased) — one idiom across Fable projects, lose pytest tooling.
3. Pilot one module to Quill+Nib, evaluate, then decide.
4. Leave as-is — 563 tests green on a standard runner.

Known limitations of the spike

• Spike.fsproj references a local ../../../Scriptorium checkout (not NuGet), so it only builds where that checkout exists. Scriptorium's Python-supporting packages would need to be consumed via NuGet/paket before this is portable. This is fine for a spike but means CI does not build it.
• Generated output (spike/scriptorium/build/) is gitignored.

🤖 Generated with Claude Code