Dashboard data contract: schema-validated exports + published release artifacts#65
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
MaxGhenis
force-pushed
the
data-plane-contract
branch
from
de68dfe to
90ac33f
Compare
Every dashboard payload write now validates against the app contract
(policybench/dashboard_schema.py) and serializes with allow_nan=False.
This catches the two observed failure classes at write time: a
per-country export copied to app/src/data.json (the analyze CLI did
exactly this by default — export_dashboard_data wrote the bare bench
object, which the app cannot load; it now wraps under {"countries"})
and NaN literals that JSON.parse rejects.
policybench publish-dashboard validates, uploads data.json as a GitHub
release asset, and writes a committed pointer
(app/src/data.artifact.json: tag, url, sha256, bytes). The app's
prepare-data resolves src/data.json first, else downloads the pointer's
asset with sha256 verification and caches it under app/.cache/. The
current payload is published as release dashboard-data-20260520 and the
download path is verified end-to-end.
policybench validate-dashboard checks any payload file from the CLI.
docs/artifacts.md records the contract and the cutover plan that stops
committing the 57MB blob once the in-flight refresh lands.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
MaxGhenis
force-pushed
the
data-plane-contract
branch
from
74669d5 to
420d2ab
Compare
MaxGhenis
added a commit
that referenced
this pull request
Jun 10, 2026
…fact (#71) app/src/data.json (57MB per refresh) is deleted and gitignored. Builds resolve app/src/data.artifact.json: prepare-data downloads the release asset and verifies its sha256 (the path proven end-to-end in #65). The snapshot integrity chain survives without the blob, machine-checked instead of prose: the manifest pins the published artifact's sha256, a test asserts the committed pointer matches the pin, and the old committed-blob equality test now recombines the committed per-country run exports and asserts their serialized bytes hash to the same pin. The score-reproduction test runs on the same recombination. Refresh flow from here: export locally, policybench publish-dashboard --tag dashboard-data-<date>, commit the 9-line pointer plus the manifest pin. In-flight refresh branches that modify data.json should adopt that flow on rebase. Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
app/src/data.jsonis a 57MB generated artifact committed as source. That's why.gitis ~290MB locally, why refresh PRs are conflict magnets — and because nothing validated the export contract, theanalyzeCLI could (and did) write an unloadable payload to the app path:export_dashboard_datawrote the bare per-country bench object instead of the{"countries": {...}}shape the app requires. A working tree with exactly this corruption exists right now.What
Contract (validation at every write):
policybench/dashboard_schema.pyvalidates payload shape, per-country required keys, scenario cross-references, and finite numbers. The per-country-copied-to-app-path mistake produces a specific, named error.export_full_run,export_country,export_dashboard_data) validate before writing and serialize withallow_nan=False(Python emitsNaNliterals thatJSON.parserejects — a second silent-corruption class, now a loud export failure).export_dashboard_databug fixed: it now wraps the bench under{"countries": {...}}.policybench validate-dashboard [path]checks any file from the CLI; the app applies the same shape guard at build time with a readable error.Published artifacts (blob out of git):
policybench publish-dashboard --tag dashboard-data-<date>validates, uploads data.json as a GitHub release asset, and writes the committed 9-line pointerapp/src/data.artifact.json(tag, url, sha256, bytes).prepare-data.tsresolvessrc/data.jsonfirst (local flows unchanged), else downloads the pointer's asset with sha256 verification and caches under gitignoredapp/.cache/.dashboard-data-20260520; the download path is verified end-to-end (fetch → hash check → split → build, then cache hit on rerun).docs/artifacts.mdrecords the contract and the cutover plan: once the in-flight refresh (Align PolicyBench runtime with 4.16.1 refresh artifacts #59) lands, refreshes publish + commit the pointer, and the committed blob gets deleted. History rewrite is explicitly out of scope.Verification
🤖 Generated with Claude Code