Skip to content

fix: accept thesis framing in design eval#2189

Open
peterostrander2 wants to merge 1 commit into
garrytan:mainfrom
peterostrander2:ci/fix-design-eval-aesthetic-synonym
Open

fix: accept thesis framing in design eval#2189
peterostrander2 wants to merge 1 commit into
garrytan:mainfrom
peterostrander2:ci/fix-design-eval-aesthetic-synonym

Conversation

@peterostrander2

Copy link
Copy Markdown

Summary

  • Broaden the design-consultation structural matcher so ## Memorable thing / ## Thesis framing counts as the aesthetic direction.
  • Keeps the quality judge intact; this only fixes a brittle synonym check from the scheduled Periodic Evals failure.

Evidence

  • Failing scheduled run: https://github.com/garrytan/gstack/actions/runs/28775340284
  • Failed job: evals (e2e-design, test/skill-e2e-design.test.ts), assertion at test/skill-e2e-design.test.ts:155 with missingSections length 1.
  • Downloaded eval artifact showed the generated DESIGN.md had ## Memorable thing and ## Thesis, plus Typography, Color, Layout, Spacing, and Motion sections; the LLM design-quality judge passed.

Verification

  • Replayed the exact failed artifact against the updated synonym list: missing=none.
  • bun test test/skill-e2e-design.test.ts with evals disabled: 0 fail, 11 skipped.

@trunk-io

trunk-io Bot commented Jul 7, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant