From 76cd34d440005c261d1d31671b6ea4e2a7eb6ebd Mon Sep 17 00:00:00 2001 From: Donald Merand Date: Tue, 31 Mar 2026 20:33:28 -0400 Subject: [PATCH] Add CI skill --- .../_shared/shopify-cli-ci-repo-contracts.md | 40 ++++++++ .agents/skills/cli-pre-submit-ci/SKILL.md | 91 +++++++++++++++++++ .claude/skills | 1 + 3 files changed, 132 insertions(+) create mode 100644 .agents/skills/_shared/shopify-cli-ci-repo-contracts.md create mode 100644 .agents/skills/cli-pre-submit-ci/SKILL.md create mode 120000 .claude/skills diff --git a/.agents/skills/_shared/shopify-cli-ci-repo-contracts.md b/.agents/skills/_shared/shopify-cli-ci-repo-contracts.md new file mode 100644 index 00000000000..9dae6696d24 --- /dev/null +++ b/.agents/skills/_shared/shopify-cli-ci-repo-contracts.md @@ -0,0 +1,40 @@ +# Shopify CLI CI repo contracts + +Use this file for durable repo-specific guidance shared by the CLI CI skills. + +## Principle + +Resolve CI behavior from the repo’s current contract files before relying on memory or check names. + +## Canonical sources + +Read sources in this order: + +1. `.github/workflows/*.yml` +2. `dev.yml` +3. `package.json` +4. helper scripts referenced by those files + +For normal PR CI work, start with `.github/workflows/tests-pr.yml`. + +## Stable repo facts + +- CI is GitHub Actions. +- PR runs use workflow-level concurrency cancellation, so new pushes can cancel in-flight runs on the same branch. +- Some visible checks are aggregate or gate checks rather than the root cause. +- Generated outputs are often part of the required change set when a workflow step verifies cleanliness after regeneration. +- `dev.yml` is a useful local workflow entrypoint, but workflow YAML is the source of truth for what CI actually enforces. + +## Gotchas + +- Do not guess from a check name alone when the workflow or job definition is available. +- Treat cancelled runs as context, not strong evidence. +- Separate queue delay from actual execution time when judging slowness. +- Reproduce the workflow-equivalent generator or check locally before inventing a fix. +- Distinguish required generated churn from optional broad normalization churn. + +## Examples + +- A docs/manifests check may resolve through workflow steps into `package.json` scripts and then helper scripts under `bin/`. +- A visible `Unit tests` failure may be an aggregate failure caused by one matrix shard. +- A GraphQL freshness check may run a schema fetch step and a codegen step, then assert a clean git state. diff --git a/.agents/skills/cli-pre-submit-ci/SKILL.md b/.agents/skills/cli-pre-submit-ci/SKILL.md new file mode 100644 index 00000000000..98870d67742 --- /dev/null +++ b/.agents/skills/cli-pre-submit-ci/SKILL.md @@ -0,0 +1,91 @@ +--- +name: cli-pre-submit-ci +description: 'Prevent predictable Shopify CLI PR CI failures before push or submit. Use when users: (1) ask what to run before pushing, submitting, restacking, opening, or updating a PR, (2) ask which generated files to commit or stage, (3) want to validate a branch against GitHub Actions, or (4) want to avoid CI churn after command, docs, workflow, schema, or test changes. At PR time, default to suggesting the minimal pre-submit checks rather than running a full local workflow automatically.' +--- + +# CLI pre-submit CI + +Start with the shared repo facts in [`../_shared/shopify-cli-ci-repo-contracts.md`](../_shared/shopify-cli-ci-repo-contracts.md). + +## Principle + +Derive the smallest sufficient pre-submit validation set from the current CI contract. Do not guess, and do not default to the broadest possible local run. + +## Pattern + +### 1) Classify the diff first +Start by inspecting both the branch diff and the current working tree. + +Use one consistent branch-diff command rather than rediscovering it each time. + +Example: + +```bash +git diff --name-only $(git merge-base HEAD origin/main)...HEAD +git status --short +``` + +Treat these as different scopes: +- branch diff vs `origin/main` = what the PR currently changes +- working tree status = what the user might still push next + +If the diff clearly maps to a narrow family, keep the investigation narrow. + +| Diff class | Default response | +|---|---| +| docs/config/wiring only, with no obvious workflow-enforced generator family | **stop there unless contradicted**: run lightweight sanity checks only (`git diff --check`, validate changed symlink targets, validate local markdown links if relevant); do not full-read large workflow/script files | +| user is at PR time (`submit`, `open`, `update`, `restack`) | advisory mode: suggest minimal checks, staging needs, and likely CI risk; ask before running anything substantial | +| user asks what to run before push | recommend the minimal high-signal checks implied by the workflow | +| user asks what to commit or stage | reproduce the relevant generator/check path, then inspect git status and diffs | +| user explicitly asks to run checks | run the minimal derived set, not the whole world | + +### 2) Resolve the contract for the relevant family +Only do this if the diff class suggests a real CI-family mapping, or if the user asks for broader confidence. + +Read sources in this order: + +1. relevant `.github/workflows/*.yml` +2. `dev.yml` +3. `package.json` +4. helper scripts referenced by those files + +For normal PR work, start with `.github/workflows/tests-pr.yml`. + +Read only the files and script sections needed for the diff class you identified. Avoid full reads of large files for docs/config/wiring-only diffs. + +### 3) Map the diff to contract families + +| Change shape | Inspect first | Likely response | +|---|---|---| +| Command/flag/help surface | docs/manifests/readme freshness jobs | derive the generator path from workflow → scripts | +| GraphQL queries or schemas | schema/codegen freshness jobs | derive the schema fetch + codegen path | +| TypeScript implementation or exports | type-check, lint, knip, bundle jobs | focused tests plus required static checks | +| Test helpers, async UI, network/auth/callback logic | unit-test jobs and nearby tests | focused tests plus a CI-risk warning | +| Workflow files or CI plumbing | affected workflow definitions | validate the changed contract directly | + +### 4) Finish with staging guidance +After any generator, freshness check, or lightweight sanity pass: + +1. inspect `git status --short` +2. inspect targeted diffs +3. say which files look like: + - required generated output + - optional broad churn worth review + - suspicious changes suggesting the wrong or incomplete generation path + +## Gotchas + +- At PR time, do **not** automatically run the full workflow-equivalent validation set unless the user asks. +- `dev.yml` is a useful local entrypoint, but workflow YAML is the source of truth for what CI enforces. +- Broad generated diffs are not automatically wrong; distinguish required churn from suspicious churn. +- Do not stop at “run this command.” Explain what likely needs staging. +- If the diff is docs/config/wiring only, do not escalate to heavyweight checks unless the workflow or the user gives a reason. +- For docs/config/wiring-only diffs, avoid full reads of large workflow or script files unless the diff clearly maps to an enforced CI family. +- If the change touches async/timing-heavy tests, local servers, callback flows, socket teardown, or workflow topology, warn that CI-only failures may still appear even after local checks pass. + +## Examples + +- "What should I run before I push this PR?" → derive the minimal checks from workflow → `dev.yml` → `package.json`, then recommend focused tests plus any required generators. +- "Submit this PR." → treat it as a pre-submit moment, suggest the minimal recommended checks and likely staging requirements first, and ask before running them. +- "Which generated files do I need to commit?" → reproduce the relevant generation path, inspect git status, and separate required generated output from optional churn. +- "I changed a command flag; what repo checks matter?" → start from the freshness job that enforces command-surface updates rather than from memory. diff --git a/.claude/skills b/.claude/skills new file mode 120000 index 00000000000..2b7a412b8fa --- /dev/null +++ b/.claude/skills @@ -0,0 +1 @@ +../.agents/skills \ No newline at end of file