feat(workflows): opt-in output_format: json exposes parsed shell stdout as output.data

[doquanghuy]

Description

Reference implementation for #2962 — for discussion, direction welcome.

Adds an opt-in output_format: json to shell steps: when declared, stdout is parsed and exposed under output.data (the raw stdout/stderr/exit_code keys are unchanged, so there is no merge/clobber ambiguity), letting later steps consume typed values — e.g. a fan-out's items: "{{ steps.emit.output.data.items }}". A parse failure fails the step with a clear error (declaring the format is a contract; silence would hide wiring bugs). Without the key, behavior is byte-identical — fully backward-compatible. validate() rejects unknown formats.

The issue lists output_file: and a declared named-outputs: schema as alternatives — happy to rework toward either if you prefer that direction.

Coordination: aware of #2443 touching the same file; this diff is isolated to the structured-output addition and I'm happy to rebase in whichever order suits.

Testing

• Ran existing tests with uv sync && uv run pytest — full suite 3729 passed
• Four new tests in TestShellStep: data exposed on valid JSON; invalid stdout fails the step; no-flag backcompat (no data key); validate rejects unknown formats — the three behavior tests are red against current main, green with the change (verified both directions)
• uvx ruff check src/ — clean
• Tested locally with uv run specify --help
• Tested with a sample project (covered by the step-level tests)

AI Disclosure

• I did not use AI assistance for this contribution
• I did use AI assistance (describe below)

Code, tests, and this description were authored with AI assistance (Claude); verified by running the repo's test suite and ruff locally in both red and green directions.

[doquanghuy]

@mnriem when you have a moment, would appreciate your thoughts on the direction here — the issue lists the alternatives considered, and I'm happy to rework toward whichever shape fits Spec Kit best.

[Copilot]

⚠️ Not ready to approve

The newly added tests use shell echo quoting that is not portable to Windows cmd.exe, which will likely break the existing Windows CI matrix.

Pull request overview

Adds an opt-in structured output mode for shell workflow steps, enabling later steps to consume typed data derived from shell stdout while preserving the existing raw stdout/stderr/exit_code outputs for backward compatibility.

Changes:

• Add output_format: json handling to ShellStep.execute() to parse stdout and expose it as output.data, failing the step on JSON parse errors.
• Extend ShellStep.validate() to reject unknown output_format values.
• Add unit tests covering JSON parsing success/failure, backward compatibility (no data key without the flag), and validation behavior.

File summaries

File	Description
src/specify_cli/workflows/steps/shell/init.py	Implements output_format: json parsing into output.data and validates allowed formats.
tests/test_workflows.py	Adds TestShellStep coverage for JSON structured output and validation.

Copilot's findings

• Files reviewed: 2/2 changed files
• Comments generated: 3

Note

Your feedback helps us improve the quality of this feature.
Please use 👍 or 👎 to tell us whether this assessment is correct.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[mnriem]

Please address Copilot feedback. Please make sure to include negative and positive tests

[doquanghuy]

@mnriem Thanks for the review — addressed the Copilot feedback:

• The output_format tests now emit JSON via sys.executable + a temp script (the established '"{py}" "{script}"' pattern used elsewhere in the suite) instead of echo '{...}', so they're portable to the Windows cmd.exe CI matrix.

Full suite green; ruff check src/ clean. Ready for another look.

[doquanghuy]

@mnriem on the positive/negative coverage you asked for — it's in TestShellStep:

• Positive: test_output_format_json_exposes_data — valid JSON stdout is parsed and surfaced as output.data (raw stdout/exit_code still present).
• Negative (parse): test_output_format_json_invalid_stdout_fails — non-JSON stdout fails the step with an output_format: json error.
• Negative (validation): test_validate_rejects_unknown_output_format — a non-json output_format is rejected at validate time.
• Back-compat: test_no_output_format_keeps_raw_output_only — no output_format ⇒ no data key.

All four emit any JSON via sys.executable now (cross-platform). Happy to add more cases if you'd like a specific one covered.