Skip to content

Add model policy frontmatter + import unioning + env policy overrides#41824

Open
Copilot wants to merge 30 commits into
mainfrom
copilot/add-frontmatter-models-fields
Open

Add model policy frontmatter + import unioning + env policy overrides#41824
Copilot wants to merge 30 commits into
mainfrom
copilot/add-frontmatter-models-fields

Conversation

Copilot AI commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

This change introduces model policy controls in workflow frontmatter (models.allowed, models.disallowed) and maps them to AWF’s allowedModels / disallowedModels config. It also makes policy behavior import-safe by unioning model sets across composed workflows, with centralized environment overrides taking precedence.

  • Frontmatter + schema support

    • Extended models frontmatter schema to support policy fields alongside optional pricing providers.
    • Added typed parsing for:
      • models.allowed
      • models.disallowed
  • Import compatibility (union semantics)

    • Extended import extraction/results to carry model policy sets from imported workflows.
    • Added workflow merge logic to union policy sets across imports + main workflow.
    • Added conflict handling so disallowed takes precedence when the same model appears in both allowed and disallowed sets.
  • Cost data cleanliness + parse warnings

    • Tightened import-side model-cost extraction to only accept valid non-empty models.providers objects.
    • Added import warnings for invalid models.allowed / models.disallowed / models.providers shapes and invalid entries, while safely skipping bad values.
  • Centralized policy overrides

    • Added compiler env overrides:
      • GHAW_POLICY_MODELS_ALLOWED
      • GHAW_POLICY_MODELS_DISALLOWED
    • Override values are parsed as model lists and applied with precedence over frontmatter/import-derived policy.
  • AWF config mapping

    • Emitted merged/effective policy into AWF config:
      • apiProxy.allowedModels
      • apiProxy.disallowedModels
# workflow frontmatter
models:
  allowed: [gpt-5, claude-sonnet]
  disallowed: [gpt-5-pro]
// generated AWF apiProxy fragment
{
  "allowedModels": ["gpt-5", "claude-sonnet"],
  "disallowedModels": ["gpt-5-pro"]
}

pr-sous-chef https://github.com/github/gh-aw/actions/runs/28333890571

Generated by 👨‍🍳 PR Sous Chef · 49.3 AIC · ⌖ 1.05 AIC · ⊞ 17.3K ·


pr-sous-chef: requested branch update via https://github.com/github/gh-aw/actions/runs/28336535351

Generated by 👨‍🍳 PR Sous Chef · 57.9 AIC · ⌖ 1.4 AIC · ⊞ 17.3K ·


pr-sous-chef: refresh branch for workflow run 28338043309

Generated by 👨‍🍳 PR Sous Chef · 47.3 AIC · ⌖ 1.47 AIC · ⊞ 17.3K ·


Generated by 👨‍🍳 PR Sous Chef · 67.9 AIC · ⌖ 1.53 AIC · ⊞ 17.6K ·


pr-sous-chef run: https://github.com/github/gh-aw/actions/runs/28381497646

Generated by 👨‍🍳 PR Sous Chef · 72.2 AIC · ⌖ 1.02 AIC · ⊞ 17.6K ·

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
@pelikhan pelikhan marked this pull request as ready for review June 27, 2026 01:19
Copilot AI review requested due to automatic review settings June 27, 2026 01:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds end-to-end “model policy” support to gh-aw workflows, allowing authors (and centralized operators via env vars) to control which models are permitted/blocked, and ensuring policies compose safely across imported workflows before being emitted into the generated AWF config.

Changes:

  • Extended workflow frontmatter models to support allowed, disallowed, and blocked policy lists (alongside optional pricing providers).
  • Propagated model policy through import extraction and merged policies across imports + main workflow using union semantics.
  • Emitted effective model policy to AWF config (apiProxy.allowedModels / apiProxy.disallowedModels) with env override precedence.
Show a summary per file
File Description
pkg/workflow/workflow_builder.go Extracts main workflow model policy and unions it with imported policy sets into WorkflowData.
pkg/workflow/workflow_builder_model_policy_test.go Adds unit tests for policy extraction and union merge behavior.
pkg/workflow/model_aliases_test.go Verifies frontmatter parsing populates parsed model policy lists.
pkg/workflow/frontmatter_types.go Adds parsed frontmatter fields for model policy lists.
pkg/workflow/frontmatter_parsing.go Parses model policy lists from raw frontmatter into typed config.
pkg/workflow/compilerenv/manager.go Adds env-driven policy overrides for allowed/blocked model sets.
pkg/workflow/compilerenv/manager_test.go Tests env override parsing and “unset” behavior.
pkg/workflow/compiler_types.go Plumbs merged model policy into WorkflowData.
pkg/workflow/awf_config.go Maps effective model policy (with env precedence) into AWF apiProxy config.
pkg/workflow/awf_config_test.go Tests AWF config emission and env override precedence.
pkg/parser/schemas/main_workflow_schema.json Updates schema for models to include policy fields and make providers optional.
pkg/parser/import_processor.go Extends ImportsResult to carry extracted model policy sets.
pkg/parser/import_field_extractor.go Extracts model policy from imported workflows and avoids treating policy keys as aliases.
pkg/parser/import_field_extractor_test.go Adds tests ensuring model policy is extracted (and not misinterpreted as aliases) and can coexist with model costs.

Review details

Tip

Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comments suppressed due to low confidence (2)

pkg/parser/import_field_extractor.go:639

  • When an imported workflow has models.providers plus model policy keys (allowed/disallowed/blocked), this appends the entire rawModels object into acc.modelCosts. That will later flow into WorkflowData.ModelCosts and into GH_AW_INFO_MODEL_COSTS, leaking policy keys into a payload that is expected to match the models.json pricing structure (providers-only). This can break downstream cost merging/parsing.
	if _, hasProviders := rawModels["providers"]; hasProviders {
		acc.modelCosts = append(acc.modelCosts, rawModels)
		if providers, ok := rawModels["providers"].(map[string]any); ok {
			parserLog.Printf("Extracted model costs from import: providers=%d", len(providers))
		} else {

pkg/workflow/workflow_builder.go:166

  • Now that models frontmatter can contain policy keys (allowed/disallowed/blocked) without providers, toolsResult.parsedFrontmatter.ModelCosts may be non-empty even when there is no pricing data (because it unmarshals the whole models object). extractMainModelCostsOverlay currently returns that map as a cost overlay, which can cause policy-only models to be emitted as GH_AW_INFO_MODEL_COSTS and/or pollute the providers overlay with policy keys.
	}

	return workflowData
}

  • Files reviewed: 14/14 changed files
  • Comments generated: 0
  • Review effort level: Low

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage

Category feature
Risk 🔴 High
Priority High (score 50/100)
Action defer · Batch: active-drafts

Score breakdown: Impact 35 · Urgency 5 · Quality 10

New model policy frontmatter controls (models.allowed/disallowed/blocked) with import-safe union semantics (+483/-11, 14 files). Just created (<1 h), draft, no CI yet. Well-described and scoped. Deferred pending CI and agent completion. Part of pr-batch:active-drafts group (#41824, #41822, #41821).

Generated by 🔧 PR Triage Agent · 87.6 AIC · ⌖ 14.9 AIC · ⊞ 5.4K ·

@github-actions

This comment has been minimized.

@github-actions

This comment has been minimized.

@github-actions

Copy link
Copy Markdown
Contributor

Hey @Copilot 👋 — great work on the model policy frontmatter feature! The addition of models.allowed, models.disallowed, and models.blocked to workflow frontmatter — along with import-safe union semantics and centralized env overrides — is a well-scoped, coherent change.

The PR is well-structured:

  • ✅ Clear, detailed description with YAML/JSON examples showing the before/after config shape.
  • ✅ Tests across all touched layers: import_field_extractor_test.go, awf_config_test.go, compilerenv/manager_test.go, model_aliases_test.go, and the newly added workflow_builder_model_policy_test.go.
  • ✅ No unrelated changes — every file touched is in service of the model policy feature.
  • ✅ No new external dependencies introduced.

This looks ready for review. 🚀

Generated by ✅ Contribution Check · 307.9 AIC · ⌖ 20.6 AIC · ⊞ 6K ·

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot merge main and recompile

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Copilot AI requested a review from pelikhan June 27, 2026 02:33
@pelikhan

Copy link
Copy Markdown
Collaborator

/review

@github-actions

github-actions Bot commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

PR Code Quality Reviewer completed the code quality review.

Copilot AI and others added 2 commits June 29, 2026 03:36
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>
Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, refresh this branch from main, address the unresolved review feedback, and rerun checks once the branch is up to date.

Unresolved review feedback:...

Done in 99bb385 (with merge refresh in baf4d7f). I refreshed this branch from main, resolved the merge conflict in smoke-claude.lock.yml, and reran local validation (make fmt, make lint, make test-unit, make test, make recompile) including wasm-golden refresh for base drift. I also checked the referenced run (28346305014) and it has no failed jobs, and PR review threads currently return no unresolved items via API.

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage — §28357644191

Field Value
Category feature
Risk 🔴 High
Score 55/100 (Impact 30 · Urgency 15 · Quality 10)
Action fast_track

⚠️ 55.1h old — CHANGES_REQUESTED blocking. Model policy frontmatter (models.allowed/models.disallowed) + import unioning + env overrides (+657/-23, 14 files). Review cited 3 correctness bugs. Author must resolve before merging. ⚠️ Label conflict: pr-action:defer + pr-action:fast_track are both set — pr-action:defer should be removed.

Generated by 🔧 PR Triage Agent · 89.9 AIC · ⌖ 12 AIC · ⊞ 5.4K ·

@github-actions

Copy link
Copy Markdown
Contributor

🤖 PR Triage — §28376613466

Field Value
Category feature
Risk 🔴 High
Priority 🔴 High
Score 51 / 100
Action 🔁 batch_review
Age 60.8h ⚠️ STALE

Score breakdown: Impact 35 + Urgency 12 + Quality 4

Rationale: Significant feature (model policy frontmatter + import unioning + env overrides), but blocked by CHANGES_REQUESTED from github-actions bot. Author has not yet addressed the review. 929+/117−, 27 files — large change requiring careful review.

Batch: Groups with #42235 and #42226 (all have CHANGES_REQUESTED pending).

⚠️ Label conflict: This PR has both pr-action:defer and pr-action:fast_track plus stale batch labels (pr-batch:active-drafts, pr-batch:feature-promo). Please manually remove conflicting labels and keep only pr-action:batch_review. Carried over from §28357644191.

Generated by 🔧 PR Triage Agent · 93.2 AIC · ⌖ 14.6 AIC · ⊞ 5.4K ·

@gh-aw-bot

Copy link
Copy Markdown
Collaborator

@copilot please run the pr-finisher skill, address the unresolved review comment, and rerun checks after the branch is refreshed.

Unresolved review:

Generated by 👨‍🍳 PR Sous Chef · 72.2 AIC · ⌖ 1.02 AIC · ⊞ 17.6K ·

…er-models-fields

# Conflicts:
#	.github/extensions/agentic-workflows-dashboard/extension.mjs
#	.github/workflows/smoke-claude.lock.yml
#	pkg/workflow/testdata/TestWasmGolden_AllEngines/claude.golden
#	pkg/workflow/testdata/TestWasmGolden_AllEngines/codex.golden
#	pkg/workflow/testdata/TestWasmGolden_AllEngines/copilot.golden
#	pkg/workflow/testdata/TestWasmGolden_AllEngines/gemini.golden
#	pkg/workflow/testdata/TestWasmGolden_AllEngines/pi.golden
#	pkg/workflow/testdata/TestWasmGolden_CompileFixtures/basic-copilot.golden
#	pkg/workflow/testdata/TestWasmGolden_CompileFixtures/playwright-cli-mode.golden
#	pkg/workflow/testdata/TestWasmGolden_CompileFixtures/smoke-copilot.golden
#	pkg/workflow/testdata/TestWasmGolden_CompileFixtures/with-imports.golden

Co-authored-by: gh-aw-bot <259018956+gh-aw-bot@users.noreply.github.com>

Copilot AI commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@copilot please run the pr-finisher skill, address the unresolved review comment, and rerun checks after the branch is refreshed.

Unresolved review:...

Done in 522c083.

  • refreshed this branch from main and resolved merge conflicts
  • reran local checks: make fmt, make lint, make test-unit, make test, make recompile
  • verified the referenced run (28381497646) has no failed jobs

The branch is current and locally validated. CI on this new HEAD still needs a maintainer-triggered rerun.

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot mark fields as experimental

@github-actions

Copy link
Copy Markdown
Contributor

🔍 PR Triage — §28395315609

Field Value
Category feature
Risk high
Score 54 / 100
Action batch_review
Batch

Score breakdown: impact 30 + urgency 16 + quality 8

Carried over (66h). Model policy frontmatter (allowed/disallowed lists + env overrides). CHANGES_REQUESTED from CI bot (dismissed approval). Priority corrected: pr-priority:highpr-priority:medium (score 54). ⚠️ Multiple conflicting action labels from prior runs (pr-action:defer, pr-action:fast_track, pr-action:batch_review) — manual label cleanup recommended. Needs CI resolution before merge.

Generated by 🔧 PR Triage Agent · 99.1 AIC · ⌖ 11.6 AIC · ⊞ 5.4K ·

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@copilot mark fields as experimental

Done in a83a52b.

Marked the model policy fields as experimental in the frontmatter schema descriptions (models.allowed, models.disallowed) and in related model-policy/env override code comments.

@pelikhan

Copy link
Copy Markdown
Collaborator

@copilot rename disallowed to blocked to match safe output vocabulary

Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>

Copilot AI commented Jun 29, 2026

Copy link
Copy Markdown
Contributor Author

@copilot rename disallowed to blocked to match safe output vocabulary

Addressed in a916dbc.

Renamed the model policy denylist vocabulary from disallowed to blocked across frontmatter parsing, import policy merging, schema/tests/env override naming (GHAW_POLICY_MODELS_BLOCKED), and updated smoke-claude (models.blocked: ["*opus*"]). The generated lock file was recompiled accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants