diff --git a/public/docs/i/1000/platform-hub/policies/policies-evaluation-details.webp b/public/docs/i/1000/platform-hub/policies/policies-evaluation-details.webp new file mode 100644 index 0000000000..760d262379 Binary files /dev/null and b/public/docs/i/1000/platform-hub/policies/policies-evaluation-details.webp differ diff --git a/public/docs/i/1000/platform-hub/policies/policies-evaluations-tab.webp b/public/docs/i/1000/platform-hub/policies/policies-evaluations-tab.webp new file mode 100644 index 0000000000..c322ebd02a Binary files /dev/null and b/public/docs/i/1000/platform-hub/policies/policies-evaluations-tab.webp differ diff --git a/public/docs/i/2000/platform-hub/policies/policies-evaluation-details.webp b/public/docs/i/2000/platform-hub/policies/policies-evaluation-details.webp new file mode 100644 index 0000000000..257059eecb Binary files /dev/null and b/public/docs/i/2000/platform-hub/policies/policies-evaluation-details.webp differ diff --git a/public/docs/i/2000/platform-hub/policies/policies-evaluations-tab.webp b/public/docs/i/2000/platform-hub/policies/policies-evaluations-tab.webp new file mode 100644 index 0000000000..fd81d25f7e Binary files /dev/null and b/public/docs/i/2000/platform-hub/policies/policies-evaluations-tab.webp differ diff --git a/public/docs/i/600/platform-hub/policies/policies-evaluation-details.webp b/public/docs/i/600/platform-hub/policies/policies-evaluation-details.webp new file mode 100644 index 0000000000..d4eeeedccf Binary files /dev/null and b/public/docs/i/600/platform-hub/policies/policies-evaluation-details.webp differ diff --git a/public/docs/i/600/platform-hub/policies/policies-evaluations-tab.webp b/public/docs/i/600/platform-hub/policies/policies-evaluations-tab.webp new file mode 100644 index 0000000000..34f62ebd59 Binary files /dev/null and b/public/docs/i/600/platform-hub/policies/policies-evaluations-tab.webp differ diff --git a/public/docs/i/x/platform-hub/policies/policies-evaluation-details.png b/public/docs/i/x/platform-hub/policies/policies-evaluation-details.png new file mode 100644 index 0000000000..0f27bcfa53 Binary files /dev/null and b/public/docs/i/x/platform-hub/policies/policies-evaluation-details.png differ diff --git a/public/docs/i/x/platform-hub/policies/policies-evaluations-tab.png b/public/docs/i/x/platform-hub/policies/policies-evaluations-tab.png new file mode 100644 index 0000000000..5de59e3ff4 Binary files /dev/null and b/public/docs/i/x/platform-hub/policies/policies-evaluations-tab.png differ diff --git a/public/docs/img/platform-hub/policies/policies-evaluation-details.png b/public/docs/img/platform-hub/policies/policies-evaluation-details.png new file mode 100644 index 0000000000..3bfd92091b Binary files /dev/null and b/public/docs/img/platform-hub/policies/policies-evaluation-details.png differ diff --git a/public/docs/img/platform-hub/policies/policies-evaluation-details.png.json b/public/docs/img/platform-hub/policies/policies-evaluation-details.png.json new file mode 100644 index 0000000000..97d7714a05 --- /dev/null +++ b/public/docs/img/platform-hub/policies/policies-evaluation-details.png.json @@ -0,0 +1 @@ +{"width":1912,"height":1368,"updated":"2026-06-19T03:38:43.046Z"} \ No newline at end of file diff --git a/public/docs/img/platform-hub/policies/policies-evaluations-tab.png b/public/docs/img/platform-hub/policies/policies-evaluations-tab.png new file mode 100644 index 0000000000..a869b0ba11 Binary files /dev/null and b/public/docs/img/platform-hub/policies/policies-evaluations-tab.png differ diff --git a/public/docs/img/platform-hub/policies/policies-evaluations-tab.png.json b/public/docs/img/platform-hub/policies/policies-evaluations-tab.png.json new file mode 100644 index 0000000000..e587aa1fcf --- /dev/null +++ b/public/docs/img/platform-hub/policies/policies-evaluations-tab.png.json @@ -0,0 +1 @@ +{"width":1912,"height":1368,"updated":"2026-06-19T03:38:43.211Z"} \ No newline at end of file diff --git a/src/pages/docs/platform-hub/policies/best-practices.md b/src/pages/docs/platform-hub/policies/best-practices.md index 6b39bd96df..87910dbbc9 100644 --- a/src/pages/docs/platform-hub/policies/best-practices.md +++ b/src/pages/docs/platform-hub/policies/best-practices.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-09-11 -modDate: 2025-09-25 +modDate: 2026-06-19 title: Policy best practices subtitle: Guidance on naming, rollout, and writing reliable policies icon: fa-solid fa-lock @@ -151,6 +151,22 @@ All policy evaluations are recorded in the Octopus audit log. If your organizati This gives your security team visibility into policy violations across your entire Octopus instance, and lets you build dashboards and alerts that match your compliance requirements. +## Preview against past executions before publishing + +:::div{.info} +Policy Evaluations is available from Octopus **2026.3.3162**. +::: + +The **Evaluations** tab on the edit policy page evaluates your policy against deployments and runbook runs that have already happened, so you can confirm it behaves as expected before you commit, publish, or activate it. The results aren't stored and don't affect real executions. + +This pairs well with the [warn before block](#use-warn-before-block) and [start narrow, then broaden](#start-narrow-then-broaden) practices: + +- **Check your scope.** Confirm the executions you expect are `In` scope and the ones you don't are `Out`. A scope that matches nothing, or matches far more than you intended, is easy to spot here. +- **Check your verdicts.** Look for executions that would be blocked or warned. If real past deployments would have been blocked, the teams that own them will need to fix their violations before you switch to `block`. +- **Catch runtime errors.** A `Runtime error` verdict usually means your Rego references a conditional field, such as `Release`, `Runbook`, or `Tenant`, without guarding against its absence. Fix these before publishing. See [Guard against conditional fields](#guard-against-conditional-fields). + +By default the tab evaluates your uncommitted draft, so you can iterate on your Rego and re-check the results without committing. See [Preview your policy](/docs/platform-hub/policies) in the getting started guide. + ## Testing your policy Before extending a policy's scope or switching from warn to block, verify it's evaluating correctly: diff --git a/src/pages/docs/platform-hub/policies/examples.md b/src/pages/docs/platform-hub/policies/examples.md index 4b4f902f13..8ea52b0e64 100644 --- a/src/pages/docs/platform-hub/policies/examples.md +++ b/src/pages/docs/platform-hub/policies/examples.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-09-11 -modDate: 2025-11-25 +modDate: 2026-06-19 title: Policy examples subtitle: Ready-to-use Rego organized by what you want to enforce icon: fa-solid fa-lock @@ -28,6 +28,12 @@ Start every new policy with `"action": "warn"` in your default result. This lets ::: +:::div{.hint} + +After you paste an example into the editor, use the **Evaluate** button to preview how it behaves against your past deployments and runbook runs before you commit. See [Preview a policy against past executions](/docs/platform-hub/policies/troubleshooting#preview-a-policy-against-past-executions). + +::: + ## Ensure required steps are present Use these examples when you want to guarantee that specific steps, such as approvals, security scans, or manual interventions, are always included in a deployment or runbook run and can't be removed or skipped. diff --git a/src/pages/docs/platform-hub/policies/index.md b/src/pages/docs/platform-hub/policies/index.md index 73f16ffc75..cc866c1f5a 100644 --- a/src/pages/docs/platform-hub/policies/index.md +++ b/src/pages/docs/platform-hub/policies/index.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-09-11 -modDate: 2025-09-11 +modDate: 2026-06-19 title: Policies subtitle: Enforce deployment standards automatically across your Octopus instance icon: fa-solid fa-lock @@ -168,7 +168,40 @@ result := {"allowed": true} if { For the full list of input fields you can reference in your conditions, see the [schema for policies](/docs/platform-hub/policies/schema). -### 7. Save your policy +### 7. Evaluate your policy + +:::div{.info} +Policy Evaluations is available from Octopus **2026.3.3064**. +::: + +Before you commit, use the **Evaluate** button to check how your policy would behave against deployments and runbook runs that have already happened. This lets you confirm your scope and conditions are correct without running a deployment, and nothing you preview is stored or affects real executions. + +By default, evaluations are for the current policy draft, including any uncommitted changes in the editor. You can also select a published version from the **Policy version** dropdown to compare behaviour. + +:::figure +![The Evaluations tab showing policy results against past executions](/docs/img/platform-hub/policies/policies-evaluations-tab.png) +::: + +Each row shows the **Verdict** for that execution and whether it was in **Scope**: + +- **Verdict:** `Compliant`, `Block`, `Warning`, or `Runtime error`. +- **Scope:** `In`, `Out`, or `Invalid`. + +Use **Show advanced filters** to narrow the results by task name, space, environment, project, tenant, execution type, or date range. + +Select **View** on any row to open the evaluation details, including the violation reason and the full policy input object that was passed to the policy engine. + +:::figure +![The evaluation details panel showing the verdict, reason, and policy input](/docs/img/platform-hub/policies/policies-evaluation-details.png) +::: + +:::div{.hint} + +The Evaluations tab evaluates against existing deployment and runbook runs. The results aren't stored, don't appear in the audit log, and don't affect the executions they're evaluated against. + +::: + +### 8. Save your policy Once you've finished writing your Rego, commit your changes to save them to your Git repository. You can commit with a description or use quick commit without one. @@ -176,7 +209,7 @@ Once you've finished writing your Rego, commit your changes to save them to your ![The commit experience for a policy](/docs/img/platform-hub/policies/policies-commit-experience.png) ::: -### 8. Publish your policy +### 9. Publish your policy After committing, publish the policy to make the changes take effect. Choose the appropriate version type: @@ -194,7 +227,7 @@ The first time you publish a policy, you can only publish a major version. ![Publish experience for a policy](/docs/img/platform-hub/policies/policies-publishing.png) ::: -### 9. Activate your policy +### 10. Activate your policy Published policies must be activated before Octopus evaluates them. You can deactivate a policy at any time to stop it from being evaluated without deleting it. @@ -208,7 +241,7 @@ Activation settings can be updated at any time from the **Versions** tab on the ![Activation status for a policy](/docs/img/platform-hub/policies/policies-activation.png) ::: -### 10. Test your policy +### 11. Validate your policy With the policy active, trigger a deployment to the Development environment for the ACME project. Because the violation action is set to `warn`, the deployment will proceed regardless. If the manual intervention step is missing, Octopus records the violation in the task log and project dashboard. Check both to confirm the policy is evaluating as expected. diff --git a/src/pages/docs/platform-hub/policies/schema.md b/src/pages/docs/platform-hub/policies/schema.md index 66eb5193c5..ef47c4264e 100644 --- a/src/pages/docs/platform-hub/policies/schema.md +++ b/src/pages/docs/platform-hub/policies/schema.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-09-11 -modDate: 2025-09-11 +modDate: 2026-06-19 title: Schema for policies subtitle: A reference for the input schema passed to the policy engine, including field descriptions, conditional fields, and example patterns. icon: fa-solid fa-lock @@ -15,6 +15,12 @@ When Octopus evaluates a policy, it passes a structured input object to the poli This page describes every field available in that input object, explains which fields are always present and which are conditional, and shows common patterns for using them. For complete working examples, see [Policy examples](/docs/platform-hub/policies/examples). +:::div{.hint} + +To see the resolved input object for a real deployment or runbook run, use the **Evaluate** button on the edit policy page and select **View** on any result. The **Policy input** section shows the exact object that was passed to the policy engine. See [Preview a policy against past executions](/docs/platform-hub/policies/troubleshooting#evaluate-a-policy-against-past-executions). + +::: + ## What's in the input object The table below summarizes every top-level field available to your policies. diff --git a/src/pages/docs/platform-hub/policies/troubleshooting.md b/src/pages/docs/platform-hub/policies/troubleshooting.md index 3c934695f0..c09f9bc0f9 100644 --- a/src/pages/docs/platform-hub/policies/troubleshooting.md +++ b/src/pages/docs/platform-hub/policies/troubleshooting.md @@ -1,7 +1,7 @@ --- layout: src/layouts/Default.astro pubDate: 2025-11-25 -modDate: 2025-11-25 +modDate: 2026-06-19 title: Troubleshooting policies subtitle: Diagnose and fix common issues with policy evaluation icon: fa-solid fa-layer-group @@ -51,6 +51,29 @@ Audit log entries only appear for executions that fall in the policy's scope. If --- +## Evaluate a policy against past executions + +:::div{.info} +Policy Evaluations is available from Octopus **2026.3.3064**. +::: + +The fastest way to diagnose a scope or conditions problem is the **Evaluations** tab on the edit policy page. It evaluates a policy version against deployments and runbook runs that have already happened, without you having to run a new deployment. The results aren't stored and don't affect real executions. + +By default it evaluates your current draft, including uncommitted changes, so you can edit your Rego and immediately re-check the results. You can also select a published version from the **Policy version** dropdown. + +Each result shows a **Verdict** and a **Scope**: + +| Column | Values | What it tells you | +| :--- | :--- | :--- | +| Verdict | `Compliant`, `Block`, `Warning`, `Runtime error` | Whether the execution passes, and what would happen if it didn't | +| Scope | `In`, `Out`, `Invalid` | Whether the execution is evaluated by this policy | + +Select **View** on any row to see the violation reason and the full policy input object that was evaluated. Use **Show advanced filters** to narrow the results by task name, space, environment, project, tenant, execution type, or date range. + +A `Runtime error` verdict or an `Invalid` scope points to an error in your Rego, often a reference to a conditional field that isn't present. See [Policy causes an evaluation error on runbook runs](#policy-causes-an-evaluation-error-on-runbook-runs). + +--- + ## Common problems ### Policy is not evaluating @@ -59,7 +82,7 @@ If a policy isn't appearing in the task log or audit log for an execution you ex 1. **Check the policy is activated.** A published policy must be activated before Octopus evaluates it. Go to the **Versions** tab on the edit policy page and confirm the policy is active. -1. **Check the scope Rego.** The scope determines which executions the policy evaluates. Open the policy editor and review your scope Rego. Use the verbose task log on a deployment you expect to be in scope to see what input fields were passed, and check whether your scope conditions would match. +1. **Check the scope Rego.** The scope determines which executions the policy evaluates. Open the policy editor and review your scope Rego. Use the verbose task log on a deployment you expect to be in scope to see what input fields were passed, and check whether your scope conditions would match. You can also use the [Evaluations tab](#evaluate-a-policy-against-past-executions) to see whether past executions resolve to `In` or `Out` of scope. ### Policy is blocking when it should warn @@ -96,7 +119,7 @@ See [Check for both existence and skipping](/docs/platform-hub/policies/best-pra ### Policy causes an evaluation error on runbook runs -If a policy works correctly for deployments but causes an error on runbook runs, it's likely referencing `input.Release` without guarding against its absence. `Release` is only present for deployments. +If a policy works correctly for deployments but causes an error on runbook runs, it's likely referencing `input.Release` without guarding against its absence. `Release` is only present for deployments. These errors show as a `Runtime error` verdict on the [Evaluations tab](#evaluate-a-policy-against-past-executions). Add a scope to limit the policy to deployments only: @@ -129,7 +152,7 @@ See [Guard against conditional fields](/docs/platform-hub/policies/best-practice If a policy is evaluating executions it shouldn't, or not evaluating ones it should, the scope Rego is likely not matching as expected. -Turn on verbose logging in the task log for an affected execution. This shows the full input object, including the exact values for `Environment.Slug`, `Project.Slug`, `Space.Slug`, and other fields your scope may be checking. Compare these against your scope Rego to identify the mismatch. +Turn on verbose logging in the task log for an affected execution. This shows the full input object, including the exact values for `Environment.Slug`, `Project.Slug`, `Space.Slug`, and other fields your scope may be checking. Compare these against your scope Rego to identify the mismatch. The [Evaluations tab](#evaluate-a-policy-against-past-executions) is also useful here: filter the results and check the **Scope** column to see exactly which past executions your scope matches. Common causes: