Skip to content

ROX-35145: Use roxie in long-running release cluster setup #98

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
porridge wants to merge 27 commits into
base: main
Choose a base branch
from
+385 −211
Open

ROX-35145: Use roxie in long-running release cluster setup #98

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
27 commits
Select commit Hold shift + click to select a range
732fff1
Replace stackrox deploy scripts with roxie in start-acs
porridge Jun 30, 2026
03a3607
Add temporary test workflow for roxie start-acs
porridge Jun 30, 2026
1ba316d
Add workflow_dispatch trigger for testing
porridge Jun 30, 2026
406da5e
Skip properties and wait-for-images on workflow_dispatch
porridge Jun 30, 2026
23e5bdf
Allow jobs to proceed when wait-for-images/properties are skipped
porridge Jun 30, 2026
cd90b1f
Install roxctl if not in PATH before roxie deploy
porridge Jun 30, 2026
616261e
Add --early-readiness flag to roxie deploy
porridge Jun 30, 2026
b9787b6
Use premium-rwo storage class instead of faster
porridge Jun 30, 2026
c0a7274
Revert test-only workflow changes, keep only roxie install step
porridge Jul 1, 2026
5becf58
Revert "Revert test-only workflow changes, keep only roxie install step"
porridge Jul 1, 2026
52a3ffa
Restore test scaffolding, add roxie install for start-secured-cluster
porridge Jul 1, 2026
465185b
Replace deploy scripts with roxie in start-secured-cluster
porridge Jul 1, 2026
d5b4f6d
Skip Slack notifications when properties output is empty
porridge Jul 1, 2026
3dfcf7d
Fix secured-cluster deploy: propagate CA cert and API_ENDPOINT
porridge Jul 1, 2026
eb10e35
Use small resource profile for both central and secured cluster
porridge Jul 1, 2026
7c8ad6a
Use acs-defaults profile with custom resource overrides
porridge Jul 1, 2026
463af65
Match legacy HPA scaling and enable scanner v2
porridge Jul 1, 2026
f080ee0
Fix: SecuredCluster scanner uses AutoSense, not Enabled
porridge Jul 2, 2026
8f769d0
Align resource specs and HPA scaling with legacy deployment
porridge Jul 2, 2026
d86a4e4
Tidy workflow dispatch comments, config comments, input order
porridge Jul 2, 2026
4cb3535
Improve testing instructions with concrete example fields
porridge Jul 2, 2026
c2cbe4a
clean up envrc
porridge Jul 6, 2026
8bbcc3d
mkdir
porridge Jul 6, 2026
3d3837e
Pause operator reconciliation, match legacy central-db resources
porridge Jul 6, 2026
ae91988
Pin roxie version to v0.4.2 to avoid GitHub API rate limits
porridge Jul 6, 2026
d525279
Enable monitoring endpoints and sensor metrics scraping
porridge Jul 7, 2026
8fe110e
review comments
porridge Jul 8, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
88 changes: 74 additions & 14 deletions .github/workflows/create-demo-clusters.yml
View file
Open in desktop
Comment thread
porridge marked this conversation as resolved.
Original file line number Diff line number Diff line change
@@ -1,5 +1,44 @@
name: Create demo clusters for version
on:
# For testing this workflow from a branch of stackrox/actions (the normal
# path is workflow_call from stackrox/stackrox). To test:
# 1. Push your branch to stackrox/actions, note the HEAD commit SHA.
# 2. Create a branch in stackrox/stackrox and edit
# .github/workflows/create-clusters.yml — change these fields:
# uses: stackrox/actions/...@<your-branch-HEAD-SHA>
# workflow-ref: <your-branch-name>
# Also change CLUSTER_WITH_FAKE_LOAD_NAME and CLUSTER_WITH_REAL_LOAD_NAME
# to unique prefixes to avoid clashing with production clusters.
# 3. Dispatch "RELEASE: Create Clusters" from that stackrox/stackrox branch
# with create-long-cluster=true and an existing ACS version (e.g. 4.11.0).

@JoukoVirtanen JoukoVirtanen Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not how I test changes to this workflow. There is a repo at stackrox/test-gh-actions that is used for testing these github actions. I create a new branch there and change .github/workflows/create-clusters.yml. I change instances of v1 to the stackrox/actions branch that I am testing. I then trigger the github action from https://github.com/stackrox/test-gh-actions/actions/workflows/create-clusters.yml

workflow_dispatch:

@JoukoVirtanen JoukoVirtanen Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this duplication. It adds to the complexity unnecessarily.

inputs:
version:
description: Version of the images
required: true
type: string
create-long-cluster:
description: Create a long-running cluster
default: true
type: boolean
workflow-ref:
description: Ref for actions checkout (defaults to current branch)
type: string
kube-burner-config-ref:
description: Ref for kube-burner config (defaults to version)
type: string
kube-burner-config-repo:
description: Repo with kube-burner config
default: stackrox
type: string
cluster-with-fake-load-name:
description: Fake-load cluster name
default: test-fl
type: string
cluster-with-real-load-name:
description: Real-load cluster name
default: test-rl
type: string
workflow_call:
inputs:
version:
Expand Down Expand Up @@ -51,6 +90,7 @@ env:

jobs:
properties:
if: github.event_name != 'workflow_dispatch'

@JoukoVirtanen JoukoVirtanen Jul 8, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I favor removing workflow_dispatch entirely. From now on I won't comment on other uses of it.

runs-on: ubuntu-latest
outputs:
slack-channel: ${{ fromJSON(format('["{0}","{1}"]', steps.fetch.outputs.dry-slack-channel, steps.fetch.outputs.slack-channel))[inputs.dry-run != true] }}
Expand All @@ -72,7 +112,7 @@ jobs:
- name: Parse refs (extract SHA from git-describe format)
env:
INPUT_VERSION: ${{ inputs.version }}
INPUT_KUBE_BURNER_VERSION: ${{ inputs.kube-burner-config-ref }}
INPUT_KUBE_BURNER_VERSION: ${{ inputs.kube-burner-config-ref || inputs.version }}
id: parse
run: |
set -euo pipefail
Expand Down Expand Up @@ -103,10 +143,12 @@ jobs:
wait-for-images:
name: Wait for images on Quay.io
runs-on: ubuntu-latest
if: >- # Skip if no clusters are going to be created.
if: >-
github.event_name != 'workflow_dispatch' && (
inputs.create-k8s-cluster != false ||
inputs.create-os4-cluster != false ||
inputs.create-long-cluster != false
)
strategy:
matrix:
image: [main, scanner, scanner-db, scanner-v4, scanner-v4-db, collector]
Expand All @@ -116,7 +158,7 @@ jobs:
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
- name: Wait for the ${{matrix.image}} image
uses: ./.actions/release/wait-for-image
with:
Expand All @@ -129,15 +171,15 @@ jobs:
create-k8s-cluster:
name: Create k8s cluster
needs: [wait-for-images]
if: inputs.dry-run != true && inputs.create-k8s-cluster != false
if: "!cancelled() && !failure() && inputs.dry-run != true && inputs.create-k8s-cluster != false"
runs-on: ubuntu-latest
steps:
- name: Checkout local action
uses: actions/checkout@v4
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
Comment thread
porridge marked this conversation as resolved.
- uses: ./.actions/infra/create-cluster
with:
token: ${{ secrets.INFRA_TOKEN }}
Expand All @@ -159,6 +201,7 @@ jobs:
echo "cluster-name=${NAME//./-}" >> "$GITHUB_OUTPUT"
echo "url=https://${NAME//[.-]/}.demos.rox.systems/login" >> "$GITHUB_OUTPUT"
- name: Post to Slack
if: needs.properties.outputs.slack-channel != ''
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
uses: slackapi/slack-github-action@v1.26.0
Expand Down Expand Up @@ -191,15 +234,15 @@ jobs:
create-os4-cluster:
name: Create OS4 cluster
needs: [wait-for-images]
if: inputs.dry-run == false && inputs.create-os4-cluster == true
if: "!cancelled() && !failure() && inputs.dry-run != true && inputs.create-k8s-cluster != false"
runs-on: ubuntu-latest
steps:
- name: Checkout local action
uses: actions/checkout@v4
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
Comment thread
porridge marked this conversation as resolved.
- uses: ./.actions/infra/create-cluster
with:
token: ${{ secrets.INFRA_TOKEN }}
Expand All @@ -221,6 +264,7 @@ jobs:
echo "cluster-name=${NAME//./-}" >> "$GITHUB_OUTPUT"
echo "url=https://central-stackrox.apps.${NAME//./-}.ocp.infra.rox.systems/login" >> "$GITHUB_OUTPUT"
- name: Post to Slack
if: needs.properties.outputs.slack-channel != ''
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
uses: slackapi/slack-github-action@v1.26.0
Expand Down Expand Up @@ -255,6 +299,7 @@ jobs:
name: Create GKE long-running cluster for fake load
needs: [wait-for-images]
if: >-
!cancelled() && !failure() &&
inputs.dry-run != true &&
inputs.create-long-cluster == true
runs-on: ubuntu-latest
Expand All @@ -264,7 +309,7 @@ jobs:
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
- uses: ./.actions/infra/create-cluster
with:
token: ${{ secrets.INFRA_TOKEN }}
Expand All @@ -280,6 +325,7 @@ jobs:
name: Create GKE long-running cluster for real load
needs: [wait-for-images]
if: >-
!cancelled() && !failure() &&
inputs.dry-run != true &&
inputs.create-long-cluster == true
runs-on: ubuntu-latest
Expand All @@ -289,7 +335,7 @@ jobs:
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
- uses: ./.actions/infra/create-cluster
with:
token: ${{ secrets.INFRA_TOKEN }}
Expand All @@ -303,10 +349,12 @@ jobs:
start-acs:
name: Start ACS
needs: [properties, parse-refs, create-long-running-cluster-for-fake-load]
if: "!cancelled() && !failure()"
runs-on: ubuntu-latest
outputs:
rox-password: ${{ steps.launch-central.outputs.rox-password }}
central-ip: ${{ steps.launch-central.outputs.central-ip }}
ca-cert: ${{ steps.launch-central.outputs.ca-cert }}
env:
NAME: ${{ inputs.cluster-with-fake-load-name }}
KUBECONFIG: artifacts/kubeconfig
Expand All @@ -330,7 +378,7 @@ jobs:
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
- uses: "google-github-actions/auth@v2"
with:
credentials_json: "${{ secrets.GCP_RELEASE_AUTOMATION_SA }}"
Expand All @@ -342,6 +390,9 @@ jobs:
id: artifacts
run: |
infractl artifacts "${NAME//./-}" -d artifacts >> "$GITHUB_STEP_SUMMARY"
- uses: ./.actions/roxie/install-cli
with:
version: v0.4.2
- name: Launch central
id: launch-central
uses: ./.actions/release/start-acs
Expand All @@ -355,6 +406,7 @@ jobs:
name: ${{ env.NAME }}

- name: Post to Slack
if: needs.properties.outputs.slack-channel != ''
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
uses: slackapi/slack-github-action@v1.26.0
Expand Down Expand Up @@ -398,6 +450,7 @@ jobs:
start-secured-cluster:
name: Start secured cluster
needs: [properties, parse-refs, start-acs, create-long-running-cluster-for-real-load]
if: "!cancelled() && !failure()"
runs-on: ubuntu-latest
env:
SECURED_CLUSTER_NAME: ${{ inputs.cluster-with-real-load-name }}
Expand All @@ -422,7 +475,7 @@ jobs:
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
- uses: "google-github-actions/auth@v2"
with:
credentials_json: "${{ secrets.GCP_RELEASE_AUTOMATION_SA }}"
Expand All @@ -440,6 +493,9 @@ jobs:
registry: quay.io
username: ${{ secrets.QUAY_RHACS_ENG_RO_USERNAME }}
password: ${{ secrets.QUAY_RHACS_ENG_RO_PASSWORD }}
- uses: ./.actions/roxie/install-cli
with:
version: v0.4.2
- name: Launch secured cluster
id: launch-secured-cluster
uses: ./.actions/release/start-secured-cluster
Expand All @@ -451,10 +507,12 @@ jobs:
registry-password: ${{ secrets.QUAY_RHACS_ENG_RO_PASSWORD }}
rox-admin-password: ${{ needs.start-acs.outputs.rox-password }}
central-ip: ${{ needs.start-acs.outputs.central-ip }}
ca-cert: ${{ needs.start-acs.outputs.ca-cert }}

start-kube-burner-for-secured-cluster:
name: Start kube-burner for the secured cluster
needs: [properties, parse-refs, start-secured-cluster]
if: "!cancelled() && !failure()"
runs-on: ubuntu-latest
env:
INFRA_NAME: ${{ inputs.cluster-with-real-load-name }}
Expand Down Expand Up @@ -483,7 +541,7 @@ jobs:
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
- uses: "google-github-actions/auth@v2"
with:
credentials_json: "${{ secrets.GCP_RELEASE_AUTOMATION_SA }}"
Expand All @@ -509,6 +567,7 @@ jobs:
start-kube-burner-for-central:
name: Start kube-burner for Central
needs: [properties, parse-refs, start-acs]
if: "!cancelled() && !failure()"
runs-on: ubuntu-latest
env:
INFRA_NAME: ${{ inputs.cluster-with-fake-load-name }}
Expand All @@ -525,13 +584,13 @@ jobs:
with:
repository: stackrox/${{ inputs.kube-burner-config-repo }}
path: .kube-burner-config
ref: ${{ needs.parse-refs.outputs.version-ref }}
ref: ${{ needs.parse-refs.outputs.burner-ref }}
- name: Checkout local action
uses: actions/checkout@v4
with:
repository: stackrox/actions
path: .actions
ref: ${{ inputs.workflow-ref }}
ref: ${{ inputs.workflow-ref || github.ref_name }}
- uses: "google-github-actions/auth@v2"
with:
credentials_json: "${{ secrets.GCP_RELEASE_AUTOMATION_SA }}"
Expand Down Expand Up @@ -575,6 +634,7 @@ jobs:
runs-on: ubuntu-latest
steps:
- name: Post to Slack
if: needs.properties.outputs.slack-channel != ''
env:
SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
uses: slackapi/slack-github-action@v1.26.0
Expand Down
18 changes: 6 additions & 12 deletions release/start-acs/action.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ inputs:
required: true
default: ""
stackrox-dir:
description: Where the stackrox directory is located
description: Where the stackrox directory is located (needed for monitoring chart)
required: true
default: ""
name:
Expand All @@ -42,24 +42,15 @@ runs:

- id: launch-central
env:
ROX_PRODUCT_BRANDING: RHACS_BRANDING
MAIN_IMAGE_TAG: ${{ inputs.main-image-tag }}
API_ENDPOINT: localhost:8000
STORAGE: pvc # Backing storage
STORAGE_CLASS: faster # Runs on an SSD type
STORAGE_SIZE: "100" # 100G
MONITORING_SUPPORT: "true" # Runs monitoring
LOAD_BALANCER: lb
SECURED_CLUSTER_AUTO_LOCK_PROCESS_BASELINES: "true"
ROX_ADMIN_USERNAME: admin

@JoukoVirtanen JoukoVirtanen Jul 8, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you comment in the PR why these environment variables are being removed and what impact removing them will have? Some of them may be in the roxie config. Others are being removed entirely.

Also there are comments for some of these in start-acs.sh. You will need to remove those.

MONITORING_SUPPORT: "true"
PAGERDUTY_INTEGRATION_KEY: ${{ inputs.pagerduty-integration-key }}
ROX_ADMIN_USERNAME: admin
REGISTRY_USERNAME: ${{ inputs.registry-username }}
REGISTRY_PASSWORD: ${{ inputs.registry-password }}
KUBECONFIG: ${{ inputs.kubeconfig }}
STACKROX_DIR: ${{ inputs.stackrox-dir }}
NAME: ${{ inputs.name }}
ROX_TELEMETRY_STORAGE_KEY_V1: R5fMyO9n0gibSGzOXtlP2qCFWCGb8uoW
ROX_SCANNER_V4: "true"
run: |
set -uo pipefail
"${GITHUB_ACTION_PATH}/../../common/common.sh" \
Expand All @@ -73,3 +64,6 @@ outputs:
central-ip:
description: The ip address of central
value: ${{steps.launch-central.outputs.central-ip}}
ca-cert:
description: Base64-encoded Central CA certificate
value: ${{steps.launch-central.outputs.ca-cert}}
45 changes: 0 additions & 45 deletions release/start-acs/patch-central.json
View file
Open in desktop

This file was deleted.

Loading
Loading