OCPBUGS-76523: Bootstrap failed in job: e2e-aws-crun-wasm#74716
OCPBUGS-76523: Bootstrap failed in job: e2e-aws-crun-wasm#74716liweinan wants to merge 1 commit intoopenshift:mainfrom
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: liweinan The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
[REHEARSALNOTIFIER]
Prior to this PR being merged, you will need to either run and acknowledge or opt to skip these rehearsals. Interacting with pj-rehearseComment: Once you are satisfied with the results of the rehearsals, comment: |
|
/pj-rehearse periodic-ci-openshift-release-main-nightly-4.22-e2e-aws-crun-wasm |
|
@liweinan: now processing your pj-rehearse request. Please allow up to 10 minutes for jobs to trigger or cancel. |
|
/jira-refresh |
|
@liweinan: all tests passed! Full PR test history. Your PR dashboard. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here. |
|
@sohankunkerkar Could you please help to review this test fix? Thanks! |
|
/jira refresh |
|
@liweinan: No Jira issue is referenced in the title of this pull request. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
liweinan
changed the title
openshift-ci-robot
added
the
jira/valid-reference
There was a problem hiding this comment.
I don't think you need this at all. You can simply remove cat > "/tmp/99-crun-wasm.conf" << EOF to disable this workflow
There was a problem hiding this comment.
@haircommander Do we need this workflow at all since we removed the wasm binary from OCP, right?
There was a problem hiding this comment.
Here's the analysis from Claude:
OCPBUGS-76523 Rebase Analysis Report
Date: 2026-02-14
Executive Summary
After rebasing OCPBUGS-76523 to the latest main branch, the analysis confirms that the crun-wasm workflow is no longer used by any CI jobs. The colleague's feedback is valid and the workflow should be removed entirely rather than fixed.
Rebase Results
Branch Status After Rebase
- Rebase: ✅ Successful (no conflicts)
- Commits before rebase: 1 commit behind main by ~20 commits
- Commits after rebase: 1 commit (5138f028356 "Bootstrap failed in job: e2e-aws-crun-wasm")
Key Finding
The commit 74d8ebb "Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+)" was merged into main on Feb 12, 2026. This commit removed all CI job configurations that referenced the crun-wasm workflow.
Current State Analysis
1. CI Job References (ci-operator/config/)
$ grep -r "workflow: openshift-e2e-aws-crun-wasm" ci-operator/config/ | wc -l
0Result: ❌ No CI configs reference this workflow anymore (removed in 74d8ebb)
2. Generated Job References (ci-operator/jobs/)
$ grep -r "crun-wasm" ci-operator/jobs/ | wc -l
0Result: ❌ No generated jobs reference this workflow
3. Step Registry Files - Critical Finding!
Main branch status: The workflow files were intentionally kept by PR #74847:
"The e2e-aws-crun-wasm workflow was left in place if anyone wants to use this in the future."
However, main branch contains the BUGGY version:
Main branch (openshift-manifests-crun-wasm-commands.sh):
[crio.runtime]
default_runtime = "crun-wasm" ← BREAKS SYSTEM CONTAINERS
[crio.runtime.runtimes.crun-wasm]
runtime_path = "/usr/bin/crun"
platform_runtime_paths = {"wasi/wasm32" = "/usr/bin/crun-wasm"}
...
extensions:
- wasm ← UNNECESSARY IN RHCOS 9.xYour branch (OCPBUGS-76523) - FIXED version:
[crio.runtime.runtimes.crun] ← CORRECT: configures existing runtime
platform_runtime_paths = {"wasi/wasm32" = "/usr/bin/crun-wasm"}
# No default_runtime override
# No extensions sectionWorkflow files present in both branches:
ci-operator/step-registry/openshift/e2e/aws/crun-wasm/openshift-e2e-aws-crun-wasm-workflow.yamlci-operator/step-registry/openshift/e2e/aws/crun-wasm/openshift-e2e-aws-crun-wasm-workflow.metadata.jsonci-operator/step-registry/openshift/manifests/crun-wasm/*(all files)
4. Cross-References in Step Registry
$ grep -r "openshift-manifests-crun-wasm" ci-operator/step-registry/ --exclude-dir="crun-wasm"Result: ❌ No other workflows or chains reference these components
Your Commit Analysis
What Your Commit Does
Commit 5138f028356 modifies openshift-manifests-crun-wasm-commands.sh to:
- Remove
default_runtime = "crun-wasm"(which was breaking system containers) - Configure the existing "crun" runtime instead of creating new "crun-wasm" runtime
- Use
platform_runtime_pathsto let crun automatically select WASM runtime - Remove unnecessary
extensions: - wasm(pre-installed in RHCOS 9.x)
The Problem
This fix is technically correct but practically useless because:
- No CI jobs will execute this code
- The workflow is completely orphaned
- The fix solves a problem for infrastructure that no longer exists
Colleague's Feedback Analysis
sohankunkerkar's Comments:
- "I don't think you need this at all. You can simply remove cat > /tmp/99-crun-wasm.conf to disable this workflow"
- "@haircommander Do we need this workflow at all since we removed the wasm binary from OCP, right?"
Assessment: ✅ Valid Feedback
The colleague is correct because:
- All jobs using this workflow were removed in commit 74d8ebb
- The wasm binary was reportedly removed from OCP
- Fixing an unused workflow provides no value
- The workflow should be removed entirely
Timeline of Events
- Earlier (unknown date): WASM binary removed from OCP
- Feb 11, 2026: You created commit dc4e31a fixing the crun-wasm workflow
- Branch created from an older main branch
- At that time, 32+ configs still referenced the workflow
- Feb 12, 2026: Commit 74d8ebb merged to main
- Removed all e2e-aws-crun-wasm job configurations (4.14+)
- Affected 55 files, removed 2,896 lines
- Feb 14, 2026: Rebase performed
- Your fix commit rebased onto latest main
- Now the workflow has zero users
Recommendations
Critical Context from PR #74847
The author (sdodson) stated: "The e2e-aws-crun-wasm workflow was left in place if anyone wants to use this in the future."
This means the workflow was intentionally preserved for potential future use, even though:
- WASM support was "only ever developer preview"
- It was removed per OCPNODE-3644
- All active CI jobs were removed
Option 1: Merge Your Fix (Recommended ✅)
Your PR fixes a real bug in the preserved workflow that could affect future users.
Action:
- Merge commit 5138f028356 to fix the buggy workflow in main
- Keep the workflow files for future use as originally intended
Rationale:
- The workflow was intentionally kept by the original deletion PR
- Main branch has a broken version that would fail if anyone tries to use it
- Your fix is technically correct and improves code quality
- No harm in fixing dormant code - it's better than leaving it broken
- If someone re-enables this in the future, they'll get working code
What the fix prevents:
- Setting
default_runtime = "crun-wasm"breaks system containers - Unnecessary
extensions: - wasm(already in RHCOS 9.x) - Proper platform-based runtime detection
Option 2: Remove the Entire Workflow
Delete all crun-wasm related files:
rm -rf ci-operator/step-registry/openshift/manifests/crun-wasm/
rm -rf ci-operator/step-registry/openshift/e2e/aws/crun-wasm/Rationale:
- WASM support permanently removed from OCP (per OCPNODE-3644)
- No plans to re-enable (colleague's question suggests this)
- Reduces maintenance burden
However:
- This contradicts the intent of PR Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847 which explicitly preserved it
- Should get approval from @sdodson or @haircommander first
Option 3: Do Nothing
Not recommended - leaves broken code in main branch
Action Items
Immediate Actions (Recommended Path)
- Push the rebased OCPBUGS-76523 branch to your remote
- Respond to @sohankunkerkar's review with context:
- Acknowledge that no CI jobs currently use this workflow
- Explain that PR Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847 intentionally preserved it for future use
- Note that main branch has broken version, your PR fixes it
- Ask: "Should we remove entirely or keep the fixed version for potential future use?"
- Wait for team decision on whether to proceed or remove
If Team Says "Merge the Fix"
- Update PR description to explain the context
- Add comment in the script explaining it's dormant but preserved
- Ensure rebase is clean (already done ✅)
- Request re-review
- Merge when approved
If Team Says "Remove It"
- Close/abandon this PR
- Create new PR to delete crun-wasm workflow files:
rm -rf ci-operator/step-registry/openshift/manifests/crun-wasm/ rm -rf ci-operator/step-registry/openshift/e2e/aws/crun-wasm/
- Run
make updateto regenerate metadata - Get approval from @sdodson (original author of Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847)
Documentation
- Add a note to PR description explaining:
- Workflow is currently unused (jobs removed in Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847)
- Workflow was intentionally preserved per Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847 comment
- This PR fixes bugs for potential future re-enablement
- Main branch has broken version, this PR provides working version
Conclusion
The Situation is Nuanced
After rebase and analysis, the situation is more complex than it initially appeared:
- Colleague's Concern is Valid: No CI jobs currently use this workflow
- BUT the workflow was intentionally preserved: PR Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847 explicitly kept it "if anyone wants to use this in the future"
- Main branch has broken code: The preserved version would fail if someone tried to use it
- Your fix is technically correct: It makes the dormant workflow actually functional
Two Valid Approaches
Approach A - Fix It (Recommended):
- Merge your PR to fix the broken workflow in main
- Better to have working dormant code than broken dormant code
- Respects the intent of PR Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847 to preserve it
- Zero risk since no jobs currently use it
- If WASM support returns, code is ready
Approach B - Remove It:
- Delete the workflow entirely
- Requires approval from @sdodson (author of Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847) or @haircommander
- Need confirmation that WASM will never return to OCP
- Contradicts the preservation intent of PR Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847
My Recommendation
Merge your fix, but add a comment to the PR explaining:
- The workflow is currently unused (all jobs removed in Remove e2e-aws-crun-wasm jobs from all CI configs (4.14+) #74847)
- The workflow was intentionally preserved for future use
- Your fix corrects bugs that would prevent future use
- This is low-risk maintenance of dormant code
Then respond to @sohankunkerkar asking: "The workflow was intentionally kept by PR #74847 for future use. Should we remove it entirely, or is it better to fix the bugs for when/if it's re-enabled?"
openshift-ci-robot
added
the
jira/invalid-bug
|
@liweinan: This pull request references Jira Issue OCPBUGS-76523, which is invalid:
Comment The bug has been updated to refer to the pull request using the external bug tracker. DetailsIn response to this: Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
|
@liweinan: This pull request references Jira Issue OCPBUGS-76523, which is invalid:
Comment DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
|
@liweinan: This pull request references Jira Issue OCPBUGS-76523, which is invalid:
Comment DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
/jira refresh |
openshift-ci-robot
added
jira/valid-bug
|
@tthvo: This pull request references Jira Issue OCPBUGS-76523, which is valid. The bug has been moved to the POST state. 3 validation(s) were run on this bug
Requesting review from QA contact: DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository. |
|
@openshift-ci-robot: GitHub didn't allow me to request PR reviews from the following users: liweinan. Note that only openshift members and repo collaborators can review this PR, and authors cannot review their own PRs. DetailsIn response to this:
Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
4 participants
No description provided.