Skip to content

[cinder_multiattach] Wait for Glance reconciliation after config patch#4060

Open
fmount wants to merge 1 commit into
openstack-k8s-operators:mainfrom
fmount:glance_cinder_multi
Open

[cinder_multiattach] Wait for Glance reconciliation after config patch#4060
fmount wants to merge 1 commit into
openstack-k8s-operators:mainfrom
fmount:glance_cinder_multi

Conversation

@fmount

@fmount fmount commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

The oc wait --for=condition=Ready on the controlplane returns immediately when the condition is already true, before the StatefulSet rolling update triggered by the config change completes.
This causes a race with subsequent hooks (e.g. amphora image upload) that depend on Glance being stable.

This patch replaces the inline oc wait with two explicit steps:

  1. Poll the Glance CR until customServiceConfig reflects the change (oc apply happened and the rollout started)
  2. Wait for the controlplane Ready condition after reconciliation starts (which is what we had previously)

Closes: OSPCIX-1454

@openshift-ci

openshift-ci Bot commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign yorabl for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@fmount
fmount requested a review from tosky July 15, 2026 20:52
@fmount
fmount force-pushed the glance_cinder_multi branch from 01cbd8f to e43ecfe Compare July 15, 2026 20:53
Comment thread hooks/playbooks/cinder_multiattach_volume_type.yml Outdated
@fmount
fmount force-pushed the glance_cinder_multi branch from e43ecfe to 3330406 Compare July 15, 2026 21:56
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/b1f330b98d6e428ca5e31605d3f36e48

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 44m 33s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 31m 06s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 52m 14s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 31m 01s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 1h 14m 15s
cifmw-crc-podified-edpm-baremetal-bootc FAILURE in 38m 33s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 42s
cifmw-pod-pre-commit FAILURE in 8m 48s

@tosky

tosky commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

The failure in the precommit job looks legit, but please wait before updating this as I'm testing it

@fmount
fmount force-pushed the glance_cinder_multi branch from 3330406 to 3b8144e Compare July 17, 2026 09:36
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/63a0cce247404d0c99c38bd685903ce3

✔️ openstack-k8s-operators-content-provider SUCCESS in 1h 54m 49s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 35m 18s
cifmw-crc-podified-edpm-baremetal NODE_FAILURE Node(set) request 099-0000145833 failed in 0s
cifmw-crc-podified-edpm-baremetal-minor-update NODE_FAILURE Node(set) request 099-0000145834 failed in 0s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 1h 04m 05s
cifmw-crc-podified-edpm-baremetal-bootc NODE_FAILURE Node(set) request 099-0000145846 failed in 0s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 45s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 45s

@fmount

fmount commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

recheck

@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/f3430c057d09425ebf9f621fa1e521e4

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 59m 06s
podified-multinode-edpm-deployment-crc FAILURE in 31m 30s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 47m 50s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 41m 36s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 17m 31s
cifmw-crc-podified-edpm-baremetal-bootc FAILURE in 1h 33m 05s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 34s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 56s

@fmount
fmount force-pushed the glance_cinder_multi branch 2 times, most recently from 77a964b to c6612d0 Compare July 17, 2026 15:24
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/0ea4b72d6de948c29d450504ac8f839f

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 30m 28s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 32m 36s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 48m 57s
cifmw-crc-podified-edpm-baremetal-minor-update NODE_FAILURE Node(set) request 099-0000146102 failed in 0s
openstack-k8s-operators-content-provider-bootc TIMED_OUT in 46m 24s
⚠️ cifmw-crc-podified-edpm-baremetal-bootc SKIPPED Skipped due to failed job openstack-k8s-operators-content-provider-bootc
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 8m 01s
✔️ cifmw-pod-pre-commit SUCCESS in 8m 47s

@fmount
fmount force-pushed the glance_cinder_multi branch from c6612d0 to aaa3ded Compare July 18, 2026 07:09
Comment thread hooks/playbooks/cinder_multiattach_volume_type.yml Outdated
@centosinfra-prod-github-app

Copy link
Copy Markdown

Build failed (check pipeline). Post recheck (without leading slash)
to rerun all jobs. Make sure the failure cause has been resolved before
you rerun jobs.

https://gateway-cloud-softwarefactory.apps.ocp.cloud.ci.centos.org/zuul/t/rdoproject.org/buildset/53e9063b3b3b43ecb3ea18a74ae7cc88

✔️ openstack-k8s-operators-content-provider SUCCESS in 2h 30m 43s
✔️ podified-multinode-edpm-deployment-crc SUCCESS in 1h 36m 41s
✔️ cifmw-crc-podified-edpm-baremetal SUCCESS in 1h 51m 47s
✔️ cifmw-crc-podified-edpm-baremetal-minor-update SUCCESS in 2h 14m 44s
✔️ openstack-k8s-operators-content-provider-bootc SUCCESS in 2h 18m 20s
✔️ cifmw-crc-podified-edpm-baremetal-bootc SUCCESS in 1h 42m 22s
✔️ noop SUCCESS in 0s
✔️ cifmw-pod-ansible-test SUCCESS in 9m 27s
cifmw-pod-pre-commit FAILURE in 9m 34s

@fmount
fmount force-pushed the glance_cinder_multi branch 2 times, most recently from b655707 to ff90252 Compare July 18, 2026 12:59
The oc wait --for=condition=Ready on the controlplane returns
immediately when the condition is already true, before the
StatefulSet rolling update triggered by the config change completes.
This causes a race with subsequent hooks (e.g. amphora image upload)
that depend on Glance being stable.

Replace the inline oc wait with two explicit steps:
1. Poll the Glance CR until customServiceConfig reflects the change
2. Wait for both the glance CR and the controlplane Ready condition after
   reconciliation starts (Ready=False)

Closes: OSPCIX-1454

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Signed-off-by: Francesco Pantano <fpantano@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants