Skip to content

<fix>[ha]: ZSTAC-84992 backport HA pre-fence core to 5.4.10 (@@3)#4496

Open
ZStack-Robot wants to merge 40 commits into
5.4.10from
sync/yingzhe.hu/cp/5.4.10/ZSTAC-84992-20260707@@3
Open

<fix>[ha]: ZSTAC-84992 backport HA pre-fence core to 5.4.10 (@@3)#4496
ZStack-Robot wants to merge 40 commits into
5.4.10from
sync/yingzhe.hu/cp/5.4.10/ZSTAC-84992-20260707@@3

Conversation

@ZStack-Robot

Copy link
Copy Markdown
Collaborator

Summary

Backport HA pre-fence core changes for ZSTAC-84992 to 5.4.10.

Notes

  • Source branch intentionally matches the premium and zstack-utility MRs: cp/5.4.10/ZSTAC-84992-20260707@@3.
  • Supersedes the previous MR created with a repo-specific source branch name.

Resolves: ZSTAC-84992

sync from gitlab !10457

liang-hanyu and others added 30 commits January 20, 2026 18:07
DBImpact

Resolves: ZSTAC-81425

Change-Id: I627a6576686f6c6972636d67786e77716e637471
<fix>[conf]: bump version to 5.4.10

See merge request zstackio/zstack!9045
Use entity alias for simple return-with-total counts so nullable
selected fields do not turn totals into zero. Keep explicit
count(field) and distinct(field) behavior unchanged. Add
CountGroupByCase coverage for nullable first selected field.

Resolves: ZSTAC-78110

Change-Id: I3b437bc686c345ba17baf256a338ce88209571f8
(cherry picked from commit 0dfe389)
The vhost-user interface (used by OVS-DPDK) strictly requires the 'virtio-net' frontend driver in Libvirt/QEMU. However, ZStack defaults Windows NIC drivers to 'e1000' (or other emulated models) when the image platform is 'Windows', causing VM start failure with Libvirt error: "vhostuser requires the virtio-net* frontend".

This fix forces the NIC driver to 'virtio' specifically when the interface type is 'vhostuser' on Windows platforms, regardless of the default platform settings. Satisfying both the legacy boot requirements of Windows and the vhost-user network requirements.

Resolves: ZSTAC-76013

Change-Id: I737a6e637273796d776e726e7969786c7a6a6478
Route AutoQuery handling through the query sync queue so bursty GraphQL refreshes cannot run unlimited APIQueryMessage requests concurrently.

Resolves: ZSTAC-80742

Change-Id: If2c052e539526cd0511e037d1718838bdb65a7a3
<fix>[kvm]: force virtio driver for windows vhostuser nic

See merge request zstackio/zstack!10342
Backport the normalizedModelName schema required by premium GPU spec deduplication to 5.4.10. This is the zstack companion change for the premium ZSTAC-86476 backport.

Constraint: ZSTAC-86476 is the 5.4.10 clone of ZSTAC-75319.
Confidence: high
Scope-risk: narrow
Tested: ./runMavenProfile premium in pr-build-bin Docker with zstack and premium mounted
Not-tested: local stability case execution

Resolves: ZSTAC-86476

Change-Id: Ic77e24e974a99842c40f29e4bd72851e3833795b
Backport ZSTAC-74748 pagination tiebreaker from 5.5.6/5.5.28 to 5.4.10.

Paginated LIMIT/OFFSET queries can return duplicate resources when the requested sort field has duplicate values. Append the primary key as a deterministic tiebreaker in both ZQL and legacy HQL query paths, and skip hidden PK fields that ZQL cannot order by.

Source-Commits: 284d049, b1156d8
Tested: git diff --check upstream/5.4.10..HEAD

Resolves: ZSTAC-86475

Change-Id: I14bef1d630a70711ef2605ea95fd37cb86e5273c
Existing host reconnect can fail before kvmagent echo.

Ansible masks libvirt sockets with systemctl during deploy.

If host systemd D-Bus is stuck, that optional step times out.

Continue only for this known mask timeout on reconnect.

New host deploy and other ansible failures still fail.

Resolves: ZSTAC-86349

Change-Id: I0ef4a535065ff797c9e4cfae5b39c2daa321a4cc
Resolves: ZSTAC-86350

Change-Id: I024ae51da6e5784fb336d8fbd5a4aeab44500565
Backport the management node hash ring fix to 5.4.10.

1. Why?
Management node lifecycle reconciliation can race with callbacks.
This can cause NodeHash or Nodes inconsistency and task stalling.

2. How?
Synchronize hash ring operations and use defensive copies.
Add lifecycle reconciliation locking and two-round missing checks.

3. Side effects?
Management node lifecycle operations are serialized by a local lock.

# Summary of changes (by module):
- core: synchronize resource destination hash ring operations.
- portal: serialize management node lifecycle reconciliation.

Source-Commit: 66ab572
Related: ZSTAC-86294, ZSTAC-78182
Change-Id: I3ba21ffc7b8f1284e17d50da7b027ffdad70828b
Backport external-storage split-brain guards to 5.4.10.

1. Why?
Starting a VM on a new host may continue even when the old storage
client still holds the volume. For xinfini Vhost volumes this can leave
two QEMU processes using the same bdev.

2. How?
Report xinfini deactivate failures instead of swallowing them, verify
the bdev is really deleted, re-check active clients after deactivate
success, and make blacklist failures propagate out of beforeStartVmOnKvm.

3. Side effects?
Deactivate failure or a still-active old client now aborts VM start for
that volume instead of silently continuing. Happy-path deactivate keeps
the previous behavior.

# Summary of changes (by module):
- header: make PrimaryStorageNodeSvc.blacklist synchronous.
- externalStorage: post-check active clients and propagate blacklist failures.
- xinfini: surface deactivate/delete failures and abort unsupported isolation.
- expon/zbs: adapt blacklist implementations to the synchronous signature.

Related: ZSTAC-86444
Related: ZSTAC-85124
Change-Id: I59812c6d68eec6533e48a8c9cf05b7c55d6b85eb
Backport VM HA skip-trace protection to 5.4.10.

1. Why?
When a management node leaves, its VM skip-trace entries are removed
immediately. If those VMs are still being started by kvmagent, the next
VM sync may treat them as stopped and trigger HA, causing split-brain.

2. How?
Move departed management node skip entries into an orphaned set, keep
them skipped for a configurable TTL, include them in VM sync skip lists,
and remove expired entries with compare-and-remove cleanup.

3. Side effects?
VMs whose controller MN departed stay skipped until the TTL expires.
The default TTL is 600 seconds and can be changed by global config
`kvm.vm.orphanedSkipTimeout`.

# Summary of changes (by module):
- kvm: add orphaned skip set, TTL global config, sync inclusion and CAS cleanup.

Related: ZSTAC-84636
Related: ZSTAC-80821
Change-Id: I7862360c7318cedb51e38939467cf2a7836e61d0
[5.4.10][ZSTAC-86350] <fix>[identity]: restrict APIQueryAccountMsg to admin-only in RBAC

See merge request zstackio/zstack!10380
Backport PMU-off handling from ZSTAC-76375 to 5.4.10 for Kunpeng openEuler guest panic.

Resolves: ZSTAC-86338

Change-Id: I3c2dbe1944c300a803f50198b4c7ba575bd77dcc
<fix>[kvm]: skip known systemd timeout on reconnect

See merge request zstackio/zstack!10362
Resolves: ZSTAC-78989

Change-Id: I0fe3a56ab724978944c69afadaab7ff7353e4c0f
(cherry picked from commit f19223a)
Pass dns-forward-max and cache-size through VR DNS commands.
Render both options in zstack-vyos dnsmasq config.

Resolves: ZSTAC-85197

Change-Id: Ib78504edc564a2ab7dbd39bcb2e0ee396d019240
(cherry picked from commit 4ada5a5)
Change-Id: If07dce918c4d7d9f62445184ff5984e30b83ce64
(cherry picked from commit 8939994)
Trigger TF port sync from TF controller ping success instead of MN startup. Keep the original one day interval and port cleanup algorithm, while scoping sync to the current controller.

Resolves: ZSTAC-85281

Change-Id: I84a88a34135cd47fc79b3019d2f0fc2953f06ac9
(cherry picked from commit e44d41a)
Remove DHCP message handlers and generated error code reference
unavailable on 5.4.10.

Change-Id: I1a15fb1eeeab60ca9c6fa239904bda89839b64e5
Add MessageSafe to the SDN message entrypoint. Move the second ping
after the first async sync window in the focused test.

Resolves: ZSTAC-86445

Change-Id: I4d7748a0de3b5d6a052a47966d2265249b2779e2
Resolves: ZSTAC-79353

Change-Id: I63706d727173706e796b64666963776a74736c61
(cherry picked from commit 2ef761d)
Resolves: ZSTAC-81450

Change-Id: I79666f616d64797a747677656762626d6a71786d
(cherry picked from commit 5c3fab5)
[ZSTAC-86394] Backport ZSTAC-78110 to 5.4.10

See merge request zstackio/zstack!10335
[ZSTAC-86484] Backport ZSTAC-78989 to 5.4.10

See merge request zstackio/zstack!10337
[ZSTAC-86501] Backport ZSTAC-79353 to 5.4.10

See merge request zstackio/zstack!10338
[ZSTAC-86489] Backport ZSTAC-85197 to 5.4.10

See merge request zstackio/zstack!10341
[ZSTAC-86445] Backport ZSTAC-85281 to 5.4.10

See merge request zstackio/zstack!10340
<fix>[kvm]: disable arm vm pmu by default

See merge request zstackio/zstack!10382
gitlab and others added 10 commits July 6, 2026 00:57
[ZSTAC-86470] Backport ZSTAC-81450 to 5.4.10

See merge request zstackio/zstack!10336
<fix>[search]: stabilize paginated query order

See merge request zstackio/zstack!10375
<fix>[gpu]: add gpu normalized model schema

See merge request zstackio/zstack!10370
The premium backport adds minAvailableDevicesPerHost to GetMdevDeviceCandidates. Commit the generated SDK action field on the companion zstack branch so update-sdk CI no longer reports an uncommitted SDK diff.

Constraint: premium and zstack companion branches must use the same @@2 suffix for cross-repo SDK validation
Tested: git diff --check
Not-tested: Full ./runMavenProfile sdk locally

Resolves: ZSTAC-86497

Change-Id: Iae56b01a4c533eeba05d10358d4944f655c5a6d4
ZSTAC-84636 backport VM HA orphaned skip-trace fix

See merge request zstackio/zstack!10401
ZSTAC-86444 backport external-storage split-brain guard

See merge request zstackio/zstack!10399
<fix>[mn]: synchronize hash ring operations

See merge request zstackio/zstack!10394
[5.4.10] cherry-pick ZSTAC query fix

See merge request zstackio/zstack!10343
<fix>[sdk]: update mdev candidates sdk

See merge request zstackio/zstack!10420
Backport the ZSTAC-83890 core HA pre-fence support to 5.4.10.

Carry the accessible peer host through HA start messages. Record the
pre-fence pending tag after HA state updates, and require a concrete
peer before creating the tag.

Test: git diff --check upstream/5.4.10..HEAD
Test: mvn -pl compute -am -DskipTests compile
Test: mvn -pl compute -am -DskipTests install

Source: ZSTAC-83890, commit 6358576
Resolves: ZSTAC-84992

Change-Id: Ic0978d214bf3996ae6fd0c112f125067ce70bed3
@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your recent review volume is higher than typical usage, so adaptive limits are currently applied.

Next review available in: 26 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 099c954a-94d9-4b12-9891-d6c85c308852

📥 Commits

Reviewing files that changed from the base of the PR and between af7a911 and eee95cb.

📒 Files selected for processing (5)
  • compute/src/main/java/org/zstack/compute/vm/VmInstanceBase.java
  • compute/src/main/java/org/zstack/compute/vm/VmStartOnHypervisorFlow.java
  • compute/src/main/java/org/zstack/compute/vm/VmSystemTags.java
  • header/src/main/java/org/zstack/header/vm/HaStartVmInstanceMsg.java
  • header/src/main/java/org/zstack/header/vm/VmBeforeStartOnHypervisorExtensionPoint.java

Warning

.coderabbit.yaml has a parsing error

The CodeRabbit configuration file in this repository has a parsing error and default settings were used instead. Please fix the error(s) in the configuration file. You can initialize chat with CodeRabbit to get help with the configuration file.

💥 Parsing errors (1)
Could not fetch remote config from http://open.zstack.ai:20001/code-reviews/zstack-cloud.yaml: TimeoutError: The operation was aborted due to timeout
⚙️ Configuration instructions
  • Please see the configuration documentation for more information.
  • You can also validate your configuration using the online YAML validator.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch sync/yingzhe.hu/cp/5.4.10/ZSTAC-84992-20260707@@3

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants