build(python): publish manylinux_2_28 wheels for broader glibc compat…#1117
Open
phamgialinhlx wants to merge 1 commit intoNVIDIA:mainfrom
Open
build(python): publish manylinux_2_28 wheels for broader glibc compat…#1117phamgialinhlx wants to merge 1 commit intoNVIDIA:mainfrom
phamgialinhlx wants to merge 1 commit intoNVIDIA:mainfrom
Conversation
…ibility
Build released Linux wheels inside a PyPA manylinux_2_28 container so the
resulting binary runs on any Linux distribution with glibc >= 2.28
(RHEL 8, Debian 10+, Ubuntu 18.04+). The previous native build on the
noble (glibc 2.39) CI image produced manylinux_2_39 wheels that uv
refused to install on common LTS hosts such as Ubuntu 22.04.
- Add deploy/docker/Dockerfile.python-wheels-linux based on
quay.io/pypa/manylinux_2_28_{x86_64,aarch64}.
- Add build:python:wheel:linux:docker mise task and per-arch aliases.
Re-point python:build:linux:{amd64,arm64} (used by release workflows)
to the Docker-based path. The legacy native build:python:wheel:linux
task remains for fast local iteration.
- Mount the host docker socket and set up buildx in the
build-python-wheels-linux job in release-tag.yml and release-dev.yml,
mirroring the existing macOS wheel job.
- Update architecture/build-containers.md to describe the new flow.
Signed-off-by: phamgialinhlx <phamgialinhlx2@gmail.com>
|
All contributors have signed the DCO ✍️ ✅ |
Author
|
I have read the DCO document and I hereby sign the DCO. |
Author
|
recheck |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Build released Linux wheels inside a PyPA
manylinux_2_28container so the binary runs on any Linux distribution shipping glibc >= 2.28 (RHEL 8, Debian 10+, Ubuntu 18.04+). The previous native build on the noble (glibc 2.39) CI image producedmanylinux_2_39_x86_64wheels thatuvandpiprefuse to install on common LTS hosts such as Ubuntu 22.04 / Debian 11.Related Issue
Reported via the install path. On Ubuntu 22.04 (glibc 2.35):
Changes
deploy/docker/Dockerfile.python-wheels-linux— builds the wheel insidequay.io/pypa/manylinux_2_28_{x86_64,aarch64}. Mirrors the layer layout (manifest copy → dummy-source dep build → real source rebuild) used byDockerfile.python-wheels-macosso cargo cache mounts behave the same way. Maturin is invoked with--compatibility manylinux_2_28so the resulting wheel is tagged accordingly.tasks/python.toml:build:python:wheel:linux:docker— driver task; takesTARGETARCHandWHEEL_OUTPUT_DIR.build:python:wheel:linux:{amd64,arm64}:docker— per-arch wrappers.python:build:linux:{amd64,arm64}are re-pointed at the Docker path.build:python:wheel:linux:{amd64,arm64}tasks remain for fast local iteration (still produce a wheel tagged for the host's glibc, which is fine for dev installs)..github/workflows/release-tag.ymlandrelease-dev.yml, in thebuild-python-wheels-linuxjob:/var/run/docker.sockso the in-containermise runstep can reach the host docker daemon to launch the manylinux build.setup-buildxstep so the Dockerfile's BuildKit cache mounts (--mount=type=cache,target=/root/.cargo/registryetc.) work.build-python-wheel-macosjob, which already uses this exact pattern for its osxcross Docker build.architecture/build-containers.mdupdated to describe the new portable wheel build path and call out that the native task is now a local-iteration convenience, not the release path.The pre-existing dead
deploy/docker/Dockerfile.python-wheels(no references in any task or workflow) is left untouched to keep this PR additive — happy to remove it in a follow-up if maintainers want.Alternatives considered
maturin build --zig --compatibility manylinux_2_28(cargo-zigbuild) was the smaller-diff option but carries real risk:bundled-z3builds Z3 from C++ source via cmake, and Zig as the C++ toolchain on a heavy C++ codebase is unverified for this repo. Switching to the manylinux container keeps Z3 building under gcc-toolset-14, which is what the Z3 project itself tests against.Dockerfile.cito an older-glibc distro would let the native build produce portable wheels, but it would simultaneously affect every other job that uses the CI image. Out of scope.Testing
End-to-end validation on Ubuntu 22.04 / glibc 2.35 / Python 3.14:
docker buildx build -f deploy/docker/Dockerfile.python-wheels-linux --target wheels --build-arg TARGETARCH=amd64 …openshell-0.0.0-py3-none-manylinux_2_28_x86_64.whl(18.8 MB)uv add <wheel>into a Python 3.14 project that previously failed with the platform-tag error aboveimport openshell; openshell.SandboxClientThe build was iterated once during testing: the first attempt failed at
dnf install perl-FindBinbecause that's not a standalone package on AlmaLinux 8 (manylinux_2_28's base). Replaced withperl-core, which provides FindBin and is the canonical "full perl" meta-package on the EL family. The Z3bundled-z3build (the highest-risk piece pre-test) worked cleanly under gcc-toolset-14 inside the manylinux container.A workflow note: the docker socket mount means the linux wheel job, like the macOS wheel job, requires
/var/run/docker.sockto be present on the self-hostedbuild-amd64/build-arm64runners. That matches thecontract the macOS job already relies on.
mise run pre-commitpassesthe new task end-to-end on the next dev/tag run
Checklist
build(python): publish manylinux_2_28 wheels for broader glibc compatibilityarchitecture/build-containers.md)