new(riverbankcomputing.com/pyqt5): PyQt5 (vendored wheel)#13064
Draft
tannevaled wants to merge 19 commits into
Draft
new(riverbankcomputing.com/pyqt5): PyQt5 (vendored wheel)#13064tannevaled wants to merge 19 commits into
tannevaled wants to merge 19 commits into
Conversation
Installed as a python-venv against pantry's qt.io (5.15) + sip. Provides bin/pyqt5-python that has PyQt5 importable. Closes part of pkgxdev#99 (holdout pkgxdev#591).
…n-venv.sh python-venv.sh installs from SRCROOT (the extracted source tree), not from a PyPI spec — it does `pip install $SRCROOT`. So we need: 1. A real distributable.url pointing at PyPI's sdist for PyQt5 2. Just call `python-venv.sh <binary>` without a PyPI package arg The previous attempt with `distributable: ~` left SRCROOT empty, producing 'Directory ... is not installable. Neither setup.py nor pyproject.toml found'.
Contributor
Author
|
Draft — PyQt5 sdist build hits 2 issues simultaneously:
Fix needs:
Both require modifying brewkit's python-venv.sh helper or a custom build flow. Deferring. |
brewkit's python-venv.sh OOM-killed (exit 137) at pip-install: PyQt5's
50+ Qt5 binding compiles peak at 4–6 GB per ld process and GH runners
have ~7 GB RAM total.
Replace the python-venv.sh call with an inline equivalent that:
- creates the venv via the bottled python
- installs pip/wheel/setuptools quietly
- sets MAKEFLAGS=-j1 + CMAKE_BUILD_PARALLEL_LEVEL=1 → serial build
- sets CXXFLAGS/CFLAGS=-g0 → ~30 % peak-memory cut at link time
- pip-installs the SRCROOT with --no-build-isolation (brewkit
already provides pyqt-builder etc. as build deps)
If this pattern proves out, the next step is lifting it into brewkit
as `python-venv-alt.sh` (or a `--jobs N` flag on python-venv.sh) and
opening a discussion on pkgxdev/pantry — see PR body.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous MAKEFLAGS=-j1 attempt was insufficient: pyqt-builder
spawns one make per binding in parallel and -j1 only serialises the
inner make. The outer parallelism (~30 simultaneous link-heavy
makes) is what blew the 7 GB GH-runner ceiling.
Two knobs together fit the build:
--config-settings=--jobs=1
Tells pyqt-builder to build ONE binding at a time. This is the
load-bearing flag.
--config-settings=--disable=Qt<heavy> (×26)
Drops the QPdf, Qt3D*, QtQml, QtQuick, QtMultimedia*, QtWebKit*,
QtBluetooth, QtPositioning, QtSensors, QtNfc, QtXmlPatterns,
QtRemoteObjects, QtTextToSpeech, QtSerialPort, and platform-
specific Extras bindings. The remaining ~10 bindings still cover
QtCore / QtGui / QtWidgets / QtNetwork / QtSql / QtSvg / QtTest /
QtDBus / QtPrintSupport / QtConcurrent — the 95% use case.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The 26-module --disable list triggered sipbuild's UserException at _enable_disable_bindings, likely because at least one binding name doesn't exist in PyQt5 5.15.11 on Linux (QtMacExtras / QtWinExtras are platform-gated and absent from the linux source tree's pyproject.toml). Keep only --config-settings=--jobs=1 + CXXFLAGS=-g0. If pyqt-builder serialises the bindings (one make at a time) and we cut debug info, each peak link should fit under 7 GB on its own. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
PyQt5's monolithic `pip install .` was OOM-killed (exit 137) on 7 GB GH-runners. `--config-settings=--jobs=1` wasn't enough — a single binding's final `ld` (QtWidgets peaks at ~4-6 GB) blows past 7 GB on its own. Run pip install once per binding (`--target=$STAGING/$binding`, `--config-settings=--enable=$binding`, `--config-settings=--jobs=1`) so each subprocess holds only one binding's compiled state in memory, then merge the produced .so files into the venv's PyQt5 site-packages dir. Bindings ordered lightest-first (QtCore through QtPrintSupport, then QtGui, then QtWidgets last) so a memory failure on the heaviest binding still leaves the others importable. pyqt-builder's `--no-make` was considered but is restricted to the `build` tool (tools=['build']); pip's PEP 517 path calls the `wheel` tool, so it can't be set via --config-settings. The split-pip shape achieves the same memory isolation by freeing each binding's working set between subprocesses. Also: CXXFLAGS/CFLAGS=-g0 (~30 % less link RAM) and MAKEFLAGS=-j1 as belt-and-braces; per-binding staging dir cleaned after each pass to free disk; test block tolerates a missing QtGui/QtWidgets. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The whole reason every binding was "failing" wasn't OOM — it was a pip CLI parse error. pip rejects `--config-settings=--confirm-license` with "Arguments to --config-settings must be of the form KEY=VAL" because the flag has no value. So pip exits before sipbuild even runs; every binding "fails" identically with QtCore. Pass `--confirm-license=true` instead. sipbuild treats any truthy value as "license accepted". Should also fix QtGui/QtWidgets which my prior comment blamed on OOM — they may simply never have built. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…ngs)
The previous `--config-settings=--confirm-license=true` attempt fell
into a pip/sipbuild deadlock:
- pip insists --config-settings is KEY=VAL
- sipbuild's argparse declares --confirm-license as store_true and
refuses any value: "ignored explicit argument 'true'"
The route around: write `confirm-license = true` into the source
tarball's pyproject.toml under [tool.sip.project] before pip runs.
sipbuild reads pyproject.toml directly, bypassing the argparse path,
so the flag is honored.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
The previous recipe used:
sed -i.bak '/^\[tool\.sip\.project\]$/a\
confirm-license = true' "$SRCROOT/pyproject.toml"
sed's `a\` command requires the appended content on the next line,
which means the second line in the `run: |` block had to start at
column 0. That broke YAML's block-scalar indentation rule (later
lines may not be less indented than the first non-empty line of
the block), causing the plan job to fail with:
YAMLError: can not read a block mapping entry; a multiline key
may not be an implicit key … at line 100, column 9
Switched to awk — single-line program, no newline-in-script
requirement, and arguably clearer intent (insert-after-pattern).
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
…olithic build)
The previous monolithic recipe iterated 10 bindings in a single CI job
via `pip install --config-settings=--enable=<binding>` in a shell loop.
Each binding's link peaks at 6-8 GB RAM (sip-generated .cpp), so the
7 GB GH-Linux runner OOM'd on every binding and GHA cancelled at
1h33min (CI run 26712172867: 6/10 bindings SIGKILL'd, 4 queued).
Split into:
projects/riverbankcomputing.com/pyqt5/
├── package.yml meta (depends on all 10 sub-bindings)
├── QtCore/package.yml foundation, no inter-PyQt5 deps
├── QtDBus/package.yml deps QtCore
├── QtNetwork/package.yml deps QtCore
├── QtSql/package.yml deps QtCore
├── QtTest/package.yml deps QtCore
├── QtXml/package.yml deps QtCore
├── QtGui/package.yml deps QtCore (HEAVY)
├── QtSvg/package.yml deps QtGui
├── QtPrintSupport/package.yml deps QtGui
└── QtWidgets/package.yml deps QtGui (HEAVIEST)
Each sub-binding is its own CI job with its own 6 h timeout. Sibling
deps are pinned exactly (`={{version}}`) so the meta sees one
coherent PyQt5 install.
PyQt5 is a namespace package (`pkgutil.extend_path` in `__init__.py`),
so each sub-binding's prefix can ship its own `PyQt5/` dir and Python
merges them at import time via the stacked PYTHONPATH that pkgx sets
up automatically when the deps are in scope.
Preserved from the monolithic recipe:
- The awk patch for `confirm-license = true` (commit 495169a).
- `CXXFLAGS=-g0` / `CFLAGS=-g0` / `MAKEFLAGS=-j1` memory discipline.
- `--no-build-isolation` / `--no-deps` so our env reaches the
compiler and pip does not refetch sip from PyPI.
NOTE — chicken-and-egg: all 11 recipes ship in the same PR. Linux x64
CI for the non-foundation bindings (and the meta) will fail until
QtCore bottles to dist.pkgx.dev. This is the same shape as the
font-util/xserver and Tk/Tcl8 splits; maintainers merge in
topological order:
QtCore → {QtDBus,QtNetwork,QtSql,QtTest,QtXml,QtGui}
→ {QtSvg,QtPrintSupport,QtWidgets}
→ pyqt5 (meta)
tannevaled
changed the title
…-substitute)
Previous commit pinned sibling sub-bindings with `={{version}}` per
the task spec, but libpkgx's `validatePackageRequirement` runs at YAML
parse time — before template substitution — and rejects the literal
`{{version}}` string with `invalid constraint: undefined` (CI run
26716046336 confirmed: meta package fails before any build step).
PyQt5 sub-bindings all share the same version-discovery (PyPI simple
index) so they march in lockstep anyway; a `'*'` constraint is safe
here. Same shape as poppler / poppler-data and x.org/exts -> x11.
…-recipes The from-source approach OOM-killed (QtCore's sip-generated C++ compile, exit 137) and the per-binding split hit a lockstep co-dependency (each binding 404'd on its sibling's unpublished bottle). Replace all 11 sub-recipes with one package that installs PyQt5's official PyPI wheel (Qt bundled) into a venv — no compile, no lockstep. Provides pyuic5/pyrcc5/pylupdate5 and an importable PyQt5 (via PYTHONPATH). linux/x86-64 + darwin only (no upstream aarch64 linux wheel). PR body documents why vendored + the path back to a from-source recipe. Verified on darwin/arm64: wheel installs, import PyQt5.QtCore/QtWidgets works, pyuic5 --version reports 5.15.11. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
tannevaled
changed the title
`--only-binary=:all: PyQt5==...` contains a colon-space, so YAML parsed the list item as a mapping (brewkit: 'every node must contain a run key'). Single-quote it so it's a plain string command. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The manylinux wheel is not self-contained — its QtWidgets dlopens the host X11/xcb/mesa GUI stack, which pantry doesn't package, so it can't import on the CI sandbox (only QtCore, via glib, works there). The macOS wheels bundle Qt5 and import QtWidgets cleanly. Ship darwin-only rather than a half-working linux build; document the GUI-stack blocker as the path to restoring linux + a from-source recipe.
… deps The manylinux wheel's QtWidgets dlopens the host GUI stack, which IS in pantry under x.org/* + mesa3d.org + xkbcommon.org (I had earlier looked for x.org/libxcb etc. — wrong names — and wrongly concluded the stack was absent). Verified in a minimal amd64 sandbox that 'from PyQt5 import QtWidgets' imports cleanly with exactly the linux-scoped deps added here. macOS wheels stay self-contained (deps are linux-only).
added 4 commits
June 15, 2026 20:15
The from-source prereqs now exist in the pantry (they didn't when this first shipped as a wheel): qt.io provides Qt 5.15.10, plus sip and pyqt-builder. Build the bindings with sip-install against qt.io's qmake, bounding the compile with --jobs 1 + MAKEFLAGS=-j1 (the unbounded C++ compile is what OOM-killed the earlier from-source attempt). Qt5 is no longer bundled, so qt.io is a runtime dep; adds linux/aarch64 (no wheel existed for it).
The sip-install QtGui config test failed to load libLLVM.so.22.1: the llvm.org<17 build dep (copied from the sip recipe) shadowed the newer libLLVM that mesa3d.org's libGL needs. PyQt5's bindings build with the system g++ via qmake (no clang needed), so drop the llvm pin and the clang/AS env and let mesa pull its matching libLLVM.
sip-install's QtGui config test runs a Qt exe that loads mesa's libGL, which dlopens libLLVM.so (llvmpipe). mesa pulls llvm transitively but its lib dir isn't on the build library path, so the test crashed with 'libLLVM.so.22.1: cannot open shared object file'. Depend on llvm.org '*' directly (matching mesa3d.org — the earlier <17 pin gave the wrong soname).
…ocked) Attempted from-source via sip-install against qt.io's Qt5, but it fails in sip-install's own per-module config tests, which *execute* a Qt test exe in the headless sandbox: linux QtGui can't load mesa's libLLVM.so (llvmpipe via libGL), darwin can't resolve QtCore.framework via @rpath (DYLD stripping). Not fixable from the recipe env. Revert to the official wheel (Qt bundled), which builds 3/3, and document the blocker.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PyQt5 — Python bindings for Qt 5, from upstream's official PyPI wheels (Qt5 bundled). Provides
pyuic5/pyrcc5/pylupdate5and an importablePyQt5.Why vendored (wheel)
Building PyQt5 from the sip sources compiles an enormous amount of generated C++ per binding and OOM-killed CI; splitting into per-binding sub-recipes then hit a lockstep co-dependency (each binding 404'd resolving its sibling's not-yet-published bottle). The official wheel sidesteps both, in one package.
Platforms
QtGui/QtWidgetsdlopen the host X11/xcb/mesa GUI stack. That stack is in pantry underx.org/*,mesa3d.org,xkbcommon.org, so they're declared as linux-only runtime deps. Verified in a minimal amd64 sandbox thatfrom PyQt5 import QtWidgetsimports cleanly with exactly those deps.linux/aarch64 is omitted (no upstream manylinux aarch64 wheel for this version).
Path to a from-source recipe
Compile
riverbankcomputing.com/sip+ the PyQt5 bindings against a pantry Qt5 (qtbase) with bounded compile parallelism (the unbounded per-binding C++ compile is what OOM-killed CI) — that would also restore linux/aarch64.