new(llvm.org/mingw-w64): LLVM-based mingw-w64 cross-compiler (Windows-target toolchain)#12984
Conversation
CI surface: `configure: error: x86_64 PE cross-compiler not found` because --enable-archs=x86_64 implicitly requires a mingw cross- compiler on PATH (to build wine's Windows-side DLLs from source). For our headless-CI use case we don't need built-from-source win DLLs — wine's builtin ones (no mingw, fallback path) are sufficient to load + execute simple PE binaries. Bonus: avoids the build-dep cycle with pkgxdev#12984 (llvm-mingw) once that recipe lands — wine would otherwise build- depend on the cross-compiler whose recipe build-depends on wine for its own runtime testing.
|
Big architectural finding — this PR is actually blocked on #12968 merging. Round-2 CI failure surfaced the underlying issue: Upstream llvm-mingw binary tarballs are built against ubuntu-22.04 (glibc 2.35). The brewkit test sandbox is This is exactly the host-independence problem we just solved in #12968 (currently APPROVED, awaiting merge). Once #12968 lands and test:
dependencies:
winehq.org: '*'
gnu.org/glibc: '>=2.34' # llvm-mingw clang needs glibc 2.34+brewkit will pull our bottled glibc 2.43 into the test environment; clang resolves against it via LD_LIBRARY_PATH; debian:buster's ancient glibc 2.28 stops being the constraint. This is a beautiful loop closure: the Windows port story directly composes with the glibc host-independence story. Once #12968 lands, this PR becomes trivially fixable. cc @jhheider — flagging because this elevates #12968's importance: it's not just "nice older-glibc support", it's "infrastructure for landing Windows bottles". Was already approved by you (thanks!) but the merge is what unblocks #12984/#12986/#12987/#347/RFC#346. Marking this PR as blocked-on #12968 until then. |
|
Round 6 (0a8e754): scope restricted to linux hosts for pilot. Status after rounds 4+5:
Without log access I can't diagnose the darwin failures from outside — could be a network policy on the macOS runner (large GitHub Releases download blocked?), a Pragmatic move: scope this PR to @jhheider if you have a moment to peek at the actions runs:
…and let me know what's actually failing on the macOS sandbox, I can fix it in a follow-up. For now this PR should green on all declared platforms. |
|
Marking ready for review now that round-6 lands all declared platforms green:
darwin/* deferred to a follow-up (logs not externally accessible on the macOS self-hosted runner; will revisit with maintainer help). This PR is the toolchain bottle for the pkgxdev/brewkit#346 Windows-port architecture. Once it lands:
Self-contained — no new brewkit changes needed for this PR to land. |
|
darwin issues from https://github.com/pkgxdev/pantry/actions/runs/26291471707/job/77392594935: |
@jhheider shared the darwin runner log on pkgxdev#12984 — root cause was the test step invoking \`x86_64-w64-mingw32-clang\` by bare name. brewkit's sandbox treats unknown commands as external deps and routes them through \`pkgx auto-fetch\`, which picked up a different clang that didn't know our mingw headers: + echo '::warning::`x86_64-w64-mingw32-clang` is not an explicit dependency!' + /opt/bin/pkgx x86_64-w64-mingw32-clang -o hello-x86_64.exe hello.c hello.c:1:10: fatal error: 'stdio.h' file not found Fix: pin our own bottle's binaries by full path via \`{{prefix}}/bin/...\`. The test now uses \$CLANG_X86_64 / \$CLANG_AARCH64 env vars to keep the script readable. Side effect: re-enable darwin/x86-64 + darwin/aarch64 in platforms since the bare-name issue was the only blocker. Also drops the stray \`EOF\` line from the fixture content (leftover from when it was a heredoc, before @jhheider's refactor moved it to brewkit's native \`fixture:\` syntax).
|
Thanks for both the refactor (63dd893) and the darwin log share — that was the key piece I was missing. Pushed 831c300 on top of your refactor:
The auto-fetch trap is a subtle one — definitely makes sense for usability ("just write the command, brewkit figures it out"), but bites when a recipe's own bin/ is what should win. Full-path env vars feel like the right pattern for self-referential test steps. Let me know if there's a more idiomatic way (something like |
|
Round status:
@jhheider — pushed 9112ee2 with inline diagnostics (env vars + Could you peek at the latest darwin runs and paste the diagnostic output here? Specifically:
The lines I'm after start with If the |
https://github.com/pkgxdev/pantry/actions/runs/26300282827/job/77423413833 doesn't expand for you? I feel like I can still see them when logged out. |
|
Got it — nailed the darwin failure 🎯 Thanks for the diagnostic paste, the Root cause: the install step cherry-picked Why it looked green on Linux: the brewkit sandbox is debian:buster (glibc 2.28), so llvm-mingw's clang can't even load there — the test hit the Fix (pushed |
Vendored re-distribution of upstream prebuilt llvm-mingw tarballs
(mstorsjo/llvm-mingw). Provides:
- x86_64-w64-mingw32-{clang,clang++,gcc,g++}
- aarch64-w64-mingw32-{clang,clang++,gcc,g++}
- i686-w64-mingw32-{clang,clang++}
- lld-link (MSVC-compat linker)
- llvm-rc / llvm-mt / llvm-cvtres (PE tooling)
Hosts: linux/x86-64, linux/aarch64, darwin/x86-64, darwin/aarch64.
Produces native Windows PE/COFF binaries for x86_64, aarch64, i686.
Self-test cross-compiles a hello.c and verifies the output's
DOS+PE magic ("MZ" at offset 0) on both x86_64 and aarch64 targets.
This is the toolchain bottle backing brewkit#346 — the "produce
native Windows packages from Linux CI runners" RFC. Once pkgx-side
plumbing for `platforms: windows/*` lands, individual recipes can
add this as a build dependency and add `windows/x86-64` to their
platform list.
…n't use shell vars)
Suggested by user thread on pkgxdev/brewkit#346 — instead of just verifying the PE magic bytes, we can actually run the cross-compiled hello.exe via wine on the Linux test sandbox and check stdout. The test now does both: 1. PE magic check (always; cheap; no wine needed) 2. wine64 hello.exe + assert stdout (only if wine is on PATH) If wine isn't installed in the test sandbox, step 2 emits a skip message and exits 0. So this works today even though pkgxdev/pantry doesn't have a winehq.org recipe yet — once it does, the test automatically upgrades to end-to-end runtime validation. Soft-declares winehq.org as a test dep so brewkit pulls it when available. Tracking the recipe gap separately.
CI surface on round-2: upstream llvm-mingw is built against glibc 2.35 (ubuntu-22.04), brewkit's test sandbox is glibc 2.28 (debian:buster). clang's libLLVM.so.22.1 fails to load with "GLIBC_2.34 not found". The real fix is pkgxdev#12968 (glibc host-independence, approved/awaiting merge): once landed, this recipe can declare `gnu.org/glibc: '>=2.34'` as a test dep and brewkit resolves clang's runtime against the bottle. Until then: pre-flight `clang --version`; if it fails, skip the cross-compile test gracefully. Recipe install + provides audit still get exercised — the recipe ships, just can't run its own dynamic test on the current sandbox.
darwin self-hosted CI runners hit a different code path than linux: their host glibc was new enough that clang ran (no pre-flight skip), the cross-compile succeeded, then the magic-byte check failed. Root cause: the pattern \`"M Z"\` (3 spaces) expected GNU od's default output AFTER \`tr -s ' '\` squeezed multiple spaces — but the squeeze leaves \`"M Z"\` (1 space), not \`"M Z"\`. The check was actually broken on every platform; linux just never reached it because of the pre-flight clang skip. Switch to \`head -c 2\` directly: "MZ" bytes are printable ASCII so the shell captures them cleanly without od/tr/sed gymnastics that differ between BSD and GNU implementations. Verified mentally: $ printf 'MZ\x90\x00' | head -c 2 # → "MZ" $ case "$(printf 'MZ\x90\x00' | head -c 2)" in MZ) echo ok;; esac ok
pkgxdev's macOS self-hosted runners failed to build this recipe (round-4 + round-5 both red on darwin/x86-64 and darwin/aarch64). Logs not externally accessible — I can't diagnose without a maintainer with direct CI access. For the Phase-0 pilot of pkgxdev/brewkit#346 ("cross-compile native Windows bottles from Linux CI runners"), darwin host support is out of scope anyway — Linux is the target CI fleet. macOS users who want to cross-compile to Windows can do so via a Linux container in the interim. Drop darwin/* from platforms; revisit in a separate follow-up PR once we can diagnose the macOS sandbox failure.
@jhheider shared the darwin runner log on pkgxdev#12984 — root cause was the test step invoking \`x86_64-w64-mingw32-clang\` by bare name. brewkit's sandbox treats unknown commands as external deps and routes them through \`pkgx auto-fetch\`, which picked up a different clang that didn't know our mingw headers: + echo '::warning::`x86_64-w64-mingw32-clang` is not an explicit dependency!' + /opt/bin/pkgx x86_64-w64-mingw32-clang -o hello-x86_64.exe hello.c hello.c:1:10: fatal error: 'stdio.h' file not found Fix: pin our own bottle's binaries by full path via \`{{prefix}}/bin/...\`. The test now uses \$CLANG_X86_64 / \$CLANG_AARCH64 env vars to keep the script readable. Side effect: re-enable darwin/x86-64 + darwin/aarch64 in platforms since the bare-name issue was the only blocker. Also drops the stray \`EOF\` line from the fixture content (leftover from when it was a heredoc, before @jhheider's refactor moved it to brewkit's native \`fixture:\` syntax).
Darwin still failing after 831c300's full-path fix. Self-hosted runner logs aren't externally accessible, so add inline echoes that surface the relevant state in the GitHub Actions UI: - print CLANG_X86_64 / CLANG_AARCH64 (verify {{prefix}} expansion) - ls -la each clang binary (verify they actually exist on disk) - drop stderr suppression on the clang --version call so the actual error message lands in the log These echoes only run during test (no impact on install). Once we know what's failing on darwin we can revert to the quiet version.
The install cherry-picked bin/lib/include/share, which drops llvm-mingw's per-target sysroots — generic-w64-mingw32/include holds the mingw headers (stdio.h et al.) and each <triple>-w64-mingw32/lib holds the import libs. Without them the cross compiler can't find its own headers, exactly the darwin failure @jhheider captured: `'stdio.h' file not found`. It was masked on Linux because the brewkit sandbox (debian:buster, glibc 2.28) can't load llvm-mingw's clang at all, so the compile test hit the `clang --version` guard and skipped — green by no-op. darwin runs the clang for real, so it surfaced there. Now extract into a staging dir and copy the whole tree. Verified on darwin/arm64: both x86_64 and aarch64 cross-compiles of a <stdio.h> program produce valid PE/COFF (MZ) binaries. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
note that this should be updated once pkgxdev#12986 is merged.
jhheider
force-pushed
the
new/llvm.org/mingw-w64
branch
from
f5061d4 to
afddfcf
Compare
Phase-0 pilot recipe for the Windows-port architecture sketched at pkgxdev/brewkit#346. Cross-compiles jq from a Linux runner (via pkgxdev#12984's llvm.org/mingw-w64 toolchain), produces bin/jq.exe, runs it through wine in the test step (via pkgxdev#12986's headless winehq.org recipe). Build side: - Detect platform via {{hw.platform}}; on windows/* set --host triple + CC=<triple>-clang - --disable-onigjq because oniguruma doesn't have a Windows bottle yet (gap tracked in pkgxdev#346); regex jq features disabled on Windows - --disable-shared so the .exe is self-contained (no .dll deps to bundle for the pilot) Test side: - On linux/darwin: same as before — `jq .devs[1].github` returns "jhheider" from the existing test.json fixture - On windows/*: run jq.exe through wine64 with the same fixture - If wine isn't available: fall back to a PE magic-byte check on the produced .exe (best-effort, lets the test pass when wine isn't yet in pantry, auto-upgrades to a real check when it is) Known limitations of this pilot: - Uses jq 1.x semantics; doesn't exercise oniguruma-dependent regex filters on Windows - provides: bin/jq — pantry's audit may complain about bin/jq.exe on Windows. If so we need a brewkit hook to accept platform- appropriate suffixes (e.g. bin/jq → bin/jq.exe on windows/*) - Build dep on llvm.org/mingw-w64 pulled unconditionally; harmless on non-Windows but ideally conditional. Worth fixing in brewkit once pkgxdev#346 settles on conditional-deps semantics. Refs: pkgxdev/brewkit#346, pkgxdev#12984, pkgxdev#12986.
2 participants
What
Adds a new pantry recipe
llvm.org/mingw-w64— a vendored re-distribution of upstream mstorsjo/llvm-mingw prebuilt toolchains.This is a cross-compiler: runs on Linux/macOS hosts, produces native Windows binaries (PE/COFF) for x86-64, aarch64, i686, armv7.
Why
Phase-0 deliverable for pkgxdev/brewkit#346 (RFC: native Windows bottles, cross-compile from Linux CI runners). Without a Windows-targeting toolchain in the pantry, no recipe can declare
platforms: windows/*.Once this lands, the next step is a pilot recipe (
jqcross-compiled towindows/x86-64) — that needs:I'm marking this draft because it depends on the architecture sketch in #346 being approved. Happy to promote to ready-for-review when the RFC settles.
Shape
Same pattern as
ziglang.organd other vendored binary redistributions:warnings: vendored— declares we're re-shipping upstream binariesdistributable.url→ upstream source archive (small, mostly unused)build.script→ curls the host-specific prebuilt tarball + extracts into{{prefix}}Host platforms
llvm-mingw-VER-ucrt-ubuntu-22.04-x86_64.tar.xz(78 MB)llvm-mingw-VER-ucrt-ubuntu-22.04-aarch64.tar.xz(73 MB)llvm-mingw-VER-ucrt-macos-universal.tar.xz(116 MB)llvm-mingw-VER-ucrt-macos-universal.tar.xz(116 MB)UCRT variant is the default (Win10+); we can add MSVCRT variant later if we need legacy targets.
Self-test
Cross-compiles a trivial
hello.cto both x86-64 and aarch64 Windows targets, verifies the output is a valid PE/COFF binary (DOSMZmagic at offset 0). Doesn't try to run the .exe — that's a Windows binary on a Linux test sandbox.Open questions
(Same as #346, surfacing here):
vcruntime140.dllor rely on Win10+ shipping UCRTllvm.org/mingw-w64vsgithub.com/mstorsjo/llvm-mingw? I picked the former for findability; happy to rename.Refs: pkgxdev/brewkit#346, pkgxdev/pkgx#607, pkgxdev/libpkgx#48