feat(cmd/duckgres-worker): add a worker-only entry point that ships libduckdb#500
Merged
Conversation
…ibduckdb Counterpart to cmd/duckgres-controlplane (PR #498). This binary: - Links libduckdb (verified: go list -deps shows duckdb-go-bindings, duckdb-go/v2 in the import graph). - Imports duckdbservice but not controlplane — no PG wire surface, no config-store / k8s pool wiring. It's the runtime that the control plane spawns into worker pods over Unix sockets / TCP. - Is the target of the planned per-DuckDB-version matrix build in CI. A worker pod ships exactly one DuckDB version, pinned via go.mod + the Dockerfile DUCKDB_EXTENSION_VERSION build arg. Like cmd/duckgres-controlplane, this is a stub that errors out at runtime — duckgres' YAML / CLI / env config resolution still lives in the package main file at the repo root alongside the all-in-one binary. The follow-up PR lifts that into a shared package both new binaries can import. The point of this PR is locking in the binary's existence and the import-graph contract. Together with PR #498 (CP-only, no libduckdb) and PR #499 (CI guard), the binary split is now structurally complete: cmd/duckgres-controlplane no libduckdb (CP-only, all DuckDB versions run on remote workers) cmd/duckgres-worker links libduckdb (worker-only, pinned to one DuckDB version per build) duckgres all-in-one (links libduckdb, all modes) Verified: - go build ./cmd/duckgres-worker/... clean - go list -deps ./cmd/duckgres-worker | grep duckdb-go shows the expected DuckDB driver imports - go list -deps ./cmd/duckgres-controlplane | grep duckdb-go is still empty (the CP-only contract held) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced May 1, 2026
fuziontech
added a commit
that referenced
this pull request
May 5, 2026
… API (#512) Two pieces of the runbook went stale during the recent binary-split work: 1. The "canary a new build" instructions still recommended direct `UPDATE duckgres_managed_warehouses SET image=...` SQL against the config store. PR #506 replaced that with `PATCH /api/v1/orgs/:id/warehouse/pinning` (row-locked transaction, audit trail, no DB shell required). Updates every example to use the API endpoint and explicitly tells operators not to use SQL. 2. The image format was the all-in-one `posthog/duckgres:vX.Y.Z`. PRs #500-#503 split that into a per-DuckDB-version `duckgres-worker` matrix. Updates examples to the new `ghcr.io/posthog/duckgres-worker:<sha>-duckdb<version>` shape and adds a tag-conventions table covering both GHCR and the ECR mirror (provisioned in posthog-cloud-infra#7848). Adds two new sections: - "Available Worker Builds" — explains the matrix CD, shows operators how to list current ECR / GHCR tags, and explains why pinning a non-default DuckDB version requires a specific `<sha>-duckdb<version>` tag (no `latest-duckdb<version>` convention exists). - "Reverting a Pin" — documents the empty-string semantics for clearing back to the global default, with a caution that clearing `ducklake_version` does NOT downgrade an already-migrated tenant (one-way door from the existing CAUTION block). Also fixes the spec-version format in examples (it's `major.minor` only — "0.4" / "1.0" / "1.1" — matching the validator added in #506) and adds a "Background" section linking back to the binary-split PRs so future maintainers understand the two-image story. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
7 tasks
fuziontech
added a commit
that referenced
this pull request
May 5, 2026
The cmd/ binary stubs were created in #498/#500 but never had their config-resolution wiring finished, so they could be built and pushed to ECR but not actually run. This wires cmd/duckgres-worker to call duckdbservice.Run via configresolve.ResolveEffective, and lifts the shared boot scaffolding (logging, tracing, otel resource, build-info) into a new internal/cliboot package so the worker, control plane, and all-in-one binaries inherit identical PostHog log shipping, password redaction, and OTLP tracing. The worker exposes a ~14-flag subset (DuckDB execution, DuckLake, query log, duckdb-service transport) — TLS/ACME, PG wire, control-plane pool, K8s, and config-store flags intentionally do not exist on this binary. YAML and env still carry the full surface so a worker pod can boot from the same ConfigMap as the control plane; CP-only fields land in Resolved slots the worker simply doesn't read. Boot scaffolding lift moves logging.go (incl. RedactingHandler), tracing.go, otel_resource.go, and the BuildInfo helper out of package main and into internal/cliboot. Each binary keeps its own package-main var version/commit/date because -ldflags -X can only target package-main symbols. just test-unit picks up ./internal/... so the redaction tests run on every CI pass. Co-authored-by: james <james@jams-spark.lan> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Counterpart to
cmd/duckgres-controlplane(PR #498). This binary:go list -depsshows duckdb-go-bindings, duckdb-go/v2 in the import graph).duckdbservicebut notcontrolplane— no PG wire surface, no config-store / k8s pool wiring. It's the runtime that the control plane spawns into worker pods over Unix sockets / TCP.go.mod+ the DockerfileDUCKDB_EXTENSION_VERSIONbuild arg.Like
cmd/duckgres-controlplane, this is a stub that errors out at runtime — duckgres' YAML / CLI / env config resolution still lives in thepackage mainfile at the repo root alongside the all-in-one binary. The follow-up PR lifts that into a shared package both new binaries can import. The point of this PR is locking in the binary's existence and the import-graph contract.Where the binary split now stands
Together with PR #498 (CP-only, no libduckdb) and PR #499 (CI guard), the binary split is structurally complete:
cmd/duckgres-controlplanecmd/duckgres-workerduckgres(all-in-one)Test plan
go build ./cmd/duckgres-worker/...cleango list -deps ./cmd/duckgres-worker | grep duckdb-goshows the expected DuckDB driver importsgo list -deps ./cmd/duckgres-controlplane | grep duckdb-gois still empty (the CP-only contract held — also enforced by the CI guard from PR ci: guard cmd/duckgres-controlplane against duckdb-go regressions #499)🤖 Generated with Claude Code