Make cache:prepare self-contained in fresh worktrees#4292
Open
Make cache:prepare self-contained in fresh worktrees#4292
Conversation
cache:prepare now works in a completely fresh worktree with no running
services. Previously it would fail if host dist, boxel-ui dist,
boxel-icons dist, or postgres weren't already available.
Changes:
- Auto-build host dist: when no pre-built host dist exists in the
worktree or root repo, build it automatically instead of erroring.
In worktrees, symlinks from the root repo's built dist when available.
- Auto-provision boxel-ui and boxel-icons dist: symlink from root repo
in worktrees (fast path), or build from source as fallback.
- Managed icon server lifecycle: instead of detecting an external icon
server and hoping it stays alive, always start our own managed
process. Falls back to the existing dev server if port 4206 is
already taken.
- Indexing progress bar: polls the boxel_index table every 2s during
template builds and renders an in-place progress bar on stderr:
indexing [=============== ] 199/373 files (53%) 80s
indexing [==============================] 603/373 files (100%) 212s
indexing complete: 603 files indexed in 229.5s
- Graceful pg unavailability: databaseExists() now returns false
instead of throwing when postgres isn't running yet, allowing
startFactorySupportServices() to start it.
- Stale context validation: cache-realm.ts now validates both hostURL
and matrixURL in cached support.json, discarding stale contexts
where either service is unreachable.
Tested: cache:prepare succeeds in a fresh worktree with zero services
running, and also succeeds alongside a running dev-all without
disrupting it.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Contributor
There was a problem hiding this comment.
Pull request overview
Updates the software-factory cache:prepare flow to be self-contained in fresh git worktrees by auto-provisioning required build artifacts, taking direct ownership of the icon server lifecycle, and adding indexing progress reporting during template builds.
Changes:
- Auto-build or (attempt to) symlink missing host/boxel-ui/boxel-icons dist artifacts to avoid manual setup in new worktrees.
- Always start a managed icon server process (with intended fallback to an already-running dev server) and ensure icon dist exists before serving.
- Add stderr progress bar for indexing by polling
boxel_index, plus makedatabaseExists()tolerant of Postgres being down and validate cached support context URLs.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| packages/software-factory/src/harness/support-services.ts | Adds dist auto-provisioning and managed icon server startup/stop logic. |
| packages/software-factory/src/harness/database.ts | Makes DB existence checks resilient and adds indexing progress reporter + file counting. |
| packages/software-factory/src/cli/cache-realm.ts | Validates cached support.json context by probing hostURL and matrixURL. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Replace spawnSync('ln', ...) with Node fs.symlinkSync for
cross-platform symlink creation in ensureBoxelUIDist and
ensureBoxelIconsDist
- Re-validate boxel-icons dist after build to catch partial output
- Add child.on('error', ...) handler to icon server process to catch
spawn failures
- Use fallback child.kill() when process group kill fails in icon
server stop
- Keep polling probe URL when icon server exits with EADDRINUSE
instead of throwing immediately (allows external server startup time)
- Guard against overlapping DB polls in progress reporter with
in-flight flag
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2cc315a to
7a91f1c
Compare
Pre-allocate the worker-manager port via findAvailablePort() and pass
it to both startIsolatedRealmStack and the progress reporter. This
lets the reporter poll the /_indexing-status JSON endpoint which has
the exact invalidation graph computed by the index runner.
Progress output shows the current realm, file counts, and queued realms:
indexing: waiting for status 3s
indexing base: discovering files... 8s (2 realms remaining)
indexing base [=====> ] 50/153 files (33%) 30s (2 realms remaining)
indexing software-factory [========> ] 22/204 files (11%) 94s
indexing test [==============================] 13/13 files (100%) 210s
indexing complete: 13/13 files in 229.5s
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
7a91f1c to
ff78820
Compare
Each harness instance now gets a fresh port via findAvailablePort() instead of falling back to the static DEFAULT_WORKER_MANAGER_PORT env var, which caused EADDRINUSE collisions in parallel test runs. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each harness instance now gets a fresh port via findAvailablePort() instead of using a pre-allocated port from the test fixture, which caused EADDRINUSE collisions due to TOCTOU races. - Remove DEFAULT_WORKER_MANAGER_PORT constant and SOFTWARE_FACTORY_WORKER_MANAGER_PORT env var entirely - Remove workerManagerPort from TestWorkerPortSet (no longer pre-allocated per worker) - startIsolatedRealmStack always uses findAvailablePort() when no explicit port is provided - cache:prepare path unchanged: still pre-allocates port for progress monitoring via /_indexing-status Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
020902f to
229d335
Compare
This was referenced Apr 1, 2026
The comment and sizing rationale now accurately reflects that only compat proxy, realm-server, and prerender ports are pre-allocated. The worker-manager port is dynamically assigned per harness instance. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…uild - startIconServerProcess: track spawn errors via spawnFailed() flag so ensureIconsReady fails immediately instead of polling for 30s - ensureBoxelUIDist/ensureBoxelIconsDist: remove leftover symlinks before falling back to pnpm build, preventing writes through the symlink into the root repo's dist directory Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
… repo Check icons.js and styles/global.css in addition to components.js and helpers.js when deciding whether the root repo's boxel-ui dist is complete enough to symlink. Prevents symlinking a partially built dist that would fail the usability check and fall through to the build fallback. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…tions The report() callback runs on a setInterval — any thrown error would surface as an unhandled promise rejection. Add a catch block so progress reporting failures are silently swallowed instead of crashing the process. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
lukemelia
approved these changes
Apr 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Resolves CS-10560:
cache:preparenow works in a completely fresh worktree with zero running services.Auto-provision build artifacts: Host dist, boxel-ui dist, and boxel-icons dist are automatically symlinked from the root repo (fast) or built from source (fallback) when missing in a worktree. Uses
fs.symlinkSyncfor cross-platform compatibility.Managed icon server lifecycle: Always starts our own icon server process instead of hoping an externally-detected server stays alive. Falls back to the existing dev server if port 4206 is already taken (with EADDRINUSE retry for slow-starting external servers).
Indexing progress bar: Pre-allocates the worker-manager port and polls its
/_indexing-statusJSON endpoint every 2s. Shows an in-place single-line progress bar that updates as each realm is indexed. File counts come from the invalidation graph so they're precise. The reporter tracks seen realms to show how many remain:Graceful pg unavailability:
databaseExists()returns false instead of throwing when postgres isn't running yet.Stale context validation: Validates both hostURL and matrixURL in cached support.json, discarding stale contexts.
Test plan
cache:preparesucceeds in a fresh worktree with no services running (pg, synapse, host, icons all auto-started)cache:preparesucceeds alongside a runningmise run dev-allwithout disrupting dev serviceslocalhost:4200remains fully functional during and aftercache:prepare/_indexing-statusendpoint directly: base 0→150/153, software-factory 4→198/204, test 0→10/12)totalFilesfrom the invalidation graph is precise🤖 Generated with Claude Code