Skip to content

ci: build CLI only when it changes, reuse prior release otherwise#36081

Open
wezell wants to merge 9 commits into
mainfrom
issue-36080-cli-build-on-change
Open

ci: build CLI only when it changes, reuse prior release otherwise#36081
wezell wants to merge 9 commits into
mainfrom
issue-36080-cli-build-on-change

Conversation

@wezell

@wezell wezell commented Jun 9, 2026

Copy link
Copy Markdown
Member

What

Performance Optimization: Build/release the dotCMS CLI only when it actually changes, and reuse the prior release otherwise.
Trying to make our automated tests and release actions as fast as possible.

Fixes #36080

Why

The CLI is rebuilt and re-released on every trunk push and every dotCMS release, even when tools/dotcms-cli/** is byte-identical to the prior release. The native build runs an expensive 3-OS GraalVM matrix (including two macOS runners), so we repeatedly pay that cost to ship identical binaries.

The CLI is self-contained under tools/dotcms-cli/** (its only com.dotcms dependency, dotcms-api-data-model, lives in that same subtree), so a path/diff change-key is correct.

Changes

Trunk — cicd_3-trunk.yml

  • build-cli now also requires fromJSON(needs.initialize.outputs.filters).cli == 'true'.
  • deployment passes deploy-cli gated on the same predicate, so it never tries to deploy artifacts that were not built.

Release — cicd_release-cli.yml

  • New detection in precheck: git diff <prev-release-tag>..<this-tag> -- tools/dotcms-cli/ sets CLI_CHANGED. Fail-safe: any uncertainty (manual dispatch, no previous tag) → CLI_CHANGED=true (full build), so a wrong "unchanged" verdict can never ship stale binaries.
  • build, build-cli (native), and release (JReleaser) are gated on CLI_CHANGED == 'true'.
  • Unchanged path republishes the prior release's binaries under the new version across all three channels JReleaser/the pipeline feed:
    • npm (publish-npm-package): repackage the published latest tarball under the new version.
    • GitHub release + Artifactory (new reuse-release job): download dotcms-cli-<prev> assets, re-stamp to the new version, create the dotcms-cli-<new> release, and upload to libs-release-local/com/dotcms/dotcms-cli/<new>/.
  • Removed the Slack announcement from the CLI release cycle.

Notes / caveats

  • The reuse-release job mirrors the channels in tools/dotcms-cli/jreleaser.yml in bash — keep the two in sync if platforms or the Artifactory path change (noted in-code).
  • On the reuse path, JReleaser-generated checksums/signatures are not reproduced (only the .zip/.jar artifacts), and the reused binaries internally report the prior version (inherent to reusing them).
  • These are workflow-logic changes not exercised locally. The reuse-release and npm-reuse steps publish irreversibly and have no dry-run guard; recommend a workflow_dispatch rehearsal before relying on the unchanged path in a real release (can add a dry-run short-circuit if wanted).



Trunk (cicd_3-trunk.yml): gate the native CLI build on the `cli` path
filter and gate deploy-cli on the same predicate, so the 3-OS GraalVM
matrix (incl. two macOS runners) is skipped when tools/dotcms-cli/** is
unchanged.

Release (cicd_release-cli.yml): detect whether the CLI changed since the
previous release tag (fail-safe: build when uncertain) and, when
unchanged, skip the native build + JReleaser and republish the prior
release's binaries under the new version across all channels — npm, the
dotcms-cli-<version> GitHub release, and Artifactory. Remove the Slack
announcement from the CLI release cycle.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@claude

claude Bot commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Claude finished @wezell's task in 52s —— View job


Rollback Safety Analysis

  • Read rollback-unsafe categories reference
  • Get full PR diff
  • Analyze diff against all unsafe categories
  • Post result and apply label

Result: ✅ Safe To Rollback

All 7 changed files are GitHub Actions workflow files (.github/filters.yaml, cicd_*.yml) and contain zero database migrations, zero Elasticsearch mapping changes, zero data model changes, and zero REST/GraphQL API contract changes.

Categories checked:

Category Verdict
C-1 Structural Data Model Change ✅ Not present — no runonce tasks, no DB schema changes
C-2 Elasticsearch Mapping Change ✅ Not present — no ES mapping code
C-3 Content JSON Serialization Bump ✅ Not present — no CURRENT_MODEL_VERSION changes
C-4 DROP TABLE / DROP COLUMN ✅ Not present — no SQL DDL
H-1 One-Way Data Migration ✅ Not present — no data transformation
H-2 RENAME TABLE / COLUMN ✅ Not present
H-3 PK Restructuring ✅ Not present
H-4 New Field Type ✅ Not present
H-5 Storage Provider Change ✅ Not present
H-6 DROP PROCEDURE / FUNCTION ✅ Not present
H-7 NOT NULL without default ✅ Not present
H-8 VTL Viewtool Contract Change ✅ Not present
M-1 Column Type Change ✅ Not present
M-2 Push Publishing Bundle Format ✅ Not present
M-3 REST/GraphQL API Contract ✅ Not present — changes are CI/CD workflow logic only
M-4 OSGi Plugin API Breakage ✅ Not present

All changes are pure CI/CD workflow and path-filter logic. Rolling back to the previous dotCMS release has no impact on the application's database state, search index, data model, or API contracts.

Label applied: AI: Safe To Rollback

…tion time

Replaces --sort=-creatordate with --sort=-version:refname so PREV_TAG is
determined by semantic version order rather than tag creation timestamp,
which can be misleading when tags are backfilled or re-created.
Extend the existing `dry-run` workflow_dispatch input to cover every
publish/upload action so the unchanged-CLI reuse path can be rehearsed
end-to-end with zero side effects:

- reuse-release: skip `gh release create` and the Artifactory upload,
  logging what would happen instead.
- publish-npm-package: pass `--dry-run` to `npm publish` on both the
  changed path and the reuse path.

JReleaser already honored JRELEASER_DRY_RUN. Updated the input
description to reflect that dry-run now skips npm, GitHub release, and
Artifactory as well.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The CLI integration tests (tools/dotcms-cli, @QuarkusTest + testcontainers
against a live dotCMS) were gated on `cli || backend`, so they ran on nearly
every backend PR — the broad `backend` filter matches almost all of dotCMS/**.

Add a `cli_test` filter = CLI sources (`cli`) + the server REST endpoints the
CLI's @path clients actually consume (assets, folder, site, contenttype,
languages v2, workflow, authentication, user, analytics event) plus the
asset-specific business APIs WebAssetHelper delegates to (fileassets, browser).
Keep `cli` (= tools/dotcms-cli/**) untouched so endpoint-only changes don't
re-trigger the GraalVM native build added in #36081.

- PR + merge-queue now gate CLI tests on `cli_test` instead of `cli || backend`.
- Register `cli_test` in the initialize-phase rewrite (build_test_filters +
  test_filters) so it defaults correctly under change-detection=disabled and
  honors CICD_SKIP_TESTS.
- Safety net: trunk (which previously ran NO CLI tests) now runs them for any
  `backend` change, so a deeper business-layer regression that slips past the
  narrowed PR filter is still caught on main before a release. Nightly continues
  to run them unconditionally.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…36080

- TempFileAPI (rest/api/v1/temp) backs CLI file uploads on the asset push
  path via WebAssetHelper; same rationale as the fileassets/browser deps.
- LTS workflow now gates CLI tests on `cli_test` (CLI sources + consumed
  endpoints) instead of bare `cli`, matching PR/merge-queue behavior so
  endpoint changes on an LTS branch still exercise the CLI suite.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…36080

Address PR review findings:

- publish-npm-package now depends on reuse-release. On the unchanged path
  (CLI_CHANGED=false) reuse-release publishes the GitHub release + Artifactory
  while publish-npm-package republishes to npm; previously they ran in parallel,
  so a reuse-release failure could leave npm published with no GitHub release.
  Adding it to `needs` makes !failure() block npm publish if reuse-release fails;
  on the changed path reuse-release is skipped (not failed) so the job still runs.
- Document, at the PR and merge-queue CLI-test gates, why the trigger is narrow
  (cli_test) and that the trunk CLI run is the compensating control for deep
  backend changes.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The clean-up job had no always() guard, so GitHub Actions skipped it whenever
any upstream job failed, leaking the temporary version-update-<ver>-<run_id>
branch. Guard with always() (AUXILIARY_BRANCH is only set when precheck
succeeded, so it still no-ops when there's nothing to clean up). Addresses PR
review follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…hecks

Replace the implicit !failure() gate with explicit
(needs.reuse-release.result == 'success' || 'skipped') &&
(needs.release.result == 'success' || 'skipped') checks so the
mutual-exclusion of the changed/unchanged paths is self-documenting
and a failure in either upstream job reliably blocks the npm publish.

https://claude.ai/code/session_018KKHBasNtegpRZy5G6cCD1

@sfreudenthaler sfreudenthaler left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged — won't fix (re: npm view "${PACKAGE_FULL_NAME}" version in the "Reuse previous CLI release and publish" step)

The feedback notes that npm view ... version returns the latest tag version rather than a version derived from PREV_RELEASE_VERSION, and suggests deriving it from the git-based output instead.

This was a deliberate choice, documented in the inline comment above that step:

"We read the source version from npm ('latest') rather than reconstructing it from the git tag, sidestepping the leading-zero normalization applied to release versions above."

Deriving from PREV_RELEASE_VERSION would require replicating the same leading-zero stripping logic from the "Dynamic configuration" step — adding complexity with no practical benefit. This workflow is the sole publisher to the latest npm tag (LTS releases are explicitly excluded, and no other code path publishes there), so the two values are guaranteed to agree under normal operation. The only divergence scenario is out-of-band manual intervention on npm, which is outside the scope of this workflow to defend against.


Generated by Claude Code

@github-actions

github-actions Bot commented Jun 26, 2026

Copy link
Copy Markdown
Contributor

🤖 dotBot Review (Bedrock)

Reviewed 7 file(s); 2 candidate(s) → 2 confirmed, 0 uncertain (unverified, kept for review).

Confirmed findings

  • 🟡 Medium .github/filters.yaml:60 — Incorrect path filter for AnalyticsAPI
    The filter path 'dotCMS/src/main/java/com/dotcms/rest/api/v1/analytics/event/**' does not match the actual package structure. The AnalyticsEventResource class is located at 'dotCMS/src/main/java/com/dotcms/rest/api/v1/analytics/content/event/AnalyticsEventResource.java', indicating the correct path should include 'content/event' instead of just 'event'. This mismatch could cause the CI to miss relevant changes to the AnalyticsAPI endpoint.
  • 🟡 Medium .github/workflows/cicd_comp_initialize-phase.yml:166 — Unnecessary CLI rebuilds due to cli_test filter
    The added 'cli_test' filter in build_test_filters (line 166) and test_filters (line 168) appears to be a distinct filter from 'cli', evidenced by separate usage in PR/trunk workflows (cicd_1-pr.yml:89 gates CLI builds on cli_test). This means CLI-related test file changes would trigger full CLI rebuilds, contradicting the PR's goal of only rebuilding when CLI source changes. The 'cli_test' filter should be removed from build_test_filters to avoid rebuilds on test-only changes.

us.deepseek.r1-v1:0 · Run: #28271011704 · tokens: in: 34103 · out: 12785 · total: 46888 · calls: 11 · est. ~$0.115

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

AI: Safe To Rollback Area : CI/CD PR changes GitHub Actions/workflows Team : Scout

Projects

Status: No status

Development

Successfully merging this pull request may close these issues.

CI: only build the CLI when it changes; reuse prior release otherwise

3 participants