chore(ci): migrate GitHub Actions CI to GitLab CI#2543
Open
universal-itengineer wants to merge 72 commits into
Open
chore(ci): migrate GitHub Actions CI to GitLab CI#2543universal-itengineer wants to merge 72 commits into
universal-itengineer wants to merge 72 commits into
Conversation
091b766 to
dcde794
Compare
nevermarine
reviewed
Jun 26, 2026
nevermarine
requested changes
Jun 26, 2026
Comment on lines
+16
to
+22
| # Upstream `.build` carries two `rules:` that fire on ANY branch or tag | ||
| # push (`if: $CI_COMMIT_BRANCH`, `if: $CI_COMMIT_TAG`). GitLab CI merges | ||
| # rules from all `extends:` parents with the parent's rules evaluated | ||
| # first, so the upstream rules always win. That regresses behavior for | ||
| # jobs like build_dev_tags, build_main, build_prod which need to gate on | ||
| # specific branch / tag patterns provided by .dev / .dev_tags / .main / | ||
| # .prod_always. |
Collaborator
There was a problem hiding this comment.
I think we need to consult SSDLC on this matter, maybe they can suggest a workaround / different job structure. Same with deploy.yml.
1f0d8d0 to
da5a921
Compare
nevermarine
requested changes
Jun 29, 2026
Comment on lines
+7
to
+12
| .dev_vars: | ||
| extends: | ||
| - .login_dev_registry | ||
| variables: | ||
| MODULES_MODULE_SOURCE: "${DEV_MODULE_SOURCE}" | ||
| ENV: DEV |
Collaborator
There was a problem hiding this comment.
we don't use .login_dev_registry anywhere, only .dev_vars. maybe remove .dev_vars entirely and move everything to .login_dev_registry?
same thing with .login_prod_registry and .prod_vars.
Comment on lines
+12
to
+15
| # Install golangci-lint v2. Keep in sync with GOLANGCI_LINT_VERSION in | ||
| # .gitlab/ci/variables.yml and GOLANGCI_LINT_MIN_VERSION in the | ||
| # _ensure:golangci-lint tasks. v2 lives under the /v2 module path. | ||
| - go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.11.1 |
Collaborator
There was a problem hiding this comment.
why do this PR needs an update of golangci-lint?
| type ebpfSpecs struct { | ||
| ebpfProgramSpecs | ||
| ebpfMapSpecs | ||
| ebpfVariableSpecs |
Collaborator
There was a problem hiding this comment.
are these changes really needed in this PR?
f71c3be to
76a42a6
Compare
76a42a6 to
1313d77
Compare
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
- auto-assign-author: run on every MR event instead of gating on changes to the job's own files, so the MR author is actually auto-assigned when no assignee is set (the script is idempotent and skips when an assignee already exists) - workflow: skip redundant branch pipelines when an open MR exists (CI_OPEN_MERGE_REQUESTS guard) to avoid duplicate branch + MR pipelines - lint-validate: drop unused RUN_ON_CHANGE matrix var from check:gens-files and fold vm-route-forge into the single COMPONENT matrix Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
test:hooks job ran 'task hooks:test' which does not exist (the task is gohooks:test), matching the migration plan and Taskfile. lint/test:virtualization-controller jobs failed at 'task virtualization-controller:init' because runner go (1.25.9, GOTOOLCHAIN=go1.25.9) is older than go.mod's 'go 1.25.11'. Add global GOTOOLCHAIN=auto so the host toolchain auto-downloads the version required by go.mod, mirroring actions/setup-go@v5. GitLab CI variables override runner environment variables. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Remove internal tracker issue IDs scattered in comments across .gitlab/ci files so they do not leak into the public PR. No behavior change. Affected files: - .gitlab/ci/stages.yml - .gitlab/ci/includes.yml - .gitlab/ci/templates/release/prod_manual.yml - .gitlab/ci/jobs/deploy-prod.yml - .gitlab/ci/jobs/gitleaks.yml Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The GitHub test_scripts_{js,python} jobs ran full unit suites, but the
GitLab migration had degraded them to a JS smoke test and a Python
py_compile check (m9e.5.16). The original GH *.test.js / charts_test.py
all live under e2e/ (out of migration scope); the migrated scripts
(mrs_notifier.mjs, changelog_collect.py, check_changelog_entry.py) never
had upstream tests, so port = write real unit tests for them.
Python (.gitlab/ci/scripts/python/tests/, 44 tests, stdlib unittest):
- check_changelog_entry: block regex, parse_block, load_allowed_sections,
validate_block (section/type/summary/impact_level rules incl. :low pin
and the feature/fix-only ALLOWED_TYPES guard from 5.4).
- changelog_collect: next_link, parse_changes_block, has_label,
group_entries, yaml_summary_scalar quoting, render_yaml (deckhouse
schema, mr-iid ordering, :low strip, unsupported-type skip),
render_markdown, minor_version_from_tag.
- test:scripts:python now runs py_compile + `unittest discover`, no
longer allow_failure.
JS (mrs_notifier): export shouldNotifyMR (open-MR filter) and formatUser
as pure helpers; add unit tests for both (29 tests total). Drop the
stale "smoke test / cannot be imported" framing in the job comment.
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Comment-only audit of .gitlab/** surfaced one internal contradiction and three stale TODOs (no behavior change): - stages.yml: the "Stage semantics" legend listed prod_check after build/scan/deploy_dev, contradicting both the actual `stages:` order (prod_check is declared before build) and its own "Runs BEFORE build" note. Reordered to match, and clarified that prod_check runs ONLY in the manual web release-channel flow (prod:check-requirements, CI_PIPELINE_SOURCE == web) — its early position is a DAG-ordering artifact so prod:build:* can `needs:` it, not a dev/MR-pipeline stage. - backport.yml (x2), changelog.yml, manual-tools.yml: dropped the "TODO: webhook-listener" framing. Reactive webhook automation is an accepted permanent gap (no webhook-listener will be built); reworded accordingly, without citing a tracker id. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Three build/deploy parity fixes for the GitHub Actions -> GitLab CI
migration, found by reviewing .github/workflows against the new
.gitlab/ci/** files.
- Drop the DEV release-channel fan-out. deploy_for_dev_tag crane-copied a
dev tag into alpha/beta/early-access/stable/rock-solid, but the DEV
registry has no release channels, only tags. The GitHub "Deploy Dev"
workflow is build-only. Remove deploy-dev.yml, its include, and the
now-unused deploy_dev stage; dev tags stay built by build_dev_tags.
- Pass secondary_repo to prod builds. GitHub builds set
secondary_repo=${DEV_MODULE_SOURCE}/${MODULE_NAME} so prod werf builds
reuse already-built layers from the DEV registry. Add
WERF_SECONDARY_REPO_1 to .dual_registry_login (a prod-only template),
so it applies to build_prod and prod:build:* but never to dev builds.
- Use "main" as the main-branch module tag instead of v0.0.0-main, to
match GitHub set_vars (MODULES_MODULE_TAG = ref_name = "main").
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…hangelog impact
Fix the functional bugs found while reviewing the GitHub Actions ->
GitLab CI migration (vs .github/workflows).
- cve-scan: the manual scan set RELEASE_IN_DEV=${SCAN_TAG:-false}, passing
the tag string into a boolean. Derive it via rules instead, mirroring
the GitHub `startsWith(tag || 'main', 'release-')` expression: a
release-* SCAN_TAG sets "true", anything else keeps "false".
- svace: svace:notify reported status from CI_JOB_STATUS of the notify job
itself (almost always success). Read the real svace:analyze status via
the pipeline jobs API, and run notify on both success and failure with
an on_success/on_failure rule pair (same idiom as prod:notify-loop).
Drop GIT_STRATEGY: none so the api.sh helper is available.
- cleanup: restore the explicit WERF_DRY_RUN="false" from
dev_registry-cleanup.yml, so an inherited WERF_DRY_RUN=true cannot turn
cleanup into a silent no-op.
- changelog_collect.py: preserve the free-text `impact` migration note for
high-impact entries (parse multi-line block values, emit `impact` after
`pull_request`), and make CHANGELOG-<minor>.md cumulative across patch
releases instead of overwriting it with only the current milestone.
Add unit tests for all four behaviours.
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The "avoid duplicate pipelines" comment sat above the default-branch rule, which does not skip anything. The branch-pipeline skip is actually done by the `push && $CI_OPEN_MERGE_REQUESTS -> when: never` rule. Move the explanation onto that rule, and give the default-branch and tag rules their own accurate comments. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Remove references to the internal planning document (section markers and tmp/ paths) from comments across .gitlab/** and the root .gitlab-ci.yml, and replace the absolute local checkout paths (/Users/...) with the canonical upstream repo name (deckhouse/3p/deckhouse/modules-gitlab-ci). Comments keep their substance; only the internal/local references are dropped. No behavior change. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…g context Two template cleanups, both behavior-preserving (same variables/rules emitted). Registry login: - Replace the misleadingly named .dual_registry_login (it only configured the DEV registry, not "two") with a single base/registry-login.yml that names each login explicitly: .login_dev_registry and .login_prod_registry (primary registry select), .login_prod_read_registry (read-only checks), and .also_login_dev_registry (the extra DEV login + werf secondary repo a prod build composes to also read DEV). This mirrors the GitHub pipeline, which logged into the prod and dev registries explicitly. - .dev_vars / .prod_vars now extend the matching .login_*_registry, so the registry credentials live in one file and the vars bundles only add MODULES_MODULE_SOURCE + ENV. Move .prod_read_registry there as .login_prod_read_registry. Prod tag context: - prod_always.yml and prod_manual.yml were identical except the rule's `when:` (always vs manual). Merge them into prod_tag.yml with a shared .prod_tag base; .prod_always / .prod_manual now only carry their rule. Anchor names used by jobs (.dev_vars, .prod_vars, .prod_always, .prod_manual) are unchanged; only .dual_registry_login -> .also_login_dev_registry and .prod_read_registry -> .login_prod_read_registry references were updated. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
- backport.yml: remove the scheduled "backport sweep" rule (Mode 3). It was beyond the GitHub workflow / migration scope and non-functional anyway (a `when: manual` job on a schedule pipeline is never triggered). Manual TARGET_BRANCH (Mode 1) and the status/backport label (Mode 2) remain the working backport paths. Drop its row from the SCHEDULE_TYPE table too. - deploy-prod.yml: remove the "inputs not yet ported" TODO — it contradicted the very next paragraph; those inputs are implemented in the web flow in release-channels.yml. - prod_tag.yml: demote the same "TODO" to a plain note (the design decision is made: tag-push uses the matrix, the input dispatch lives in release-channels). Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Pin the mrs_notifier JS test dependencies with a committed package-lock.json and switch the test:scripts:js job from `npm install` to `npm ci` for reproducible, lockfile-verified installs. package-lock.json is ignored globally at the repo root, so a scoped .gitlab/scripts/js/.gitignore re-includes this one. Verified locally: `npm ci` + `npm test` pass (29 tests). Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…release/mutating jobs Centralize the cancel-in-progress behavior (GitHub concurrency equivalent): - defaults.yml now sets `interruptible: true` for every job, so a new commit cancels the redundant in-flight pipeline once the project enables "Auto-cancel redundant pipelines". Drop the resolved TODO in workflow.yml. - Remove the now-redundant per-job `interruptible: true` lines. - Override `interruptible: false` where a run must not be cancelled mid-flight: prod release build/deploy (via .prod_vars) and the whole web release flow (via .prod_release_rules), plus state-mutating/long jobs: cleanup, changelog (both), backport, translate-changelog (already: precache, cve daily/manual, gitleaks full, svace build/analyze). build_prod drops its explicit false (inherited from .prod_vars). Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The decision is to keep tracking the v13.0 branch (upstream fixes flow in automatically). Replace the "pin to SHA" TODO in includes.yml with a plain note that records the branch HEAD SHA to use if pinning is ever wanted. Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Replace the leftover GitHub username default ("z9r5") with the GitLab handle
"gitlab-virt-bot" — the one reviewer kept as a real @-mention in the Loop MR
summary. Still overridable via the DOC_REVIEWER CI/CD variable. Removes the
last open TODO in the GitLab CI scripts.
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…build jobs only WERF_SECONDARY_REPO_1 is a werf build-time var, consumed only by `werf build`. In the GitHub workflows the equivalent `secondary_repo` input is passed to build@v4 only, never to deploy@v2. It was bundled into .also_login_dev_registry, which deploy jobs also extend, so every prod deploy carried a dead WERF_SECONDARY_REPO_1. Pull it out into a dedicated .werf_secondary_repo template (home for the WERF_SECONDARY_REPO_* family) and add it only to prod build jobs (build_prod, prod:build:ce/ee/se-plus/fe). .also_login_dev_registry now holds just the DEV login vars, needed by both builds and deploys.
…ogin The template only adds the DEV_MODULES_REGISTRY* vars so the second 'werf cr login' authenticates against the DEV registry for read/pull (prod jobs reuse DEV-built layers / crane-copy source images). The new name states that role directly instead of the vague 'also login'. Pure rename: template definition, all extends: references, and the example comments. No behavior change.
The GitLab-only vX.Y.Z-dev* tag scheme was carried over from the previous
root .gitlab-ci.yml and never matched a real GitHub tag. Gate tag pipelines
on the effective GitHub scheme instead:
vX.Y.Z build into DEV (build_dev_tags) AND PROD (build_prod),
mirroring dev_module_build-and-registration.yml +
release_module_build-and-registration.yml firing on the
same tag.
vX.Y.Z-rc.N build into DEV only; the prod rule stays strict ^vX.Y.Z$.
workflow.yml creates pipelines for ^v\d+\.\d+\.\d+(-rc\.\d+)?$, and the
.dev_tags / set_vars rules match the same set so the DEV build and its
set_vars dotenv dependency stay in sync.
The four DEV-registry build jobs (build_dev, build_dev_tags, build_main, build_release) were identical except for their context template (.dev / .dev_tags / .main / .release), which supplies MODULES_MODULE_TAG and the trigger rule. Move the shared body (stage, needs:set_vars, WERF_VIRTUAL_MERGE, .base_build) into a hidden .dev_build base; each job is now extends: [.dev_build, <context>]. The jobs stay distinct (not collapsed into one rule-variables job) so each context remains a named, independently retriable job. .dev / .main are left standalone because they are reused as context mixins by tests, lint:dmt, show_*_manifest and precache.
1313d77 to
092250b
Compare
…pying Include the upstream modules-gitlab-ci Build.gitlab-ci.yml and Deploy.gitlab-ci.yml and drive our .base_build / .base_deploy from their rules-free .build_base / .deploy_base via `!reference`, replacing the verbatim copies of the upstream build/deploy script bodies. `!reference [.build_base, script]` (rather than `extends`) keeps the local check-runner-tools guard prepended to the upstream script, so behavior is unchanged except that the build now also copies the bundle attestation (the copies had drifted from upstream). Job-level rules from .dev / .main / .prod still drive gating because the *_base templates carry no rules. The upstream include ref temporarily points at the branch carrying the base split (deckhouse/modules-gitlab-ci#89); return it to v13.0 once that lands and mirrors into the v13.0 branch on fox.
The rules-free base split (.build_base / .deploy_base / .lint_base) has been merged upstream (deckhouse/modules-gitlab-ci#89) and is now in the v13.0 branch, so the temporary branch ref is no longer needed.
…es .lint_base Update stale comments left over from the .build_base / .deploy_base / .lint_base migration: drop dead HEAD SHAs, and describe .base_build / .base_deploy as consuming the rules-free upstream bases rather than copying/mirroring them. lint:dmt now extends the upstream .lint_base (from the now-included Build.gitlab-ci.yml) instead of inlining `dmt lint ./` — the last leftover inline copy of an upstream body. Behavior is unchanged: same script, allow_failure, and lint stage with .dev gating.
…e to EE On a vX.Y.Z tag, set_vars (info stage) emits MODULE_EDITION=EE. Without an explicit needs:, build_prod ran in stage-mode and inherited that dotenv, which overrides variables: set in YAML including parallel:matrix -> the ce leg's MODULE_EDITION: CE was clobbered to EE and the ce prod image was built as EE (.werf/consts.yaml uses MODULE_EDITION as a Go build tag and EE gate). needs: [] opts build_prod into DAG mode with no dependencies, blocking all prior-stage dotenv inheritance. build_prod consumes nothing from set_vars; all its vars come from .prod_tag/.prod_vars and the matrix. Mirrors the same guard the dev build jobs already have (466e80f).
svace:set-vars runs in the info stage and, on any web "Run pipeline", emits MODULES_MODULE_TAG=<ref>-svace + MODULE_EDITION=EE unconditionally. Jobs in later stages without an explicit needs: inherit that dotenv, and dotenv vars override variables: set in YAML: - precache:build:main / precache:build:branch (build stage, web trigger) had their MODULES_MODULE_TAG clobbered to the -svace tag, so a manual precache run pushed the wrong image tag. Confirmed. - cve:scan:manual (scan stage, web trigger) inherited the same dotenv, which could shift the scan target away from its SOURCE_TAG. Defensive guard. needs: [] opts each into DAG mode with no dependencies; none has an in-pipeline data dependency. Same guard as build_prod (24fc663) and the dev build jobs.
notify-release-loop.sh interpolated SUMMARY (which embeds RELEASE_TAG and
RELEASE_CHANNEL from manual UI input) raw into --data "{\"text\": ...}".
Any ", \ or newline broke the JSON, and a crafted value could inject
arbitrary top-level fields into the payload. Build the body with
jq -nc --arg instead, and use real newlines in SUMMARY (jq encodes them)
so the manual \n escaping is no longer needed.
mrs:summary called check-runner-tools.sh node npm, but the shell runner host
has no Node.js (documented in test:scripts:js and variables.yml NODE_VERSION),
so the job failed at the tool check. Run it inside node:${NODE_VERSION} via
docker run, mirroring test:scripts:js: repo mounted read-only and copied to an
ephemeral tmpdir (keeps root-owned node_modules out of the shared workspace),
runtime env forwarded with -e, container has outbound network for the GitLab
API and Loop webhook.
The generated-files check ran as a single parallel:matrix job over three components, and GitLab cannot gate individual matrix legs with rules:changes. So any matching change ran all three legs, including the heavy vm-route-forge bpf2go leg (docker) for e.g. an api-only change: wasted cost and an extra flake surface. Split into one job per component sharing a hidden .gens_check base, each with its own rules:changes. Mirrors the GitHub per-component gating: the vm-route-forge docker leg now fires only on vm-route-forge changes, api only on api/go.mod changes. docker is checked inside the vm-route-forge branch (the only leg that needs it) instead of for every leg.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Prepares the
deckhouse/virtualizationmodule to move from GitHub to GitLab by porting the CI. All development pipelines that ran on GitHub Actions now run on GitLab CI with the same behavior: build and deploy (dev/prod, all editions), lint, tests, secret/CVE/Svace scanning, changelog and backport automation, MR auto-assign, and manual release-channel dispatch.The root
.gitlab-ci.ymlis a thin entrypoint; the pipeline lives in.gitlab/ci/**.Not migrated in this iteration
Correctness fixes (from review)
mrs:summarycould never run — it expected Node.js on the shell runner, which has none; it now runs in thenodecontainer like the other JS job.vm-route-forge(bpf2go, docker) check now runs only onvm-route-forgechanges, not on every change.Why do we need it, and what problem does it solve?
Development of the module is moving to GitLab. Porting the CI first keeps the pipeline working once the repository moves.
What is the expected result?
main/release-*builds and deploys the dev image.Checklist
Changelog entries