Skip to content

chore(ci): migrate GitHub Actions CI to GitLab CI#2543

Open
universal-itengineer wants to merge 72 commits into
mainfrom
chore/ci/add-gitlab-ci
Open

chore(ci): migrate GitHub Actions CI to GitLab CI#2543
universal-itengineer wants to merge 72 commits into
mainfrom
chore/ci/add-gitlab-ci

Conversation

@universal-itengineer

@universal-itengineer universal-itengineer commented Jun 25, 2026

Copy link
Copy Markdown
Member

Description

Prepares the deckhouse/virtualization module to move from GitHub to GitLab by porting the CI. All development pipelines that ran on GitHub Actions now run on GitLab CI with the same behavior: build and deploy (dev/prod, all editions), lint, tests, secret/CVE/Svace scanning, changelog and backport automation, MR auto-assign, and manual release-channel dispatch.

The root .gitlab-ci.yml is a thin entrypoint; the pipeline lives in .gitlab/ci/**.

Not migrated in this iteration

  • e2e workflows — out of scope.
  • Reactive slash-commands / webhook listener — GitLab can't start pipelines from MR comments or label changes natively. Manual and scheduled jobs cover the same surface; a webhook listener is a possible follow-up.

Correctness fixes (from review)

  • Editions could be built/tagged wrong on release and manual pipelines. Some jobs picked up stray pipeline variables that overrode their own settings (a release tag built the CE prod image as EE; a manual precache pushed the wrong tag). The affected build/scan jobs are now isolated to their own config.
  • mrs:summary could never run — it expected Node.js on the shell runner, which has none; it now runs in the node container like the other JS job.
  • Malformed Loop release notification — the message is assembled safely, so a tag or channel value with quotes/newlines can't break or inject into it.
  • Generated-files check ran too much — the heavy vm-route-forge (bpf2go, docker) check now runs only on vm-route-forge changes, not on every change.

Why do we need it, and what problem does it solve?

Development of the module is moving to GitLab. Porting the CI first keeps the pipeline working once the repository moves.

What is the expected result?

  • MR push runs lint, tests, dev build, and the secret/CVE scans.
  • Merge to main/release-* builds and deploys the dev image.
  • Tag push builds and deploys the prod image per edition.
  • Manual/scheduled jobs cover backport, changelog, MR summary, release-channel dispatch, and registry cleanup.

Checklist

  • The code is covered by unit tests.
  • e2e tests passed.
  • Documentation updated according to the changes.
  • Changes were tested in the Kubernetes cluster manually.

Changelog entries

section: ci
type: feature
summary: "Migrate repository CI from GitHub Actions to GitLab CI with full parity: build/deploy (dev/prod, all editions incl. se-plus), lint, tests, secrets/CVE/Svace scanning, changelog/backport automation, MR auto-assign, and manual release-channel dispatch."
impact_level: low

@universal-itengineer universal-itengineer self-assigned this Jun 25, 2026
@universal-itengineer universal-itengineer changed the title Chore/ci/add gitlab ci chore(ci): migrate GitHub Actions CI to GitLab CI Jun 25, 2026
@universal-itengineer universal-itengineer added this to the v1.10.0 milestone Jun 26, 2026
Comment thread .gitlab/ci/templates/dev/main.yml Outdated
Comment thread .gitlab/ci/workflow.yml Outdated
Comment thread .gitlab/ci/templates/release/prod_always.yml Outdated
Comment thread .gitlab/ci/templates/base/dual_registry_login.yml Outdated
Comment thread .gitlab/ci/templates/base/build.yml Outdated
Comment on lines +16 to +22
# Upstream `.build` carries two `rules:` that fire on ANY branch or tag
# push (`if: $CI_COMMIT_BRANCH`, `if: $CI_COMMIT_TAG`). GitLab CI merges
# rules from all `extends:` parents with the parent's rules evaluated
# first, so the upstream rules always win. That regresses behavior for
# jobs like build_dev_tags, build_main, build_prod which need to gate on
# specific branch / tag patterns provided by .dev / .dev_tags / .main /
# .prod_always.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need to consult SSDLC on this matter, maybe they can suggest a workaround / different job structure. Same with deploy.yml.

@universal-itengineer universal-itengineer force-pushed the chore/ci/add-gitlab-ci branch 4 times, most recently from 1f0d8d0 to da5a921 Compare June 29, 2026 12:07
Comment thread .gitlab/ci/templates/base/registry-login.yml Outdated
Comment on lines +7 to +12
.dev_vars:
extends:
- .login_dev_registry
variables:
MODULES_MODULE_SOURCE: "${DEV_MODULE_SOURCE}"
ENV: DEV

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we don't use .login_dev_registry anywhere, only .dev_vars. maybe remove .dev_vars entirely and move everything to .login_dev_registry?
same thing with .login_prod_registry and .prod_vars.

Comment on lines +12 to +15
# Install golangci-lint v2. Keep in sync with GOLANGCI_LINT_VERSION in
# .gitlab/ci/variables.yml and GOLANGCI_LINT_MIN_VERSION in the
# _ensure:golangci-lint tasks. v2 lives under the /v2 module path.
- go install github.com/golangci/golangci-lint/v2/cmd/golangci-lint@v2.11.1

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do this PR needs an update of golangci-lint?

type ebpfSpecs struct {
ebpfProgramSpecs
ebpfMapSpecs
ebpfVariableSpecs

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are these changes really needed in this PR?

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
- auto-assign-author: run on every MR event instead of gating on
  changes to the job's own files, so the MR author is actually
  auto-assigned when no assignee is set (the script is idempotent and
  skips when an assignee already exists)
- workflow: skip redundant branch pipelines when an open MR exists
  (CI_OPEN_MERGE_REQUESTS guard) to avoid duplicate branch + MR pipelines
- lint-validate: drop unused RUN_ON_CHANGE matrix var from check:gens-files
  and fold vm-route-forge into the single COMPONENT matrix

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
test:hooks job ran 'task hooks:test' which does not exist (the task is
gohooks:test), matching the migration plan and Taskfile.

lint/test:virtualization-controller jobs failed at 'task virtualization-controller:init'
because runner go (1.25.9, GOTOOLCHAIN=go1.25.9) is older than go.mod's
'go 1.25.11'. Add global GOTOOLCHAIN=auto so the host toolchain
auto-downloads the version required by go.mod, mirroring actions/setup-go@v5.
GitLab CI variables override runner environment variables.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Remove internal tracker issue IDs scattered in comments across
.gitlab/ci files so they do not leak into the public PR. No behavior change.

Affected files:
- .gitlab/ci/stages.yml
- .gitlab/ci/includes.yml
- .gitlab/ci/templates/release/prod_manual.yml
- .gitlab/ci/jobs/deploy-prod.yml
- .gitlab/ci/jobs/gitleaks.yml

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The GitHub test_scripts_{js,python} jobs ran full unit suites, but the
GitLab migration had degraded them to a JS smoke test and a Python
py_compile check (m9e.5.16). The original GH *.test.js / charts_test.py
all live under e2e/ (out of migration scope); the migrated scripts
(mrs_notifier.mjs, changelog_collect.py, check_changelog_entry.py) never
had upstream tests, so port = write real unit tests for them.

Python (.gitlab/ci/scripts/python/tests/, 44 tests, stdlib unittest):
- check_changelog_entry: block regex, parse_block, load_allowed_sections,
  validate_block (section/type/summary/impact_level rules incl. :low pin
  and the feature/fix-only ALLOWED_TYPES guard from 5.4).
- changelog_collect: next_link, parse_changes_block, has_label,
  group_entries, yaml_summary_scalar quoting, render_yaml (deckhouse
  schema, mr-iid ordering, :low strip, unsupported-type skip),
  render_markdown, minor_version_from_tag.
- test:scripts:python now runs py_compile + `unittest discover`, no
  longer allow_failure.

JS (mrs_notifier): export shouldNotifyMR (open-MR filter) and formatUser
as pure helpers; add unit tests for both (29 tests total). Drop the
stale "smoke test / cannot be imported" framing in the job comment.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Comment-only audit of .gitlab/** surfaced one internal contradiction and
three stale TODOs (no behavior change):

- stages.yml: the "Stage semantics" legend listed prod_check after
  build/scan/deploy_dev, contradicting both the actual `stages:` order
  (prod_check is declared before build) and its own "Runs BEFORE build"
  note. Reordered to match, and clarified that prod_check runs ONLY in the
  manual web release-channel flow (prod:check-requirements,
  CI_PIPELINE_SOURCE == web) — its early position is a DAG-ordering
  artifact so prod:build:* can `needs:` it, not a dev/MR-pipeline stage.

- backport.yml (x2), changelog.yml, manual-tools.yml: dropped the
  "TODO: webhook-listener" framing. Reactive webhook automation is an
  accepted permanent gap (no webhook-listener will be built); reworded
  accordingly, without citing a tracker id.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Three build/deploy parity fixes for the GitHub Actions -> GitLab CI
migration, found by reviewing .github/workflows against the new
.gitlab/ci/** files.

- Drop the DEV release-channel fan-out. deploy_for_dev_tag crane-copied a
  dev tag into alpha/beta/early-access/stable/rock-solid, but the DEV
  registry has no release channels, only tags. The GitHub "Deploy Dev"
  workflow is build-only. Remove deploy-dev.yml, its include, and the
  now-unused deploy_dev stage; dev tags stay built by build_dev_tags.

- Pass secondary_repo to prod builds. GitHub builds set
  secondary_repo=${DEV_MODULE_SOURCE}/${MODULE_NAME} so prod werf builds
  reuse already-built layers from the DEV registry. Add
  WERF_SECONDARY_REPO_1 to .dual_registry_login (a prod-only template),
  so it applies to build_prod and prod:build:* but never to dev builds.

- Use "main" as the main-branch module tag instead of v0.0.0-main, to
  match GitHub set_vars (MODULES_MODULE_TAG = ref_name = "main").

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…hangelog impact

Fix the functional bugs found while reviewing the GitHub Actions ->
GitLab CI migration (vs .github/workflows).

- cve-scan: the manual scan set RELEASE_IN_DEV=${SCAN_TAG:-false}, passing
  the tag string into a boolean. Derive it via rules instead, mirroring
  the GitHub `startsWith(tag || 'main', 'release-')` expression: a
  release-* SCAN_TAG sets "true", anything else keeps "false".

- svace: svace:notify reported status from CI_JOB_STATUS of the notify job
  itself (almost always success). Read the real svace:analyze status via
  the pipeline jobs API, and run notify on both success and failure with
  an on_success/on_failure rule pair (same idiom as prod:notify-loop).
  Drop GIT_STRATEGY: none so the api.sh helper is available.

- cleanup: restore the explicit WERF_DRY_RUN="false" from
  dev_registry-cleanup.yml, so an inherited WERF_DRY_RUN=true cannot turn
  cleanup into a silent no-op.

- changelog_collect.py: preserve the free-text `impact` migration note for
  high-impact entries (parse multi-line block values, emit `impact` after
  `pull_request`), and make CHANGELOG-<minor>.md cumulative across patch
  releases instead of overwriting it with only the current milestone.
  Add unit tests for all four behaviours.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The "avoid duplicate pipelines" comment sat above the default-branch rule,
which does not skip anything. The branch-pipeline skip is actually done by
the `push && $CI_OPEN_MERGE_REQUESTS -> when: never` rule. Move the
explanation onto that rule, and give the default-branch and tag rules their
own accurate comments.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Remove references to the internal planning document (section markers and
tmp/ paths) from comments across .gitlab/** and the root .gitlab-ci.yml,
and replace the absolute local checkout paths (/Users/...) with the
canonical upstream repo name (deckhouse/3p/deckhouse/modules-gitlab-ci).
Comments keep their substance; only the internal/local references are
dropped. No behavior change.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…g context

Two template cleanups, both behavior-preserving (same variables/rules emitted).

Registry login:
- Replace the misleadingly named .dual_registry_login (it only configured the
  DEV registry, not "two") with a single base/registry-login.yml that names
  each login explicitly: .login_dev_registry and .login_prod_registry (primary
  registry select), .login_prod_read_registry (read-only checks), and
  .also_login_dev_registry (the extra DEV login + werf secondary repo a prod
  build composes to also read DEV). This mirrors the GitHub pipeline, which
  logged into the prod and dev registries explicitly.
- .dev_vars / .prod_vars now extend the matching .login_*_registry, so the
  registry credentials live in one file and the vars bundles only add
  MODULES_MODULE_SOURCE + ENV. Move .prod_read_registry there as
  .login_prod_read_registry.

Prod tag context:
- prod_always.yml and prod_manual.yml were identical except the rule's
  `when:` (always vs manual). Merge them into prod_tag.yml with a shared
  .prod_tag base; .prod_always / .prod_manual now only carry their rule.

Anchor names used by jobs (.dev_vars, .prod_vars, .prod_always, .prod_manual)
are unchanged; only .dual_registry_login -> .also_login_dev_registry and
.prod_read_registry -> .login_prod_read_registry references were updated.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
- backport.yml: remove the scheduled "backport sweep" rule (Mode 3). It was
  beyond the GitHub workflow / migration scope and non-functional anyway
  (a `when: manual` job on a schedule pipeline is never triggered). Manual
  TARGET_BRANCH (Mode 1) and the status/backport label (Mode 2) remain the
  working backport paths. Drop its row from the SCHEDULE_TYPE table too.
- deploy-prod.yml: remove the "inputs not yet ported" TODO — it contradicted
  the very next paragraph; those inputs are implemented in the web flow in
  release-channels.yml.
- prod_tag.yml: demote the same "TODO" to a plain note (the design decision is
  made: tag-push uses the matrix, the input dispatch lives in release-channels).

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Pin the mrs_notifier JS test dependencies with a committed package-lock.json
and switch the test:scripts:js job from `npm install` to `npm ci` for
reproducible, lockfile-verified installs.

package-lock.json is ignored globally at the repo root, so a scoped
.gitlab/scripts/js/.gitignore re-includes this one. Verified locally:
`npm ci` + `npm test` pass (29 tests).

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…release/mutating jobs

Centralize the cancel-in-progress behavior (GitHub concurrency equivalent):
- defaults.yml now sets `interruptible: true` for every job, so a new commit
  cancels the redundant in-flight pipeline once the project enables
  "Auto-cancel redundant pipelines". Drop the resolved TODO in workflow.yml.
- Remove the now-redundant per-job `interruptible: true` lines.
- Override `interruptible: false` where a run must not be cancelled mid-flight:
  prod release build/deploy (via .prod_vars) and the whole web release flow
  (via .prod_release_rules), plus state-mutating/long jobs: cleanup,
  changelog (both), backport, translate-changelog (already: precache, cve
  daily/manual, gitleaks full, svace build/analyze). build_prod drops its
  explicit false (inherited from .prod_vars).

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
The decision is to keep tracking the v13.0 branch (upstream fixes flow in
automatically). Replace the "pin to SHA" TODO in includes.yml with a plain
note that records the branch HEAD SHA to use if pinning is ever wanted.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
Replace the leftover GitHub username default ("z9r5") with the GitLab handle
"gitlab-virt-bot" — the one reviewer kept as a real @-mention in the Loop MR
summary. Still overridable via the DOC_REVIEWER CI/CD variable. Removes the
last open TODO in the GitLab CI scripts.

Signed-off-by: Nikita Korolev <nikita.korolev@flant.com>
…build jobs only

WERF_SECONDARY_REPO_1 is a werf build-time var, consumed only by
`werf build`. In the GitHub workflows the equivalent `secondary_repo`
input is passed to build@v4 only, never to deploy@v2. It was bundled
into .also_login_dev_registry, which deploy jobs also extend, so every
prod deploy carried a dead WERF_SECONDARY_REPO_1.

Pull it out into a dedicated .werf_secondary_repo template (home for the
WERF_SECONDARY_REPO_* family) and add it only to prod build jobs
(build_prod, prod:build:ce/ee/se-plus/fe). .also_login_dev_registry now
holds just the DEV login vars, needed by both builds and deploys.
…ogin

The template only adds the DEV_MODULES_REGISTRY* vars so the second
'werf cr login' authenticates against the DEV registry for read/pull
(prod jobs reuse DEV-built layers / crane-copy source images). The new
name states that role directly instead of the vague 'also login'.

Pure rename: template definition, all extends: references, and the
example comments. No behavior change.
The GitLab-only vX.Y.Z-dev* tag scheme was carried over from the previous
root .gitlab-ci.yml and never matched a real GitHub tag. Gate tag pipelines
on the effective GitHub scheme instead:

  vX.Y.Z       build into DEV (build_dev_tags) AND PROD (build_prod),
               mirroring dev_module_build-and-registration.yml +
               release_module_build-and-registration.yml firing on the
               same tag.
  vX.Y.Z-rc.N  build into DEV only; the prod rule stays strict ^vX.Y.Z$.

workflow.yml creates pipelines for ^v\d+\.\d+\.\d+(-rc\.\d+)?$, and the
.dev_tags / set_vars rules match the same set so the DEV build and its
set_vars dotenv dependency stay in sync.
The four DEV-registry build jobs (build_dev, build_dev_tags, build_main,
build_release) were identical except for their context template
(.dev / .dev_tags / .main / .release), which supplies MODULES_MODULE_TAG and
the trigger rule. Move the shared body (stage, needs:set_vars,
WERF_VIRTUAL_MERGE, .base_build) into a hidden .dev_build base; each job is now
extends: [.dev_build, <context>].

The jobs stay distinct (not collapsed into one rule-variables job) so each
context remains a named, independently retriable job. .dev / .main are left
standalone because they are reused as context mixins by tests, lint:dmt,
show_*_manifest and precache.
…pying

Include the upstream modules-gitlab-ci Build.gitlab-ci.yml and
Deploy.gitlab-ci.yml and drive our .base_build / .base_deploy from their
rules-free .build_base / .deploy_base via `!reference`, replacing the verbatim
copies of the upstream build/deploy script bodies.

`!reference [.build_base, script]` (rather than `extends`) keeps the local
check-runner-tools guard prepended to the upstream script, so behavior is
unchanged except that the build now also copies the bundle attestation (the
copies had drifted from upstream). Job-level rules from .dev / .main / .prod
still drive gating because the *_base templates carry no rules.

The upstream include ref temporarily points at the branch carrying the base
split (deckhouse/modules-gitlab-ci#89); return it to v13.0 once that lands and
mirrors into the v13.0 branch on fox.
The rules-free base split (.build_base / .deploy_base / .lint_base) has been
merged upstream (deckhouse/modules-gitlab-ci#89) and is now in the v13.0
branch, so the temporary branch ref is no longer needed.
…es .lint_base

Update stale comments left over from the .build_base / .deploy_base / .lint_base
migration: drop dead HEAD SHAs, and describe .base_build / .base_deploy as
consuming the rules-free upstream bases rather than copying/mirroring them.

lint:dmt now extends the upstream .lint_base (from the now-included
Build.gitlab-ci.yml) instead of inlining `dmt lint ./` — the last leftover inline
copy of an upstream body. Behavior is unchanged: same script, allow_failure, and
lint stage with .dev gating.
…e to EE

On a vX.Y.Z tag, set_vars (info stage) emits MODULE_EDITION=EE. Without an
explicit needs:, build_prod ran in stage-mode and inherited that dotenv, which
overrides variables: set in YAML including parallel:matrix -> the ce leg's
MODULE_EDITION: CE was clobbered to EE and the ce prod image was built as EE
(.werf/consts.yaml uses MODULE_EDITION as a Go build tag and EE gate).

needs: [] opts build_prod into DAG mode with no dependencies, blocking all
prior-stage dotenv inheritance. build_prod consumes nothing from set_vars; all
its vars come from .prod_tag/.prod_vars and the matrix. Mirrors the same guard
the dev build jobs already have (466e80f).
svace:set-vars runs in the info stage and, on any web "Run pipeline", emits
MODULES_MODULE_TAG=<ref>-svace + MODULE_EDITION=EE unconditionally. Jobs in
later stages without an explicit needs: inherit that dotenv, and dotenv vars
override variables: set in YAML:

- precache:build:main / precache:build:branch (build stage, web trigger) had
  their MODULES_MODULE_TAG clobbered to the -svace tag, so a manual precache run
  pushed the wrong image tag. Confirmed.
- cve:scan:manual (scan stage, web trigger) inherited the same dotenv, which
  could shift the scan target away from its SOURCE_TAG. Defensive guard.

needs: [] opts each into DAG mode with no dependencies; none has an in-pipeline
data dependency. Same guard as build_prod (24fc663) and the dev build jobs.
notify-release-loop.sh interpolated SUMMARY (which embeds RELEASE_TAG and
RELEASE_CHANNEL from manual UI input) raw into --data "{\"text\": ...}".
Any ", \ or newline broke the JSON, and a crafted value could inject
arbitrary top-level fields into the payload. Build the body with
jq -nc --arg instead, and use real newlines in SUMMARY (jq encodes them)
so the manual \n escaping is no longer needed.
mrs:summary called check-runner-tools.sh node npm, but the shell runner host
has no Node.js (documented in test:scripts:js and variables.yml NODE_VERSION),
so the job failed at the tool check. Run it inside node:${NODE_VERSION} via
docker run, mirroring test:scripts:js: repo mounted read-only and copied to an
ephemeral tmpdir (keeps root-owned node_modules out of the shared workspace),
runtime env forwarded with -e, container has outbound network for the GitLab
API and Loop webhook.
The generated-files check ran as a single parallel:matrix job over three
components, and GitLab cannot gate individual matrix legs with rules:changes.
So any matching change ran all three legs, including the heavy vm-route-forge
bpf2go leg (docker) for e.g. an api-only change: wasted cost and an extra
flake surface.

Split into one job per component sharing a hidden .gens_check base, each with
its own rules:changes. Mirrors the GitHub per-component gating: the
vm-route-forge docker leg now fires only on vm-route-forge changes, api only on
api/go.mod changes. docker is checked inside the vm-route-forge branch (the
only leg that needs it) instead of for every leg.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants