From 07e95937d6546ff1e4a14bada521d94383dcb779 Mon Sep 17 00:00:00 2001 From: Ariel Schulz Date: Mon, 11 May 2026 16:02:03 +0200 Subject: [PATCH 1/7] Add schedule to view and augment text a bit --- .../features/github_workflows/index.rst | 25 ++++++++++++++----- 1 file changed, 19 insertions(+), 6 deletions(-) diff --git a/doc/user_guide/features/github_workflows/index.rst b/doc/user_guide/features/github_workflows/index.rst index abdbccf2a..8ba63cd48 100644 --- a/doc/user_guide/features/github_workflows/index.rst +++ b/doc/user_guide/features/github_workflows/index.rst @@ -146,13 +146,26 @@ CI Actions Dependency Update ^^^^^^^^^^^^^^^^^ -The ``dependency-update.yml`` workflow is used to resolve vulnerabilities by updating our project dependencies. +Once a week, the ``dependency-update.yml`` workflow runs on the default branch. Its +purpose is to keep project dependencies current and reduce risk from known +vulnerabilities by checking for insecure packages and automatically opening an update +pull request when the lock file changes. -It can be triggered manually and is also scheduled to run weekly. -The workflow first audits dependencies for known vulnerabilities. If vulnerabilities -are detected, it updates the dependencies using Poetry. When the ``poetry.lock`` is changed, -then it creates a pull request with the update. + +.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/dependency-update.yml + :language: yaml + :start-at: on: + :end-at: workflow_dispatch: + +The workflow first audits dependencies for known vulnerabilities: + +* If no vulnerabilities are detected, then no update is needed. +* If vulnerabilities are detected, it updates the dependencies using Poetry. + + * If the ``poetry.lock`` is unchanged, then no further action is taken. + * If the ``poetry.lock`` is changed, then it creates a branch, stages the commit, creates a pull request, and sends a Slack notification. + .. _pr_merge_yml: @@ -173,7 +186,7 @@ When a pull request is merged to main, then the ``pr-merge.yml`` workflow is act Periodic Validation ^^^^^^^^^^^^^^^^^^^ -Once a week, this `periodic-validation.yml` is triggered on the default branch. Its main +Once a week, this ``periodic-validation.yml`` is triggered on the default branch. Its main purpose is to ensure that our critical checks and tests continue to run, but it also sends the results of the linting tools and test coverage to Sonar for an overall report. From ca1c25b1354068b939f1e6c00b31dc6371674bc4 Mon Sep 17 00:00:00 2001 From: Ariel Schulz Date: Tue, 12 May 2026 10:09:33 +0200 Subject: [PATCH 2/7] Restructure documentation to make clearer the general types of workflows we support --- .../features/github_workflows/index.rst | 179 ++++++++++-------- 1 file changed, 98 insertions(+), 81 deletions(-) diff --git a/doc/user_guide/features/github_workflows/index.rst b/doc/user_guide/features/github_workflows/index.rst index 8ba63cd48..7220728ca 100644 --- a/doc/user_guide/features/github_workflows/index.rst +++ b/doc/user_guide/features/github_workflows/index.rst @@ -25,8 +25,9 @@ workflows from the templates. .. attention:: - In most cases, we recommend using _all_ workflows without change to ensure - consistent interdependencies. For more details, see :ref:`ci_actions`. + In most cases, we recommend using **all** workflows without change to ensure + consistent interdependencies. For more details, see :ref:`manual_activities` + and :ref:`automated_activities`. Workflows @@ -136,92 +137,27 @@ and is maintained by the PTB and what is project-specific. tests. It's encouraged to add to this workflow extension additional approval requests, similar to ``approve-run-slow-tests``. -.. _ci_actions: -CI Actions ----------- -.. _dependency_update: - -Dependency Update -^^^^^^^^^^^^^^^^^ - -Once a week, the ``dependency-update.yml`` workflow runs on the default branch. Its -purpose is to keep project dependencies current and reduce risk from known -vulnerabilities by checking for insecure packages and automatically opening an update -pull request when the lock file changes. - - - -.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/dependency-update.yml - :language: yaml - :start-at: on: - :end-at: workflow_dispatch: - -The workflow first audits dependencies for known vulnerabilities: - -* If no vulnerabilities are detected, then no update is needed. -* If vulnerabilities are detected, it updates the dependencies using Poetry. +.. _manual_activities: - * If the ``poetry.lock`` is unchanged, then no further action is taken. - * If the ``poetry.lock`` is changed, then it creates a branch, stages the commit, creates a pull request, and sends a Slack notification. - - -.. _pr_merge_yml: +Manual Activities +----------------- -Merge -^^^^^ - -When a pull request is merged to main, then the ``pr-merge.yml`` workflow is activated. - -.. mermaid:: - :name: merge-diagram +The following workflows are typically triggered directly by a developer during normal +project maintenance. - graph TD - %% Workflow Triggers (Solid Lines) - pr-merge[pr-merge.yml] --> publish-docs[gh-pages.yml] - -.. _periodic_validation_yml: +.. _ci_yml: -Periodic Validation -^^^^^^^^^^^^^^^^^^^ +Create a PR +^^^^^^^^^^^ -Once a week, this ``periodic-validation.yml`` is triggered on the default branch. Its main -purpose is to ensure that our critical checks and tests continue to run, but it also -sends the results of the linting tools and test coverage to Sonar for an overall report. +When any pull request is modified, then the ``ci.yml`` will be triggered. -.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/periodic-validation.yml +.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/ci.yml :language: yaml - :start-at: schedule: - :end-at: - cron: "0 0 * * 6" - -.. mermaid:: - - graph TD - %% Define Nodes - checks[checks.yml] - periodic_validation[periodic-validation.yml] - fast-tests[fast-tests.yml] - slow_checks[slow-checks.yml] - report[report.yml] - - %% Workflow Triggers - periodic_validation --> checks - periodic_validation --> fast-tests - periodic_validation --> slow_checks - - %% Dependencies - checks -.->|needs| report - fast-tests -.->|needs| report - slow_checks -.->|needs| report - -.. _ci_yml: - -Pull Request -^^^^^^^^^^^^ - -When any pull request is opened, synchronized, or reopened, then the ``ci.yml`` will be -triggered. + :start-at: on: + :end-at: types: [opened, synchronize, reopened] When configured as described on :ref:`github_project_configuration`, the ``run-slow-tests`` job requires a developer to manually approve executing the slower @@ -277,10 +213,25 @@ then the subsequent jobs will not be started. style approver fill:#fff,stroke:#333,stroke-dasharray: 5 5 style slow_run fill:#fff,stroke:#333,stroke-dasharray: 5 5 +.. _pr_merge_yml: + +Merge a PR +^^^^^^^^^^ + +When you merge a pull request to the default branch, then the ``pr-merge.yml`` workflow is +activated. + +.. mermaid:: + :name: merge-diagram + + graph TD + %% Workflow Triggers (Solid Lines) + pr-merge[pr-merge.yml] --> publish-docs[gh-pages.yml] + .. _cd_yml: -Release -^^^^^^^ +Create a Release +^^^^^^^^^^^^^^^^ When the nox session ``release:trigger`` is used, a new tag is created & pushed to main. This starts the release process by activating the ``cd.yml`` workflow. @@ -295,3 +246,69 @@ to main. This starts the release process by activating the ``cd.yml`` workflow. %% Dependencies / Waiting (Dotted Lines) check-release-tag -.->|needs| build-and-publish[build-and-publish.yml] build-and-publish -.->|needs| gh-pages[gh-pages.yml] + +.. _automated_activities: + +Automated Activities +-------------------- + +The following workflows are triggered automatically by schedules. + +.. _dependency_update: + +Update Dependencies +^^^^^^^^^^^^^^^^^^^ + +The ``dependency-update.yml`` workflow runs on the default branch. Its +purpose is to keep project dependencies up-to-date and reduce risk from known +vulnerabilities by checking for insecure packages and automatically opening an update +pull request. + +.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/dependency-update.yml + :language: yaml + :start-at: on: + :end-at: workflow_dispatch: + +The workflow first audits dependencies for known vulnerabilities: + +* If no vulnerabilities are detected, then no update is needed. +* If vulnerabilities are detected, it updates the dependencies using ``poetry update``. + + * If the ``poetry.lock`` is unchanged, then no further action is taken. + * If the ``poetry.lock`` is changed, then it creates a branch, stages the commit, + creates a pull request, and sends a Slack notification. + +.. _periodic_validation_yml: + +Validate Periodically +^^^^^^^^^^^^^^^^^^^^^ + +Once a week, this ``periodic-validation.yml`` is triggered on the default branch. Its +main purpose is to ensure that our critical checks and tests continue to run, but it +also sends the results of the linting tools and test coverage to Sonar for an overall +report. + +.. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/periodic-validation.yml + :language: yaml + :start-at: schedule: + :end-at: - cron: "0 0 * * 6" + +.. mermaid:: + + graph TD + %% Define Nodes + checks[checks.yml] + periodic_validation[periodic-validation.yml] + fast-tests[fast-tests.yml] + slow_checks[slow-checks.yml] + report[report.yml] + + %% Workflow Triggers + periodic_validation --> checks + periodic_validation --> fast-tests + periodic_validation --> slow_checks + + %% Dependencies + checks -.->|needs| report + fast-tests -.->|needs| report + slow_checks -.->|needs| report From 7c6e27a4086cf181365cd1cce95be53d1e260ca4 Mon Sep 17 00:00:00 2001 From: Ariel Schulz Date: Tue, 12 May 2026 10:11:23 +0200 Subject: [PATCH 3/7] Add user caution --- doc/user_guide/features/github_workflows/index.rst | 2 ++ 1 file changed, 2 insertions(+) diff --git a/doc/user_guide/features/github_workflows/index.rst b/doc/user_guide/features/github_workflows/index.rst index 7220728ca..72820bd67 100644 --- a/doc/user_guide/features/github_workflows/index.rst +++ b/doc/user_guide/features/github_workflows/index.rst @@ -278,6 +278,8 @@ The workflow first audits dependencies for known vulnerabilities: * If the ``poetry.lock`` is changed, then it creates a branch, stages the commit, creates a pull request, and sends a Slack notification. +Afterwards, users need to perform some manual steps which are described in the PR description. + .. _periodic_validation_yml: Validate Periodically From 59e5b28f2664a78ff6d2b9b61060f8a5c81dee3e Mon Sep 17 00:00:00 2001 From: Ariel Schulz Date: Tue, 12 May 2026 10:28:34 +0200 Subject: [PATCH 4/7] Modify PR description --- .../templates/github/workflows/dependency-update.yml | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-) diff --git a/exasol/toolbox/templates/github/workflows/dependency-update.yml b/exasol/toolbox/templates/github/workflows/dependency-update.yml index c6783bfd8..5bc1b4b2f 100644 --- a/exasol/toolbox/templates/github/workflows/dependency-update.yml +++ b/exasol/toolbox/templates/github/workflows/dependency-update.yml @@ -91,9 +91,13 @@ jobs: BASE_BRANCH=$(gh repo view --json defaultBranchRef -q .defaultBranchRef.name) PR_BODY="Automated dependency update for \`poetry.lock\`. - This PR was created by the dependency update workflow after running: - - \`poetry run -- nox -s dependency:audit\` - - \`poetry update\`" + This PR was created by the dependency update workflow after running the \`dependency-update.yml\` + + Please perform the following actions on a locally checked out branch: + - [ ] Execute \`poetry run -- nox -s workflow:generate -- --all\` + - [ ] Use \`poetry run -- nox -s dependency:audit\` to check for vulnerabilities requiring manual action + - [ ] Add a change entry to the \`unreleased.md\` + " PR_URL=$(gh pr create \ --base "$BASE_BRANCH" \ From 23158189187c935c7e773f04c3125262794dfb8c Mon Sep 17 00:00:00 2001 From: Ariel Schulz Date: Tue, 12 May 2026 10:30:07 +0200 Subject: [PATCH 5/7] Fix unreleased.md --- doc/changes/unreleased.md | 12 +++++------- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/doc/changes/unreleased.md b/doc/changes/unreleased.md index 82a9043e2..b95a549eb 100644 --- a/doc/changes/unreleased.md +++ b/doc/changes/unreleased.md @@ -19,16 +19,14 @@ details, check out the [Workflow Extensions](https://exasol.github.io/python-too ## Features -* #756: Added `dependency-update.yml` to automate resolving vulnerabilities with a generated pull request - -## Bugfix - -* #563: Fixed merge-gate to prevent auto-merges from happening when integration tests failed - -## Feature * #829: Extended removing a job from a workflow to also remove it from the `needs` of another job * #825: Created two workflows by splitting up previous ones: * Moved the periodic jobs in `ci.yml` to its own `periodic-validation.yml` * Moved the unit tests job in `checks.yml` to its own `fast-tests.yml` * #730: Added workflow extensions to `fast-tests` and `merge-gate` +* #756: Added `dependency-update.yml` to automate resolving vulnerabilities with a generated pull request + +## Bugfix + +* #563: Fixed merge-gate to prevent auto-merges from happening when integration tests failed From 92370db3ff2b6424ef28e965555f175ef885b088 Mon Sep 17 00:00:00 2001 From: Ariel Schulz Date: Tue, 12 May 2026 10:30:32 +0200 Subject: [PATCH 6/7] Add changelog entry --- doc/changes/unreleased.md | 1 + 1 file changed, 1 insertion(+) diff --git a/doc/changes/unreleased.md b/doc/changes/unreleased.md index b95a549eb..82dc04fb6 100644 --- a/doc/changes/unreleased.md +++ b/doc/changes/unreleased.md @@ -26,6 +26,7 @@ details, check out the [Workflow Extensions](https://exasol.github.io/python-too * Moved the unit tests job in `checks.yml` to its own `fast-tests.yml` * #730: Added workflow extensions to `fast-tests` and `merge-gate` * #756: Added `dependency-update.yml` to automate resolving vulnerabilities with a generated pull request +* #792: Improved `dependency-update.yml` documentation ## Bugfix From 2b7edd2f2c06511164dc283455a4d4214f3a9b34 Mon Sep 17 00:00:00 2001 From: Ariel Schulz Date: Tue, 12 May 2026 10:56:42 +0200 Subject: [PATCH 7/7] Apply changes from review --- .../features/github_workflows/index.rst | 18 +++++++----------- .../github/workflows/dependency-update.yml | 4 ++-- 2 files changed, 9 insertions(+), 13 deletions(-) diff --git a/doc/user_guide/features/github_workflows/index.rst b/doc/user_guide/features/github_workflows/index.rst index 72820bd67..c763d1925 100644 --- a/doc/user_guide/features/github_workflows/index.rst +++ b/doc/user_guide/features/github_workflows/index.rst @@ -152,7 +152,7 @@ project maintenance. Create a PR ^^^^^^^^^^^ -When any pull request is modified, then the ``ci.yml`` will be triggered. +Modifying any pull request triggers the workflow ``ci.yml``. .. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/ci.yml :language: yaml @@ -218,8 +218,7 @@ then the subsequent jobs will not be started. Merge a PR ^^^^^^^^^^ -When you merge a pull request to the default branch, then the ``pr-merge.yml`` workflow is -activated. +Merging a pull request to the default branch, activates the ``pr-merge.yml`` workflow. .. mermaid:: :name: merge-diagram @@ -259,10 +258,8 @@ The following workflows are triggered automatically by schedules. Update Dependencies ^^^^^^^^^^^^^^^^^^^ -The ``dependency-update.yml`` workflow runs on the default branch. Its -purpose is to keep project dependencies up-to-date and reduce risk from known -vulnerabilities by checking for insecure packages and automatically opening an update -pull request. +The ``dependency-update.yml`` workflow runs on the default branch. It +checks for known vulnerabilities and tries to fix them by updating dependencies. .. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/dependency-update.yml :language: yaml @@ -285,10 +282,9 @@ Afterwards, users need to perform some manual steps which are described in the P Validate Periodically ^^^^^^^^^^^^^^^^^^^^^ -Once a week, this ``periodic-validation.yml`` is triggered on the default branch. Its -main purpose is to ensure that our critical checks and tests continue to run, but it -also sends the results of the linting tools and test coverage to Sonar for an overall -report. +The ``periodic-validation.yml`` runs on the default branch. It ensures that critical +checks and tests continue to run and sends the results of the linting tools and test +coverage to Sonar for an overall report. .. literalinclude:: ../../../../exasol/toolbox/templates/github/workflows/periodic-validation.yml :language: yaml diff --git a/exasol/toolbox/templates/github/workflows/dependency-update.yml b/exasol/toolbox/templates/github/workflows/dependency-update.yml index 5bc1b4b2f..1419bff35 100644 --- a/exasol/toolbox/templates/github/workflows/dependency-update.yml +++ b/exasol/toolbox/templates/github/workflows/dependency-update.yml @@ -91,12 +91,12 @@ jobs: BASE_BRANCH=$(gh repo view --json defaultBranchRef -q .defaultBranchRef.name) PR_BODY="Automated dependency update for \`poetry.lock\`. - This PR was created by the dependency update workflow after running the \`dependency-update.yml\` + This PR was created by the workflow \`dependency-update.yml\` Please perform the following actions on a locally checked out branch: - [ ] Execute \`poetry run -- nox -s workflow:generate -- --all\` - [ ] Use \`poetry run -- nox -s dependency:audit\` to check for vulnerabilities requiring manual action - - [ ] Add a change entry to the \`unreleased.md\` + - [ ] Update file \`doc/changes/unreleased.md\` " PR_URL=$(gh pr create \