Skip to content

refactor(runners): split runner provider layout#5203

Open
edersonbrilhante wants to merge 14 commits into
mainfrom
refactor-runner-provider-split
Open

refactor(runners): split runner provider layout#5203
edersonbrilhante wants to merge 14 commits into
mainfrom
refactor-runner-provider-split

Conversation

@edersonbrilhante

@edersonbrilhante edersonbrilhante commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Description

Refactors the existing EC2 runner implementation behind provider-specific boundaries while preserving EC2 as the only supported/default runner provider in this PR.

This prepares the codebase for future AWS runner providers without adding MicroVM support yet:

  • renames the control-plane AWS runner implementation to EC2-specific files (ec2-runners.ts, ec2-runners.d.ts)
  • adds provider interfaces and registries for scale-up, scale-down, and pool flows
  • moves EC2-specific scale-up, scale-down, pool, runner creation, and dynamic-label logic into EC2 modules while keeping orchestration code provider-neutral
  • extracts shared GitHub runner/JIT config flow and scale-runner types
  • decouples webhook dynamic-label dispatch through AWS/provider helpers and keeps the EC2 label policy implementation behind the EC2 provider
  • renames dynamic label policy configuration from ec2_dynamic_labels_policy / ec2DynamicLabelsPolicy to aws_dynamic_labels_policy / awsDynamicLabelsPolicy
  • wires Terraform defaults so scale-up and pool payloads carry the EC2 provider type where needed, while idle_config remains provider-neutral
  • updates configuration docs and generated module README tables
  • keeps the existing test file layout to reduce review noise

Test Plan

Replace your config to use aws_dynamic_labels_policy and run apply. Everything should work as today

Related Issues

N/A

@github-actions

github-actions Bot commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

Dependency Review

✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.

Scanned Files

None

@edersonbrilhante edersonbrilhante force-pushed the refactor-runner-provider-split branch from 37edc84 to ef4f7e7 Compare July 7, 2026 21:37
Comment thread lambdas/functions/webhook/src/runners/dispatch.ts
Comment thread lambdas/functions/webhook/src/runners/dispatch.ts
Comment thread lambdas/functions/control-plane/src/pool/pool.ts

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Refactors the runner orchestration code to be provider-based (while keeping EC2 as the only/default provider), splitting EC2-specific logic into dedicated modules and introducing provider registries for scale-up, scale-down, and pool management. This also renames the dynamic label policy configuration to be AWS/provider-scoped in preparation for future non-EC2 providers.

Changes:

  • Introduces provider interfaces/registries and moves EC2 scale-up, scale-down, pool, and dynamic-label handling behind EC2 provider implementations.
  • Renames dynamic label policy configuration from EC2-specific to AWS/provider-scoped (ec2_*aws_*) across Terraform and webhook config.
  • Adds explicit/default EC2 provider type wiring in scheduler/pool inputs and scale-up lambda environment.

Reviewed changes

Copilot reviewed 58 out of 60 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
variables.tf Adds provider type to idle config objects; renames dynamic label policy input to aws_dynamic_labels_policy; updates scale error description text.
README.md Regenerates input docs reflecting aws_dynamic_labels_policy, idle config type, and updated allocation strategy description.
main.tf Wires webhook matcher config to aws_dynamic_labels_policy.
docs/configuration.md Updates examples/docs for idle config provider type and renamed dynamic label policy variable/fields.
modules/webhook/variables.tf Renames matcher policy field to awsDynamicLabelsPolicy in module input schema/docs.
modules/webhook/README.md Regenerates module docs for awsDynamicLabelsPolicy.
modules/termination-watcher/termination/README.md Regenerates docs to include new policy resource listing.
modules/termination-watcher/notification/README.md Regenerates docs to include additional event/policy resources.
modules/runners/variables.tf Adds provider type to idle config objects; updates scale error description text.
modules/runners/scale-up.tf Adds explicit RUNNER_PROVIDER_TYPE=ec2 environment variable to scale-up lambda.
modules/runners/README.md Regenerates docs for idle config type, allocation strategy text, and new inputs.
modules/runners/pool/README.md Updates pool module config schema docs (adds instance type priorities).
modules/runners/pool/main.tf Adds type: "ec2" to scheduler input payload.
modules/multi-runner/variables.tf Adds provider type to idle config objects; renames matcher policy field to awsDynamicLabelsPolicy; updates scale error description text.
modules/multi-runner/README.md Regenerates docs reflecting idle config type and awsDynamicLabelsPolicy rename.
lambdas/functions/webhook/src/sqs/index.ts Renames matcher policy type usage to awsDynamicLabelsPolicy and introduces optional runner provider field on queue config.
lambdas/functions/webhook/src/runners/labels.ts Extracts/centralizes workflow label splitting + sanitization + label matching logic.
lambdas/functions/webhook/src/runners/ec2-dynamic-labels.ts Implements EC2-specific dynamic label queue selection using the new provider strategy pattern.
lambdas/functions/webhook/src/runners/ec2-dynamic-labels-policy.ts Refactors EC2 policy typing to reuse AWS policy types while retaining EC2 policy evaluation behavior.
lambdas/functions/webhook/src/runners/dynamic-labels-policy.test.ts Updates tests to reference EC2 policy module path after refactor.
lambdas/functions/webhook/src/runners/dispatch.ts Refactors webhook dispatch logic to use shared label helpers and provider-based dynamic label selection.
lambdas/functions/webhook/src/runners/dispatch.test.ts Updates dispatch tests for awsDynamicLabelsPolicy and adds coverage for default EC2 provider selection.
lambdas/functions/webhook/src/runners/aws-dynamic-labels.ts Adds provider-based dynamic label selection orchestration (currently only EC2 strategy registered).
lambdas/functions/webhook/src/runners/aws-dynamic-labels-provider.ts Defines the provider strategy interface for dynamic label queue selection.
lambdas/functions/webhook/src/runners/aws-dynamic-labels-policy.ts Introduces provider-agnostic AWS dynamic label policy type definitions.
lambdas/functions/control-plane/src/scale-runners/types.ts Extracts shared types from scale-up into a dedicated types module.
lambdas/functions/control-plane/src/scale-runners/ScaleError.ts Updates type import source to the new shared types module.
lambdas/functions/control-plane/src/scale-runners/ScaleError.test.ts Updates type import source to the new shared types module.
lambdas/functions/control-plane/src/scale-runners/scale-up.ts Refactors scale-up to use provider registry + GitHub runner helpers and EC2 provider implementation.
lambdas/functions/control-plane/src/scale-runners/scale-up-provider.ts Defines the scale-up provider interface and strategy creation contract.
lambdas/functions/control-plane/src/scale-runners/scale-up-provider-registry.ts Adds registry for scale-up provider strategies (EC2 default).
lambdas/functions/control-plane/src/scale-runners/scale-up-config.ts Adds helper for choosing scale-up provider type from env/default.
lambdas/functions/control-plane/src/scale-runners/scale-down.ts Refactors scale-down to use provider registry, provider runner abstraction, and GitHub runner helper utilities.
lambdas/functions/control-plane/src/scale-runners/scale-down.test.ts Updates scale-down tests for renamed EC2 runner modules and mocks.
lambdas/functions/control-plane/src/scale-runners/scale-down-provider.ts Introduces provider interface for listing/marking/terminating runners during scale-down.
lambdas/functions/control-plane/src/scale-runners/scale-down-provider-registry.ts Adds registry for scale-down provider strategies (EC2 default).
lambdas/functions/control-plane/src/scale-runners/scale-down-config.ts Adds optional provider type to scale-down config and logic to enforce a single provider type per config.
lambdas/functions/control-plane/src/scale-runners/scale-down-config.test.ts Adds unit tests for determining scale-down provider type.
lambdas/functions/control-plane/src/scale-runners/job-retry.ts Updates imports to use GitHub runner helpers and shared types.
lambdas/functions/control-plane/src/scale-runners/job-retry.test.ts Updates type imports to use the new shared types module.
lambdas/functions/control-plane/src/scale-runners/github-runner.ts Extracts shared GitHub runner registration/JIT/token logic from scale-up into reusable helper module.
lambdas/functions/control-plane/src/scale-runners/ec2.ts Extracts EC2-specific runner creation + metadata tagging into its own module.
lambdas/functions/control-plane/src/scale-runners/ec2-scale-up.ts Implements EC2 scale-up provider strategy including dynamic EC2 override label parsing integration.
lambdas/functions/control-plane/src/scale-runners/ec2-scale-down.ts Implements EC2 scale-down provider strategy (list/terminate/orphan tag operations).
lambdas/functions/control-plane/src/scale-runners/ec2-labels.ts Extracts EC2 override label parsing and launch template block device lookup into a dedicated module.
lambdas/functions/control-plane/src/pool/pool.ts Refactors pool adjuster to use provider registry + provider-specific logic and requires provider type in pool events.
lambdas/functions/control-plane/src/pool/pool.test.ts Updates pool tests to use provider-driven pool logic and adds tests for EC2 pool provider behavior.
lambdas/functions/control-plane/src/pool/pool-provider.ts Introduces provider interface for pool runner listing/counting/creation.
lambdas/functions/control-plane/src/pool/pool-provider-registry.ts Adds registry for pool provider strategies (EC2 default).
lambdas/functions/control-plane/src/pool/ec2-pool.ts Implements EC2 pool provider (list running runners, count “available”, create runners).
lambdas/functions/control-plane/src/modules.d.ts Adds optional RUNNER_PROVIDER_TYPE env var typing.
lambdas/functions/control-plane/src/local-pool.ts Updates local pool runner to include explicit type: 'ec2' in the event.
lambdas/functions/control-plane/src/lambda.ts Updates imports to use shared types and refactored scale-up module.
lambdas/functions/control-plane/src/lambda.test.ts Updates adjust pool tests to include pool event type.
lambdas/functions/control-plane/src/github/octokit.ts Updates type import source to shared types module.
lambdas/functions/control-plane/src/github/octokit.test.ts Updates type import source to shared types module.
lambdas/functions/control-plane/src/aws/runners.test.ts Updates imports to renamed EC2 runner modules and shared types.
lambdas/functions/control-plane/src/aws/ec2-runners.ts Updates internal type import path for renamed EC2 runners declarations.
lambdas/functions/control-plane/src/aws/ec2-runners.d.ts Updates type import source to shared types module.
Comments suppressed due to low confidence (1)

modules/webhook/variables.tf:40

  • The runner_matcher_config validation currently uses try(var.runner_matcher_config.matcherConfig.priority, 999), but runner_matcher_config is a map so this expression errors and try(...) always falls back to 999. As a result, invalid priority values in individual matcher entries won’t be validated. Consider validating over all map values instead.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread lambdas/functions/webhook/src/sqs/index.ts
Comment thread lambdas/functions/webhook/src/runners/ec2-dynamic-labels.ts
Comment thread variables.tf
@edersonbrilhante edersonbrilhante marked this pull request as ready for review July 7, 2026 22:48
@edersonbrilhante edersonbrilhante requested review from a team as code owners July 7, 2026 22:48

@Brend-Smits Brend-Smits left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the idea for this, I need a bit more time to review the changes though.
Have you been running this change in your environment?

My reviews are a little bit more delayed at the moment due to my vacation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants