Skip to content

[rb] streamline tests on github actions runners#17550

Draft
titusfortner wants to merge 8 commits into
trunkfrom
ci-ruby
Draft

[rb] streamline tests on github actions runners#17550
titusfortner wants to merge 8 commits into
trunkfrom
ci-ruby

Conversation

@titusfortner
Copy link
Copy Markdown
Member

🔗 Related Issues

Ruby implementation of #17539
Supersedes #16813

💥 What does this PR do?

  • Passes affected targets from ci.yml to ci-ruby.yml and the OS jobs run all affected with filters as described in the linked issue.
  • OS tests include both integration and unit tests, so unit test matrix can be significantly reduced
  • Relying on build target to happen on RBE and if there is an mri issue we'll see it in nightly run
  • Removes processing commit messages and PR titles for [rb] to decide what to run for a PR, everything is gated off affected targets evaluated

🔧 Implementation Notes

These OS tests will include multiple browsers, but need to specify browser name as safari to get safaridriver setup, and browser name can't be blank so I used true

🤖 AI assistance

  • No substantial AI assistance used
  • AI assisted (complete below)
    • Tool(s):
    • What was generated:
    • I reviewed all AI output and can explain the change

💡 Additional Considerations

Currently running local and remote, considering limiting it to just local, especially for safari. I'll time the runs to compare

@selenium-ci selenium-ci added C-rb Ruby Bindings B-build Includes scripting, bazel and CI integrations labels May 22, 2026
@qodo-code-review
Copy link
Copy Markdown
Contributor

Review Summary by Qodo

Streamline Ruby CI tests on GitHub Actions runners

✨ Enhancement 🧪 Tests

Grey Divider

Walkthroughs

Description
• Streamline Ruby CI by passing affected targets from main workflow
• Reduce unit test matrix by relying on OS-specific integration tests
• Replace commit message/PR title parsing with target-based filtering
• Consolidate browser-specific tests into unified OS test matrix
• Update test tags from no-sandbox to semantic labels (unit, lint)
Diagram
flowchart LR
  A["ci.yml<br/>read-targets"] -->|"rb_targets"| B["ci-ruby.yml<br/>workflow_call"]
  B --> C["unit-tests<br/>reduced matrix"]
  B --> D["os-tests<br/>ubuntu/windows/macos"]
  D --> E["browser-specific<br/>tag filters"]
  F["rb/spec/tests.bzl<br/>rb_unit_test"] -->|"tags: unit"| C
  G["rb/BUILD.bazel<br/>lint"] -->|"tags: lint"| B

Loading

File Changes

1. .github/workflows/ci-ruby.yml ⚙️ Configuration changes +23/-35

Refactor CI workflow with target-based filtering

• Add targets workflow input parameter with default value //rb/...
• Remove standalone build job and consolidate into other jobs
• Reduce unit test matrix from 5 configurations to 2 (Ruby 4.0.4 and truffleruby)
• Replace integration-tests-local with unified os-tests job using tag filters
• Update test command to use inputs.targets and tag-based filtering instead of browser filters
• Add exit code handling for test execution (exit 0 on code 4)

.github/workflows/ci-ruby.yml


2. .github/workflows/ci.yml ✨ Enhancement +11/-3

Extract and pass Ruby targets to workflow

• Rename output variable from rb to rb_targets
• Add process_binding function to extract language-specific targets from affected targets
• Replace check_binding for Ruby with process_binding to capture actual target paths
• Pass extracted rb_targets to ci-ruby.yml workflow via with clause

.github/workflows/ci.yml


3. rb/spec/tests.bzl ⚙️ Configuration changes +1/-1

Update unit test tag to semantic label

• Change rb_unit_test tag from no-sandbox to unit for semantic clarity
• Remove TODO comment about sandbox requirement

rb/spec/tests.bzl


View more (2)
4. rb/BUILD.bazel ⚙️ Configuration changes +1/-1

Update lint tag to semantic label

• Change lint job tag from no-sandbox to lint for consistency

rb/BUILD.bazel


5. .skipped-tests Miscellaneous +0/-1

Remove obsolete skipped test entry

• Remove one skipped test entry for Ruby BiDi browsing context test with Firefox beta

.skipped-tests


Grey Divider

Qodo Logo

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Code Review by Qodo

🐞 Bugs (5) 📘 Rule violations (2) 📎 Requirement gaps (0)

Grey Divider


Action required

1. Invalid matrix.tag-filters access 📘 Rule violation ☼ Reliability
Description
The workflow references matrix.tag-filters, but matrix keys containing - cannot be accessed with
dot-notation, causing expression evaluation failure or empty --test_tag_filters and potentially
running the wrong test set. This makes CI behavior non-deterministic and undermines safe CI gating.
Code

.github/workflows/ci-ruby.yml[76]

Evidence
PR Compliance ID 14 requires hardening CI/scripts to be robust and deterministic. The workflow uses
a hyphenated matrix key (tag-filters) but references it via matrix.tag-filters, which is not a
safe/deterministic GitHub Actions expression access pattern for such keys, risking misconfigured
Bazel arguments.

.github/workflows/ci-ruby.yml[53-77]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
`.github/workflows/ci-ruby.yml` defines a matrix key named `tag-filters`, but later references it as `${{ matrix.tag-filters }}`. In GitHub Actions expressions, keys with hyphens must be accessed using bracket notation (e.g., `matrix['tag-filters']`) or renamed to an underscore key.

## Issue Context
If this expression fails (or resolves unexpectedly), the `--test_tag_filters=...` argument may be empty/incorrect, leading to incorrect test selection and unreliable CI outcomes.

## Fix Focus Areas
- .github/workflows/ci-ruby.yml[53-77]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


2. Unvalidated targets shell injection ✓ Resolved 📘 Rule violation ⛨ Security
Description
workflow_dispatch introduces a user-provided targets input that is later interpolated into a
shell command without validation/quoting, enabling command injection and unsafe argument splitting.
This violates the CI/script hardening requirement and can lead to unexpected command execution or
incorrect Bazel invocation.
Code

.github/workflows/ci-ruby.yml[R10-15]

Evidence
PR Compliance ID 14 requires safe argument handling in CI scripts. The PR adds a manually-supplied
targets input and later expands ${{ inputs.targets }} directly into a shell command, which is
unsafe because the expanded text can be interpreted by the shell as additional commands/flags.

.github/workflows/ci-ruby.yml[10-15]
.github/workflows/ci-ruby.yml[73-80]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
A `workflow_dispatch` string input (`inputs.targets`) is user-controlled and later inserted directly into a `bazel test` shell command, allowing shell metacharacters (e.g., `;`, `&&`) to be interpreted as commands.

## Issue Context
This workflow now accepts `targets` via manual dispatch. The value is used in the `os-tests` job’s `run` script, so it must be handled as data (arguments) rather than executable shell text.

## Fix Focus Areas
- .github/workflows/ci-ruby.yml[10-15]
- .github/workflows/ci-ruby.yml[73-80]

## Suggested approach
- Read `inputs.targets` into a variable inside quotes.
- Split into a bash array safely (e.g., `read -r -a targets_arr <<< "$targets"`).
- Invoke Bazel with `"${targets_arr[@]}"` so each target is a separate argument and shell metacharacters are not executed.
- Optionally validate that each entry matches expected Bazel target patterns (e.g., starts with `//`).

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


3. IE remote tests included ✓ Resolved 🐞 Bug ≡ Correctness
Description
The Windows os-tests job uses --test_tag_filters=edge,edge-remote,unit,skip-rbe,-ie, but IE
remote targets are tagged ie-remote (not ie) and also include skip-rbe, so they can still be
selected and executed on Windows despite the -ie exclusion.
Code

.github/workflows/ci-ruby.yml[59]

Evidence
The workflow’s Windows tag filter includes skip-rbe and only excludes ie. In the Ruby test rule
generator, IE is defined with the skip-rbe tag, and remote tests are tagged as {browser}-remote
while inheriting the browser tags. Also, some integration tests do not override the default
browsers=BROWSERS.keys(), so IE/IE-remote targets exist and are eligible for selection.

.github/workflows/ci-ruby.yml[54-62]
rb/spec/tests.bzl[130-141]
rb/spec/tests.bzl[170-171]
rb/spec/tests.bzl[195-215]
rb/spec/integration/selenium/webdriver/BUILD.bazel[30-37]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
The Windows `os-tests` matrix entry tries to exclude IE by adding `-ie`, but IE remote targets use the tag `ie-remote` and inherit `skip-rbe`, so they still match the positive filters.

### Issue Context
`rb_integration_test` generates both local and `*-remote` test targets, and remote targets inherit the browser-specific tag list (including `skip-rbe` for IE).

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[57-60]

### Suggested fix
Update the Windows `tag-filters` to also exclude the remote IE tag, e.g.:
- `edge,edge-remote,unit,skip-rbe,-ie,-ie-remote`

(Alternatively, remove `skip-rbe` from the Windows include list if it is not intended to pull in unrelated `skip-rbe` tests.)

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View more (5)
4. Boolean passed as browser ✓ Resolved 🐞 Bug ≡ Correctness
Description
ci-ruby.yml sets matrix.browser: true (boolean) and passes it into bazel.yml where browser is
declared as a string input; this can break reusable-workflow input validation or lead to unexpected
conditional behavior (inputs.browser != '').
Code

.github/workflows/ci-ruby.yml[R54-59]

Evidence
ci-ruby sets browser: true in the matrix (boolean) and forwards it as `browser: ${{ matrix.browser
}}. bazel.yml declares browser as a string input and uses it in conditions like inputs.browser
!= ''`, so passing a boolean is unsafe.

.github/workflows/ci-ruby.yml[52-66]
.github/workflows/bazel.yml[19-28]
.github/workflows/bazel.yml[252-267]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`matrix.browser` is set to YAML boolean `true` for ubuntu/windows, then forwarded to reusable workflow input `browser` which is typed as `string`. This risks workflow-call validation errors and also changes behavior of steps gated on `inputs.browser != ''`.

### Issue Context
The called workflow declares `browser` as `type: string`.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[52-66]
- .github/workflows/bazel.yml[19-28]

### Suggested fix
- Change `browser: true` to a string value:
 - If you want “no specific browser but enable UI setup”, use `browser: 'enabled'` (string) and update bazel.yml docs accordingly.
 - If you actually intend Edge on Windows, set `browser: 'edge'`.
 - If Ubuntu is unit-only, set `browser: ''` and adjust tag filters accordingly.
- Keep macOS as `browser: safari` for safaridriver enablement.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


5. Dispatch missing targets input ✓ Resolved 🐞 Bug ≡ Correctness
Description
ci-ruby.yml uses ${{ inputs.targets }} in the os-tests Bazel command, but targets is only
defined under workflow_call; a manual workflow_dispatch run can expand to an empty target list
and cause bazel test to fail.
Code

.github/workflows/ci-ruby.yml[R74-76]

Evidence
The workflow defines targets only for workflow_call (lines 4-9) but the os-tests run command
always interpolates ${{ inputs.targets }} (line 75).

.github/workflows/ci-ruby.yml[3-11]
.github/workflows/ci-ruby.yml[47-76]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`os-tests` references `inputs.targets`, but `targets` is only declared for `workflow_call`. For `workflow_dispatch` runs, this input is not defined, so the Bazel command may receive no explicit targets.

### Issue Context
The workflow declares `workflow_dispatch:` with no inputs, but still relies on `inputs.targets`.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[3-11]
- .github/workflows/ci-ruby.yml[47-76]

### Suggested fix
Add `workflow_dispatch.inputs.targets` mirroring the `workflow_call` input (same default `//rb/...`), or change the command to fall back to `//rb/...` when `inputs.targets` is empty.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


6. Bazel exit code 4 ignored ✓ Resolved 📘 Rule violation ☼ Reliability
Description
The os-tests job treats bazel test exit code 4 as success, which can mask misconfiguration
(e.g., filters producing “no tests found”) and allow CI to go green without running any Ruby tests.
This violates the requirement to avoid error-swallowing constructs in CI/scripts that hide failures.
Code

.github/workflows/ci-ruby.yml[R74-76]

Evidence
PR Compliance ID 15 forbids error-swallowing in CI/scripts that can hide failures. The added shell
handler explicitly exits 0 when bazel test returns code 4, masking “no tests found”
conditions.

.github/workflows/ci-ruby.yml[74-76]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
The `os-tests` Bazel invocation converts exit code `4` into success (`exit 0`). This can hide “no tests found” situations caused by incorrect target selection or tag filters, resulting in false-green CI.

## Issue Context
`bazel test` exit code `4` typically indicates that no tests were executed. Treating this as success in PR CI can allow Ruby changes to merge without test coverage.

## Fix Focus Areas
- .github/workflows/ci-ruby.yml[74-76]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


7. macOS runs safari-preview tests 🐞 Bug ☼ Reliability
Description
The macOS os-tests job includes skip-rbe as a positive --test_tag_filters value, which matches
safari-preview tests because they carry the skip-rbe tag; the workflow only performs Safari setup
when inputs.browser == 'safari', so safari-preview targets may run without intended setup.
Code

.github/workflows/ci-ruby.yml[R60-62]

Evidence
macOS tag filters include skip-rbe. In Ruby test definitions, safari-preview is tagged with
skip-rbe and is macOS-compatible, and safari integration BUILD files explicitly request
safari-preview targets. Meanwhile, the runner setup only enables Safari when the workflow input is
exactly safari.

.github/workflows/ci-ruby.yml[47-76]
rb/spec/tests.bzl[142-167]
rb/spec/integration/selenium/webdriver/safari/BUILD.bazel[3-13]
.github/workflows/bazel.yml[265-267]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`skip-rbe` is included as a positive tag filter on macOS. Ruby safari-preview targets are tagged `skip-rbe`, so they can be selected even though the job is intended to run Safari.

### Issue Context
- `safari-preview` browser definition includes `skip-rbe` and is macOS-compatible.
- Some integration specs explicitly generate both `safari` and `safari-preview` targets.
- GitHub Actions setup only enables safaridriver when `inputs.browser == 'safari'`.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[52-63]

### Suggested fix
- Remove `skip-rbe` from macOS `tag-filters`, or
- Add an explicit exclusion: `-safari-preview,-safari-preview-remote` (and `-safari-preview-bidi` if applicable).
- Optionally, if you *do* want safari-preview coverage, add explicit setup steps and set `browser: safari-preview` so bazel.yml can configure it.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


8. grep exits read-targets ✓ Resolved 🐞 Bug ☼ Reliability
Description
In ci.yml, process_binding assigns lang_targets via a pipeline containing grep; when there are no
matching Ruby targets, grep exits non-zero and can fail the entire “Read targets” step instead of
simply producing no rb_targets output.
Code

.github/workflows/ci.yml[R89-94]

Evidence
The process_binding() function uses a grep -E pipeline to compute lang_targets (line 92). If
no targets match ^//rb[:/], grep returns non-zero; without || true this can fail the step
rather than just leaving rb_targets unset.

.github/workflows/ci.yml[81-100]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`process_binding()` uses `grep` inside a command substitution without guarding for the “no matches” exit code. When no Ruby targets exist, this can terminate the step early and fail CI.

### Issue Context
The workflow should continue and simply omit `rb_targets` when there are no matching targets.

### Fix Focus Areas
- .github/workflows/ci.yml[81-100]

### Suggested fix
- Make the pipeline tolerant of no matches, e.g. append `|| true` to the *whole* pipeline, or use `grep ... || :`.
 - Example:
   - `lang_targets=$(echo "$targets" | tr ' ' '\n' | grep -E "^${pattern}[:/]" | tr '\n' ' ' | sed 's/ *$//' || true)`
- Consider switching `echo` to `printf '%s\n' "$targets"` to avoid echo edge cases.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools



Remediation recommended

9. Broken Ruby lint instructions 🐞 Bug ⚙ Maintainability
Description
The Bazel test target //rb:lint was removed, but the README still instructs running `bazel test
//rb:lint and claims bazel test //rb/...` runs lint. This will now fail (missing target) and
misleads contributors about how to run Ruby linting.
Code

rb/BUILD.bazel[R213-215]

Evidence
The Ruby BUILD file no longer defines a lint test target, while the README still documents `bazel
test //rb:lint` as a supported command.

rb/BUILD.bazel[192-221]
README.md[435-445]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`//rb:lint` no longer exists (the lint `rb_test` wrapper was removed), but the README Ruby command table still references `bazel test //rb:lint` and implies `bazel test //rb/...` runs lint.

### Issue Context
Lint is now driven via Bazel `run` on `//rb:rubocop` (and the CI job uses `./go rb:lint`). Documentation should reflect the supported invocation.

### Fix Focus Areas
- rb/BUILD.bazel[192-221]
- README.md[435-445]

### Suggested fix
- Update README Ruby section:
 - Replace `bazel test //rb:lint` with `bazel run //rb:rubocop` and/or `./go rb:lint`.
 - Update `bazel test //rb/...` description to remove “and lint” (or reintroduce a `rb_test(name="lint")` wrapper if you want `bazel test` to run it).
- Ensure any other docs referring to `//rb:lint` are updated similarly.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


10. IE default browser expansion 🐞 Bug ☼ Reliability
Description
rb_integration_test now defaults to browsers=BROWSERS.keys(), which includes "ie", so any
BUILD.bazel that omits an explicit browsers list will start generating *-ie and *-ie-remote test
targets. This likely widens execution/analysis scope unintentionally (DEFAULT_BROWSERS excluding IE
is still defined but no longer used).
Code

rb/spec/tests.bzl[R171-172]

Evidence
The default browser set now includes IE, and multiple BUILD files omit an explicit browsers list so
they will inherit that new default; the prior IE-excluding default list still exists but is unused,
suggesting an unintended behavior change.

rb/spec/tests.bzl[169-193]
rb/spec/integration/selenium/webdriver/remote/BUILD.bazel[3-9]
rb/spec/integration/selenium/webdriver/bidi/BUILD.bazel[3-16]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`rb_integration_test` changed its default `browsers` parameter from `DEFAULT_BROWSERS` (which excludes IE) to `BROWSERS.keys()` (which includes IE). This causes IE test variants to be generated across packages that don’t explicitly pass `browsers`, expanding the test matrix and leaving `DEFAULT_BROWSERS` unused.

### Issue Context
Several integration BUILD files call `rb_integration_test(...)` without specifying `browsers`, so they inherit the new default.

### Fix Focus Areas
- rb/spec/tests.bzl[169-193]
- rb/spec/integration/selenium/webdriver/remote/BUILD.bazel[3-9]
- rb/spec/integration/selenium/webdriver/bidi/BUILD.bazel[3-16]

### Suggested fix
1. Change the default parameter back to `DEFAULT_BROWSERS` (or explicitly define a new default list that excludes IE if that’s the intent).
2. If any specific suites truly need IE coverage, pass `browsers=["ie", ...]` explicitly in those BUILD files instead of enabling it globally.
3. Remove or repurpose `DEFAULT_BROWSERS` if you intend to keep the new behavior, so the code doesn’t carry misleading dead constants.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


11. Re-enabled skipped Ruby BiDi 🐞 Bug ☼ Reliability
Description
The Ruby target //rb/spec/integration/selenium/webdriver/bidi:browsing_context-firefox-beta-bidi
was removed from .skipped-tests, so it will now run in the RBE CI invocation that consumes this
file. This is a behavior change for CI stability/coverage and should be done only if the test is now
reliably passing.
Code

.skipped-tests[26]

Evidence
ci-build.sh passes .skipped-tests as negative target patterns to Bazel; removing an entry means
Bazel will no longer exclude it. The Ruby BiDi package still defines BiDi-tagged integration tests
that produce the referenced *-bidi variants.

scripts/github-actions/ci-build.sh[18-25]
.skipped-tests[1-25]
rb/spec/integration/selenium/webdriver/bidi/BUILD.bazel[3-16]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
A Ruby BiDi target was removed from `.skipped-tests`, which is used to exclude known-problematic tests from the RBE CI run (`scripts/github-actions/ci-build.sh`). This change will cause the target to start running again in CI.

### Issue Context
If the skip was removed because the underlying issue is fixed, consider adding a reference (issue/PR) or ensuring the fix is part of this PR/series.

### Fix Focus Areas
- scripts/github-actions/ci-build.sh[18-25]
- .skipped-tests[1-25]

### Suggested fix
- If the test is still flaky/failing on RBE, restore the `-//rb/spec/integration/selenium/webdriver/bidi:browsing_context-firefox-beta-bidi` entry.
- If it is intentionally re-enabled, add a brief note in the PR description (or adjacent docs) explaining why it’s safe to remove and what changed to make it stable.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


View more (2)
12. Ungated presses pointer twice 📘 Rule violation ☼ Reliability
Description
The ActionBuilder integration tests it 'presses pointer twice' and it 'sends keys to element'
were changed to run unconditionally, no longer excluding Safari/Safari Preview, which can cause
cross-browser CI failures when Actions/double-click or Actions+sendKeys behavior is not consistently
supported by a given browser/driver. These tests should be conditionally gated/skipped based on
runtime/browser feature support where unsupported.
Code

rb/spec/integration/selenium/webdriver/action_builder_spec.rb[162]

Evidence
PR Compliance ID 13 requires guarding tests by runtime/browser feature support to avoid
cross-environment failures. In rb/spec/integration/selenium/webdriver/action_builder_spec.rb, the
modified tests at lines 41 and 162 are now unconditional (run for all browsers), while nearby
ActionBuilder/double-click-related tests in the same file still explicitly exclude Safari/Safari
Preview via except: {browser: %i[safari safari_preview]}, indicating there is known or expected
variance in support that these two tests should also account for.

rb/spec/integration/selenium/webdriver/action_builder_spec.rb[162-168]
rb/spec/integration/selenium/webdriver/action_builder_spec.rb[170-179]
rb/spec/integration/selenium/webdriver/action_builder_spec.rb[41-50]
rb/spec/integration/selenium/webdriver/action_builder_spec.rb[28-29]
Best Practice: Learned patterns

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

## Issue description
Two browser-sensitive ActionBuilder integration tests (`it 'presses pointer twice'` and `it 'sends keys to element'`) were changed to run unconditionally. If Safari/Safari Preview (or another CI browser/driver) does not reliably support the specific Actions double-click / pointer behavior or Actions+send_keys behavior under test, this can introduce cross-browser CI failures.

## Issue Context
PR Compliance ID 13 requires gating tests by runtime/browser feature support to prevent cross-environment failures. Other ActionBuilder tests in the same spec file remain gated with `except: {browser: %i[safari safari_preview]}`, and an adjacent double-click equivalence test still excludes Safari/Safari Preview, implying known incompatibility or flakiness that should be handled consistently.

## Fix Focus Areas
- rb/spec/integration/selenium/webdriver/action_builder_spec.rb[41-50]
- rb/spec/integration/selenium/webdriver/action_builder_spec.rb[162-179]

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


13. Affected browser tests can skip 🐞 Bug ≡ Correctness
Description
os-tests tag filters do not include chrome or firefox, but Ruby integration tests are tagged
with their browser name (e.g., chrome, firefox); if the affected targets include those tests,
--test_tag_filters can filter them out so they never run in ci-ruby despite being affected.
Code

.github/workflows/ci-ruby.yml[R53-62]

Evidence
Ruby integration tests are tagged with their browser name (local) and ${browser}-remote (remote).
ci-ruby os-tests only includes edge/safari tags, so chrome/firefox-tagged integration tests can be
filtered out even when passed via inputs.targets.

.github/workflows/ci-ruby.yml[47-76]
rb/spec/tests.bzl[170-215]

Agent prompt
The issue below was found during a code review. Follow the provided context and guidance below and implement a solution

### Issue description
`os-tests` uses `--test_tag_filters` that select only edge/safari/unit/skip-rbe (plus exclusions). Ruby integration tests are tagged with `[browser]` and `[browser-remote]`, so affected chrome/firefox tests may be excluded and not exercised on GitHub-hosted OS runners.

### Issue Context
Ruby integration test generation appends the browser tag (e.g., `chrome`) to each test target.

### Fix Focus Areas
- .github/workflows/ci-ruby.yml[47-76]

### Suggested fix
- If you intend to run affected integration tests for chrome/firefox on OS runners, add `chrome,chrome-remote,firefox,firefox-remote` (and beta variants if needed) to the appropriate matrix entries.
- If you intentionally rely on RBE for chrome/firefox coverage, consider documenting that explicitly in the workflow and ensure required CI checks still cover those targets.

ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools


Grey Divider

Qodo Logo

Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci.yml
Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci-ruby.yml Outdated
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit fa50a25

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Streamlines Ruby CI on GitHub Actions runners by switching Ruby workflow execution to be driven by the Bazel “affected targets” list, and consolidating more signal into OS-specific test jobs (instead of a broader unit-test OS matrix).

Changes:

  • Pass Ruby affected targets from ci.yml into ci-ruby.yml and gate Ruby CI on those targets being present.
  • Replace the prior Ruby CI job layout with OS-specific test runs using Bazel tag filters, plus a smaller Ruby interpreter unit-test matrix.
  • Update Bazel tags for Ruby unit and lint targets to support the new filtering approach.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
rb/spec/tests.bzl Retags Ruby unit test targets to support CI tag filtering.
rb/BUILD.bazel Retags the Ruby lint target to be filterable via Bazel tag filters.
.skipped-tests Removes a previously skipped Ruby BiDi test label from the skip list.
.github/workflows/ci.yml Emits Ruby affected targets (rb_targets) and passes them into the Ruby reusable workflow.
.github/workflows/ci-ruby.yml Restructures Ruby CI to run OS-specific Bazel tests over affected targets with tag filters and a reduced unit-test matrix.

Comment thread .github/workflows/ci-ruby.yml
Comment thread .github/workflows/ci-ruby.yml Outdated
Comment thread .github/workflows/ci-ruby.yml Outdated
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 2f0d00f

Comment thread .github/workflows/ci-ruby.yml Outdated
@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 035aaf6

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 7e9d57a

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 6c8325e

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 22, 2026

Persistent review updated to latest commit 2f4b024

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit e6d425f

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit 6d6cb6f

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit f0a30f7

@qodo-code-review
Copy link
Copy Markdown
Contributor

qodo-code-review Bot commented May 23, 2026

Persistent review updated to latest commit 02d2d5c

Comment thread .github/workflows/ci-ruby.yml
@titusfortner titusfortner marked this pull request as draft May 25, 2026 13:31
@titusfortner titusfortner force-pushed the ci-ruby branch 3 times, most recently from c9df9f6 to f05ec3d Compare May 26, 2026 20:47
@qodo-code-review
Copy link
Copy Markdown
Contributor

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: windows-tests / windows-tests

Failed stage: Run Bazel [❌]

Failed test name: //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi

Failure summary:

The GitHub Action failed because a Bazel/RSpec test target failed:
- Bazel reported FAIL:
//rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi (Exit 1), causing the
overall Bazel run to finish with Build completed, 1 test FAILED and the job to exit with code 3.
-
The failing spec is Selenium::WebDriver::BiDi::BrowsingContext sets the viewport in
./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb (failed example at line 77;
assertion at line 81):
- expect(driver.execute_script('return window.devicePixelRatio')).to
eq(2.0) but it returned 1 on Windows + stable Edge (MicrosoftEdge=148.0.3967.83).
Notes:
- Several
other specs in the same file show TimeoutError waiting for an alert, but they are marked PENDING
(guarded) and are explicitly stated as not affecting suite status.

Relevant error logs:
1:  ##[group]Runner Image Provisioner
2:  Hosted Compute Agent
...

495:  Received 591396864 of 1702756241 (34.7%), 140.1 MBs/sec
496:  Received 746586112 of 1702756241 (43.8%), 141.6 MBs/sec
497:  Received 931135488 of 1702756241 (54.7%), 147.0 MBs/sec
498:  Received 1073741824 of 1702756241 (63.1%), 144.7 MBs/sec
499:  Received 1207959552 of 1702756241 (70.9%), 142.6 MBs/sec
500:  Received 1379926016 of 1702756241 (81.0%), 145.0 MBs/sec
501:  Received 1535115264 of 1702756241 (90.2%), 145.1 MBs/sec
502:  Received 1673527296 of 1702756241 (98.3%), 143.8 MBs/sec
503:  Received 1702756241 of 1702756241 (100.0%), 143.5 MBs/sec
504:  Cache Size: ~1624 MB (1702756241 B)
505:  [command]"C:\Program Files\Git\usr\bin\tar.exe" -xf D:/a/_temp/397d1b17-b8f4-4653-b70b-db50c6175dba/cache.tzst -P -C D:/a/selenium/selenium --force-local --use-compress-program "zstd -d"
506:  Cache restored successfully
507:  Successfully restored cache from setup-bazel-2-win32-x64-repository-986d40d98abf71f837036d95fe60c57c5040a9d863b9b3a8ffff8a0e10b6b31b
508:  ##[endgroup]
509:  ##[group]Restore cache for external-ci---ruby-bazel-manifest
510:  Failed to restore external-ci---ruby-bazel-manifest cache
511:  ##[endgroup]
512:  ##[group]Run if [[ "success" == "failure" ]]; then
513:  �[36;1mif [[ "success" == "failure" ]]; then�[0m
514:  �[36;1m  echo "::warning::Bazel cache setup failed — fell back to setup without cache"�[0m
515:  �[36;1mfi�[0m
...

567:  shell: C:\Program Files\PowerShell\7\pwsh.EXE -command ". '{0}'"
568:  env:
569:  GITHUB_TOKEN: ***
570:  SEL_M2_USER: 
571:  SEL_M2_PASS: 
572:  TWINE_PASSWORD: 
573:  TWINE_NIGHTLY_PASSWORD: 
574:  GEM_HOST_API_KEY: 
575:  NUGET_API_KEY: 
576:  BAZELISK_GITHUB_TOKEN: ***
577:  ##[endgroup]
578:  The registry state is now: 0 (Enable 8dot3 name creation on all volumes).
579:  ##[group]Run . ./scripts/github-actions/disk-status.sh
580:  �[36;1m. ./scripts/github-actions/disk-status.sh�[0m
581:  �[36;1mif [ "$AVAIL_GB" -lt 10 ]; then�[0m
582:  �[36;1m  echo "::error::Insufficient disk space: ${AVAIL_GB}GB available (need >=10GB)"�[0m
583:  �[36;1m  exit 1�[0m
...

1240:  ^
1241:  where T is a type-variable:
1242:  T extends Object declared in method <T>sendAsyncNative(HttpRequest,BodyHandler<T>)
1243:  java\src\org\openqa\selenium\remote\tracing\TracedHttpClient.java:70: warning: [removal] <T>sendNative(HttpRequest,BodyHandler<T>) in HttpClient has been deprecated and marked for removal
1244:  return delegate.sendNative(request, handler);
1245:  ^
1246:  where T is a type-variable:
1247:  T extends Object declared in method <T>sendNative(HttpRequest,BodyHandler<T>)
1248:  �[32m[2,052 / 2,457]�[0m Building java/src/org/openqa/selenium/bidi/speculation/libspeculation.jar (4 source files); 0s multiplex-worker ... (3 actions, 1 running)
1249:  �[32m[2,059 / 2,459]�[0m Building java/src/org/openqa/selenium/bidi/browsingcontext/libbrowsingcontext.jar (22 source files); 0s multiplex-worker ... (3 actions, 1 running)
1250:  �[32m[2,065 / 2,460]�[0m Building java/src/org/openqa/selenium/bidi/storage/libstorage.jar (11 source files); 0s multiplex-worker ... (4 actions, 1 running)
1251:  �[32m[2,072 / 2,461]�[0m Building java/src/org/openqa/selenium/remote/tracing/opentelemetry/libopentelemetry.jar (7 source files) and running annotation processors (AutoServiceProcessor); 0s multiplex-worker ... (3 actions, 1 running)
1252:  �[32m[2,077 / 2,463]�[0m Building java/src/org/openqa/selenium/remote/libapi-class.jar (70 source files); 0s multiplex-worker ... (2 actions, 1 running)
1253:  �[32m[2,079 / 2,465]�[0m Building java/src/org/openqa/selenium/remote/libapi-class.jar (70 source files); 1s multiplex-worker ... (3 actions, 1 running)
1254:  �[32mINFO: �[0mFrom Building java/src/org/openqa/selenium/remote/libapi-class.jar (70 source files):
1255:  java\src\org\openqa\selenium\remote\ErrorHandler.java:49: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1256:  private final ErrorCodes errorCodes;
1257:  ^
1258:  java\src\org\openqa\selenium\remote\ErrorHandler.java:63: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1259:  this.errorCodes = new ErrorCodes();
1260:  ^
1261:  java\src\org\openqa\selenium\remote\ErrorHandler.java:71: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1262:  public ErrorHandler(ErrorCodes codes, boolean includeServerErrors) {
1263:  ^
1264:  java\src\org\openqa\selenium\remote\RemoteWebDriverBuilder.java:309: warning: [removal] <T>sendNative(HttpRequest,BodyHandler<T>) in HttpClient has been deprecated and marked for removal
1265:  public <T> java.net.http.HttpResponse<T> sendNative(
1266:  ^
1267:  where T is a type-variable:
1268:  T extends Object declared in method <T>sendNative(HttpRequest,BodyHandler<T>)
1269:  java\src\org\openqa\selenium\remote\RemoteWebDriverBuilder.java:302: warning: [removal] <T>sendAsyncNative(HttpRequest,BodyHandler<T>) in HttpClient has been deprecated and marked for removal
1270:  sendAsyncNative(
1271:  ^
1272:  where T is a type-variable:
1273:  T extends Object declared in method <T>sendAsyncNative(HttpRequest,BodyHandler<T>)
1274:  java\src\org\openqa\selenium\remote\Response.java:99: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1275:  ErrorCodes errorCodes = new ErrorCodes();
1276:  ^
1277:  java\src\org\openqa\selenium\remote\Response.java:99: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1278:  ErrorCodes errorCodes = new ErrorCodes();
1279:  ^
1280:  java\src\org\openqa\selenium\remote\ProtocolHandshake.java:182: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1281:  response.setStatus(ErrorCodes.SUCCESS);
1282:  ^
1283:  java\src\org\openqa\selenium\remote\ProtocolHandshake.java:183: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1284:  response.setState(ErrorCodes.SUCCESS_STRING);
1285:  ^
1286:  java\src\org\openqa\selenium\remote\W3CHandshakeResponse.java:54: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1287:  new ErrorCodes().toStatus((String) rawError, Optional.of(tuple.getStatusCode())));
1288:  ^
1289:  java\src\org\openqa\selenium\remote\W3CHandshakeResponse.java:57: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1290:  new ErrorCodes().getExceptionType((String) rawError);
1291:  ^
1292:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:43: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1293:  private final ErrorCodes errorCodes = new ErrorCodes();
1294:  ^
1295:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:43: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1296:  private final ErrorCodes errorCodes = new ErrorCodes();
1297:  ^
1298:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:55: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1299:  int status = responseStatus == ErrorCodes.SUCCESS ? HTTP_OK : HTTP_INTERNAL_ERROR;
1300:  ^
1301:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:100: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1302:  response.setStatus(ErrorCodes.UNKNOWN_COMMAND);
1303:  ^
1304:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:102: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1305:  response.setStatus(ErrorCodes.UNHANDLED_ERROR);
1306:  ^
1307:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:118: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1308:  response.setStatus(ErrorCodes.SUCCESS);
1309:  ^
1310:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:119: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1311:  response.setState(errorCodes.toState(ErrorCodes.SUCCESS));
1312:  ^
1313:  java\src\org\openqa\selenium\remote\codec\AbstractHttpResponseCodec.java:125: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1314:  response.setState(errorCodes.toState(ErrorCodes.SUCCESS));
1315:  ^
1316:  java\src\org\openqa\selenium\remote\codec\w3c\W3CHttpResponseCodec.java:70: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1317:  private final ErrorCodes errorCodes = new ErrorCodes();
1318:  ^
1319:  java\src\org\openqa\selenium\remote\codec\w3c\W3CHttpResponseCodec.java:70: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1320:  private final ErrorCodes errorCodes = new ErrorCodes();
1321:  ^
1322:  java\src\org\openqa\selenium\remote\codec\w3c\W3CHttpResponseCodec.java:98: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1323:  response.setStatus(ErrorCodes.UNKNOWN_COMMAND);
1324:  ^
1325:  java\src\org\openqa\selenium\remote\codec\w3c\W3CHttpResponseCodec.java:103: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1326:  response.setStatus(ErrorCodes.UNHANDLED_ERROR);
1327:  ^
1328:  java\src\org\openqa\selenium\remote\codec\w3c\W3CHttpResponseCodec.java:150: warning: [removal] ErrorCodes in org.openqa.selenium.remote has been deprecated and marked for removal
1329:  response.setStatus(ErrorCodes.SUCCESS);
1330:  ^
...

1457:  �[32m[2,419 / 2,513]�[0m 8 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:takes_screenshot-chrome-beta; 1s local ... (4 actions, 1 running)
1458:  �[32m[2,419 / 2,513]�[0m 8 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:takes_screenshot-chrome-beta; 11s local ... (4 actions, 1 running)
1459:  �[32m[2,419 / 2,513]�[0m 8 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:takes_screenshot-chrome-beta; 36s local ... (4 actions, 2 running)
1460:  �[32m[2,420 / 2,513]�[0m 9 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote:features; 1s local ... (4 actions, 1 running)
1461:  �[32m[2,420 / 2,513]�[0m 9 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote:features; 11s local ... (4 actions, 1 running)
1462:  �[32m[2,420 / 2,513]�[0m 9 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote:features; 14s local ... (4 actions, 2 running)
1463:  �[32m[2,421 / 2,513]�[0m 10 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver:zipper; 1s local ... (4 actions, 1 running)
1464:  �[32m[2,421 / 2,513]�[0m 10 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver:zipper; 11s local ... (4 actions, 1 running)
1465:  �[32m[2,421 / 2,513]�[0m 10 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver:zipper; 13s local ... (4 actions, 2 running)
1466:  �[32m[2,422 / 2,513]�[0m 11 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/common/fedcm:dialog; 1s local ... (4 actions, 1 running)
1467:  �[32m[2,422 / 2,513]�[0m 11 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/common/fedcm:dialog; 11s local ... (4 actions, 1 running)
1468:  �[32m[2,422 / 2,513]�[0m 11 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/common/fedcm:dialog; 13s local ... (4 actions, 2 running)
1469:  �[32m[2,423 / 2,513]�[0m 12 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver:wait; 1s local ... (4 actions, 1 running)
1470:  �[32m[2,423 / 2,513]�[0m 12 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver:wait; 11s local ... (4 actions, 1 running)
1471:  �[32m[2,423 / 2,513]�[0m 12 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver:wait; 15s local ... (4 actions, 2 running)
1472:  �[32m[2,424 / 2,513]�[0m 13 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:error-edge; 1s local ... (4 actions, 1 running)
1473:  �[32m[2,424 / 2,513]�[0m 13 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:error-edge; 11s local ... (4 actions, 1 running)
1474:  �[32m[2,424 / 2,513]�[0m 13 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:error-edge; 16s local ... (4 actions, 2 running)
1475:  �[32m[2,425 / 2,513]�[0m 14 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver:window-firefox-beta; 1s local ... (4 actions, 1 running)
...

1735:  �[32m[2,510 / 2,513]�[0m 99 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote/http:default; 13s local ... (3 actions, 2 running)
1736:  �[32m[2,511 / 2,513]�[0m 100 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote:capabilities; 1s local ... (2 actions, 1 running)
1737:  �[32m[2,511 / 2,513]�[0m 100 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote:capabilities; 11s local ... (2 actions, 1 running)
1738:  �[32m[2,511 / 2,513]�[0m 100 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote:capabilities; 13s local ... (2 actions running)
1739:  �[32m[2,512 / 2,513]�[0m 101 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote/http:common; 1s local
1740:  �[32m[2,512 / 2,513]�[0m 101 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote/http:common; 11s local
1741:  �[32m[2,512 / 2,513]�[0m 101 / 110 tests;�[0m Testing //rb/spec/unit/selenium/webdriver/remote/http:common; 13s local
1742:  �[32m[2,513 / 2,514]�[0m 102 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:browser-edge-bidi; 1s local
1743:  �[32m[2,513 / 2,514]�[0m 102 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:browser-edge-bidi; 25s local
1744:  �[32m[2,514 / 2,515]�[0m 103 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi; 1s local
1745:  �[32m[2,514 / 2,515]�[0m 103 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi; 59s local
1746:  �[31m�[1mFAIL: �[0m//rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi (Exit 1) (see D:/b/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi/browsing_context-edge-bidi/test_attempts/attempt_1.log)
1747:  �[32m[2,514 / 2,515]�[0m 103 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi; 60s local
1748:  �[32m[2,514 / 2,515]�[0m 103 / 110 tests;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi; 106s local
1749:  �[31m�[1mFAIL: �[0m//rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi (Exit 1) (see D:/b/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi/browsing_context-edge-bidi/test.log)
1750:  �[31m�[1mFAILED: �[0m//rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi (Summary)
1751:  D:/b/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi/browsing_context-edge-bidi/test.log
1752:  D:/b/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi/browsing_context-edge-bidi/test_attempts/attempt_1.log
1753:  �[32mINFO: �[0mFrom Testing //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi:
1754:  ==================== Test output for //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi:
1755:  Running Ruby specs:
1756:  browser: edge
1757:  driver: edge
1758:  version: stable
1759:  platform: windows
1760:  ci: github
1761:  rbe: false
1762:  ruby: ruby 3.3.11 (2026-03-26 revision 1f2d15125a) [x64-mingw-ucrt]
1763:  Selenium::WebDriver::BiDi::BrowsingContext
1764:  closes a window
1765:  sets the viewport (FAILED - 1)
1766:  accepts users prompts without text (PENDING: Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};)
1767:  accepts users prompts with text (PENDING: Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};)
1768:  rejects users prompts (PENDING: Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};)
1769:  activates a browser context
1770:  #create
1771:  without arguments
1772:  accepts a tab type
1773:  accepts a window type
1774:  errors on unknown type
1775:  accepts a reference context
1776:  Pending: (Failures listed here are expected and do not affect your suite's status)
1777:  1) Selenium::WebDriver::BiDi::BrowsingContext accepts users prompts without text
1778:  # Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};
1779:  Failure/Error: wait_for_alert
1780:  Selenium::WebDriver::Error::TimeoutError:
1781:  timed out after 5 seconds (no such alert
1782:  (Session info: MicrosoftEdge=148.0.3967.83))
1783:  # ./rb/lib/selenium/webdriver/common/wait.rb:76:in `until'
1784:  # //?/D:/a/selenium/selenium/rb/spec/integration/selenium/webdriver/spec_support/helpers.rb:79:in `wait_for_alert'
1785:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:91:in `block (2 levels) in <class:BiDi>'
1786:  2) Selenium::WebDriver::BiDi::BrowsingContext accepts users prompts with text
1787:  # Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};
1788:  Failure/Error: wait_for_alert
1789:  Selenium::WebDriver::Error::TimeoutError:
1790:  timed out after 5 seconds (no such alert
1791:  (Session info: MicrosoftEdge=148.0.3967.83))
1792:  # ./rb/lib/selenium/webdriver/common/wait.rb:76:in `until'
1793:  # //?/D:/a/selenium/selenium/rb/spec/integration/selenium/webdriver/spec_support/helpers.rb:79:in `wait_for_alert'
1794:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:105:in `block (2 levels) in <class:BiDi>'
1795:  3) Selenium::WebDriver::BiDi::BrowsingContext rejects users prompts
1796:  # Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};
1797:  Failure/Error: wait_for_alert
1798:  Selenium::WebDriver::Error::TimeoutError:
1799:  timed out after 5 seconds (no such alert
1800:  (Session info: MicrosoftEdge=148.0.3967.83))
1801:  # ./rb/lib/selenium/webdriver/common/wait.rb:76:in `until'
1802:  # //?/D:/a/selenium/selenium/rb/spec/integration/selenium/webdriver/spec_support/helpers.rb:79:in `wait_for_alert'
1803:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:118:in `block (2 levels) in <class:BiDi>'
1804:  Failures:
1805:  1) Selenium::WebDriver::BiDi::BrowsingContext sets the viewport
1806:  Failure/Error: expect(driver.execute_script('return window.devicePixelRatio')).to eq(2.0)
1807:  expected: 2.0
1808:  got: 1
1809:  (compared using ==)
1810:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:81:in `block (2 levels) in <class:BiDi>'
1811:  Finished in 45.44 seconds (files took 0.74816 seconds to load)
1812:  11 examples, 1 failure, 3 pending
1813:  Failed examples:
1814:  rspec ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:77 # Selenium::WebDriver::BiDi::BrowsingContext sets the viewport
1815:  ================================================================================
1816:  ==================== Test output for //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi:
1817:  Running Ruby specs:
1818:  browser: edge
1819:  driver: edge
1820:  version: stable
1821:  platform: windows
1822:  ci: github
1823:  rbe: false
1824:  ruby: ruby 3.3.11 (2026-03-26 revision 1f2d15125a) [x64-mingw-ucrt]
1825:  Selenium::WebDriver::BiDi::BrowsingContext
1826:  closes a window
1827:  sets the viewport (FAILED - 1)
1828:  accepts users prompts without text (PENDING: Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};)
1829:  accepts users prompts with text (PENDING: Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};)
1830:  rejects users prompts (PENDING: Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};)
1831:  activates a browser context
1832:  #create
1833:  without arguments
1834:  accepts a tab type
1835:  accepts a window type
1836:  errors on unknown type
1837:  accepts a reference context
1838:  Pending: (Failures listed here are expected and do not affect your suite's status)
1839:  1) Selenium::WebDriver::BiDi::BrowsingContext accepts users prompts without text
1840:  # Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};
1841:  Failure/Error: wait_for_alert
1842:  Selenium::WebDriver::Error::TimeoutError:
1843:  timed out after 5 seconds (no such alert
1844:  (Session info: MicrosoftEdge=148.0.3967.83))
1845:  # ./rb/lib/selenium/webdriver/common/wait.rb:76:in `until'
1846:  # //?/D:/a/selenium/selenium/rb/spec/integration/selenium/webdriver/spec_support/helpers.rb:79:in `wait_for_alert'
1847:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:91:in `block (2 levels) in <class:BiDi>'
1848:  2) Selenium::WebDriver::BiDi::BrowsingContext accepts users prompts with text
1849:  # Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};
1850:  Failure/Error: wait_for_alert
1851:  Selenium::WebDriver::Error::TimeoutError:
1852:  timed out after 5 seconds (no such alert
1853:  (Session info: MicrosoftEdge=148.0.3967.83))
1854:  # ./rb/lib/selenium/webdriver/common/wait.rb:76:in `until'
1855:  # //?/D:/a/selenium/selenium/rb/spec/integration/selenium/webdriver/spec_support/helpers.rb:79:in `wait_for_alert'
1856:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:105:in `block (2 levels) in <class:BiDi>'
1857:  3) Selenium::WebDriver::BiDi::BrowsingContext rejects users prompts
1858:  # Test guarded; Guarded by {:browser=>[:edge, :chrome], :reason=>"https://github.com/GoogleChromeLabs/chromium-bidi/issues/3281"};
1859:  Failure/Error: wait_for_alert
1860:  Selenium::WebDriver::Error::TimeoutError:
1861:  timed out after 5 seconds (no such alert
1862:  (Session info: MicrosoftEdge=148.0.3967.83))
1863:  # ./rb/lib/selenium/webdriver/common/wait.rb:76:in `until'
1864:  # //?/D:/a/selenium/selenium/rb/spec/integration/selenium/webdriver/spec_support/helpers.rb:79:in `wait_for_alert'
1865:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:118:in `block (2 levels) in <class:BiDi>'
1866:  Failures:
1867:  1) Selenium::WebDriver::BiDi::BrowsingContext sets the viewport
1868:  Failure/Error: expect(driver.execute_script('return window.devicePixelRatio')).to eq(2.0)
1869:  expected: 2.0
1870:  got: 1
1871:  (compared using ==)
1872:  # ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:81:in `block (2 levels) in <class:BiDi>'
1873:  Finished in 45.35 seconds (files took 0.73566 seconds to load)
1874:  11 examples, 1 failure, 3 pending
1875:  Failed examples:
1876:  rspec ./rb/spec/integration/selenium/webdriver/bidi/browsing_context_spec.rb:77 # Selenium::WebDriver::BiDi::BrowsingContext sets the viewport
1877:  ================================================================================
1878:  �[32m[2,515 / 2,516]�[0m 104 / 110 tests, �[31m�[1m1 failed�[0m;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:network-edge-bidi; 1s local
1879:  �[32m[2,515 / 2,516]�[0m 104 / 110 tests, �[31m�[1m1 failed�[0m;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:network-edge-bidi; 44s local
1880:  �[32m[2,516 / 2,517]�[0m 105 / 110 tests, �[31m�[1m1 failed�[0m;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:script-edge-bidi; 1s local
1881:  �[32m[2,516 / 2,517]�[0m 105 / 110 tests, �[31m�[1m1 failed�[0m;�[0m Testing //rb/spec/integration/selenium/webdriver/bidi:script-edge-bidi; 30s local
1882:  �[32m[2,517 / 2,518]�[0m 106 / 110 tests, �[31m�[1m1 failed�[0m;�[0m Testing //rb/spec/integration/selenium/webdriver:devtools-edge; 1s local
1883:  �[32m[2,517 / 2,518]�[0m 106 / 110 tests, �[31m�[1m1 failed�[0m;�[0m Testing //rb/spec/integration/selenium/webdriver:devtools-edge; 170s local
1884:  �[32mINFO: �[0mFound 324 targets and 110 test targets...
1885:  �[32mINFO: �[0mElapsed time: 2730.087s, Critical Path: 320.31s
1886:  �[32mINFO: �[0m2430 processes: 1688 internal, 753 local, 100 worker.
1887:  �[32mINFO: �[0mBuild completed, 1 test FAILED, 2430 total actions
1888:  //rb/spec/integration/selenium/webdriver:action_builder-safari          �[0m�[35mSKIPPED�[0m
1889:  //rb/spec/integration/selenium/webdriver:takes_screenshot-safari        �[0m�[35mSKIPPED�[0m
1890:  //rb/spec/integration/selenium/webdriver:window-safari                  �[0m�[35mSKIPPED�[0m
1891:  //rb/spec/integration/selenium/webdriver:action_builder-chrome-beta      �[0m�[32mPASSED�[0m in 17.9s
1892:  //rb/spec/integration/selenium/webdriver:action_builder-edge             �[0m�[32mPASSED�[0m in 17.6s
1893:  //rb/spec/integration/selenium/webdriver:action_builder-firefox-beta     �[0m�[32mPASSED�[0m in 24.0s
1894:  //rb/spec/integration/selenium/webdriver:bidi-edge-bidi                  �[0m�[32mPASSED�[0m in 10.7s
1895:  //rb/spec/integration/selenium/webdriver:devtools-edge                   �[0m�[32mPASSED�[0m in 159.8s
1896:  //rb/spec/integration/selenium/webdriver:driver-edge                     �[0m�[32mPASSED�[0m in 16.9s
1897:  //rb/spec/integration/selenium/webdriver:element-edge                    �[0m�[32mPASSED�[0m in 30.8s
1898:  //rb/spec/integration/selenium/webdriver:error-edge                      �[0m�[32mPASSED�[0m in 5.0s
1899:  //rb/spec/integration/selenium/webdriver:fedcm-edge                      �[0m�[32mPASSED�[0m in 32.0s
...

1982:  //rb/spec/unit/selenium/webdriver/ie:driver                              �[0m�[32mPASSED�[0m in 2.8s
1983:  //rb/spec/unit/selenium/webdriver/ie:options                             �[0m�[32mPASSED�[0m in 2.7s
1984:  //rb/spec/unit/selenium/webdriver/ie:service                             �[0m�[32mPASSED�[0m in 2.7s
1985:  //rb/spec/unit/selenium/webdriver/remote:bridge                          �[0m�[32mPASSED�[0m in 2.8s
1986:  //rb/spec/unit/selenium/webdriver/remote:capabilities                    �[0m�[32mPASSED�[0m in 2.8s
1987:  //rb/spec/unit/selenium/webdriver/remote:driver                          �[0m�[32mPASSED�[0m in 2.8s
1988:  //rb/spec/unit/selenium/webdriver/remote:features                        �[0m�[32mPASSED�[0m in 2.8s
1989:  //rb/spec/unit/selenium/webdriver/remote/http:common                     �[0m�[32mPASSED�[0m in 2.8s
1990:  //rb/spec/unit/selenium/webdriver/remote/http:default                    �[0m�[32mPASSED�[0m in 2.7s
1991:  //rb/spec/unit/selenium/webdriver/safari:driver                          �[0m�[32mPASSED�[0m in 2.8s
1992:  //rb/spec/unit/selenium/webdriver/safari:options                         �[0m�[32mPASSED�[0m in 2.8s
1993:  //rb/spec/unit/selenium/webdriver/safari:service                         �[0m�[32mPASSED�[0m in 2.8s
1994:  //rb/spec/unit/selenium/webdriver/support:color                          �[0m�[32mPASSED�[0m in 2.8s
1995:  //rb/spec/unit/selenium/webdriver/support:event_firing                   �[0m�[32mPASSED�[0m in 2.9s
1996:  //rb/spec/unit/selenium/webdriver/support:select                         �[0m�[32mPASSED�[0m in 2.7s
1997:  //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi �[0m�[31m�[1mFAILED�[0m in 2 out of 2 in 47.6s
1998:  Stats over 2 runs: max = 47.6s, min = 47.2s, avg = 47.4s, dev = 0.2s
1999:  D:/b/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi/browsing_context-edge-bidi/test.log
2000:  D:/b/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi/browsing_context-edge-bidi/test_attempts/attempt_1.log
2001:  Executed 107 out of 110 tests: 106 tests pass, �[0m�[31m�[1m1 fails locally�[0m, and �[0m�[35m3 were skipped�[0m.
2002:  There were tests whose specified size is too big. Use the --test_verbose_timeout_warnings command line option to see which ones these are.
2003:  �[0m
2004:  ##[error]Process completed with exit code 3.
2005:  ##[group]Run ./scripts/github-actions/rerun-failures.sh './scripts/github-actions/bazel-test-if-targets.sh --keep_going --flaky_test_attempts 2 --local_test_jobs 1 --pin_browsers=false --skip_incompatible_explicit_targets --test_env=SE_FORCE_BROWSER_DOWNLOAD=true --test_env=SE_SKIP_DRIVER_IN_PATH=true --test_tag_filters=edge,edge-bidi,unit,os-sensitive //rb/...
...

2038:  bazel-bin/external/rules_ruby++ruby+bundle/vendor/bundle
2039:  bazel-bin/rb/lib/selenium/devtools/v146.rb
2040:  bazel-bin/rb/lib/selenium/devtools/v146
2041:  bazel-bin/rb/lib/selenium/devtools/v147.rb
2042:  bazel-bin/rb/lib/selenium/devtools/v147
2043:  bazel-bin/rb/lib/selenium/devtools/v148.rb
2044:  bazel-bin/rb/lib/selenium/devtools/v148
2045:  bazel-bin/rb/lib/selenium/webdriver/atoms/findElements.js
2046:  bazel-bin/rb/lib/selenium/webdriver/atoms/getAttribute.js
2047:  bazel-bin/rb/lib/selenium/webdriver/atoms/isDisplayed.js
2048:  bazel-bin/rb/bin/linux/selenium-manager
2049:  bazel-bin/rb/bin/macos/selenium-manager
2050:  bazel-bin/rb/bin/windows/selenium-manager.exe
2051:  �[32mINFO: �[0mElapsed time: 69.203s, Critical Path: 51.84s
2052:  �[32mINFO: �[0m2 processes: 4 action cache hit, 1 internal, 2 local.
2053:  �[32mINFO: �[0mBuild completed, 1 test FAILED, 2 total actions
2054:  //rb/spec/integration/selenium/webdriver/bidi:browsing_context-edge-bidi �[0m�[31m�[1mFAILED�[0m in 48.3s
2055:  D:/b/execroot/_main/bazel-out/x64_windows-fastbuild/testlogs/rb/spec/integration/selenium/webdriver/bidi/browsing_context-edge-bidi/test.log
2056:  Executed 1 out of 1 test: �[0m�[31m�[1m1 fails locally�[0m.
2057:  �[0m
2058:  ##[error]Process completed with exit code 3.
2059:  ##[group]Run ./scripts/github-actions/collect-test-logs.sh

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

B-build Includes scripting, bazel and CI integrations C-rb Ruby Bindings

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants