Skip to content

TEST: tackles some CI inefficiencies#139

Draft
umair-ably wants to merge 1 commit intomainfrom
improveCI
Draft

TEST: tackles some CI inefficiencies#139
umair-ably wants to merge 1 commit intomainfrom
improveCI

Conversation

@umair-ably
Copy link
Contributor

@umair-ably umair-ably commented Feb 25, 2026

Summary by CodeRabbit

  • Chores

    • Consolidated end-to-end testing workflow configuration with centralized build artifact handling and Playwright browser caching to improve CI performance.
    • Reorganized test job dependencies and naming conventions for a clearer, more efficient pipeline structure.
    • Removed redundant workflow files.
  • Tests

    • Enabled parallel test execution by removing sequential execution constraints, improving test suite performance.

@vercel
Copy link

vercel bot commented Feb 25, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
cli-web-cli Ready Ready Preview, Comment Feb 25, 2026 0:03am

Request Review

@cursor
Copy link

cursor bot commented Feb 25, 2026

You have run out of free Bugbot PR reviews for this billing cycle. This will reset on March 4.

To receive reviews on all of your PRs, visit the Cursor dashboard to activate Pro and start your 14-day free trial.

@coderabbitai
Copy link

coderabbitai bot commented Feb 25, 2026

Important

Review skipped

Auto incremental reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Walkthrough

The changes consolidate E2E testing workflows by removing a dedicated e2e-tests.yml file, refactoring the main e2e.yml to introduce a centralized build job with shared artifacts and Playwright caching, reorganizing test jobs with updated naming conventions, and relaxing sequential test execution constraints in Vitest configuration to enable parallelism.

Changes

Cohort / File(s) Summary
E2E Workflow Consolidation
.github/workflows/e2e-tests.yml
Complete removal of the standalone E2E CLI test workflow that previously handled dependency setup, code quality checks, environment configuration, and CLI test execution with timeout handling.
E2E Workflow Refactoring
.github/workflows/e2e.yml
Restructured workflow to introduce a centralized build job, added Playwright browser caching across jobs (~/.cache/ms-playwright), expanded path triggers, created new e2e-cli-tests job, renamed test jobs (auth-tests → web-cli-auth-tests, session-tests → web-cli-session-tests, ui-tests → web-cli-ui-tests, rate-limit-test → web-cli-rate-limit-test), updated all test job dependencies to depend on build, and removed explicit artifact exclusions.
Test Configuration Optimization
vitest.config.ts
Removed sequence and fileParallelism constraints from the e2e project configuration, enabling parallel test execution and flexible test ordering.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Poem

🐰 Build once, test in parallel delight,
Caching browsers keeps the pipeline tight,
E2E CLI tests now have a home,
Shared artifacts let dependencies roam,
Workflows consolidated, execution runs faster still! ✨

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'TEST: tackles some CI inefficiencies' is vague and uses generic language that doesn't clearly describe the specific changes made. Consider a more specific title such as 'TEST: consolidate E2E workflow and enable Playwright caching' or 'TEST: refactor E2E tests to improve CI performance' to clearly convey the main changes.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch improveCI

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
.github/workflows/e2e.yml (1)

145-344: Consider a matrix/composite refactor for repeated test-job setup.

The four web-cli jobs duplicate checkout/setup/install/cache/download steps. Converting auth/session/ui into a matrix job (and optionally extracting common setup into a composite action) would reduce drift and simplify future maintenance.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/e2e.yml around lines 145 - 344, Multiple web-cli jobs
duplicate the same checkout/setup/install/cache/download steps (see jobs
web-cli-auth-tests, web-cli-session-tests, web-cli-ui-tests); refactor by either
(A) combining them into a single matrix job (e.g., job web-cli-tests with
strategy.matrix including TEST_GROUP values like auth/session/ui and a matrix
variable listing the test file paths, then run pnpm exec playwright test using
the matrix-provided files) or (B) extracting the repeated steps into a reusable
composite action (create an action that performs Checkout, Set up pnpm, Set up
Node.js, Install dependencies, Cache Playwright browsers, Install Playwright,
Download build artifacts) and call that composite from each job before running
the group-specific test command; update web-cli-rate-limit-test to reuse the
composite or the new matrix if appropriate.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/workflows/e2e.yml:
- Around line 293-294: The web-cli-rate-limit-test job can run concurrently with
e2e-cli-tests causing flaky rate-limit failures; update the job's needs list to
include e2e-cli-tests so it waits for that job to finish before starting (modify
the web-cli-rate-limit-test needs array to add "e2e-cli-tests", referencing the
web-cli-rate-limit-test job name and the e2e-cli-tests job).
- Around line 6-27: The path filters in the e2e workflow (the two "paths:"
blocks under push and pull_request in .github/workflows/e2e.yml) omit root-level
test/runtime config files so changes to those (e.g., vitest.config.ts) won't
trigger the job; update both path filter lists to include the root config
filenames or globs (for example add entries like "vitest.config.ts" and/or
"vitest.config.*" and any other root test config names you use) so edits to
those files will run the workflow.

---

Nitpick comments:
In @.github/workflows/e2e.yml:
- Around line 145-344: Multiple web-cli jobs duplicate the same
checkout/setup/install/cache/download steps (see jobs web-cli-auth-tests,
web-cli-session-tests, web-cli-ui-tests); refactor by either (A) combining them
into a single matrix job (e.g., job web-cli-tests with strategy.matrix including
TEST_GROUP values like auth/session/ui and a matrix variable listing the test
file paths, then run pnpm exec playwright test using the matrix-provided files)
or (B) extracting the repeated steps into a reusable composite action (create an
action that performs Checkout, Set up pnpm, Set up Node.js, Install
dependencies, Cache Playwright browsers, Install Playwright, Download build
artifacts) and call that composite from each job before running the
group-specific test command; update web-cli-rate-limit-test to reuse the
composite or the new matrix if appropriate.

ℹ️ Review info

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Disabled knowledge base sources:

  • Jira integration is disabled

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 00a97fd and c03789c.

📒 Files selected for processing (3)
  • .github/workflows/e2e-tests.yml
  • .github/workflows/e2e.yml
  • vitest.config.ts
💤 Files with no reviewable changes (2)
  • vitest.config.ts
  • .github/workflows/e2e-tests.yml

@umair-ably
Copy link
Contributor Author

will revisit later... seem to be a couple flaky tests

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant