-
Notifications
You must be signed in to change notification settings - Fork 3
Open
Description
Background
Issue #10 revealed that our CI tests were insufficient — they only verified vp --version after installation, which doesn't exercise critical code paths like vp exec and vp run. The Cannot find module 'which' error on Windows went undetected because no test actually ran commands through vite-plus.
Goal
Design and implement an e2e testing workflow that runs real-world ecosystem repositories against the action on all platforms (ubuntu, macos, windows) to catch regressions before they reach users.
Testing Plan
1. Ecosystem Repository Matrix
Include real repositories that use setup-vp in their CI to validate the full workflow:
| Repository | Key Commands | Purpose |
|---|---|---|
voidzero-dev/vite-plus |
vp install, vp run build, vp run test |
Core toolchain — validates install + build + test |
cloudflare/vinext (or similar) |
vp exec, vp install, vp run |
Real consumer — validates exec path that broke in #10 |
2. Test Scenarios
Each ecosystem repo should be tested with:
- Fresh install (no cache) on all 3 platforms
- Cached install (cache hit) on all 3 platforms
-
vp exec— the code path that triggered Windows runner failures with latest action version #10 -
vp run— script execution -
vp install— dependency installation -
vp env use— Node.js version switching - Multiple Node.js versions (e.g., 22, 24)
3. Workflow Design
- Create a separate
e2e.ymlworkflow (to avoid slowing down the maintest.yml) - Trigger on
pushto main,pull_request, andschedule(e.g., daily) to catch upstream breakage - Clone ecosystem repos at pinned refs for reproducibility
- Use
fail-fast: falseso all platform results are visible even if one fails
4. Example Structure
jobs:
e2e-vite-plus:
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v6
- uses: ./
with:
version: latest
cache: false
- uses: actions/checkout@v6
with:
repository: voidzero-dev/vite-plus
path: ecosystem/vite-plus
- name: Install and build
working-directory: ecosystem/vite-plus
run: |
vp install
vp run build
- name: Run tests
working-directory: ecosystem/vite-plus
run: vp run testAcceptance Criteria
- e2e workflow exists and runs on all 3 platforms
- At least 2 ecosystem repositories are included in the matrix
-
vp exec,vp run, andvp installare all exercised - Scheduled runs catch upstream breakage early
- The e2e test would have caught Windows runner failures with latest action version #10 before it reached users
Related
- Windows runner failures with latest action version #10 — Windows runner failures with latest action version
- fix: use staging install scripts to fix Windows runner failures #10 #11 — Temporary fix using staging install scripts
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels
Type
Fields
Give feedbackPriority
None yet