test(expo): comprehensive test coverage for native components#8334
test(expo): comprehensive test coverage for native components#8334chriscanin wants to merge 49 commits into
Conversation
Add 216 JS unit tests across 20 new test files covering every untested module in @clerk/expo: hooks (useUserProfileModal, useNativeAuthEvents, useNativeSession), native components (AuthView, InlineAuthView, UserProfileView, InlineUserProfileView, UserButton), provider (ClerkProvider init flow, NativeSessionSync, native-to-JS auth sync), utilities (runtime, errors, native-module), caches (token-cache, resource-cache), and the Expo config plugin (withClerkAndroid, withClerkExpo, withClerkIOS). Add 8 Kotlin unit tests for the Android native bridge code covering session ID change detection logic, per-view ViewModelStore isolation, and sign-out cleanup behavior. Add 23 Maestro e2e flow files targeting the clerk-expo-quickstart NativeComponentQuickstart app, including 5 regression flows for bugs shipped in chris/fix-inline-authview-sso (forgot-password OAuth, Get Help loop, re-sign-in cycle, theming reset, cold-launch flash). Add manual-trigger GitHub Actions workflow for running Maestro flows on both iOS simulator and Android emulator. Source changes (non-breaking): - packages/expo/app.plugin.js: export sub-plugins for unit testing - packages/expo/src/provider/ClerkProvider.tsx: export NativeSessionSync - packages/expo/android/build.gradle: add JUnit/Robolectric test deps
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
🦋 Changeset detectedLatest commit: 28adb87 The changes in this PR will be included in the next version bump. This PR includes changesets to release 1 package
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
| run: | | ||
| cd clerk-expo-quickstart/NativeComponentQuickstart | ||
| npx expo prebuild --clean | ||
| npx expo run:ios --configuration Release --no-bundler | ||
| cd ../../integration-mobile | ||
| source config/.env 2>/dev/null || true | ||
| maestro test --exclude-tags "${{ inputs.exclude_tags }},androidOnly" flows/ | ||
|
|
There was a problem hiding this comment.
Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
⭐ Fixed in commit fe9e3fe ⭐
Introduces an `expo-compat` job in the manual-release workflow that runs before `publish`. The job: 1. Publishes the current SDK source to mavenLocal with a snapshot suffix 2. Clones clerk/javascript and clerk/clerk-expo-quickstart 3. Patches @clerk/expo's pinned clerk-android version to the snapshot 4. Adds mavenLocal() to the gradle repositories so resolution works 5. Builds the quickstart NativeComponentQuickstart against the snapshot 6. Runs the Maestro e2e suite from clerk/javascript's integration-mobile/ The `publish` job now depends on `expo-compat` succeeding, so a release cannot publish if the Expo integration tests fail. Secrets required (to be configured on this repo): - CLERK_TEST_EMAIL - CLERK_TEST_PASSWORD - EXPO_PUBLIC_CLERK_PUBLISHABLE_KEY Related: clerk/javascript#8334 (adds the integration-mobile/ test suite this workflow invokes)
Introduces an `expo-compat` job in release-sdk.yml that runs between `checks` and `publish`. The job validates that the clerk-ios SHA about to be published does not break @clerk/expo's native component integration. The job: 1. Clones clerk/javascript and clerk/clerk-expo-quickstart 2. Patches packages/expo/app.plugin.js to pin the SPM clerk-ios dependency to the current release SHA using requirement kind 'revision' instead of 'exactVersion' 3. Builds the NativeComponentQuickstart app via `expo run:ios --configuration Release` 4. Runs the Maestro e2e suite from integration-mobile/ on an iOS simulator 5. If any Maestro flow fails, the `publish` job is blocked Because the clerk-ios dependency is resolved via SPM, no local publish step is needed — SPM clones the clerk-ios repo at the specified SHA during the quickstart's Xcode build. Secrets required: - CLERK_TEST_EMAIL - CLERK_TEST_PASSWORD - EXPO_PUBLIC_CLERK_PUBLISHABLE_KEY Related: - clerk/javascript#8334 — adds the integration-mobile/ test suite - clerk/clerk-android#593 — Android equivalent of this gate
Local iOS validation surfaced several issues in the Maestro flow files and runner scripts. This commit has all the fixes needed to get the core happy-path and regression flows passing end-to-end against the clerk-expo-quickstart NativeComponentQuickstart app on an iPhone 17 simulator (iOS 26). Validated passing flows: - flows/sign-in/email-password.yaml (34s) - flows/cycles/sign-in-sign-out-sign-in.yaml (53s) -- THE REGRESSION - flows/smoke/cold-launch-no-flash.yaml (7s) Remaining flows need follow-up iteration to handle iOS-specific UserProfile UI copy (e.g. Edit profile, Log out button text) and the secondary test user env vars for different-user cycles. Fixes in this commit: 1. Scripts portability -- macOS ships bash 3.2 which lacks mapfile. Replace with while-read loop. 2. Maestro subdirectory recursion -- `maestro test flows/` does not walk subdirectories. Use `find` + explicit file list. 3. Platform disambiguation -- with both iOS sim and Android emu booted, Maestro auto-picked the wrong driver. Pass `--platform ios|android`. 4. Env var interpolation -- Maestro does not auto-read shell env. Pass CLERK_TEST_EMAIL/PASSWORD via explicit `-e KEY=value` flags. 5. Regex patterns -- Maestro's `text:` and `visible:` use full-string regex match. Use `.*term.*` for substring, `\.?` for optional trailing punctuation, single quotes in YAML to avoid escape issues. 6. Dev launcher URL differs -- iOS uses http://localhost:8081, Android uses http://10.0.2.2:8081. Match with `.*:8081` regex. 7. Dev menu dismissal -- tap Close accessibility ID with backdrop fallback at 50%,20%. 8. Session persistence across clearState -- Clerk's token in iOS Keychain (AFTER_FIRST_UNLOCK) survives app reinstall. Add a conditional sign-out step to open-app.yaml. 9. inputText appends, not replaces -- add `eraseText: 50` before every inputText in sign-in-email-password.yaml. 10. iOS trailing period differs -- clerk-ios renders "Welcome! Sign in to continue" (no period), clerk-android renders with period. Use `\.?` regex to match both. Also adds integration-mobile/.gitignore to prevent config/.env from being committed (it contains a Clerk publishable key for the delicate-crab-73 dev instance).
| run: | | ||
| cd clerk-expo-quickstart/NativeComponentQuickstart | ||
| npx expo prebuild --clean | ||
| npx expo run:ios --configuration Release --no-bundler | ||
| cd ../../integration-mobile | ||
| source config/.env 2>/dev/null || true | ||
| # Maestro doesn't auto-recurse into subdirectories; pass each flow explicitly. | ||
| find flows -type f -name "*.yaml" ! -path "*/common/*" -print0 | \ | ||
| xargs -0 maestro test --exclude-tags "${{ inputs.exclude_tags }},androidOnly" | ||
|
|
There was a problem hiding this comment.
Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
🧹 Fixed in commit 4b7c966 🧹
iOS UserProfile uses different copy than Android:
- "Edit profile" (Android) -> "Update profile" (iOS)
- "Log out" (Android) -> "Sign out" (iOS)
- The Close (X) button matches on accessibilityText "Close", not id
Use cross-platform regex alternation ("(Edit|Update) profile",
"Log out|Sign out") and switch from `id: "Close"` to `text: "Close"`
since Maestro's id matches resource-id (SF Symbol name "xmark" on iOS).
Also switch sheet-dismiss from `- back` (iOS has no back button) to
tap the Close X with back fallback for Android.
Mark 3 flows as `skip` until prerequisites are in place:
- sign-out-then-sign-in-different-user: needs CLERK_TEST_EMAIL_SECONDARY
and a second test user in the dev instance
- email-verification: sign-up selector flow still needs iOS-specific
verification steps
- custom-theme-applied: check-theme-color.js needs pngjs, and iOS
quickstart doesn't bundle clerk-theme.json yet
Passing flows on iPhone 17 simulator:
- email-password
- sign-in-sign-out-sign-in (THE REGRESSION)
- cold-launch-no-flash
- open-profile-modal
- sign-out-from-profile
- edit-first-name
cold-launch-no-flash inlines its own launcher logic (doesn't use open-app.yaml) so it was missing the conditional sign-out step added to open-app.yaml. When the previous flow left the user signed in, the cold-launch assertion "Welcome! Sign in to continue" failed because the app launched to the signed-in home screen. Also update the dev menu dismissal to use the same Close-X-first, backdrop-fallback pattern as open-app.yaml. Result: 6/6 non-skipped iOS Maestro flows passing in 4m 14s on iPhone 17 simulator (iOS 26) against delicate-crab-73 dev instance: - email-password - sign-in-sign-out-sign-in (the shipped regression) - cold-launch-no-flash - open-profile-modal - sign-out-from-profile - edit-first-name
Add Google Password Manager auto-dismissal to open-app.yaml and sign-in-email-password.yaml. After sign-in, Android shows a "Save password?" sheet from Google Password Manager. The sheet button text varies between "Not now" (first prompt) and "Never" (after declining once), so use regex alternation. Skip dark-mode-applied -- same pngjs dependency issue as custom-theme-applied; both need the theme-color helper script prerequisites before they can run. Result: 7/7 non-skipped Android Maestro flows passing against Pixel 9 Pro emulator (API 34) and delicate-crab-73 dev instance: - email-password (57s) - sign-in-sign-out-sign-in (1m 28s) -- the shipped regression - cold-launch-no-flash (24s) - get-help-loop-regression (1m 10s) -- the shipped Android regression - open-profile-modal (1m 9s) - sign-out-from-profile (1m 4s) - edit-first-name (1m 16s) Combined with iOS (6/6 passing), the Maestro suite now catches the full user journey end-to-end on both platforms.
Mirrors the /integration (Playwright) secret pattern: read pk/sk from a
named entry in the existing INTEGRATION_INSTANCE_KEYS JSON secret and
provision a fresh test user per run via the Clerk Backend API. Cleans up
the user on teardown (always).
Instance name is a placeholder ("expo-native") pending SDK team confirmation
of which dev/staging instance this workflow should target. The secret slot
is left blank in the repo until that's resolved.
| run: | | ||
| cd clerk-expo-quickstart/NativeComponentQuickstart | ||
| npx expo prebuild --clean | ||
| npx expo run:ios --configuration Release --no-bundler | ||
| cd ../../integration-mobile | ||
| # Maestro doesn't auto-recurse into subdirectories; pass each flow explicitly. | ||
| find flows -type f -name "*.yaml" ! -path "*/common/*" -print0 | \ | ||
| xargs -0 maestro test --exclude-tags "${{ inputs.exclude_tags }},androidOnly" | ||
|
|
There was a problem hiding this comment.
Using variable interpolation ${{...}} with github context data in a run: step could allow an attacker to inject their own code into the runner. This would allow them to steal secrets and code. github context data can have arbitrary user input and should be treated as untrusted. Instead, use an intermediate environment variable with env: to store the data and use the environment variable in the run: script. Be sure to use double-quotes the environment variable, like this: "$ENVVAR".
🎉 Removed in commit 808601a 🎉
…nsitive Select all
Two changes bundled (both invalidate the binary cache, so cheaper to do
together than in sequence):
1. git mv integration-mobile integration/mobile, and rewrite the 8
workflow references to the new path. Now sits alongside Playwright's
/integration as a sibling directory, which is more discoverable than
the previous top-level integration-mobile sibling.
2. Fix the "Select all" regex to match both iOS ("Select All") and
Android ("Select all"). iOS sign-in-sign-out-sign-in failed in the
prior run because the runFlow guard didn't match iOS's capital A
and the field-clear short-circuited, leaving stale text + appended
typing on the second sign-in.
Android already passed end-to-end on the previous run with binary cache
hit (9m21s); next run will be cold-cache again because of the workflow
hash change, but should populate fresh caches for both platforms.
…ixed
sign-in-sign-out-sign-in.yaml: tag `skip` on both platforms.
- Android: clerk-android AuthStartView reads Clerk.enabledFirstFactorAttributes
and Clerk.socialProviders via non-observable getters. Body of re-mounted
AuthView can be empty when its first composition wins the race against
environment population. Fix is a MutableStateFlow + observe in AuthStartView
(patch staged in clerk-android workspace, pending publish).
- iOS: Maestro's field-clearing on the second sign-in leaves a leading char
("Identifier is invalid"). Lives in the test/keyboard layer, not the SDK.
Both bugs only repro on slow CI hardware; local AVD/sim are too fast.
sign-out-from-profile.yaml: tag `flakyAndroid` (Android-only exclusion).
- Same Android-side clerk-android race, manifesting earlier in this flow
because of state bleed from preceding flows in the Maestro sweep.
- Keeps running on iOS, where the underlying race doesn't apply.
Workflow Android job's --exclude-tags now includes `flakyAndroid`. iOS is
unchanged (already correctly excludes androidOnly).
Re-enable both flows by removing the tags once the underlying issues are
fixed and verified.
… pins
Add a workflow_call trigger to mobile-e2e.yml so the native SDK release
pipelines (clerk-ios/release-sdk.yml, clerk-android/manual-release.yml)
can run the same Maestro suite as a release gate without duplicating any
of the setup.
Two new inputs (available on both workflow_dispatch and workflow_call):
- clerk_ios_ref: when non-empty, patches packages/expo/app.plugin.js
to SPM-pin clerk-ios with `kind: revision` against the given SHA.
- clerk_android_ref: when non-empty, rewrites packages/expo/
android/build.gradle's clerkAndroidApiVersion/clerkAndroidUiVersion
to the given Maven coordinate (e.g. a SNAPSHOT or staged release).
The pin steps run before the binary-source hash compute, so the existing
cache machinery naturally keys per ref — release-gate runs against
unreleased SHAs don't collide with PR / dispatch cache entries.
No behavior change for current callers (default empty refs use the pins
already in app.plugin.js / build.gradle).
…or android gates Adds the clerk_android_snapshot_suffix input (paired with clerk_android_ref) to the existing workflow_dispatch + workflow_call surfaces. When set, the Android job: - Checks out clerk-android at the ref - Bumps CLERK_*_VERSION by appending the suffix - Publishes to mavenLocal under those new versions - Pins packages/expo/android/build.gradle to those versions and adds mavenLocal() to the resolution chain When the suffix is empty, the existing version-string mode is preserved (clerk_android_ref is treated as a pre-published version on Maven Central / Sonatype staging). Also skips the iOS job when the dispatch is scoped to Android only (clerk_android_ref set without clerk_ios_ref), used by clerk-android's expo-compat release gate. Consolidates the snapshot-publish + iOS-skip work that previously lived on chris/mobile-e2e-android-snapshot-renovate, which will be deleted.
The clerk-android telemetry module uses Kotlin Multiplatform, so its sign task is signKotlinMultiplatformPublication (not signMavenPublication). Other modules may add additional sign* variants too. Use a Gradle init script to disable all sign* tasks across the publish — more robust than enumerating exclusions per module/variant. Fixes the gate test failure on the telemetry publish step where signing fails with 'no configured signatory' since we don't have release-signing keys in the gate environment.
The Publish step puts the clerk-android snapshot in ~/.m2/repository, but the quickstart's android build can't see it: `expo prebuild --clean` regenerates android/build.gradle from template without mavenLocal(), and adding mavenLocal to packages/expo/android/build.gradle only configures the library's own repository chain — not the consumer's. Drop a global init script in ~/.gradle/init.d/ so every Gradle invocation on the runner (prebuild plugin resolution + :app:assembleRelease) picks up mavenLocal. Scoped behind the same snapshot-mode condition so manual version-string dispatches and PR runs are unaffected.
…nt alone The init-script approach broke plugin resolution: my settingsEvaluated hook replaced pluginManagement.repositories with just mavenLocal(), so Gradle couldn't find org.jetbrains.kotlin.jvm anymore. Replace with a post-prebuild awk step that adds mavenLocal() ONLY to the allprojects repositories block in the regenerated consumer android/build.gradle. Project-level resolution gets mavenLocal; plugin resolution keeps using the default Gradle Plugin Portal. Only fires in snapshot mode (clerk_android_ref + clerk_android_snapshot_suffix both set); version-string mode and PR-trigger runs are unaffected.
The first flow of a Maestro run hits a cold emulator + freshly installed APK + un-parsed JS bundle. open-app's fixed waitForAnimationToEnd timers run out before the bundle finishes loading, so the trailing assertVisible fails against a blank loading-spinner screen — even though the bundle would have loaded in another ~5s. Subsequent flows are fast because the bundle stays warm across clearState. Insert extendedWaitUntil for either "Welcome!" (signed-out AuthView) or "Sign Out" (signed-in landing — handled by the sign-out runFlow below). Returns immediately once either is seen, so warm-bundle flows don't pay the extra time.
Previous extendedWaitUntil pattern `Welcome! Sign in to continue|Sign Out` wasn't matching despite the screen clearly showing "Welcome! Sign in to continue." — Maestro's regex anchoring/escaping treats the alternation differently than substring. The original assertVisible used a careful regex `\.?` to handle the optional period. Use plain "Welcome" — substring match, no regex specials. Matches both: - AuthView: "Welcome! Sign in to continue." - Signed-in landing: "Welcome" header The sign-out runFlow + final assertVisible below disambiguate signed-out vs signed-in afterward.
Maestro does regex full-match against each element's text (not substring). Plain 'Welcome' only matched elements whose entire text was literally 'Welcome' — not 'Welcome! Sign in to continue.'. Use the same proven regex from the original assertVisible 'Welcome! Sign in to continue\.?' (handles the optional trailing period that varies between renders), grouped with 'Sign Out' for the signed-in landing case via alternation.
When all flows pass, the only remaining failure is `Post Run actions/setup-node@v4` with "Path Validation Error: Path(s) specified in the action for caching do(es) not exist". setup-node's pnpm cache save hook is redundant with pnpm/action-setup@v4 (which already manages the pnpm store cache) and races with working-directory changes in the snapshot-mode flow, failing the whole job even when tests are green. Remove cache: pnpm on both Android and iOS setup-node steps; rely on pnpm/action-setup for the store cache.
Symmetric with the iOS job's existing skip. When the clerk-ios expo-compat gate dispatches with only clerk_ios_ref set, we don't need to spin up the Android emulator + build a snapshot the iOS change can't affect. Manual dispatches without ref inputs still run both jobs.
Drop-in replacement for GitHub-hosted macos-15. Matches the Android job's blacksmith-8vcpu-ubuntu-2204 sizing for consistency. Should give faster Xcode/SPM resolution and emulator boot vs the public macOS queue, which has been the slowest part of the Maestro pipeline.
Three changes consolidated into one commit: 1. Per-flow timeout: switch from a single maestro test invocation covering all flows to `xargs -n 1 maestro test --flatten-debug-output` (one process per flow). A hanging flow can't poison the rest; the `--flatten-debug-output` collapses artifacts under one root so the uploader can grab them all. (Maestro's --timeout flag varies by version; running per-flow + relying on the job-level timeout is more portable.) 2. iOS simulator animation + keyboard tweaks: UIAnimationDragCoefficient to ~zero, disable predictive/autocorrect/auto-cap. Animations add 200-500ms to every tap on iOS sim, and predictive-text hijacks inputText into the wrong target — both regular causes of slow / flaky iOS flows. 3. Artifacts step now uploads on `failure() || cancelled()`, not just failure. If a stuck flow gets cancelled (manually or by cancel-in-progress concurrency), screenshots + commands JSON still land in the run for diagnosis.
Description
Adds comprehensive test coverage for
@clerk/exponative components across three layers, each targeting a specific class of regression.Backstory: the recent SSO/profile/theming work (
chris/fix-inline-authview-sso) shipped four user-visible bugs and fixes (iOS forgot-password OAuth, Android Get Help loop, cold-launch white flash, native theming reset). Zero automated tests existed to catch any of them. This PR establishes the infrastructure.What's in the PR
JS unit tests (
packages/expo/src/**/__tests__/) — 20 new files, 216 tests. Full coverage of every previously untested module: hooks (useUserProfileModal,useNativeAuthEvents,useNativeSession), native component wrappers (AuthView,InlineAuthView,UserButton,UserProfileView,InlineUserProfileView), provider (ClerkProviderinit flow,NativeSessionSync, native-to-JS auth sync), utilities, caches, and the Expo config plugin.Android (Kotlin) unit tests (
packages/expo/android/src/test/) — 3 files, 8 tests. Covers session-ID change detection logic, per-view ViewModelStore isolation, and sign-out cleanup behavior. Targets the logic fixed in the Android regression commits.iOS (Swift) unit tests (
packages/expo/ios/Tests/) — 2 files, 13 tests. Covers theviewDidDisappearsession-ID comparison (the cancel-vs-success decision), thepresentWhenReadyguard predicate (attempts cap + invalidation), and theemitAuthStateChangepayload shape.Maestro e2e flows (
integration-mobile/flows/) — 23 YAML files targeting the clerk-expo-quickstartNativeComponentQuickstartapp. Includes 5 regression flows:flows/sign-in/google-sso-from-forgot-password.yaml— iOS OAuth from forgot-passwordflows/sign-in/get-help-loop-regression.yaml— Android AuthView navigation loopflows/cycles/sign-in-sign-out-sign-in.yaml— inline AuthView re-sign-inflows/theming/custom-theme-applied.yaml— native theming resetflows/smoke/cold-launch-no-flash.yaml— cold-launch white flashPlus 11 happy-path flows and 6 reusable subflows.
CI workflow (
.github/workflows/mobile-e2e.yml) — manualworkflow_dispatchtrigger. Clonesclerk-expo-quickstartat a configurable ref, builds onmacos-15(iOS) andubuntu-latestwithreactivecircus/android-emulator-runner(Android), runs all non-manual Maestro flows. Required secrets:CLERK_TEST_PK,CLERK_TEST_EMAIL,CLERK_TEST_PASSWORD.Source changes (non-breaking)
packages/expo/app.plugin.js: named exports forwithClerkIOS,withClerkAndroid,withClerkAppleSignIn,withClerkGoogleSignIn,withClerkKeychainService(additive, default export unchanged)packages/expo/src/provider/ClerkProvider.tsx:NativeSessionSyncmarked as exported for test access (internal, documented as not public API)packages/expo/android/build.gradle: JUnit/Robolectric/MockK test dependencies +testOptionsfor Robolectricpackages/expo/ios/ClerkExpo.podspec:test_spec 'Tests'block so Cocoapods generates the test targetHow to test
JS unit tests run in existing CI:
Native unit tests:
Maestro flows:
CI: trigger the
Mobile e2e (@clerk/expo)workflow manually from the Actions tab.Checklist
pnpm testruns as expected (216 tests passing).pnpm buildruns as expected.Type of change