diff --git a/.claude/skills/uts-to-kotlin/SKILL.md b/.claude/skills/uts-to-kotlin/SKILL.md index f2e8ffe35..ca1cda668 100644 --- a/.claude/skills/uts-to-kotlin/SKILL.md +++ b/.claude/skills/uts-to-kotlin/SKILL.md @@ -1,85 +1,161 @@ --- -description: "Translate a UTS pseudocode test spec into Kotlin tests in the uts module. Usage: /uts-to-kotlin " -allowed-tools: Bash, Read, Edit, Write +description: "Translate the UTS pseudocode test specs in a whole module directory into runnable Kotlin tests in the ably-java uts module. Takes a UTS module directory (e.g. .../specification/uts/objects), validates its structure, resolves the target ably-java module, lets you pick a tier (unit/integration/proxy) and which specs, then derives a Kotlin test per spec. Usage: /uts-to-kotlin " +allowed-tools: Bash, Read, Edit, Write, WebFetch --- -Translate the UTS pseudocode test spec at `$ARGUMENTS` into a runnable Kotlin test in the `uts` module. +Translate the UTS pseudocode test specs under the **module directory** `$ARGUMENTS` into runnable Kotlin +tests in the ably-java `uts` module. -Reference: [Writing Derived Tests](https://raw.githubusercontent.com/ably/specification/refs/heads/main/uts/docs/writing-derived-tests.md) +`$ARGUMENTS` is a UTS *module* directory — a directory sitting directly under `.../specification/uts/`, +e.g. `/Users/sachinsh/ably-specification/specification/uts/objects`. Its name (`objects`, `realtime`, +`rest`, …) is the **source module**. A module directory holds many spec files, organised into tiers +(`unit/`, `integration/`, and `integration/proxy/`). + +The work happens in two phases: + +- **Phase 1 — Selection (Steps A–E below):** a bundled resolver script validates the directory and works + out the target ably-java package, the spec files, and their class names; you then pick a tier, pick which + specs, and choose whether to also evaluate. +- **Phase 2 — Per-spec translation (Steps 1–7):** for each selected spec file, derive a Kotlin test. + +## Required reading — fetch first + +Always fetch [writing-derived-tests.md](https://raw.githubusercontent.com/ably/specification/refs/heads/main/uts/docs/writing-derived-tests.md) first (once per run) — don't rely on memory or the inlined summaries; the manual is updated over time. --- -## Step 0 — Validate arguments +# Phase 1 — Selection -**If `$ARGUMENTS` is empty or blank**, stop immediately and tell the user: +Path validation, the package mapping, spec discovery, and class-name derivation are all mechanical, so a +bundled script does them — that keeps selection byte-for-byte deterministic instead of relying on the model +to re-eyeball regexes, join paths, and hand-convert `snake_case` → `PascalCase` each run. -``` -Usage: /uts-to-kotlin +> **If `$ARGUMENTS` is empty or blank**, stop and show: `Usage: /uts-to-kotlin ` +> — with the example `/uts-to-kotlin /Users/sachinsh/ably-specification/specification/uts/objects`. -Example: - /uts-to-kotlin lib/src/spec/uts/test/realtime/unit/connection/connection_state_machine_test.md +## Step A — Resolve the module -Please re-run the command with the path to a UTS pseudocode spec file. +Run the resolver on the directory passed in (substitute the real path; the script path is relative to the +ably-java repo root): + +```bash +python3 .claude/skills/uts-to-kotlin/scripts/resolve_uts.py "" ``` -Do not proceed to Step 1. +It prints one JSON object. **If `ok` is `false`, relay `message` to the user and stop** — error codes: +`NOT_A_UTS_MODULE_PATH` (not a `.../uts/` directory), `DIR_NOT_FOUND`, `NO_TIER_DIRS` (no `unit/` +or `integration/`). On success it gives `sourceModule`, `mapped`, `testRoot`, `translationNotes`, and a +`tiers` object with one entry per tier (`unit` / `integration` / `proxy`), each carrying `present`, +`sourceDir`, `targetDir`, `package`, and `specs` (a list of `{file, className}`). Everything downstream +reads from this output — treat it as the single source of truth and don't recompute paths or names by hand. -**If `$ARGUMENTS` is provided but does not end in `.md`**, stop and tell the user: +`translationNotes` is the path to a per-module ably-js → ably-java type/interface map when the module +declares one (its `notes` field in `uts-package-mapping.json`, e.g. `objects` → +`references/objects-mapping.md`), else `null`. When it's non-null, it is **required reading before +Phase 2** — see Step 1. -``` -Error: "" does not look like a spec file path (expected a .md file). +## Step B — Confirm or create the target mapping -Usage: /uts-to-kotlin -``` +The target dirs come from `uts-package-mapping.json` (alongside this skill); spec and ably-java module names +don't always match (e.g. `objects` → `liveobjects`), which is why it's explicit. -Do not proceed to Step 1. +- **If `mapped` is `true`**: show the resolved `targetDir` for each present tier and ask the user to confirm. + If they say the mapping is wrong, ask for the correct ably-java module base name and re-run with `--create` + (below) to overwrite the entry, then re-resolve. +- **If `mapped` is `false`**: there's no mapping for `sourceModule` yet. Ask for the target ably-java module + base name (default to `sourceModule`; suggest a rename only when the SDK uses different terminology, e.g. + `objects` → `liveobjects`), then create it deterministically and re-resolve: -**If `$ARGUMENTS` ends in `.md` but the file does not exist** (check with `test -f "$ARGUMENTS"`), stop and tell the user: + ```bash + python3 .claude/skills/uts-to-kotlin/scripts/resolve_uts.py "" --create + ``` -``` -Error: file not found: "" + This adds `unit/`, `integration/standard/`, and `integration/proxy/` under + `packages` and re-prints the resolved output. (`` must be a simple module base name — letters, + digits, underscore; the script returns `BAD_TARGET_NAME` otherwise, so just ask again.) -Check the path and try again. -``` +## Step C — Choose the tier -Do not proceed to Step 1. +Offer the tiers whose `present` is `true`. The chosen tier fixes the `targetDir`, `package`, and `specs` +(from Step A) **and** the translation flow Phase 2 uses — don't re-detect any of it per spec: -Only continue to Step 1 once the file is confirmed to exist. +| Tier | Translation flow | +|---|---| +| **unit** | mocked transport — Steps 3–4 below | +| **integration** (direct sandbox) | real sandbox, no faults — **Proxy integration tests** section, but drop the `ProxySession`/`connectThroughProxy` wiring | +| **proxy** | real sandbox + fault injection — **Proxy integration tests** section | ---- +## Step D — Choose which specs to translate -## Step 1 — Read the spec +The chosen tier's `specs` list (from Step A) is the candidate set — each entry already has its source `file` +and derived `className`. Present it and ask whether to translate **all** of them or a **selected subset**. +Then continue to Step E. -Read the file at `$ARGUMENTS`. Identify: -- All test cases — each has a structured ID like `realtime/unit/RSA4c2/callback-error-connecting-disconnected-0` and a description -- The protocol used (WebSocket for Realtime, HTTP for REST) -- Any timer usage (`enable_fake_timers`, `ADVANCE_TIME`) +## Step E — Translate only, or also evaluate? + +`writing-derived-tests.md` splits the work into **Translation** (always) and **Evaluation** (only +meaningful once the SDK implementation for this module exists). Ask the user which they want, and carry the +answer into Phase 2: + +- **Translate only** — generate each test and make it **compile** (Steps 1–5 and the Step 7 review). Don't + run the tests. Use this when the SDK feature isn't implemented yet, so there's nothing to run against. +- **Translate and evaluate** — all of the above **plus** running the tests and **fixing until they pass** + (Step 6): work the decision tree, and where the SDK genuinely diverges, gate/adapt the assertion and + record a deviation. Use this when the implementation exists. + +If you can't tell whether the implementation exists, ask the user rather than guessing. --- -## Step 2 — Determine output path and package +# Phase 2 — Per-spec translation -Map the spec path to a test path: +Run this for **each** spec file selected in Step D. **Step 6 only applies in "translate and evaluate" mode +(Step E)** — in "translate only" mode, stop after compiling (Step 5) and reviewing (Step 7), and skip +Step 6 entirely. -| Spec location | Test location | -|---|---| -| `.../uts/test/rest/unit/.md` | `uts/src/test/kotlin/io/ably/lib/rest/unit/Test.kt` | -| `.../uts/test/realtime/unit//.md` | `uts/src/test/kotlin/io/ably/lib/realtime/unit//Test.kt` | -| `.../uts/test/realtime/integration//.md` | `uts/src/test/kotlin/io/ably/lib/realtime/integration//Test.kt` | +When translating several specs, do Steps 1–4 (generate the file) for every spec first, then run Step 5 +(compile) once for the whole module, then per file run Step 6 (only if evaluating) and the Step 7 review — +compiling once is faster than per-file and surfaces cross-file issues together. For a single spec, just go +through the steps in order. -Class name: take the file name, strip `_test` suffix, convert `snake_case` → `PascalCase`, append `Test`. +## Step 1 — Read the spec (and any module translation notes) -**Integration specs that drive traffic through the programmable proxy** (they reference `create_proxy_session()`, proxy rules, or `uts/test/realtime/integration/helpers/proxy.md`) follow a different translation flow — see the **Proxy integration tests** section at the end of this skill instead of the unit-test rules below. +**If Step A reported a non-null `translationNotes`, read that file first (once per run).** UTS specs are +written in a language-agnostic pseudocode that mirrors the *ably-js* API; for modules whose ably-java types +diverge (e.g. `objects` → `liveobjects`, where ably-java is a typed SDK with a partitioned `PathObject` / +`Instance` hierarchy and a `LiveMapValue` write union), the notes map each spec symbol to its ably-java +equivalent. Skipping them yields tests that read like ably-js and won't compile. + +Then read the current spec file (the one being translated from the Step D selection). Identify: +- All test cases — each has a structured ID like `realtime/unit/RSA4c2/callback-error-connecting-disconnected-0` and a description +- The protocol used (WebSocket for Realtime, HTTP for REST) +- Any timer usage (`enable_fake_timers`, `ADVANCE_TIME`) + +--- -Example: `connection_state_machine_test.md` → `ConnectionStateMachineTest` +## Step 2 — Output path and package -Package: derived from the output path under `kotlin/`. +Don't derive anything here — the resolver (Step A) already produced it. For the chosen tier use its +`targetDir` and `package`, and for the spec being translated use its `className` from that tier's `specs` +list. Write the test to `/.kt` with `package ` at the top. + +The spec's own `` grouping (e.g. `connection/`, `channels/`) is **not** reflected in the output — every +test sits directly in `targetDir` (the resolver flattens it). The chosen tier also fixes the translation +flow: **unit** → the rules in Steps 3–4 below; **integration** (direct sandbox) and **proxy** → the **Proxy +integration tests** section (direct sandbox drops the `ProxySession`/`connectThroughProxy` wiring; see +Step C). --- ## Step 3 — Read infrastructure files -Read ALL of files in `uts/src/test/kotlin/io/ably/lib/uts/infra` before generating any code (you need exact method signatures). +Infrastructure is split by tier under `uts/src/test/kotlin/io/ably/lib/uts/infra/`: + +- `infra/Utils.kt` — shared async helpers (`awaitState`, `awaitChannelState`, `pollUntil`), package `io.ably.lib.uts.infra`. +- `infra/unit/` — unit-test mocks/factories (`ClientFactories.kt`, `MockWebSocket.kt`, `MockHttpClient.kt`, `FakeClock.kt`, `MockEvent.kt`, the `PendingConnection`/`PendingRequest` pairs, and `Utils.kt` with the `ConnectionDetails { }` builder), package `io.ably.lib.uts.infra.unit`. +- `infra/integration/` + `infra/integration/proxy/` — proxy/sandbox helpers (`SandboxApp.kt`, `ProxyManager.kt`, `ProxySession.kt`) — see the **Proxy integration tests** section. + +For a **unit** test, read all files under `infra/unit/` plus `infra/Utils.kt` before generating any code (you need exact method signatures). ## Step 4 — Generate the Kotlin test file @@ -249,8 +325,10 @@ yield() ### Test naming and annotation -- KDoc comment immediately above `@Test` using `/** @UTS */` format -- Method name: backtick string `` ` - ` `` +- KDoc comment immediately above `@Test` using `/** @UTS */` — copy the spec's **full** structured + id verbatim (e.g. `realtime/unit/RTN4a/some-description-0`; for the objects module it would start + `objects/unit/…`). Don't hand-build the prefix — use what the spec file declares. +- Method name: backtick string `` ` - ` `` — the spec point (e.g. `RTN4a`) plus a short description. - Use `runTest { }` from `kotlinx.coroutines.test` for all async tests ```kotlin @@ -266,16 +344,14 @@ fun `RTN4a - description of what is being tested`() = runTest { ### File template ```kotlin -package io.ably.lib..unit[.] +package // the resolver's package for the chosen tier (Step 2) -import io.ably.lib.TestRealtimeClient // or TestRestClient -import io.ably.lib.awaitChannelState // if testing channels -import io.ably.lib.awaitState -import io.ably.lib.realtime.ChannelState // if testing channels +import io.ably.lib.uts.infra.unit.* // TestRealtimeClient/TestRestClient, MockWebSocket, MockHttpClient, FakeClock, CONNECTED_MESSAGE, ConnectionDetails { } builder +import io.ably.lib.uts.infra.awaitState +import io.ably.lib.uts.infra.awaitChannelState // if testing channels +import io.ably.lib.uts.infra.pollUntil // if polling on a predicate +import io.ably.lib.realtime.ChannelState // if testing channels import io.ably.lib.realtime.ConnectionState -import io.ably.lib.test.mock.FakeClock // if using fake timers -import io.ably.lib.test.mock.MockWebSocket // or MockHttpClient -import io.ably.lib.types.ConnectionDetails import io.ably.lib.types.ErrorInfo import io.ably.lib.types.ProtocolMessage import kotlinx.coroutines.launch @@ -283,13 +359,13 @@ import kotlinx.coroutines.test.runTest import kotlin.test.* import kotlin.time.Duration.Companion.seconds // if using Duration literals -class Test { +class { /** - * @UTS realtime/unit// + * @UTS */ @Test - fun ` - `() = runTest { + fun ` - `() = runTest { val mock = MockWebSocket { onConnectionAttempt = { conn -> conn.respondWithSuccess(ProtocolMessage().apply { @@ -332,13 +408,26 @@ Fix any compilation errors and recompile until clean. Common issues: --- -## Step 6 — Run tests +## Step 6 — Run tests *(evaluate mode only)* + +Skip this whole step in "translate only" mode. In "translate and evaluate" mode, run the test and **keep +fixing until it passes** — either the spec-correct assertion passes, or it's deliberately gated/adapted as a +documented deviation (below). A red test is never an acceptable end state here. + +Use the per-tier task that matches the chosen tier (both are registered in `uts/build.gradle.kts`), and the +resolver's `package` + the spec's `className` for the `--tests` filter: ```bash -./gradlew :uts:test --tests "." +# unit tier → io.ably.lib.uts.unit.* +./gradlew :uts:runUtsUnitTests --tests "." + +# integration / proxy → io.ably.lib.uts.integration.* +./gradlew :uts:runUtsIntegrationTests --tests "." ``` -Handle test failures using this decision tree (see [reference doc](https://github.com/ably/specification/blob/main/uts/docs/writing-derived-tests.md) for full detail): +(`./gradlew :uts:test` still runs all tiers — unit, standard, and proxy.) + +Handle test failures using this decision tree (the **Required reading** doc you fetched up front has the full detail): ``` Test fails @@ -379,7 +468,7 @@ assertEquals(40160, error.errorInfo.code) ### Deviations file -Append to `uts/src/test/kotlin/io/ably/lib/deviations.md`. Each entry needs: +Append to `uts/src/test/kotlin/io/ably/lib/uts/deviations.md`. Each entry needs: 1. The spec point (e.g. `RSA4c2`) 2. What the spec says 3. What the SDK does @@ -413,13 +502,15 @@ For each test case, verify: - [ ] Timer setup (`enableFakeTimers`, `fakeClock.advance(...)`) matches every `enable_fake_timers` / `ADVANCE_TIME` in the spec - [ ] Channel operations (attach, detach, publish) are performed in the order the spec requires -### Deviation honesty +### Deviation honesty *(evaluate mode)* -For any place where the generated test diverges from the spec pseudocode (adapted assertion, env-gated skip, or omitted step): +Deviations are discovered by running, so this check applies in evaluate mode. For any place where the +generated test diverges from the spec pseudocode (adapted assertion, env-gated skip, or omitted step): - [ ] A `// DEVIATION:` comment explains why -- [ ] The deviation is recorded in `uts/src/test/kotlin/io/ably/lib/deviations.md` +- [ ] The deviation is recorded in `uts/src/test/kotlin/io/ably/lib/uts/deviations.md` -If you find gaps during this review, fix them and re-run Steps 5–6 before finishing. +If you find gaps during this review, fix them and re-run Step 5 (compile) — and, in evaluate mode, Step 6 — +before finishing. --- @@ -427,7 +518,7 @@ If you find gaps during this review, fix them and re-run Steps 5–6 before fini Some specs are **integration tests** that exercise fault-handling behaviour against the **real Ably sandbox** instead of a mocked transport. They route the SDK through the [`ably/uts-proxy`](https://github.com/ably/uts-proxy) — a programmable HTTP/WebSocket proxy that forwards traffic transparently by default but can inject faults (dropped connections, modified/injected/delayed frames, error responses) via rules. -Recognise them by: a reference to `create_proxy_session()`, proxy `rules`, `trigger_action`, `get_log`, or a pointer to `uts/test/realtime/integration/helpers/proxy.md`. +Recognise them by: a reference to `create_proxy_session()`, proxy `rules`, `trigger_action`, `get_log`, or a pointer to `uts/realtime/integration/helpers/proxy.md`. ### When proxy tests are the right tool @@ -439,11 +530,13 @@ Recognise them by: a reference to `create_proxy_session()`, proxy `rules`, `trig ### Infrastructure -Three helpers live in `uts/src/test/kotlin/io/ably/lib/test/helper/`. **Read them before translating a proxy spec** — they hold the exact method signatures. +Three helpers live under `uts/src/test/kotlin/io/ably/lib/uts/infra/integration/`. **Read them before translating a proxy spec** — they hold the exact method signatures. + +- **`ProxyManager`** (`infra/integration/proxy/ProxyManager.kt`, package `io.ably.lib.uts.infra.integration.proxy`) — downloads/starts the shared `uts-proxy` process and exposes the sandbox host. Call `ProxyManager.ensureProxy()` once per suite in setup. `ProxyManager.sandboxRealtimeHost` / `sandboxRestHost` are the upstream sandbox hosts (the default target of every session). +- **`ProxySession`** (`infra/integration/proxy/ProxySession.kt`, same package) — one programmable session wrapping the proxy control API; also defines the `connectThroughProxy` extension and the rule-builder helpers. +- **`SandboxApp`** (`infra/integration/SandboxApp.kt`, package `io.ably.lib.uts.infra.integration`) — provisions/deletes a sandbox test app from the shared `test-app-setup.json` in ably-common. `SandboxApp.create()` returns a `SandboxApp` with `appId`, `defaultKey`, and `keys` (`defaultKey` is a full-capability `appId.keyId:keySecret`); `app.delete()` tears it down. Provision in suite setup, delete in teardown. -- **`ProxyManager`** — downloads/starts the shared `uts-proxy` process and exposes the sandbox host. Call `ProxyManager.ensureProxy()` once per suite in setup. `ProxyManager.sandboxRealtimeHost` / `sandboxRestHost` are the upstream sandbox hosts (the default target of every session). -- **`ProxySession`** — one programmable session wrapping the proxy control API. -- **`SandboxApp`** — provisions/deletes a sandbox test app from the shared `test-app-setup.json` in ably-common. `SandboxApp.create()` returns a `SandboxApp` with `appId`, `defaultKey`, and `keys` (`defaultKey` is a full-capability `appId.keyId:keySecret`); `app.delete()` tears it down. Provision in suite setup, delete in teardown. +Import these into a proxy test from their packages, e.g. `io.ably.lib.uts.infra.integration.SandboxApp`, `io.ably.lib.uts.infra.integration.proxy.{ProxyManager, ProxySession, connectThroughProxy}`, plus `io.ably.lib.uts.infra.unit.TestRealtimeClient` and `io.ably.lib.uts.infra.{awaitState, pollUntil}`. `ensureProxy()`, the `ProxySession` methods, and the `SandboxApp` methods are all **`suspend`** functions. Per-test bodies use `runTest { }`; JUnit5 `@BeforeAll`/`@AfterAll` (with `@TestInstance(Lifecycle.PER_CLASS)`) wrap their suspend calls in `runBlocking { }`. @@ -455,9 +548,9 @@ Give every proxy integration test class this KDoc: /** * Proxy integration test against Ably Sandbox endpoint. * - * Uses the programmable proxy (`uts/test/proxy/`) to inject transport-level faults while the + * Uses the programmable uts-proxy to inject transport-level faults while the * SDK communicates with the real Ably backend. See - * `uts/test/realtime/integration/helpers/proxy.md` for proxy infrastructure details. + * `uts/realtime/integration/helpers/proxy.md` for proxy infrastructure details. */ ``` @@ -589,4 +682,4 @@ assertNotNull(queryParams["resume"]) 5. Use generous timeouts (10–30s) — real network is involved: `awaitState(client, ConnectionState.connected, 15.seconds)`. 6. Don't set `fallbackHosts`; explicit hosts already disable fallbacks. -Steps 5 (compile) and 6 (run) still apply. Note that proxy tests hit the live sandbox and download the proxy binary on first run, so they are slower and require network access. +Step 5 (compile) still applies; Step 6 (run) applies only in evaluate mode (Step E). Note that proxy tests hit the live sandbox and download the proxy binary on first run, so they are slower and require network access. diff --git a/.claude/skills/uts-to-kotlin/references/objects-mapping.md b/.claude/skills/uts-to-kotlin/references/objects-mapping.md new file mode 100644 index 000000000..cbe977f0c --- /dev/null +++ b/.claude/skills/uts-to-kotlin/references/objects-mapping.md @@ -0,0 +1,628 @@ +# `objects` UTS → ably-java `liveobjects`: ably-js ⇄ ably-java type/interface map + +Read this **before translating any spec from the `objects` module** (target ably-java module +`liveobjects`). The `objects` UTS specs are written in a language-agnostic pseudocode that mirrors the +**ably-js** LiveObjects API — a dynamically-typed surface with a single polymorphic `PathObject` / +`Instance`, `Promise`-returning mutators, and raw JS values. ably-java is a **statically-typed SDK** and +implements the *Typed-SDK variant* of the spec (`RTTS1`–`RTTS10` in `objects-features.md`): it partitions +that one polymorphic class into a typed hierarchy and wraps write values in a union type. So almost every +spec line needs a mechanical rewrite, not a literal transcription. This doc is that rewrite table. + +The canonical bridge is the spec's own Interface Definition (`## Interface Definition {#idl}`) and its +`=== Typed-SDK variant (RTTS1-RTTS10) ===` block — ably-java follows the typed variant verbatim. When in +doubt, that IDL is the source of truth; this doc is the applied version of it for ably-java. + +## Table of contents + +1. [The three layers (don't conflate them)](#1-the-three-layers) +2. [Entry point & channel access](#2-entry-point) +3. [Async: Promise/await → CompletableFuture](#3-async) +4. [Dynamic `PathObject` → typed `PathObject` hierarchy](#4-pathobject) +5. [Dynamic `Instance` → typed `Instance` hierarchy](#5-instance) +6. [Creation value types & the `LiveMapValue` union](#6-value-types) +7. [Mutations (set / remove / increment / decrement)](#7-mutations) +8. [Subscriptions, listeners & events](#8-subscriptions) +9. [Sync-state events (`object.on('synced')`)](#9-sync-state) +10. [`ValueType` & type discrimination](#10-valuetype) +11. [Message / operation types (`PublicAPI::ObjectMessage` →)](#11-messages) +12. [Errors & error codes](#12-errors) +13. [Internal-graph types (unit specs) — important caveat](#13-internal-graph) +14. [Worked example](#14-worked-example) +15. [Quick symbol index](#15-symbol-index) + +--- + +## 1. The three layers + +The single biggest source of confusion: the spec uses the names `LiveMap` / `LiveCounter` for **two +different things**, and a third *internal* layer underneath. Keep them straight: + +| Layer | Spec name | ably-js | ably-java | Package | +|---|---|---|---|---| +| **Creation value type** — immutable blueprint you pass *into* `set` | `LiveMap` / `LiveCounter` (the `RTLMV*` / `RTLCV*` classes) | `LiveMap.create()` / `LiveCounter.create()` | `LiveMap` / `LiveCounter` | `io.ably.lib.liveobjects.value` | +| **Public read/write view** — what you navigate & subscribe on | `PathObject`, `Instance` | `PathObject`, `Instance` | typed hierarchy (§4, §5) | base in `io.ably.lib.liveobjects.path` / `.instance`; **typed subtypes in `.path.types` / `.instance.types`** | +| **Internal graph object** — the live CRDT node | `InternalLiveMap` / `InternalLiveCounter` (`RTLM*` / `RTLC*`), `ObjectsPool` | internal | `DefaultLiveMap` / `DefaultLiveCounter` etc. (impl, `:liveobjects` module) | not public — see §13 | + +So when a spec says `counter = LiveCounter.create(5)` and passes it to `set`, that's the **value type** +(`io.ably.lib.liveobjects.value.LiveCounter`). When a spec says "the resolved value is an +`InternalLiveCounter` with `.data == 5`", that's the **internal graph node** (§13). When a spec navigates +`root.get("counter").value()`, that's the **public view** (`PathObject`). + +--- + +## 2. Entry point & channel access + +| Spec / ably-js | ably-java (Kotlin) | +|---|---| +| `channel.object` (objects entry point) | `` channel.`object` `` — a **public field** of type `RealtimeObject`, *not* a method. (Declared `public RealtimeObject object;` on `ChannelBase`.) | +| `root = AWAIT channel.object.get()` | `` val root: LiveMapPathObject = channel.`object`.get().await() `` — returns `CompletableFuture` (always a `LiveMapPathObject`, per `RTTS6d`/`RTO23f`). | +| `channel.object.get()` (ably-js generic) | **No generic.** ably-java is untyped at the root; you always get a `LiveMapPathObject` and narrow downstream with `as*` casts (§4). Drop the type parameter entirely. | +| Channel needs object modes | `TestRealtimeClient { … }` then `channels.get(name, ChannelOptions().apply { modes = arrayOf(ChannelMode.object_subscribe, ChannelMode.object_publish) })`. (`ChannelMode` constants are lower-case: `object_subscribe`, `object_publish`; `ChannelOptions.modes` is a `ChannelMode[]`.) | + +> ⚠️ **`object` is a hard keyword in Kotlin.** The entry-point field is named `object` in Java, so from +> Kotlin you **must** escape it with backticks: `` channel.`object` ``. Bare `channel.object` is a compile +> error. This applies everywhere in this doc and in generated tests. + +`RealtimeObject` extends `ObjectStateChange` (sync-state subscription, §9). When the plugin isn't installed +the field is `RealtimeObject.Unavailable.INSTANCE`; real tests install it. + +--- + +## 3. Async: Promise / await → CompletableFuture + +Every spec `AWAIT`/Promise-returning call maps to a `CompletableFuture<…>` in ably-java: + +| Spec / ably-js | ably-java return type | +|---|---| +| `AWAIT channel.object.get()` | `CompletableFuture` | +| `AWAIT pathObj.set(k, v)` / `.remove(k)` | `CompletableFuture` | +| `AWAIT counterObj.increment(n)` / `.decrement(n)` | `CompletableFuture` | +| `AWAIT instance.set(...)` etc. | `CompletableFuture` | + +Subscriptions are **not** futures — `subscribe(...)` returns a `Subscription` synchronously (`@NonBlocking`). + +**Awaiting a `CompletableFuture` inside a `runTest { }` body:** use `future.await()` with +`import kotlinx.coroutines.future.await` — the `future` integration ships inside `kotlinx-coroutines-core` +(verified on the version the uts module resolves), so no extra dependency is needed. Use the blocking +`future.get(timeout, unit)` only if a specific test needs to assert synchronously; prefer `await()` so the +test stays within structured concurrency. + +--- + +## 4. Dynamic `PathObject` → typed `PathObject` hierarchy + +In the spec/ably-js a `PathObject` is polymorphic: `get`, `at`, `value`, `set`, `increment`, `entries`… +all hang off the one object. In ably-java the base `PathObject` exposes **only** the type-agnostic methods; +everything type-specific is moved onto a sub-interface you reach via an `as*` cast. + +**Base `PathObject`** (`io.ably.lib.liveobjects.path.PathObject`) — always available. The typed sub-types +returned by the `as*` casts (`LiveMapPathObject`, `LiveCounterPathObject`, `NumberPathObject`, +`StringPathObject`, `BooleanPathObject`, `BinaryPathObject`, `JsonObjectPathObject`, `JsonArrayPathObject`) +live in **`io.ably.lib.liveobjects.path.types`** (not `.path`) — import them from there. + +| Spec / ably-js | ably-java | +|---|---| +| `pathObj.path()` | `pathObj.path(): String` | +| `pathObj.instance()` | `pathObj.instance(): Instance?` (null if path resolves to a primitive, or doesn't resolve) | +| `pathObj.compactJson()` | `pathObj.compactJson(): JsonElement?` | +| `pathObj.compact()` | **Not implemented in ably-java** (`RTTS3f`: typed SDKs need not implement `compact`). Use `compactJson()` for snapshot assertions; if a spec genuinely needs the non-JSON `compact()` shape, that's a deviation — flag it. | +| `pathObj.subscribe(listener[, opts])` | `pathObj.subscribe(PathObjectListener[, PathObjectSubscriptionOptions]): Subscription` (§8) | +| *(typed-SDK addition)* exists check | `pathObj.exists(): Boolean` (`RTTS4a`) | +| `pathObj.getType()` | `pathObj.getType(): ValueType?` — null when nothing resolves (§10) | +| — cast helpers — | `asLiveMap()`, `asLiveCounter()`, `asNumber()`, `asString()`, `asBoolean()`, `asBinary()`, `asJsonObject()`, `asJsonArray()` | + +**`PathObject` casts never throw** (`RTTS5d`) — they only re-wrap. A wrong cast surfaces later: read ops on +the wrong-typed view return `null`/empty; write ops throw (§12). So `root.get("k").asNumber().value()` +returns `null` if `k` isn't a number, rather than throwing. + +**Map-only methods** — require `asLiveMap()` → `LiveMapPathObject`: + +| Spec / ably-js (on a `PathObject`) | ably-java | +|---|---| +| `pathObj.get(key)` | `pathObj.asLiveMap().get(key): PathObject` | +| `pathObj.at("a.b.c")` | `pathObj.asLiveMap().at("a.b.c"): PathObject` | +| `pathObj.entries()` | `pathObj.asLiveMap().entries(): Iterable>` | +| `pathObj.keys()` / `.values()` | `pathObj.asLiveMap().keys(): Iterable` / `.values(): Iterable` | +| `pathObj.size()` | `pathObj.asLiveMap().size(): Long?` | +| `pathObj.set(key, value)` | `pathObj.asLiveMap().set(key, LiveMapValue.of(value))` (§6, §7) | +| `pathObj.remove(key)` | `pathObj.asLiveMap().remove(key)` | + +> The **root** is already a `LiveMapPathObject` (from `channel.object.get()`), so `root.get(...)` / +> `root.set(...)` need no cast — only deeper, freshly-navigated `PathObject`s do. + +**Iterating & membership.** `entries()` returns `Iterable>`; `keys()` / +`values()` return `Iterable<…>`. The spec's tuple-destructuring loops and `IN` membership map to Kotlin +directly: + +``` +# spec +FOR [key, pathObj] IN root.entries(): … +ASSERT "name" IN root.keys() +keys = list(root.keys()) +``` +```kotlin +for ((key, pathObj) in root.entries()) { … } // Map.Entry destructures into (key, value) +assertTrue("name" in root.keys()) // Kotlin `in` -> Iterable.contains +val keys = root.keys().toList() // when the spec materialises a list / checks length +``` + +These live on `LiveMapPathObject`, so a *navigated* node needs `asLiveMap()` first +(`root.get("score").asLiveMap().entries()`); `root` itself doesn't. + +**Counter-only methods** — require `asLiveCounter()` → `LiveCounterPathObject`: + +| Spec / ably-js | ably-java | +|---|---| +| `pathObj.value()` *(when it's a counter)* | `pathObj.asLiveCounter().value(): Double?` (counter value, else null) | +| `pathObj.increment([n])` | `pathObj.asLiveCounter().increment()` / `.increment(n: Number)` | +| `pathObj.decrement([n])` | `pathObj.asLiveCounter().decrement()` / `.decrement(n: Number)` | + +**Primitive value reads** — the spec's single `pathObj.value()` splits by primitive type. Cast to the +matching primitive sub-type (`NumberPathObject`, `StringPathObject`, `BooleanPathObject`, `BinaryPathObject`, +`JsonObjectPathObject`, `JsonArrayPathObject`) and call `value()` (each returns its type or `null`): + +| Spec resolves to | ably-java | +|---|---| +| number | `pathObj.asNumber().value(): Number?` | +| string | `pathObj.asString().value(): String?` | +| boolean | `pathObj.asBoolean().value(): Boolean?` | +| binary | `pathObj.asBinary().value(): ByteArray?` | +| JSON object | `pathObj.asJsonObject().value(): JsonObject?` | +| JSON array | `pathObj.asJsonArray().value(): JsonArray?` | + +> The dynamic `PathObject#value` (`RTPO7`) returns "the resolved counter value *or* any primitive". The +> typed `value()` accessors are **stricter** (`RTTS6g`): each returns `null` unless the resolved value is +> exactly that type. Translate "ASSERT pathObj.value() == 5" against a counter as +> `assertEquals(5.0, root.get("c").asLiveCounter().value())`, not `asNumber()`. +> +> **Number comparison gotcha.** Specs assert against integer literals (`value() == 110`, `size() == 7`), +> but ably-java returns wider numeric types: counter `value()` is `Double` (assert `110.0`); primitive +> `asNumber().value()` is a boxed `Number` whose runtime type follows JSON decoding; and `size()` is `Long` +> (assert `7L`). `assertEquals` treats `110.0`/`110`/`110L` as unequal across `Double`/`Int`/`Long`, so +> normalise: `assertEquals(110.0, obj.asNumber().value()?.toDouble())`, `assertEquals(7L, root.size())`. A +> spec `size() == null` (called on a non-map) is `assertNull(node.asLiveMap().size())` — the cast doesn't +> throw and `size()` returns null off-map. + +**Path strings & dot-escaping (`RTPO4`/`RTPO4b`/`RTPO6`).** `path()` returns a dot-delimited `String`; the +root's is `""`. A literal dot *inside* a segment is escaped as `\.`, and `at()` parses `\.` back to a +literal dot — so `path()` round-trips. Mind Kotlin's own backslash escaping (`"a\\.b.c"` is the string +`a\.b.c`): + +``` +# spec # ably-java (Kotlin) +ASSERT root.path() == "" assertEquals("", root.path()) +ASSERT root.get("a").get("b").path() assertEquals("a.b", root.get("a").asLiveMap().get("b").path()) + == "a.b" +po = root.at("a\.b.c") val po = root.at("a\\.b.c") // segments ["a.b", "c"] +ASSERT po.path() == "a\.b.c" assertEquals("a\\.b.c", po.path()) +``` + +--- + +## 5. Dynamic `Instance` → typed `Instance` hierarchy + +Same partition as `PathObject`, with two differences: the base is **abstract / never instantiated** +directly (`RTTS7e`), and the casts **throw** on mismatch instead of degrading (`RTTS9d`) — because an +`Instance` wraps an already-resolved value of known type. + +**Base `Instance`** (`io.ably.lib.liveobjects.instance.Instance`): + +| Spec / ably-js | ably-java | +|---|---| +| `instance.getType()` | `instance.getType(): ValueType` (non-null — never `UNKNOWN` in normal operation, `RTTS8a`) | +| `instance.compactJson()` | `instance.compactJson(): JsonElement` (**non-null**, `RTINS11c`) | +| `instance.compact()` | **Not implemented in ably-java** (`RTTS7d`, same as `PathObject`). Use `compactJson()`; flag a deviation if a spec needs `compact()`. | +| — casts — | `asLiveMap()`, `asLiveCounter()`, `asNumber()`, `asString()`, `asBoolean()`, `asBinary()`, `asJsonObject()`, `asJsonArray()` — **throw `IllegalStateException`** (or `AblyException` 400/92007) on type mismatch | + +> `subscribe` is **not** on the base `Instance` (`RTTS7b`) — only on `LiveMapInstance` / `LiveCounterInstance`. +> `id`, `value`, `get`, `set`, … are all partitioned onto sub-classes too (`RTTS7c`). + +**`LiveMapInstance`** (`…instance.types.LiveMapInstance`): + +| Spec / ably-js | ably-java | +|---|---| +| `instance.id` | `getId(): String` (non-null, `RTTS10a`) | +| `instance.get(key)` | `get(key): Instance?` | +| `instance.entries()` / `.keys()` / `.values()` | `entries(): Iterable>` / `keys(): Iterable` / `values(): Iterable` | +| `instance.size()` | `size(): Long` (non-null here, `RTTS10a`) | +| `instance.set(key, value)` / `.remove(key)` | `set(key, LiveMapValue.of(value))` / `remove(key)` → `CompletableFuture` | +| `instance.subscribe(listener)` | `subscribe(InstanceListener): Subscription` | +| `instance.compactJson()` | `compactJson(): JsonObject` (narrowed) | + +**`LiveCounterInstance`** (`…instance.types.LiveCounterInstance`): + +| Spec / ably-js | ably-java | +|---|---| +| `instance.id` | `getId(): String` | +| `instance.value()` | `value(): Double` (non-null, `RTTS10b`) | +| `instance.increment([n])` / `.decrement([n])` | `increment()` / `increment(n)` / `decrement()` / `decrement(n)` → `CompletableFuture` | +| `instance.subscribe(listener)` | `subscribe(InstanceListener): Subscription` | +| `instance.compactJson()` | `compactJson(): JsonPrimitive` (narrowed) | + +**Primitive instances** (`NumberInstance`, `StringInstance`, `BooleanInstance`, `BinaryInstance`, +`JsonObjectInstance`, `JsonArrayInstance`) are **read-only**: each exposes a non-null `value()` of its type +and a narrowed `compactJson()`; no `id`, `get`, `set`, `subscribe`, etc. + +--- + +## 6. Creation value types & the `LiveMapValue` union + +ably-java can't accept "any JS value" into `set`, so it uses a tagged union `LiveMapValue` and dedicated +immutable creation value types. + +| Spec / ably-js | ably-java | +|---|---| +| `LiveCounter.create()` | `LiveCounter.create(): LiveCounter` (`io.ably.lib.liveobjects.value`) | +| `LiveCounter.create(5)` | `LiveCounter.create(5)` (arg is `Number`) | +| `LiveMap.create()` | `LiveMap.create(): LiveMap` | +| `LiveMap.create({ a: 1, b: "x" })` | `LiveMap.create(mapOf("a" to LiveMapValue.of(1), "b" to LiveMapValue.of("x")))` — entries are `Map` | +| a raw value passed to `set` | wrap it: `LiveMapValue.of(value)` | + +`LiveMapValue.of(...)` overloads: `Boolean`, `ByteArray` (binary), `Number`, `String`, `JsonArray`, +`JsonObject`, `LiveCounter` (value type), `LiveMap` (value type). Inspect with `isNumber()` / `getAsNumber()` +etc. when a spec asserts on a constructed value's contents. + +> **Type-safety turns several "invalid input" spec cases into compile errors, not runtime assertions.** +> Where a spec feeds a deliberately wrong type and expects an `ErrorInfo`, ably-java's signatures reject it +> at compile time, so the test isn't expressible — note it as a deviation rather than forcing it: +> - Passing a **graph object / public view** (`PathObject`, `Instance`, a live object) as a map value +> (`RTLMV4c1`, runtime `40013` in the dynamic API) — blocked by the `LiveMapValue` union. +> - **Wrong-typed `create` args**, e.g. `LiveCounter.create("not_a_number")` (spec expects `40003`) — +> blocked by `create(Number)`; `LiveMap.create` likewise takes `Map` so non-`Dict` +> / non-`String`-key / unsupported-value entry cases (`RTLMV4a`/`b`/`c`) can't be constructed either. +> +> Validation cases on *values the type system still allows* (e.g. a NaN / out-of-range `Number`) remain +> real runtime assertions — only the cases the signature outright forbids become deviations. + +--- + +## 7. Mutations (set / remove / increment / decrement) + +Putting §4 + §6 together — the canonical write translations: + +``` +# spec +AWAIT root.set("count", LiveCounter.create(0)) +AWAIT root.get("count").increment(5) +AWAIT root.set("name", "alice") +AWAIT root.remove("name") +``` +```kotlin +// ably-java (root is a LiveMapPathObject) +root.set("count", LiveMapValue.of(LiveCounter.create(0))).await() +root.get("count").asLiveCounter().increment(5).await() +root.set("name", LiveMapValue.of("alice")).await() +root.remove("name").await() +``` + +- `set` / `remove` live on `LiveMapPathObject` (or `LiveMapInstance`); navigate+`asLiveMap()` first unless + you're on the root or an already-typed map view. +- `increment` / `decrement` live on `LiveCounterPathObject` (or `LiveCounterInstance`); `asLiveCounter()` + first. +- Default-amount forms exist: `increment()` ≡ `increment(1)`, `decrement()` ≡ `decrement(1)`. + +### Wrong-type write failures still go *through* the cast + +A common spec shape is a write on the wrong kind of object, expecting a runtime error — e.g. +`AWAIT root.increment(5) FAILS WITH error` (increment on a map) or `counter.set("k", v) FAILS WITH error`. +In the dynamic API every method exists on every `PathObject`, so the call is expressible and throws at +runtime. In ably-java the typed view **doesn't have that method at all** (`LiveMapPathObject` has no +`increment`; `LiveCounterPathObject` has no `set`), so calling it directly is a *compile* error — not the +runtime failure the spec is testing. + +To translate these, cast to the view whose write method you need (the `PathObject` cast never throws, +`RTTS5d`), then assert the **operation** throws — that's where the `92007` surfaces: + +``` +# spec: increment on a map fails +AWAIT root.increment(5) FAILS WITH error # code 92007 +``` +```kotlin +val ex = assertFailsWith { root.asLiveCounter().increment(5).await() } +assertEquals(92007, ex.errorInfo.code) +``` + +So "can't call increment on a map" is **not** "not expressible" — it's `asLiveCounter().increment(...)` +plus an assertion on the throw. (Contrast §6: invalid *value* / *argument-type* cases genuinely aren't +expressible, because the union/`create(Number)` signatures reject them at compile time.) + +--- + +## 8. Subscriptions, listeners & events + +ably-js passes a closure and gets back a `Subscription` with `.unsubscribe()`. ably-java uses named +single-method listener interfaces; the event is an object with getters. + +| Spec / ably-js | ably-java | +|---|---| +| `sub = pathObj.subscribe((event) => { … })` | `val sub = pathObj.subscribe(PathObjectListener { event -> … })` | +| `pathObj.subscribe(listener, { depth: 2 })` | `pathObj.subscribe(listener, PathObjectSubscriptionOptions(2))` (no-arg ctor = unlimited depth) | +| `sub = instance.subscribe((event) => { … })` | `val sub = mapOrCounterInstance.subscribe(InstanceListener { event -> … })` | +| `sub.unsubscribe()` | `sub.unsubscribe()` | +| `event.object` | `event.getObject()` — a `PathObject` (path sub) / `Instance` (instance sub) | +| `event.message` | `event.getMessage(): ObjectMessage?` (§11) | + +Listener SAMs: `PathObjectListener.onUpdated(PathObjectSubscriptionEvent)`, +`InstanceListener.onUpdated(InstanceSubscriptionEvent)`. In Kotlin you can pass a lambda for either (SAM +conversion). `PathObjectSubscriptionOptions(depth)` throws `AblyException` 400/`40003` for non-positive +depth (`RTPO19c1`). + +> **`LiveObjectUpdate` is not the public event.** `live_object_subscribe.md` cites the internal `RTLO4b` +> `LiveObjectUpdate` (fields `update` / `noop` / `objectMessage` / `tombstone`), but it subscribes through +> the *public* `instance.subscribe(...)`, whose ably-java event is `InstanceSubscriptionEvent` — only +> `getObject()` + `getMessage()`, **no diff/`noop`/`tombstone` accessors**. So "listener fired N times" and +> "returns a `Subscription`" translate directly, but any assertion on the `LiveObjectUpdate` *diff* fields +> is internal (§13) — adapt or skip with a deviation. + +--- + +## 9. Sync-state events (`object.on('synced')`) + +`RealtimeObject` extends `ObjectStateChange`. The ably-js string-event API becomes an enum + listener: + +| Spec / ably-js | ably-java | +|---|---| +| `channel.object.on('synced', cb)` | `` channel.`object`.on(ObjectStateEvent.SYNCED, listener): Subscription `` | +| `channel.object.on('syncing', cb)` | `` channel.`object`.on(ObjectStateEvent.SYNCING, listener) `` | +| `channel.object.off(cb)` | `` channel.`object`.off(listener) `` | +| remove all | `` channel.`object`.offAll() `` | + +Listener: `ObjectStateChange.Listener.onStateChanged(ObjectStateEvent)`. Enum `ObjectStateEvent { SYNCING, +SYNCED }` (`io.ably.lib.liveobjects.state`). + +--- + +## 10. `ValueType` & type discrimination + +ably-js uses string-literal type tags; ably-java has an enum `io.ably.lib.liveobjects.ValueType`: + +| Spec value category | `ValueType` | +|---|---| +| string / number / boolean / binary | `STRING` / `NUMBER` / `BOOLEAN` / `BINARY` | +| JSON object / JSON array | `JSON_OBJECT` / `JSON_ARRAY` | +| live map / live counter | `LIVE_MAP` / `LIVE_COUNTER` | +| present but unrecognised | `UNKNOWN` | + +`pathObj.getType()` returns `null` when nothing resolves (distinct from `UNKNOWN`); `instance.getType()` is +non-null. Use `getType()` for "what is it" assertions and the matching `as*` cast to read it. + +--- + +## 11. Message / operation types + +The spec's `PublicAPI::ObjectMessage` / `PublicAPI::ObjectOperation` (the `PAOM*` / `PAOOP*` types, +delivered to subscription listeners) map to ably-java interfaces with getters (package +`io.ably.lib.liveobjects.message`). The `PublicAPI::` prefix is dropped — ably-java exposes them as +`ObjectMessage` / `ObjectOperation`. + +> **Getter-only, no *public* constructor — use the `buildPublicObjectMessage` helper.** In normal use you +> obtain an `ObjectMessage` from a subscription event (`event.getMessage()`, §8); there is no public +> factory. The spec's explicit construction-from-wire (`PublicObjectMessage.fromObjectMessage(source, +> channel)` / `PublicObjectOperation.fromObjectOperation(op)`, `PAOM3`/`PAOOP3`, in +> `public_object_message.md`) is `internal` to `:liveobjects` — but the unit helpers expose it **by +> reflection** as `buildPublicObjectMessage(wireJson, channelName)` (§13). So `public_object_message.md` is +> translatable: build the source with the op builders (`buildMapSet(...)`, `buildCounterInc(...)`, …) and +> assert the public getters on the result. + +`ObjectMessage`: `getId()`, `getClientId()`, `getConnectionId()`, `getTimestamp(): Long?`, `getChannel(): +String`, `getOperation(): ObjectOperation`, `getSerial()`, `getSerialTimestamp(): Long?`, `getSiteCode()`, +`getExtras(): JsonObject?`. (Timestamps are epoch-millis `Long`, not a `Time` object.) + +`ObjectOperation`: `getAction(): ObjectOperationAction`, `getObjectId(): String`, and one non-null payload +getter matching the action — `getMapCreate()`, `getMapSet()`, `getMapRemove()`, `getCounterCreate()`, +`getCounterInc()`, `getObjectDelete()`, `getMapClear()`. + +**The spec accesses these as dotted property chains and compares `action` to a *string literal*; ably-java +uses getters and an *enum constant*.** Translate the chain getter-by-getter and the string tag to the enum: + +``` +# spec +ASSERT msg.operation.action == "MAP_SET" +ASSERT msg.operation.mapSet.key == "name" +ASSERT msg.operation.mapSet.value.string == "blue" +ASSERT msg.operation.counterInc.number == 42 +ASSERT msg.operation.mapCreate == null +``` +```kotlin +val op = msg.operation +assertEquals(ObjectOperationAction.MAP_SET, op.action) // string "MAP_SET" -> enum constant +assertEquals("name", op.mapSet!!.key) +assertEquals("blue", op.mapSet!!.value.string) // ObjectData.getString() +assertEquals(42.0, op.counterInc!!.number) // getNumber(): Double -> use .0 +assertNull(op.mapCreate) +``` + +(Java getters read as Kotlin properties: `msg.operation` ≡ `getOperation()`, `op.mapSet` ≡ `getMapSet()`, +etc. The payload getter for the non-matching actions returns `null`, so `mapCreate == null` → `assertNull`.) +Every spec string action tag maps to its `ObjectOperationAction` constant: `"MAP_SET"`→`MAP_SET`, +`"COUNTER_INC"`→`COUNTER_INC`, `"OBJECT_DELETE"`→`OBJECT_DELETE`, etc. The same string-tag→enum rule applies +to map semantics (`"lww"`→`ObjectsMapSemantics.LWW`) and value types (§10). + +| Spec type | ably-java | Notable getters | +|---|---|---| +| `ObjectOperationAction` | enum `ObjectOperationAction` | `MAP_CREATE, MAP_SET, MAP_REMOVE, COUNTER_CREATE, COUNTER_INC, OBJECT_DELETE, MAP_CLEAR, UNKNOWN` | +| `MapSet` | `MapSet` | `getKey(): String`, `getValue(): ObjectData` | +| `MapRemove` | `MapRemove` | `getKey(): String` | +| `MapCreate` | `MapCreate` | `getSemantics(): ObjectsMapSemantics`, `getEntries(): Map` | +| `CounterCreate` | `CounterCreate` | `getCount(): Double` | +| `CounterInc` | `CounterInc` | `getNumber(): Double` | +| `ObjectDelete` / `MapClear` | marker interfaces | no methods | +| `ObjectData` (leaf value) | `ObjectData` | `getObjectId()`, `getString()`, `getNumber(): Double?`, `getBoolean()`, `getBytes(): ByteArray?`, `getJson(): JsonElement?` | +| `ObjectsMapEntry` | `ObjectsMapEntry` | `getTombstone(): Boolean?`, `getTimeserial()`, `getSerialTimestamp(): Long?`, `getData(): ObjectData?` | +| map semantics | enum `ObjectsMapSemantics` | `LWW, UNKNOWN` | + +> Note `PublicAPI::ObjectOperation` carries only `mapCreate`/`counterCreate` (the `*WithObjectId` outbound +> variants are resolved back to their `MapCreate`/`CounterCreate` forms, `PAOOP1`). Don't expect a +> `getMapCreateWithObjectId()` on the public type. + +--- + +## 12. Errors & error codes + +Spec assertions like `FAILS WITH error code 92007` map to ably-java exceptions: + +| Spec failure | ably-java | +|---|---| +| async op rejects with `ErrorInfo` code N | the `CompletableFuture` completes exceptionally with `AblyException`. With `.await()` the cause is rethrown directly, so `assertFailsWith { … .await() }` then `ex.errorInfo.code == N`. With blocking `.get()` you instead catch `ExecutionException` and read `.cause` | +| wrong write method for the type (e.g. `increment` on a map, `set` on a counter) | the typed view lacks the method, so cast first (`asLiveCounter()` / `asLiveMap()` — never throws, `RTTS5d`) then the **operation** throws `AblyException` 400/`92007`. See §7 "Wrong-type write failures" | +| `Instance` `as*` cast on wrong type | **`IllegalStateException`** (or `AblyException` 400/`92007`) — thrown synchronously (`RTTS9d`) | +| `PathObject` `as*` cast on wrong type | **never throws** (`RTTS5d`) — failure shows up on the subsequent read (null) or write (throws, above) | +| invalid value into `set` (graph object / view) | `AblyException` 400 / code `40013` (`RTLMV4c1`) — usually not expressible in ably-java's typed `set`; treat as a deviation | +| non-positive subscription `depth` | `AblyException` 400 / code `40003` (`PathObjectSubscriptionOptions(int)`) | +| write where path doesn't resolve | `AblyException` 400 / code `92005` | +| write where value isn't the required type | `AblyException` 400 / code `92007` | +| `get()` / op when channel lacks the object mode (`RTO23a`/`RTO2a2`) | `AblyException` 400 / code `40024` | +| `get()` / access when channel is DETACHED or FAILED (`RTO23b`/`RTO25`) | `AblyException` 400 / code `90001` | +| channel enters DETACHED/SUSPENDED/FAILED while awaiting SYNCED (`RTO20e`/`RTO23c`) | `AblyException` 400 / code `92008` | +| write while `echoMessages` is false (`RTO26c`) | `AblyException` 400 / code `40000` | + +Assert the code as a plain int — `assertEquals(90001, ex.errorInfo.code)` — matching the spec's +`error.code == 90001`; error codes are int literals, not enums (unlike the action / semantics / value-type +tags). The `90000` a spec injects via a mocked `ERROR`/`DETACHED` `ProtocolMessage` is the channel-level +error, not an objects code — it's what drives the channel into the state that makes the objects call fail. + +--- + +## 13. Internal-graph types (unit specs) — important caveat + +Several **unit** specs assert on the **internal CRDT graph**, not the public API: +`InternalLiveCounter.data`, `InternalLiveMap.siteTimeserials`, `ObjectsPool.syncState`, `LiveObjectUpdate`, +`applyOperation(msg, source)`, object-id generation (`RTO14`), the `*CreateWithObjectId` wire variants and +`generateObjectId`, etc. Specs that are wholly or mostly internal: + +- `objects_pool.md`, `parent_references.md`, `object_id.md` — pool sync state, the reverse parent-reference + graph, and object-id generation: entirely internal. +- the internal-state assertions in `live_counter.md` / `live_map.md` (`.data`, `.siteTimeserials`, + `.createOperationIsMerged`, `.isTombstone`, `applyOperation`, `replaceData`) — internal; their + public-facing read/write counterparts live in `live_counter_api.md` / `live_map_api.md`. +- `value_types.md` — the *public* `LiveMap.create` / `LiveCounter.create` surface maps via §6, but the + evaluation half (`COUNTER_CREATE` / `MAP_CREATE` `ObjectMessage` generation, nonce/`initialValue`/ + `objectId` derivation, the `*WithObjectId` wire forms) is internal/wire-level. +- `realtime_object.md` — **mixed**: `get()` (`RTO23`, incl. the `40024`/`90001`/`92008` precondition cases) + is public and maps via §2/§12, but `publish` / `publishAndApply` (`RTO15`/`RTO20`, marked `internal` in the + IDL) and the OBJECT/ACK wire assertions are internal. +- `public_object_message.md` — **translatable** via the `buildPublicObjectMessage` helper (below), which + reflectively performs the `PAOM3`/`PAOOP3` construction (`WireObjectMessage` → `DefaultObjectMessage`) + that is otherwise `internal`. Build the source with the op builders and assert the public getters (§11). + +In ably-java these are **not public**. They live in the `:liveobjects` module as `Default*` / `Wire*` / +`ResolvedValue` / `Leaf` / `MapRef` / `CounterRef` classes (package `io.ably.lib.liveobjects.*`, +implementation source set). The `uts` module keeps them **off its compile classpath** (it compiles against +`:java` only) but now has `testRuntimeOnly(project(":liveobjects"))`, so the helpers reach the internal +wire/message classes **by reflection** at runtime. Consequences when translating: + +- **Public-API unit specs** (`path_object*.md`, `instance.md`, `live_object_subscribe.md`, + `public_object_message.md`, and the public-surface parts of `realtime_object.md` and `value_types.md`) + translate cleanly against the §1–§12 map + the helpers below, and compile against `:java`. (Note + `path_object.md` / `instance.md` also contain `compact()` cases, which are deviations per §4/§5 since + ably-java implements only `compactJson()`.) +- **Internal-graph unit specs** (`objects_pool.md`, `parent_references.md`, the internal-state assertions in + `live_counter.md` / `live_map.md`) assert on internal CRDT state the public API can't see. Options: (a) + add reflective accessors to the helpers for the `Default*`/internal classes (the technique + `buildPublicObjectMessage` and `infra/unit/Utils.kt` already use), (b) translate them in the + `:liveobjects` module's own test source where the types are directly accessible, or (c) skip them. Flag + rather than forcing a public-API assertion that can't reach internal state. +- Spec name → ably-java impl (for orientation, not public use): `InternalLiveMap` → `DefaultLiveMap`, + `InternalLiveCounter` → `DefaultLiveCounter`, the public-view impls are `DefaultPathObject` / + `DefaultLiveMapPathObject` / `DefaultInstance` / …, wire form is `WireObjectMessage` / + `WireObjectOperation` / `WireObjectState` etc. + +### Unit-test helpers — `standard_test_pool.md` → `helpers.kt` + +Every objects unit spec opens with `setup_synced_channel` and constructs protocol/object messages with the +`build_*` helpers. These are implemented in +`uts/src/test/kotlin/io/ably/lib/uts/unit/liveobjects/helpers.kt` — **call them; don't hand-roll the mock +setup or message JSON.** + +| Spec helper | `helpers.kt` | +|---|---| +| `{ client, channel, root, mock_ws } = AWAIT setup_synced_channel("test")` | `val (client, channel, root, mockWs) = setupSyncedChannel("test")` (`suspend`, returns `SyncedChannel`) | +| `setup_synced_channel_no_ack(...)` | `setupSyncedChannelNoAck(...)` | +| `build_object_sync_message` / `build_object_message` / `build_ack_message` | `buildObjectSyncMessage` / `buildObjectMessage` / `buildAckMessage` → `ProtocolMessage` | +| `build_counter_inc` / `build_map_set` / `build_map_remove` / `build_map_clear` / `build_object_delete` / `build_counter_create` / `build_map_create` | same names camelCased → wire `JsonObject` | +| `build_object_state` / `build_object_message_with_state` | `buildObjectState` / `buildObjectMessageWithState` | +| `build_public_object_message(msg, channel)` | `buildPublicObjectMessage(wireJson, channel)` (reflective; §11) | +| `STANDARD_POOL_OBJECTS` | `STANDARD_POOL_OBJECTS` | +| inline ObjectData / map-entry / state fragments | `dataString` / `dataNumber` / `dataBoolean` / `dataObjectId` / `dataBytes` / `dataJson`, `mapEntry`, `mapState`, `counterState`, `mapCreateOp`, `counterCreateOp` | + +`mock_ws.send_to_client(...)` is the existing `mockWs.sendToClient(...)` (§ mock API in the main skill). The +wire `action` / `semantics` are integer enum codes — the builders emit the codes for you. + +> **Runtime caveat:** `setupSyncedChannel` returns only once `RealtimeObject.get()` resolves, which needs +> the `:liveobjects` SDK's OBJECT_SYNC processing. Until that lands the helpers **compile** and the test +> structure is correct, but the setup throws at runtime — i.e. translate-only today, runnable once the SDK +> is implemented. (`buildPublicObjectMessage` does *not* depend on this — the message/operation layer is +> implemented, so those tests can run now.) + +--- + +## 14. Worked example + +Spec pseudocode (public-API style): + +``` +test "increments a nested counter and observes it" + root = AWAIT channel.object.get() + AWAIT root.set("game", LiveMap.create({ score: LiveCounter.create(0) })) + scoreSub = root.at("game.score").subscribe((event) => { received = event }) + AWAIT root.at("game.score").increment(10) + ASSERT root.at("game.score").value() == 10 + ASSERT received.object.value() == 10 + scoreSub.unsubscribe() +``` + +ably-java / Kotlin translation: + +```kotlin +val root: LiveMapPathObject = channel.`object`.get().await() // `object` is a Kotlin keyword — backticks required + +root.set( + "game", + LiveMapValue.of(LiveMap.create(mapOf("score" to LiveMapValue.of(LiveCounter.create(0))))), +).await() + +var received: PathObjectSubscriptionEvent? = null +val scoreSub = root.at("game.score").subscribe(PathObjectListener { event -> received = event }) + +root.at("game.score").asLiveCounter().increment(10).await() + +assertEquals(10.0, root.at("game.score").asLiveCounter().value()) +assertEquals(10.0, received!!.getObject().asLiveCounter().value()) +scoreSub.unsubscribe() +``` + +Note the four mechanical rewrites: `get()` → `.await()`; nested `LiveMap.create`/`LiveCounter.create` +wrapped in `LiveMapValue.of`; `at(...)` followed by `asLiveCounter()` before counter ops; `event.object` +→ `event.getObject()` and re-cast. + +--- + +## 15. Quick symbol index + +| ably-js / spec symbol | ably-java | +|---|---| +| `channel.object` | field `` channel.`object` `` : `RealtimeObject` (Kotlin keyword → backticks) | +| `channel.object.get()` | `` channel.`object`.get() `` → `CompletableFuture` | +| `PathObject` (polymorphic) | base `PathObject` + `asLiveMap()`/`asLiveCounter()`/`as()` | +| `Instance` (polymorphic) | abstract `Instance` + `as*` (throwing) | +| `pathObj.get(k)` / `.at(p)` | `pathObj.asLiveMap().get(k)` / `.at(p)` | +| `pathObj.value()` | `pathObj.as().value()` (typed, null on mismatch) | +| `pathObj.set(k, v)` / `.remove(k)` | `pathObj.asLiveMap().set(k, LiveMapValue.of(v))` / `.remove(k)` | +| `pathObj.increment(n)` / `.decrement(n)` | `pathObj.asLiveCounter().increment(n)` / `.decrement(n)` | +| `op FAILS WITH ` (wrong method for type) | cast to the needed view, then assert the op throws: `assertFailsWith { node.asLiveCounter().increment(n).await() }` (§7, §12) | +| `FOR [k, v] IN x.entries()` | `for ((k, v) in x.asLiveMap().entries())` | +| `"k" IN x.keys()` / `list(x.keys())` | `"k" in x.asLiveMap().keys()` / `x.asLiveMap().keys().toList()` | +| `size() == 7` / `== null` | `assertEquals(7L, …size())` (Long) / `assertNull(node.asLiveMap().size())` | +| `op.action == "MAP_SET"` | `assertEquals(ObjectOperationAction.MAP_SET, op.action)` (string tag → enum) | +| `op.mapSet.value.string` | `op.mapSet!!.value.string` (ObjectData getters) | +| `LiveMap.create(entries)` | `LiveMap.create(Map)` (value type) | +| `LiveCounter.create(n)` | `LiveCounter.create(Number)` (value type) | +| raw value into `set` | `LiveMapValue.of(value)` | +| `subscribe(cb)` → `Subscription` | `subscribe(PathObjectListener / InstanceListener)` → `Subscription` | +| `{ depth: n }` | `PathObjectSubscriptionOptions(n)` | +| `event.object` / `event.message` | `event.getObject()` / `event.getMessage(): ObjectMessage?` | +| `object.on('synced', cb)` | `object.on(ObjectStateEvent.SYNCED, listener)` | +| type tag `'LiveMap'` etc. | `ValueType.LIVE_MAP` etc. | +| `PublicAPI::ObjectMessage` | `ObjectMessage` (getters) | +| `PublicAPI::ObjectOperation` | `ObjectOperation` (getters, one payload non-null) | +| `InternalLiveMap` / `InternalLiveCounter` / `ObjectsPool` | internal `:liveobjects` impl — see §13 | diff --git a/.claude/skills/uts-to-kotlin/scripts/resolve_uts.py b/.claude/skills/uts-to-kotlin/scripts/resolve_uts.py new file mode 100644 index 000000000..a61d0d91c --- /dev/null +++ b/.claude/skills/uts-to-kotlin/scripts/resolve_uts.py @@ -0,0 +1,179 @@ +#!/usr/bin/env python3 +"""Resolve a UTS spec module directory to its ably-java test targets. + +Deterministic helper for the uts-to-kotlin skill. Given a UTS spec *module* +directory (a directory directly under .../specification/uts/), it: + + - validates the path and the module's tier structure, + - reads uts-package-mapping.json (next to this script's skill dir), + - resolves, per tier, the target output directory and Kotlin package, and + - lists the candidate spec files with their derived Kotlin class names. + +Doing this in code (rather than asking the model to eyeball regexes, join +paths, and hand-convert snake_case -> PascalCase every run) keeps the skill's +selection phase byte-for-byte deterministic. + +Usage: + resolve_uts.py # validate + resolve + list specs + resolve_uts.py --create NAME # add a mapping entry for this + # module (target base name NAME), + # then resolve + +Always prints a single JSON object to stdout. On failure: {"ok": false, +"error": , "message": ...} and a non-zero exit. +""" +import argparse +import json +import re +import sys +from pathlib import Path + +SKILL_DIR = Path(__file__).resolve().parent.parent +MAPPING = SKILL_DIR / "uts-package-mapping.json" +TIERS = ("unit", "integration", "proxy") + + +def fail(code, message): + print(json.dumps({"ok": False, "error": code, "message": message}, indent=2)) + sys.exit(1) + + +def class_name(md_path: Path) -> str: + """objects_batch_test.md -> ObjectsBatchTest; instance.md -> InstanceTest.""" + stem = md_path.stem + if stem.endswith("_test"): + stem = stem[: -len("_test")] + return "".join(part.capitalize() for part in stem.split("_") if part) + "Test" + + +def is_nonspec_doc(name: str) -> bool: + """README/PLAN/*_SUMMARY markdown are docs, not test specs.""" + if re.fullmatch(r"(README|PLAN)\.md", name, re.IGNORECASE): + return True + return name.upper().endswith("_SUMMARY.MD") + + +def list_specs(base: Path, exclude_proxy: bool = False): + """List spec .md files under `base`, deterministically. + + All exclusions are checked against the path **relative to base**, so they + can't be tripped by an ancestor directory in the checkout path (e.g. a + clone living under some `.../helpers/...` path). + """ + if not base.is_dir(): + return [] + specs = [] + for p in sorted(base.rglob("*.md")): + rel_parts = p.relative_to(base).parts + if "helpers" in rel_parts: + continue + if exclude_proxy and "proxy" in rel_parts: + continue + if is_nonspec_doc(p.name): + continue + specs.append(p) + return specs + + +def package_for(target_dir: str) -> str: + marker = "src/test/kotlin/" + idx = target_dir.find(marker) + return target_dir[idx + len(marker):].replace("/", ".") if idx != -1 else "" + + +def main(): + ap = argparse.ArgumentParser() + ap.add_argument("module_dir") + ap.add_argument( + "--create", + metavar="NAME", + help="add a mapping for this source module using NAME as the ably-java " + "module base name, then resolve", + ) + args = ap.parse_args() + + raw = args.module_dir.rstrip("/") + if not re.search(r"/uts/[^/]+$", raw): + fail("NOT_A_UTS_MODULE_PATH", + f"{raw!r} is not a module directory directly under uts/ " + f"(expected .../uts/).") + module_dir = Path(raw) + if not module_dir.is_dir(): + fail("DIR_NOT_FOUND", f"{raw!r} does not exist or is not a directory.") + if not (module_dir / "unit").is_dir() and not (module_dir / "integration").is_dir(): + fail("NO_TIER_DIRS", + f"{raw!r} has no unit/ or integration/ sub-directory; " + f"not a valid UTS module.") + + source_module = module_dir.name + + if not MAPPING.is_file(): + fail("MAPPING_NOT_FOUND", f"mapping file not found at {MAPPING}") + data = json.loads(MAPPING.read_text()) + packages = data.setdefault("packages", {}) + test_root = data.get("testRoot", "") + + if args.create: + target = args.create + if not re.fullmatch(r"[A-Za-z][A-Za-z0-9_]*", target): + fail("BAD_TARGET_NAME", + f"--create target {target!r} must be a simple module base name " + f"(letters/digits/underscore, e.g. 'liveobjects') so it forms a " + f"valid path and Kotlin package.") + new_entry = { + "unit": f"unit/{target}", + "integration": f"integration/standard/{target}", + "proxy": f"integration/proxy/{target}", + } + # preserve a hand-maintained "notes" pointer when re-creating an existing entry + notes = packages.get(source_module, {}).get("notes") + if notes: + new_entry["notes"] = notes + packages[source_module] = new_entry + MAPPING.write_text(json.dumps(data, indent=2) + "\n") + + mapped = source_module in packages + entry = packages.get(source_module, {}) + + # Per-module translation notes (ably-js -> ably-java type map etc.), declared by + # the module's "notes" field in the mapping (a path relative to this skill dir). + # Read it before translating when present. None when the module declares no notes, + # or the declared file is missing. + notes_rel = entry.get("notes") + notes_path = SKILL_DIR / notes_rel if notes_rel else None + translation_notes = str(notes_path) if (notes_path and notes_path.is_file()) else None + + src = { + "unit": module_dir / "unit", + "integration": module_dir / "integration", + "proxy": module_dir / "integration" / "proxy", + } + specs = { + "unit": list_specs(src["unit"]), + "integration": list_specs(src["integration"], exclude_proxy=True), + "proxy": list_specs(src["proxy"]), + } + + tiers_out = {} + for tier in TIERS: + target_dir = f"{test_root}/{entry[tier]}" if (mapped and tier in entry) else None + tiers_out[tier] = { + "present": src[tier].is_dir(), + "sourceDir": str(src[tier]), + "targetDir": target_dir, + "package": package_for(target_dir) if target_dir else None, + "specs": [{"file": str(p), "className": class_name(p)} for p in specs[tier]], + } + + print(json.dumps({ + "ok": True, + "sourceModule": source_module, + "mapped": mapped, + "testRoot": test_root, + "translationNotes": translation_notes, + "tiers": tiers_out, + }, indent=2)) + + +if __name__ == "__main__": + main() diff --git a/.claude/skills/uts-to-kotlin/uts-package-mapping.json b/.claude/skills/uts-to-kotlin/uts-package-mapping.json new file mode 100644 index 000000000..6e3c8172a --- /dev/null +++ b/.claude/skills/uts-to-kotlin/uts-package-mapping.json @@ -0,0 +1,22 @@ +{ + "_comment": "Maps each UTS spec module (a dir under specification/uts/) to its target test packages. Output dir = testRoot + '/' + tier entry; Kotlin package = that path after 'src/test/kotlin/' with '/' -> '.'. An optional 'notes' field points (relative to this skill dir) to a per-module ably-js -> ably-java translation reference, read before translating that module. Used by the uts-to-kotlin skill.", + "testRoot": "uts/src/test/kotlin/io/ably/lib/uts", + "packages": { + "realtime": { + "unit": "unit/realtime", + "integration": "integration/standard/realtime", + "proxy": "integration/proxy/realtime" + }, + "objects": { + "unit": "unit/liveobjects", + "integration": "integration/standard/liveobjects", + "proxy": "integration/proxy/liveobjects", + "notes": "references/objects-mapping.md" + }, + "rest": { + "unit": "unit/rest", + "integration": "integration/standard/rest", + "proxy": "integration/proxy/rest" + } + } +} diff --git a/.github/workflows/check.yml b/.github/workflows/check.yml index c60db1d41..a2fe3a2bb 100644 --- a/.github/workflows/check.yml +++ b/.github/workflows/check.yml @@ -23,4 +23,4 @@ jobs: distribution: 'temurin' - name: Set up Gradle uses: gradle/actions/setup-gradle@d9c87d481d55275bb5441eef3fe0e46805f9ef70 # v3 - - run: ./gradlew checkWithCodenarc checkstyleMain checkstyleTest runUnitTests runLiveObjectsUnitTests :uts:test + - run: ./gradlew checkWithCodenarc checkstyleMain checkstyleTest runUnitTests runLiveObjectsUnitTests :uts:runUtsUnitTests diff --git a/.github/workflows/integration-test.yml b/.github/workflows/integration-test.yml index 8250c30aa..df5e399a8 100644 --- a/.github/workflows/integration-test.yml +++ b/.github/workflows/integration-test.yml @@ -123,3 +123,24 @@ jobs: uses: gradle/actions/setup-gradle@d9c87d481d55275bb5441eef3fe0e46805f9ef70 # v3 - run: ./gradlew runLiveObjectsIntegrationTests + + check-uts: + runs-on: ubuntu-latest + permissions: + contents: read + steps: + - uses: actions/checkout@34e114876b0b11c390a56381ad16ebd13914f8d5 # v4 + with: + submodules: 'recursive' + persist-credentials: false + + - name: Set up the JDK + uses: actions/setup-java@c1e323688fd81a25caa38c78aa6df2d33d3e20d9 # v4 + with: + java-version: '17' + distribution: 'temurin' + + - name: Set up Gradle + uses: gradle/actions/setup-gradle@d9c87d481d55275bb5441eef3fe0e46805f9ef70 # v3 + + - run: ./gradlew :uts:runUtsIntegrationTests diff --git a/uts/README.md b/uts/README.md new file mode 100644 index 000000000..615e1f936 --- /dev/null +++ b/uts/README.md @@ -0,0 +1,894 @@ +# UTS in ably-java — A Human-Readable Guide + +> A practical, end-to-end explanation of the **Universal Test Specification (UTS)** and how it is +> realised in the `ably-java` repository. Written for a developer who has never touched UTS before +> and needs to understand *what it is*, *why it exists*, and *exactly how the Java/Kotlin code under +> `uts/` makes the unit and proxy-integration tests work*. + +--- + +## Table of Contents + +1. [Introduction: What is UTS?](#1-introduction-what-is-uts) +2. [The Three Test Tiers](#2-the-three-test-tiers) +3. [The UTS Documents (the source of truth)](#3-the-uts-documents-the-source-of-truth) +4. [The Java Setup: the `uts/` module](#4-the-java-setup-the-uts-module) +5. [How a Test Reaches the SDK: the hook points](#5-how-a-test-reaches-the-sdk-the-hook-points) +6. [Unit-Test Infrastructure (mocked transports)](#6-unit-test-infrastructure-mocked-transports) +7. [Proxy-Integration Infrastructure (real backend + fault injection)](#7-proxy-integration-infrastructure-real-backend--fault-injection) +8. [Shared Async Helpers](#8-shared-async-helpers) +9. [Walkthrough: the Unit Test (`ConnectionRecoveryTest`)](#9-walkthrough-the-unit-test-connectionrecoverytest) +10. [Walkthrough: the Proxy Test (`AuthReauthTest`)](#10-walkthrough-the-proxy-test-authreauthtest) +11. [Deviations: when the SDK disagrees with the spec](#11-deviations-when-the-sdk-disagrees-with-the-spec) +12. [How to Run the Tests](#12-how-to-run-the-tests) +13. [Quick Reference / Cheat-Sheet](#13-quick-reference--cheat-sheet) +14. [Appendix A: Request-Flow Diagrams](#14-appendix-a-request-flow-diagrams) +15. [Appendix B: Per-File API Reference](#15-appendix-b-per-file-api-reference) + +--- + +## 1. Introduction: What is UTS? + +**UTS (Universal Test Specification)** is Ably's language-neutral catalogue of tests for its client +SDKs. The problem it solves: Ably ships many SDKs (JavaScript, Dart, Kotlin/Java, Swift, Go, …), and +every one of them must obey the *same* behavioural contract — the **Ably features spec** +(`specification/specifications/features.md`, whose requirements are tagged `RSC7`, `RTN15a`, `RTL4f`, +etc.). Without a shared test definition, each SDK would re-invent its own tests, drift apart, and +leave gaps. + +UTS fixes this by separating **what to test** from **how to test it in a given language**: + +``` + ┌──────────────────────────────┐ + │ Ably features spec │ ← the ultimate authority (RSC*, RTN*, RTL* …) + │ (features.md) │ + └──────────────┬───────────────┘ + │ distilled into portable test specs + ▼ + ┌──────────────────────────────┐ + │ UTS test specs (.md) │ ← language-neutral pseudocode, one file per feature + │ "writing-test-specs" │ e.g. realtime/unit/connection/connection_recovery_test.md + └──────────────┬───────────────┘ + │ translated ("derived") per SDK + ▼ + ┌──────────────────────────────┐ + │ Derived tests │ ← concrete, runnable tests in the SDK's language + │ (this repo: Kotlin in uts/) │ e.g. ConnectionRecoveryTest.kt + └──────────────────────────────┘ +``` + +Three concepts you will see constantly: + +| Term | Meaning | +|------|---------| +| **Spec point** | A tagged requirement in the features spec, e.g. `RTN16g`, `RTN22`, `RTL4f`. Test names embed these. | +| **UTS spec** | A markdown file of portable pseudocode describing the setup, steps, and assertions for one feature. The *source of truth for what to test.* | +| **Derived test** | A faithful translation of a UTS spec into a real test in a specific SDK/language. This is what lives in `ably-java/uts/`. | +| **Deviation** | A documented case where the SDK's actual behaviour diverges from the spec. Recorded in `deviations.md`. | + +The golden rule (from [`writing-derived-tests.md`](https://github.com/ably/specification/blob/main/uts/docs/writing-derived-tests.md)): **translate the UTS spec faithfully** — same +structure, same assertions, same naming — don't optimise or skip steps. Every derived test carries a +`// UTS: ` (here `@UTS …`) comment linking it back to its spec. + +--- + +## 2. The Three Test Tiers + +UTS divides tests into three tiers by *what infrastructure they need* and *what confidence they +give*. Understanding this split is the key to understanding the whole `uts/` module, because the two +tests you asked about sit in two different tiers. + +| Tier | Transport | Backend | Purpose | Example in this repo | +|------|-----------|---------|---------|----------------------| +| **Unit** | **Mocked** (`MockWebSocket`, `MockHttpClient`) | none | Client-side logic: state machines, request formation, response parsing, timer behaviour. Fast & deterministic. | `unit/realtime/ConnectionRecoveryTest.kt` | +| **Direct sandbox integration** | Real network | Real Ably sandbox | Happy-path interop: connect, publish, subscribe. No fault injection. | `integration/standard//` *(tier exists; no tests yet)* | +| **Proxy integration** | Real network **through a programmable proxy** | Real Ably sandbox | Fault behaviour: dropped connections, injected errors, timeouts, re-auth. | `integration/proxy/realtime/AuthReauthTest.kt` | + +Each tier folder is further organised **by module** (`realtime`, `liveobjects`, …): `unit//`, +`integration/standard//`, and `integration/proxy//`. So a feature's tests sit together +by SDK area — the two example tests live at `unit/realtime/` and `integration/proxy/realtime/`. + +Key principles (from [`integration-testing.md`](https://github.com/ably/specification/blob/main/uts/docs/integration-testing.md)): + +- **Integration tests do not replace unit tests.** A spec point covered by a proxy test should + *also* have a unit test. The unit test proves the client logic; the proxy test proves the client + and the real server agree. +- **Proxy tests prefer "late fault injection".** Let the real handshake complete against the real + server, *then* inject the fault as the final interaction. This maximises how much of the test + exercises genuine client-server behaviour (otherwise you've just written a slow unit test). +- **Proxy tests always use JSON** (`useBinaryProtocol = false`). The spec corpus gives two reasons: + the proxy only supports **text** WebSocket frames so it can't inspect/modify msgpack + ([`integration-testing.md`](https://github.com/ably/specification/blob/main/uts/docs/integration-testing.md) §Protocol Variants), and the SDK under test doesn't implement msgpack + ([`helpers/proxy.md`](https://github.com/ably/specification/blob/main/uts/realtime/integration/helpers/proxy.md)). + +--- + +## 3. The UTS Documents (the source of truth) + +These four documents live in the **specification repo** at +[`uts/docs/`](https://github.com/ably/specification/blob/main/uts/docs/) (in a local +`ably-specification` checkout, under `specification/uts/docs/`). They are the policy/authoring guides; +the Kotlin code in this repo is the *implementation* of what they describe. Each title below links to +the file on GitHub. + +### 3.1 [`writing-test-specs.md`](https://github.com/ably/specification/blob/main/uts/docs/writing-test-specs.md) — how to author a portable UTS spec +The authoring manual. Defines: +- **Test types** (unit / integration / proxy) and when each applies. +- **Test IDs** — the format `//-`, e.g. + `realtime/proxy/RTN22/server-initiated-reauth-0`. These IDs are what appear in the `@UTS` + comments in the Kotlin tests. +- **Mock infrastructure pseudocode interfaces** — `MockHttpClient`, `MockWebSocket`, + `PendingConnection`, `PendingRequest`, with `respond_with_success()`, `send_to_client()`, + `simulate_disconnect()`, etc. The Kotlin classes in `uts/infra/unit/` are direct realisations of + these interfaces. +- **Handler vs await patterns** for mocks (see §6). +- **WebSocket closing semantics** — the crucial rule: `send_to_client_and_close()` for + DISCONNECTED / connection-level ERROR (server closes the socket); `send_to_client()` for a + channel-level ERROR (connection stays open). +- **Anti-flake conventions** — no fixed `WAIT`s; use polling, `AWAIT_STATE`, fake timers, and the + **record-and-verify** pattern (`CONTAINS_IN_ORDER`) for transient states. + +### 3.2 [`writing-derived-tests.md`](https://github.com/ably/specification/blob/main/uts/docs/writing-derived-tests.md) — how to translate a spec into a real SDK test +The translation manual. Two phases: +1. **Translation** (always): faithfully render the spec into the target language; map pseudocode to + the SDK's API and test framework; flag ambiguities in comments; make sure it compiles. +2. **Evaluation** (when an implementation exists): run the test and, if it fails, work the + **decision tree**: + - *Is the UTS spec wrong* (contradicts features spec)? → fix the test, record a **UTS spec error**. + - *Is the translation wrong*? → fix the test, no deviation. + - *Is the SDK non-compliant*? → keep the spec-correct assertion but adapt/gate it, and record a + **deviation**. +- Defines the **env-gated skip** pattern (`RUN_DEVIATIONS`) — the test holds the *spec-correct* + assertion but only runs it when the env var is set, so normal runs stay green while each deviation + stays individually reproducible. This is exactly what `ConnectionRecoveryTest` uses for RTN16f. + +### 3.3 [`integration-testing.md`](https://github.com/ably/specification/blob/main/uts/docs/integration-testing.md) — the policy for integration & proxy tests +Defines what *deserves* an integration test (request/response interop, error interop, data +round-trips, stateful protocol sequences), the directory layout, sandbox provisioning, proxy session +lifecycle, timeout strategy, and the **late-fault-injection** philosophy. The `integration/proxy/` +segregation exists because proxy tests have different infra needs, CI cadence, and failure modes. + +### 3.4 [`completion-status.md`](https://github.com/ably/specification/blob/main/uts/docs/completion-status.md) — the coverage matrix +A big table mapping every features-spec group (`RSC`, `RTN`, `RTL`, `RTP`, …) to the UTS specs that +cover it, with a per-tier summary (`unit:✓ proxy:✓`). This is the tracker for "what's done and +what's missing". The two tests you asked about correspond to these rows: +- `RTN16` (connection recovery) → unit spec `connection_recovery_test.md` → **`ConnectionRecoveryTest.kt`**. +- `RTN22` / `RTC8a` (server-initiated re-auth) → proxy spec + `realtime/integration/proxy/auth_reauth.md` → **`AuthReauthTest.kt`**. + +> There is also a fifth, *referenced* spec: +> [`realtime/integration/helpers/proxy.md`](https://github.com/ably/specification/blob/main/uts/realtime/integration/helpers/proxy.md) +> (in the spec repo under `uts/realtime/integration/helpers/`). It defines the proxy's control API, rule format, +> action types, and the **protocol message action-number table** (CONNECTED=4, ATTACH=10, AUTH=17, +> …). The Kotlin `ProxySession` is the client for exactly that API. + +--- + +## 4. The Java Setup: the `uts/` module + +The `uts/` directory is a **standalone Gradle module** (`include("uts")` in +`settings.gradle.kts`) whose only job is to host UTS-derived tests. It contains *no production code* — +everything lives under `uts/src/test/`. + +### 4.1 `uts/build.gradle.kts` +```kotlin +plugins { alias(libs.plugins.kotlin.jvm) } + +dependencies { + testImplementation(project(":java")) // the SDK under test + testImplementation(project(":network-client-core")) // HttpEngine / WebSocketEngine interfaces + testImplementation(kotlin("test")) + testImplementation(libs.mockk) + testImplementation(libs.coroutine.core) // kotlinx.coroutines + testImplementation(libs.coroutine.test) // runTest, virtual time + testImplementation(libs.ktor.client.core) // HTTP client for proxy/sandbox control + testImplementation(libs.ktor.client.cio) +} + +tasks.withType().configureEach { + useJUnitPlatform() // JUnit 5 + jvmArgs("--add-opens", "java.base/java.time=ALL-UNNAMED") + jvmArgs("--add-opens", "java.base/java.lang=ALL-UNNAMED") + // Propagate a local proxy build override (see ProxyManager): + systemProperty("uts.proxy.localPath", /* -Duts.proxy.localPath=… or $UTS_PROXY_LOCAL_PATH */ …) +} +``` +Takeaways: +- Tests are **Kotlin + JUnit 5**, using **kotlinx.coroutines** for async control and **Ktor** as the + HTTP client that talks to the sandbox REST API and the proxy control API. +- It depends on `:java` (the SDK) and `:network-client-core` (the pluggable transport interfaces the + mocks implement). +- The `--add-opens java.base/java.time` and `java.base/java.lang` flags grant reflective access into + those JDK packages for the test runtime. They mirror the same flags set in `java/build.gradle.kts` + for the SDK's own test module (which additionally opens `java.net` and `java.lang.reflect`). +- A system property carries an optional path to a **locally built** proxy binary (so you can test + against an unreleased proxy). + +### 4.2 Directory layout + +Everything lives under the `io.ably.lib.uts` package, split cleanly into **infrastructure** (`infra/`, +no `@Test`s) and the **tests** themselves. Tests are organised **by tier, then by module**: `unit/` for +mocked-transport tests, and `integration/` for real-backend tests — the latter splitting again into +`standard/` (direct sandbox, happy-path) and `proxy/` (sandbox through the fault-injecting proxy). Under +each, a per-module folder (`realtime`, `liveobjects`, …) holds the actual test classes: + +``` +uts/src/test/kotlin/io/ably/lib/uts/ +├── deviations.md # the catalogue of SDK-vs-spec divergences +│ +├── infra/ # ── TEST INFRASTRUCTURE (no @Test methods) ── +│ ├── Utils.kt # awaitState / awaitChannelState / pollUntil (shared) +│ │ +│ ├── unit/ # UNIT infra (mocked transports) +│ │ ├── ClientFactories.kt # TestRealtimeClient / TestRestClient / ClientOptionsBuilder +│ │ ├── MockWebSocket.kt # fake WS transport + WebSocketMockConfig + CONNECTED_MESSAGE +│ │ ├── MockWebSocketEngineFactory.kt# plugs the mock into the SDK's WebSocketEngine SPI +│ │ ├── MockHttpClient.kt # fake HTTP engine + HttpMockConfig +│ │ ├── MockHttpEngine.kt # plugs the mock into the SDK's HttpEngine SPI +│ │ ├── MockEvent.kt # sealed log of everything on a mock transport +│ │ ├── PendingConnection.kt # interface: a connection attempt awaiting a response +│ │ ├── DefaultPendingConnection.kt # WS implementation of PendingConnection +│ │ ├── PendingRequest.kt # interface: an in-flight HTTP request awaiting a response +│ │ ├── DefaultPendingRequest.kt # HTTP implementation of PendingRequest +│ │ ├── FakeClock.kt # virtual clock + virtual timers (deterministic time) +│ │ └── Utils.kt # ConnectionDetails { } builder (reflective constructor) +│ │ +│ └── integration/ # INTEGRATION infra (real backend) +│ ├── SandboxApp.kt # provisions/deletes a sandbox app +│ └── proxy/ +│ ├── ProxyManager.kt # downloads/launches the uts-proxy binary +│ └── ProxySession.kt # proxy session: rules, actions, log + connectThroughProxy +│ +├── unit/ # ── UNIT TESTS (mock transport) ── · per module +│ ├── realtime/ +│ │ └── ConnectionRecoveryTest.kt # ← the UNIT test (RTN16*) +│ └── liveobjects/ # (further modules as coverage grows) +│ +└── integration/ # ── INTEGRATION TESTS (real backend) ── · per module + ├── standard/ # direct sandbox: happy-path, no fault injection + │ ├── realtime/ + │ └── liveobjects/ + └── proxy/ # sandbox through the fault-injecting uts-proxy + ├── realtime/ + │ └── AuthReauthTest.kt # ← the PROXY test (RTN22, RTC8a) + └── liveobjects/ +``` + +The mental model: **`infra/unit/` powers the unit tests, `infra/integration/` powers both integration +kinds (`standard` + `proxy`), and `infra/Utils.kt` serves all of them.** Every tier is sub-divided **by +module** (`realtime`, `liveobjects`, …) so a feature's tests sit together regardless of SDK area. The +top-level `unit/` ↔ `infra/unit/` and `integration/` ↔ `infra/integration/` pairing is what the +`runUtsUnitTests` / `runUtsIntegrationTests` Gradle tasks key off (§12) — `runUtsIntegrationTests` +covers **both** `integration/standard/` and `integration/proxy/`. + +--- + +## 5. How a Test Reaches the SDK: the hook points + +A test can only mock transports because the SDK was designed with **pluggable seams**. They live on +`io.ably.lib.debug.DebugOptions` (a subclass of `ClientOptions`): + +```java +public class DebugOptions extends ClientOptions { + public HttpEngine httpEngine; // ← MockHttpClient installs here + public WebSocketEngineFactory webSocketEngineFactory; // ← MockWebSocket installs here + public Clock clock; // ← FakeClock installs here + … +} +``` + +and the `Clock` interface: + +```java +public interface Clock { + long currentTimeMillis(); + long nanoTime(); + AblyTimer newTimer(String name); // every SDK timer is created through this + void waitOn(Object target, long timeout) throws InterruptedException; // every blocking wait +} +``` + +So the recipe is: +- Want to fake the **WebSocket**? Set `webSocketEngineFactory` to a factory that produces a mock + engine. +- Want to fake **HTTP**? Set `httpEngine` to a mock engine. +- Want to control **time** (timeouts, retries, TTL expiry) deterministically? Set `clock` to a + `FakeClock`. + +The `ClientOptionsBuilder` (next section) wraps all three so tests never touch `DebugOptions` +directly. + +--- + +## 6. Unit-Test Infrastructure (mocked transports) + +### 6.1 The client builder — `ClientFactories.kt` +Every unit test builds its client through a tiny DSL: + +```kotlin +class ClientOptionsBuilder : DebugOptions("appId.keyId:keySecret") { + init { useBinaryProtocol = false } // JSON so mocks can decode frames + fun install(mock: MockWebSocket) = mock.installOn(this) + fun install(mock: MockHttpClient) = mock.installOn(this) + fun enableFakeTimers(fakeClock: FakeClock) { clock = fakeClock } +} + +fun TestRealtimeClient(block: ClientOptionsBuilder.() -> Unit): AblyRealtime = + AblyRealtime(ClientOptionsBuilder().apply(block)) +fun TestRestClient(block: ClientOptionsBuilder.() -> Unit): AblyRest = + AblyRest(ClientOptionsBuilder().apply(block)) +``` + +- It seeds a **dummy API key** (`appId.keyId:keySecret`) — fine, because unit tests never hit a real + server and tokens are opaque. +- It forces **JSON** so the mock can parse protocol frames. +- `install(mock)` / `enableFakeTimers(clock)` wire the seams from §5. + +A typical unit test reads: +```kotlin +val mock = MockWebSocket { onConnectionAttempt = { it.respondWithSuccess(CONNECTED_MESSAGE) } } +val client = TestRealtimeClient { + autoConnect = false + install(mock) +} +``` + +### 6.2 `MockWebSocket` — the fake realtime transport +This is the heart of realtime unit testing. It plugs into the SDK via +`MockWebSocketEngineFactory` (which implements the SDK's `WebSocketEngineFactory` SPI from +`network-client-core`), and exposes two complementary control styles: + +**(a) Callback style** — handle inline, synchronously on the SDK thread. Set fields on +`WebSocketMockConfig`: +```kotlin +val mock = MockWebSocket { + onConnectionAttempt = { conn -> conn.respondWithSuccess(CONNECTED_MESSAGE) } + onMessageFromClient = { msg -> /* inspect frames the SDK sent */ } +} +``` +Best when every connection attempt should behave the same way. + +**(b) Await style** — suspend until the SDK triggers something, then respond. Leave the callbacks +null and call the `await*` methods: +```kotlin +val pending = mock.awaitConnectionAttempt() // suspend until SDK opens a socket +pending.respondWithRefused() // …then decide how to answer +val frame = mock.awaitNextMessageFromClient() // suspend until SDK sends a frame +``` +Required when the *first* connection and a *reconnection* need different answers (e.g. +"connect succeeds, then all retries are refused" — exactly the SUSPENDED scenario in the unit test). + +> ⚠️ You cannot mix the two styles for the same event type — a callback consumes the event before the +> queue ever sees it. + +**Server → client direction** (driving the SDK), matching the spec's closing semantics: + +| Method | What it does | Use for | +|--------|--------------|---------| +| `sendToClient(msg)` | deliver a frame, connection stays open | CONNECTED, ATTACHED, channel-level ERROR, normal messages | +| `sendToClientAndClose(msg)` | deliver a frame then close (code 1000) | DISCONNECTED, connection-level ERROR (fatal) | +| `simulateDisconnect()` | close with code 1006, no message | unexpected network drop → triggers DISCONNECTED/resume | + +**Everything is logged.** `mock.events` is an ordered `List` (a sealed class in +`MockEvent.kt`: `ConnectionAttempt`, `ConnectionEstablished`, `ConnectionRefused`, `SentToClient`, +`MessageFromClient`, `ClientClose`, `Disconnected`, …). Tests assert against it, e.g. +`mock.events.filterIsInstance().size`. + +**`CONNECTED_MESSAGE`** is a ready-to-use CONNECTED `ProtocolMessage` (connectionId +`test-connection-id`, a connection key, TTL 120 s, max-idle 15 s) so most tests don't hand-build it. +(It is a `val` with a custom getter, so each access returns a **fresh** instance — not a shared +singleton; safe to mutate per test, e.g. `CONNECTED_MESSAGE.apply { … }`.) + +One subtlety encoded in `DefaultPendingConnection.respondWithSuccess(message)`: the CONNECTED frame is +delivered **asynchronously** on a separate `mock-ws-delivery` thread. That mirrors reality — the SDK +must store the WebSocket reference *before* it processes CONNECTED, so the mock must not deliver it +synchronously inside the connect call. + +### 6.3 `MockHttpClient` — the fake REST transport +The HTTP analogue, plugged in via `MockHttpEngine` (implements the SDK's `HttpEngine` SPI). Same two +styles (`onConnectionAttempt`/`onRequest` callbacks, or `awaitConnectionAttempt()`/`awaitRequest()`). +A request flows in two phases inside `MockHttpCall.execute()`: +1. **Connect phase** → produces a `PendingConnection` (`respondWithSuccess/Refused/Timeout/DnsError`). +2. **Request phase** → produces a `PendingRequest` exposing `url`, `method`, `headers`, `body`, and + `respondWith(status, body, headers)` / `respondWithDelay(...)` / `respondWithTimeout()`. + +This lets REST unit tests assert on outgoing request shape (path, headers, query) and feed canned +responses back — all without a socket. + +### 6.4 `FakeClock` — deterministic time +`FakeClock` implements the SDK's `Clock`. Time is frozen until you call `advance(ms)`; on each +advance it fires any due virtual timers **synchronously**, and wakes any `waitOn` sleepers. This is +how the unit test drives reconnection backoff and `connectionStateTtl` expiry **without real +sleeping**: +```kotlin +val fakeClock = FakeClock() +val client = TestRealtimeClient { enableFakeTimers(fakeClock); … } +… +fakeClock.advance(2.seconds) // jump forward; due timers fire now +``` +`pendingTaskCount(timerName)` lets you assert how many tasks are scheduled — useful for verifying +retry state. + +--- + +## 7. Proxy-Integration Infrastructure (real backend + fault injection) + +Proxy tests connect the **real SDK** to the **real Ably sandbox**, but route the traffic through a +small Go program — [`ably/uts-proxy`](https://github.com/ably/uts-proxy) — that can be told to inject +faults. Three Kotlin helpers make this work. + +### 7.1 `ProxyManager` — gets the proxy binary running +A singleton (`object`) responsible for the proxy *process*: +- Pins a proxy version (`v0.3.0`) and knows the **SHA-256 checksums** for each + OS/arch archive. +- `ensureProxy()` (called in `@BeforeAll`) is idempotent: if a proxy is already healthy on the + control port (**10100**) it's a no-op; otherwise it **downloads** the right + `uts-proxy___.tar.gz` from GitHub releases, **verifies the checksum**, extracts the + binary with a hand-rolled tar/gzip reader (JDK-only, no extra deps), caches it under + `~/.cache/uts-proxy//`, and launches it with `--port 10100`. +- The download is serialised **across JVMs** by a `FileLock` and **within a JVM** by a `Mutex`. + Because process startup shares the control port, `ProxyManager`'s KDoc **advises** running proxy + suites single-fork (`maxParallelForks = 1`) to avoid two Gradle workers racing to bind the control + port. ⚠️ Note: this is currently only a documented recommendation — it is **not** set in + `uts/build.gradle.kts`. With a single proxy test class today the race is not yet triggered, but it + should be configured before a second proxy suite is added. +- A **JVM shutdown hook** force-kills the spawned process on exit (a `ProcessBuilder` child does not + die with its parent). +- Override knob: set `-Duts.proxy.localPath=…` or `$UTS_PROXY_LOCAL_PATH` to use a **locally built** + proxy binary or `.tar.gz` (skips download + checksum). The build script forwards this property + into the test JVM. + +### 7.2 `ProxySession` — one test's window into the proxy +The proxy exposes a **control REST API** on the control port; `ProxySession` is the typed Kotlin +client for it (via Ktor). One session per test. + +- `ProxySession.create(rules, …)` → `POST /sessions` with a `target` (the sandbox realtime/REST + hosts) and an initial **rule list**; the proxy assigns a `sessionId` and a fresh **listening + port**. +- `addRules(rules, position)` → add rules mid-test (`POST /sessions/{id}/rules`). +- `triggerAction(action)` → fire an **imperative** action *right now* (`POST + /sessions/{id}/actions`) — e.g. inject a frame or drop the connection at a precise moment. +- `getLog()` → `GET /sessions/{id}/log`, returning a typed `List`. Each `Event` carries + `type` (`ws_connect`, `ws_frame`, `http_request`, …), `direction`, `queryParams`, and the parsed + protocol `message` (a `JsonObject`, introspected via `message?.get("action")?.asInt`). +- `close()` → `DELETE /sessions/{id}`, always called in a `finally`. + +**Rules** = `match` + `action` (+ optional `times`). Builder helpers keep tests readable: +`wsConnectRule`, `wsFrameToClientRule`, `wsFrameToServerRule`, `httpRequestRule`. Rules evaluate in +order, first match wins, unmatched traffic passes through, and `times: N` auto-removes a rule after N +firings. Common actions: `refuse_connection`, `suppress`, `replace`, `inject_to_client[_and_close]`, +`disconnect`, `http_respond`. + +**Wiring the client to the proxy** — the `connectThroughProxy(session)` extension does exactly what +the proxy spec prescribes: +```kotlin +fun ClientOptionsBuilder.connectThroughProxy(session: ProxySession) { + realtimeHost = session.proxyHost // "localhost" + restHost = session.proxyHost + port = session.proxyPort // the session's assigned port + tls = false // proxy serves plain HTTP/WS; TLS is only upstream +} +``` +Explicit hosts auto-disable fallback hosts (REC2c2), so no `fallbackHosts` juggling is needed. + +### 7.3 `SandboxApp` — a throwaway app on the real sandbox +Provisioning helper for the real backend (provisioned **directly**, not through the proxy, so it's +independent of the fault rules): +- `SandboxApp.create()` fetches the canonical `test-app-setup.json` from `ably-common`, + `POST`s it to `https://sandbox.realtime.ably-nonprod.net/apps`, and exposes `appId`, `defaultKey` + (full-capability `appId.keyId:keySecret`), and the full `keys` list. +- `delete()` removes the app in teardown (best-effort — errors are swallowed since sandbox apps + auto-expire). +- The Ktor client retries only **idempotent GETs** (never re-POSTs `/apps`, to avoid duplicate + apps). + +`SandboxApp` is the shared backbone of *both* integration kinds: **proxy** tests pair it with a +`ProxySession`, while **direct sandbox** tests (`integration/standard//`) use it alone — +connecting straight to `ProxyManager.sandboxRealtimeHost` / `sandboxRestHost` with no proxy and no +fault rules, for happy-path interop. + +--- + +## 8. Shared Async Helpers + +`Utils.kt` provides the coroutine glue every tier relies on (unit, direct sandbox, and proxy). All +three run on a **single-thread real dispatcher** so their timeouts measure **wall-clock** time (not the virtual time of +`kotlinx.coroutines.test`). The two state-waiters (`awaitState`/`awaitChannelState`) register their +listener *before* checking current state, to avoid a check-then-register race; `pollUntil` has no +listener — it re-evaluates the predicate every `interval` until it holds or the timeout fires. + +| Helper | Signature | Purpose | +|--------|-----------|---------| +| `awaitState` | `(client, target, timeout=5s)` | suspend until `connection.state == target` (or already there) | +| `awaitChannelState` | `(channel, target, timeout=5s)` | same, for a channel's state | +| `pollUntil` | `(timeout=15s, interval=100ms) { condition }` | suspend until a boolean predicate holds — used in proxy tests to wait on real network/proxy state, e.g. `pollUntil { authCallbackCount.get() > original }` | + +A second `Utils.kt` under `infra/unit/` adds the `ConnectionDetails { … }` builder DSL so tests can +write `ConnectionDetails { connectionKey = "key-1"; connectionStateTtl = 120000L }`. Since this file +no longer sits in the `io.ably.lib.types` package, it can't call `ConnectionDetails`'s package-private +constructor directly — it obtains an instance **reflectively** (the same package-private-access +technique used by `liveobjects/.../TestUtils.kt`). See Appendix B.1. + +--- + +## 9. Walkthrough: the Unit Test (`ConnectionRecoveryTest`) + +**File:** `uts/.../uts/unit/realtime/ConnectionRecoveryTest.kt` (package `io.ably.lib.uts.unit.realtime`) +**Tier:** Unit (mocked WebSocket, no network). +**Spec area:** RTN16 — connection recovery via the `recover` option and `createRecoveryKey()`. + +It contains six tests; each carries an `@UTS realtime/unit/RTN16…/…` tag. Here's what each proves and +the technique it uses: + +### 9.1 `RTN16g, RTN16g1` — recovery-key structure (incl. Unicode) +Connects (mock returns CONNECTED with a known key), attaches two channels — one ASCII, one Unicode +(`channel-éàü-世界`) — feeding each an `ATTACHED` with a `channelSerial` via `sendToClient`. Then calls +`connection.createRecoveryKey()`, decodes it with `RecoveryKeyContext.decode`, and asserts the +connection key, `msgSerial == 0`, and both channel serials survive — including a full +**encode→decode round-trip** to prove the Unicode name isn't corrupted (RTN16g1). +*Technique: callback-style `onConnectionAttempt`, `sendToClient` for ATTACHED, `awaitChannelState`.* + +### 9.2 `RTN16g2` — `createRecoveryKey()` returns null in inactive states +The most elaborate test — it walks the connection through **five** states and asserts the key is null +in each inactive one: +- **INITIALIZED** (before connect) → null. +- **CONNECTED** → non-null (sanity). +- **CLOSING / CLOSED** → null (close nulls the key immediately). +- **FAILED** → null. *(Contains a documented **deviation** — see §11: the spec's fatal error + code 50000/500 isn't treated as fatal by the SDK, and `send_to_client_and_close` races the FAILED + transition; the test uses code 40000/400 and plain `sendToClient`.)* +- **SUSPENDED** → null. Built with a `FakeClock`: connect succeeds, then `simulateDisconnect()`, + then a coroutine **refuses every reconnection attempt** while `fakeClock.advance(2.seconds)` loops + until the short `connectionStateTtl` (800 ms) expires and the client gives up to SUSPENDED. +*Technique: this is the textbook example of **await-style** mocking — the first connection succeeds +via `awaitConnectionAttempt()`, but reconnections need the *refused* response, so a separate +`refuseJob` coroutine drives them; mixing this with fake timers gives deterministic SUSPENDED.* + +### 9.3 `RTN16k` — `recover` adds the `recover` query param +Constructs the client with `recover = `, captures `conn.queryParams` on each connection +attempt, then `simulateDisconnect()` and reconnect. Asserts the **first** attempt carries +`recover=` (and no `resume`), while the **second** (post-reconnect) carries `resume=` +(and no `recover`) — i.e. recover is a one-shot bootstrap, subsequent reconnections use resume. + +### 9.4 `RTN16f` — `recover` initialises `msgSerial` *(env-gated deviation)* +Asserts the recovered `msgSerial` (42) is preserved. The SDK resets it to 0, so the spec-correct +assertion `assertEquals(42L, …)` runs only under `RUN_DEVIATIONS`; otherwise a regression-guard +`assertEquals(0L, …)` runs. (See §11.) + +### 9.5 `RTN16f1` — malformed `recover` key degrades gracefully +`recover = "this-is-not-valid-json!!!"`. Asserts the client still connects normally with a fresh +identity, **no** `recover`/`resume` query params, and exactly one connection attempt — i.e. a bad key +is logged and ignored, not fatal. + +### 9.6 `RTN16j` — `recover` instantiates channels with their serials (RTN16i too) +Recovery key carries three channels (incl. Unicode). Asserts each `channels.get(name).properties. +channelSerial` matches the key, that the channels are **NOT auto-attached** (state INITIALIZED — +RTN16i), and that a manual `attach()` sends an ATTACH frame carrying the recovered serial (verified +via `awaitNextMessageFromClient()`). + +**What this test teaches about the infra:** callback vs await styles side by side, `FakeClock`-driven +SUSPENDED, `sendToClient` for server frames, `events`/`awaitNextMessageFromClient` for inspecting +client output, and the env-gated deviation pattern. + +--- + +## 10. Walkthrough: the Proxy Test (`AuthReauthTest`) + +**File:** `uts/.../uts/integration/proxy/realtime/AuthReauthTest.kt` (package `io.ably.lib.uts.integration.proxy.realtime`) +**Tier:** Proxy integration (real sandbox + uts-proxy). +**Spec points:** RTN22 (server-initiated re-authentication) and RTC8a (the client sends an AUTH +frame with renewed auth details). Unit-test counterparts: `server_initiated_reauth_test.md`, +`realtime_authorize.md`. + +### 10.1 Suite setup/teardown +```kotlin +@TestInstance(TestInstance.Lifecycle.PER_CLASS) // one instance, so @BeforeAll can be non-static +class AuthReauthTest { + @BeforeAll fun setUpAll() = runBlocking { + ProxyManager.ensureProxy() // download+launch proxy if needed + app = SandboxApp.create() // provision a real sandbox app + } + @AfterAll fun tearDownAll() = runBlocking { if (::app.isInitialized) app.delete() } +} +``` + +### 10.2 The test, step by step +1. **Create a session with no rules** — the fault will be injected *imperatively* later (late + injection — the connect handshake runs against the real server unmodified): + ```kotlin + val session = ProxySession.create(rules = emptyList()) + ``` +2. **Auth via `authCallback`** — the spec generates a JWT from the sandbox key; the idiomatic + ably-java equivalent is a locally-signed `TokenRequest` from the same key (no external JWT + library). A counter records how many times the callback is invoked: + ```kotlin + val tokenSigner = AblyRest(app.defaultKey) + val authCallback = Auth.TokenCallback { params -> + authCallbackCount.incrementAndGet() + tokenSigner.auth.createTokenRequest(params, null) + } + ``` +3. **Build the client through the proxy** and connect (JSON stays on so the proxy can inspect + frames): + ```kotlin + val client = TestRealtimeClient { + this.authCallback = authCallback + connectThroughProxy(session) + autoConnect = false + } + client.connect() + awaitState(client, ConnectionState.connected, 15.seconds) + ``` +4. **Snapshot identity** — `connection.id` and the callback count, and assert the callback already + ran ≥ 1 (initial auth). +5. **Start recording state changes**, then **inject a server-initiated AUTH** (protocol action 17) + imperatively — simulating Ably asking the client to re-authenticate: + ```kotlin + session.triggerAction(mapOf("type" to "inject_to_client", + "message" to mapOf("action" to 17))) + ``` +6. **Wait for the re-auth round-trip** with `pollUntil { stateChanges.size > 1 }` (real network, so + poll — don't sleep). +7. **Assertions** prove RTN22 + RTC8a: + - `authCallback` was invoked **again** (count incremented) → re-auth was triggered. + - Connection is still **CONNECTED** and `connection.id` is **unchanged** → re-auth does not + reconnect. + - **No** transitions away from CONNECTED were recorded. + - The **proxy event log** contains a client→server **AUTH frame (action 17) carrying non-null + `auth` details** (RTC8a) — verified by filtering `session.getLog()`. +8. **Nested teardown** in `finally`: close the client and wait for CLOSED, then always close the + session and the token signer. + +**What this test teaches about the infra:** `ProxyManager.ensureProxy` + `SandboxApp` setup, +`connectThroughProxy`, **late imperative fault injection** via `triggerAction`, real-network waiting +with `pollUntil`, and **proxy-log assertions** as the primary verification (`getLog()` → +filter by `type`/`direction`/`message.action`). + +--- + +## 11. Deviations: when the SDK disagrees with the spec + +`uts/.../io/ably/lib/uts/deviations.md` is the single catalogue of every place the ably-java SDK behaves +differently from the features spec, discovered during translation. Each entry records: the **spec +point**, **what the spec requires**, **what the SDK does**, the **root cause** (file/function, where +known), the **workaround in tests**, and the **affected tests**. + +The mechanism (from [`writing-derived-tests.md`](https://github.com/ably/specification/blob/main/uts/docs/writing-derived-tests.md)): the test keeps the **spec-correct** assertion but +gates it behind the `RUN_DEVIATIONS` env var, with a regression-guard assertion for the SDK's actual +behaviour running by default. Normal runs stay green; `RUN_DEVIATIONS=1` turns the failing assertions +on so the gap is reproducible and the test flips automatically once the SDK is fixed. + +Current entries relevant to the two tests: + +| Spec point | Gist | Touches | +|------------|------|---------| +| **RTN16f** | SDK resets `msgSerial` to 0 on connect even with `recover`; spec says preserve it (42). | `ConnectionRecoveryTest` (§9.4) — `assertEquals(42L,…)` gated, `assertEquals(0L,…)` default guard. | +| **RTN16g2** | Spec's fatal error 50000/500 isn't fatal to the SDK (`isFatalError()` needs code 40000–49999 or status < 500); also `send_to_client_and_close` races the FAILED transition. | `ConnectionRecoveryTest` (§9.2) — uses 40000/400 + plain `sendToClient`. | +| **RTL13b** | `ATTACHING → SUSPENDED` via `realtimeRequestTimeout` not implemented for channel attach. | various channel tests (not the two here). | +| **RTL13c** | `channelRetryTimeout` not cancelled when the connection leaves CONNECTED. | various channel tests; assertions gated behind `RUN_DEVIATIONS`. | + +> These deviations are **valuable output**, not failures — each one is a precise, reproducible bug +> report the SDK team can act on, and the gated test becomes the acceptance test for the fix. + +--- + +## 12. How to Run the Tests + +There are two custom Gradle tasks (registered in `uts/build.gradle.kts`), filtered by package — they +mirror `runLiveObjectsUnitTests` / `runLiveObjectsIntegrationTests` in the `liveobjects` module: + +```bash +# Unit tests only — io.ably.lib.uts.unit.* (fast, no network). This is the PR gate. +./gradlew :uts:runUtsUnitTests + +# Integration tests only — io.ably.lib.uts.integration.* (real sandbox; covers both +# integration/standard/ and integration/proxy/ — proxy tests also download/launch the uts-proxy). +./gradlew :uts:runUtsIntegrationTests + +# Everything (the default Test task still runs both): +./gradlew :uts:test + +# Just one test class (works with any of the tasks above): +./gradlew :uts:runUtsUnitTests --tests "io.ably.lib.uts.unit.realtime.ConnectionRecoveryTest" +./gradlew :uts:runUtsIntegrationTests --tests "io.ably.lib.uts.integration.proxy.realtime.AuthReauthTest" + +# Turn on the spec-correct (currently failing) deviation assertions: +RUN_DEVIATIONS=1 ./gradlew :uts:runUtsUnitTests --tests "*ConnectionRecoveryTest*" + +# Run proxy tests against a locally built proxy instead of a GitHub release: +./gradlew :uts:runUtsIntegrationTests -Duts.proxy.localPath=/path/to/uts-proxy # or .tar.gz +# (equivalently: export UTS_PROXY_LOCAL_PATH=/path/to/uts-proxy) +``` + +**Where CI runs them:** `runUtsUnitTests` is part of the `check.yml` gate (alongside +`runLiveObjectsUnitTests`); `runUtsIntegrationTests` runs in the `check-uts` job of +`integration-test.yml` (alongside `check-liveobjects`). + +Notes: +- `ProxyManager` **advises** running proxy suites single-fork (`maxParallelForks = 1`) because they + share the control port (10100). This is not currently set in `uts/build.gradle.kts`; it isn't + exercised yet because there is only one proxy test class. +- Proxy/sandbox tests need outbound network (sandbox + GitHub releases on first run; the binary is + then cached under `~/.cache/uts-proxy/`). +- Before pushing, run the project's static-analysis gate (from `CLAUDE.md`): + `./gradlew checkWithCodenarc checkstyleMain checkstyleTest` — Checkstyle is Java-only and easy to + miss; remember **no star imports**. + +--- + +## 13. Quick Reference / Cheat-Sheet + +**The two seams that make unit tests possible** (`DebugOptions`): +`webSocketEngineFactory` (WS), `httpEngine` (HTTP), `clock` (time). + +**Build a unit-test client:** +```kotlin +val mock = MockWebSocket { onConnectionAttempt = { it.respondWithSuccess(CONNECTED_MESSAGE) } } +val client = TestRealtimeClient { autoConnect = false; install(mock) } +client.connect(); awaitState(client, ConnectionState.connected) +``` + +**Build a proxy-test client:** +```kotlin +ProxyManager.ensureProxy(); val app = SandboxApp.create() +val session = ProxySession.create(rules = emptyList()) +val client = TestRealtimeClient { authCallback = …; connectThroughProxy(session); autoConnect = false } +``` + +**Server→client (mock):** `sendToClient` (stays open) · `sendToClientAndClose` (DISCONNECTED / +fatal ERROR) · `simulateDisconnect` (1006 drop). + +**Inspect what the SDK did:** `mock.events` (unit) · `session.getLog()` (proxy). + +**Wait (never sleep):** `awaitState` · `awaitChannelState` · `pollUntil { … }` · `FakeClock.advance(…)`. + +**Protocol action numbers** (used in rules & log assertions): CONNECTED=4, DISCONNECTED=6, ERROR=9, +ATTACH=10, ATTACHED=11, DETACH=12, DETACHED=13, **AUTH=17**. + +**Test ID format:** `//-` → +`@UTS realtime/proxy/RTN22/server-initiated-reauth-0`. + +**The decision tree when a translated test fails:** spec wrong → fix test + record UTS spec error; +translation wrong → fix test; SDK non-compliant → gate spec-correct assertion behind `RUN_DEVIATIONS` +and record in `deviations.md`. + +--- + +## 14. Appendix A: Request-Flow Diagrams + +### A.1 Unit test — mocked WebSocket (no network) + +A unit test installs `MockWebSocket` into `DebugOptions.webSocketEngineFactory`. The SDK believes it +is talking to a real socket; in fact every byte is intercepted by the mock and surfaced to the test. + +``` + ┌──────────────────────────────────── TEST (Kotlin coroutine) ────────────────────────────────────┐ + │ │ + │ TestRealtimeClient { install(mock); autoConnect = false } │ + │ │ client.connect() ▲ awaitState(client, connected) │ + │ ▼ │ │ + │ ┌───────────┐ webSocketEngineFactory ┌──────────────────────────┐ │ + │ │ AblyRealtime (SDK :java) │──────────▶ │ MockWebSocketEngineFactory │ (implements SDK SPI) │ + │ │ ConnectionManager, etc. │ └─────────────┬────────────┘ │ + │ └───────────┬────────────────┘ │ create() │ + │ │ send(frame) ───────────────────────────▶ │ │ + │ │ ▼ │ + │ │ ┌────────────────────────┐ │ + │ │ │ MockWebSocket │ │ + │ onMessage(frame) ◀───────────────────────│ • records MockEvent[] │ │ + │ ▲ │ • onConnectionAttempt │ ◀── PendingConnection ──┐ │ + │ │ │ • onMessageFromClient │ │ │ + │ │ └───────────┬────────────┘ │ │ + │ │ │ │ │ + │ TEST drives the "server" side: │ TEST inspects/responds: │ │ + │ mock.sendToClient(CONNECTED) ───────────────────────┘ pending.respondWithSuccess(msg) ───┘ │ + │ mock.sendToClientAndClose(DISCONNECTED) mock.awaitNextMessageFromClient() │ + │ mock.simulateDisconnect() mock.events (assert) │ + │ │ + │ FakeClock (DebugOptions.clock): fakeClock.advance(2.s) ── fires due timers synchronously │ + └───────────────────────────────────────────────────────────────────────────────────────────────┘ + + No TCP, no DNS, no real time. Everything is in-process and deterministic. +``` + +(The HTTP path is identical in shape: `MockHttpClient` → `DebugOptions.httpEngine` → +`MockHttpEngine` → `PendingConnection` then `PendingRequest`, with `respondWith(status, body)`.) + +### A.2 Proxy integration test — real backend through the fault-injecting proxy + +A proxy test uses the **real** SDK transport but points its host/port at the local `uts-proxy` +process, which forwards to the Ably sandbox and can inject faults on command. + +``` + ┌─────────────────── TEST (Kotlin) ───────────────────┐ + │ @BeforeAll: ProxyManager.ensureProxy() │ downloads/launches binary, control :10100 + │ SandboxApp.create() ─────────────────────────────────────────────┐ POST /apps (direct, TLS) + │ session = ProxySession.create(rules) ──────────── control REST :10100 ───┐ │ + │ client = TestRealtimeClient { connectThroughProxy(session) } │ │ + └──────────────┬───────────────────────────────────────────────────────────┘ │ + │ client.connect() (host=localhost, port=session.port, tls=false) │ + ▼ │ ▼ + ┌──────────────────┐ ws/http (plain) ┌───────────────────────┐ │ ┌───────────────────────┐ + │ AblyRealtime │ ◀──────────────────▶ │ uts-proxy │ ◀─┼▶│ Ably sandbox │ + │ (REAL transport) │ │ • forwards traffic │ │ │ sandbox.realtime. │ + └──────────────────┘ │ • applies rules │ │ │ ably-nonprod.net (TLS) │ + ▲ │ • records event log │ │ └───────────────────────┘ + │ TEST controls the proxy: └──────────┬────────────┘ │ + │ session.triggerAction({inject_to_client, action:17}) │ control REST :10100 + │ session.addRules([...]) │ + │ TEST verifies via: │ + │ session.getLog() ── filter type/direction/message.action ─┘ + │ awaitState(...) / pollUntil { ... } + └── (everything before the injected fault is REAL client↔server traffic) +``` + +**Why two channels to the proxy?** The **data plane** (the SDK's ws/http traffic on +`session.proxyPort`) is separate from the **control plane** (the test's REST calls on +`CONTROL_PORT = 10100` to create sessions, add rules, trigger actions, read the log). The SDK never +sees the control plane; the test never speaks the data plane directly. + +--- + +## 15. Appendix B: Per-File API Reference + +A one-stop table of every Kotlin source file under `uts/src/test/` and the SDK seams they use, so +nothing is left implicit. + +### B.1 Unit-test infrastructure — `io.ably.lib.uts.infra.unit` + +| File | Key public surface | Role | +|------|--------------------|------| +| `ClientFactories.kt` | `ClientOptionsBuilder` (extends `DebugOptions`), `TestRealtimeClient { }`, `TestRestClient { }`, `install(mock)`, `enableFakeTimers(clock)` | Entry point for building a mocked SDK client; seeds dummy key, forces JSON. | +| `MockWebSocket.kt` | `MockWebSocket`, `WebSocketMockConfig` (`onConnectionAttempt`, `onMessageFromClient`, `onTextDataFrame`, `onBinaryDataFrame`), `events`, `installOn`, `awaitConnectionAttempt`, `awaitNextMessageFromClient`, `awaitClientClose`, `sendToClient`, `sendToClientAndClose`, `simulateDisconnect`, `reset`; top-level `MockWebSocket { }`, `CONNECTED_MESSAGE` | Fake realtime transport (callback + await styles). | +| `MockWebSocketEngineFactory.kt` | `MockWebSocketEngineFactory`, `MockWebSocketEngine`, `MockWebSocketClient` (implement `WebSocketEngineFactory`/`Engine`/`Client`) | Adapts the mock to the SDK's WebSocket SPI; parses URL → host/port/tls/query. | +| `MockHttpClient.kt` | `MockHttpClient`, `HttpMockConfig` (`onConnectionAttempt`, `onRequest`), `engine`, `installOn`, `awaitConnectionAttempt`, `awaitRequest`, `reset`; top-level `MockHttpClient { }` | Fake REST transport. | +| `MockHttpEngine.kt` | `MockHttpEngine`, `MockHttpCall`, `DefaultHttpPendingConnection` (implement `HttpEngine`/`HttpCall`) | Adapts the mock to the SDK's HTTP SPI; two-phase connect→request in `execute()`. | +| `PendingConnection.kt` | `interface PendingConnection` (`host`,`port`,`tls`,`queryParams`, `respondWithSuccess[ (message) ]`, `respondWithRefused/Timeout/DnsError`); plus the top-level helper `parseQueryString()` (not an interface member) | Abstract connection attempt awaiting a verdict (shared WS + HTTP). | +| `DefaultPendingConnection.kt` | `DefaultPendingConnection : PendingConnection` | WS impl; **async** CONNECTED delivery on `mock-ws-delivery` thread. | +| `PendingRequest.kt` | `interface PendingRequest` (`url`,`method`,`headers`,`body`, `respondWith`, `respondWithDelay`, `respondWithTimeout`) | Abstract in-flight HTTP request awaiting a response. | +| `DefaultPendingRequest.kt` | `DefaultPendingRequest : PendingRequest` | HTTP impl backed by a `CompletableDeferred`. | +| `MockEvent.kt` | `sealed class MockEvent`: `ConnectionAttempt`, `ConnectionEstablished`, `ConnectionRefused`, `ConnectionTimeout`, `DnsError`, `HttpRequest`, `SentToClient`, `Disconnected`, `ClientClose`, `MessageFromClient` | Ordered, typed log of everything that happened on a mock transport. | +| `FakeClock.kt` | `FakeClock : Clock` (`advance(ms\|Duration)`, `pendingTaskCount(name)`, `currentTimeMillis`, `nanoTime`, `newTimer`, `waitOn`) | Virtual clock + virtual timers; deterministic time. | +| `Utils.kt` | `ConnectionDetails { }` builder | Test-only `ConnectionDetails` DSL; instantiates the type via its **package-private constructor reflectively** (see §8). | + +### B.2 Integration infrastructure — `io.ably.lib.uts.infra.integration` (and `…integration.proxy`) + +| File | Key public surface | Role | +|------|--------------------|------| +| `proxy/ProxyManager.kt` | `object ProxyManager`: `ensureProxy(timeoutMs)`, `stopProxy()`, `CONTROL_PORT=10100`, `sandboxRealtimeHost`, `sandboxRestHost`; pinned `PROXY_VERSION=v0.3.0` + per-arch checksums; `uts.proxy.localPath` override | Downloads/verifies/launches the `uts-proxy` binary; one shared process per run. *(package `…integration.proxy`)* | +| `proxy/ProxySession.kt` | `class ProxySession` (`create(rules,port,timeoutMs,realtimeHost,restHost)`, `addRules`, `triggerAction`, `getLog(): List`, `close`, `sessionId`, `proxyPort`, `proxyHost`); `data class Event`; `typealias ProxyRule`; rule builders `wsConnectRule`/`wsFrameToClientRule`/`wsFrameToServerRule`/`httpRequestRule`; `ClientOptionsBuilder.connectThroughProxy(session)` | Typed client for the proxy control REST API + client wiring. *(package `…integration.proxy`)* | +| `SandboxApp.kt` | `class SandboxApp` (`create()`, `delete()`, `appId`, `defaultKey`, `keys`) | Provisions/tears down a throwaway sandbox app from `ably-common`'s `test-app-setup.json`. *(package `…integration`)* | + +### B.3 Shared helpers & tests + +| File | Key public surface | Role | +|------|--------------------|------| +| `infra/Utils.kt` | `awaitState(client,target,timeout=5s)`, `awaitChannelState(channel,target,timeout=5s)`, `pollUntil(timeout=15s,interval=100ms){ }` | Shared wall-clock coroutine waits (package `io.ably.lib.uts.infra`); listener registered before state check. | +| `unit/realtime/ConnectionRecoveryTest.kt` | 6 `@Test`s: RTN16g/g1, RTN16g2, RTN16k, RTN16f, RTN16f1, RTN16j | Unit tier (`io.ably.lib.uts.unit.realtime`) — connection recovery (mocked WS, FakeClock, env-gated deviations). | +| `integration/proxy/realtime/AuthReauthTest.kt` | 1 `@Test` (two `@UTS`: RTN22, RTC8a) | Integration tier (`io.ably.lib.uts.integration.proxy.realtime`) — server-initiated re-authentication. | +| `deviations.md` | RTN16f, RTN16g2, RTL13b, RTL13c | Catalogue of SDK-vs-spec divergences. | + +> **Coverage note:** at the time of writing, the `uts/` module contains exactly **two test classes** +> (**7** `@Test` methods total: 6 in `ConnectionRecoveryTest` + 1 in `AuthReauthTest`). The infrastructure under +> `infra/unit/` and `infra/integration/` is built out far beyond what these two tests exercise (full HTTP +> mock, all four rule builders, REST proxy wiring, etc.), anticipating the broader UTS coverage +> catalogued in [`completion-status.md`](https://github.com/ably/specification/blob/main/uts/docs/completion-status.md). + +--- + +### Source map (where each fact in this doc comes from) + +| Topic | File | +|-------|------| +| Authoring portable specs, test IDs, mock pseudocode | [`uts/docs/writing-test-specs.md`](https://github.com/ably/specification/blob/main/uts/docs/writing-test-specs.md) | +| Translating specs, deviation patterns, decision tree | [`uts/docs/writing-derived-tests.md`](https://github.com/ably/specification/blob/main/uts/docs/writing-derived-tests.md) | +| Integration/proxy policy, late fault injection, tiers | [`uts/docs/integration-testing.md`](https://github.com/ably/specification/blob/main/uts/docs/integration-testing.md) | +| Coverage matrix | [`uts/docs/completion-status.md`](https://github.com/ably/specification/blob/main/uts/docs/completion-status.md) | +| Proxy control API, rule format, action numbers | [`uts/realtime/integration/helpers/proxy.md`](https://github.com/ably/specification/blob/main/uts/realtime/integration/helpers/proxy.md) | +| SDK seams | `lib/.../debug/DebugOptions.java`, `lib/.../util/Clock.java` | +| Module wiring | `uts/build.gradle.kts`, `settings.gradle.kts` | +| Unit mocks | `uts/.../uts/infra/unit/*` | +| Integration helpers | `uts/.../uts/infra/integration/*` (+ `…/integration/proxy/*`) | +| Async helpers | `uts/.../uts/infra/Utils.kt` (awaits), `…/uts/infra/unit/Utils.kt` (ConnectionDetails builder) | +| The two example tests | `…/uts/unit/realtime/ConnectionRecoveryTest.kt`, `…/uts/integration/proxy/realtime/AuthReauthTest.kt` | +| Deviations | `uts/.../io/ably/lib/uts/deviations.md` | diff --git a/uts/build.gradle.kts b/uts/build.gradle.kts index 585d1f239..5286e59eb 100644 --- a/uts/build.gradle.kts +++ b/uts/build.gradle.kts @@ -7,6 +7,9 @@ plugins { dependencies { testImplementation(project(":java")) testImplementation(project(":network-client-core")) + // Runtime-only so compile-time stays decoupled from the plugin internals; the LiveObjects test + // helpers reach the internal wire/message classes (e.g. for build_public_object_message) by reflection. + testRuntimeOnly(project(":liveobjects")) testImplementation(kotlin("test")) testImplementation(libs.mockk) testImplementation(libs.coroutine.core) @@ -35,3 +38,15 @@ tasks.withType().configureEach { .getOrElse(""), ) } + +tasks.register("runUtsUnitTests") { + filter { + includeTestsMatching("io.ably.lib.uts.unit.*") + } +} + +tasks.register("runUtsIntegrationTests") { + filter { + includeTestsMatching("io.ably.lib.uts.integration.*") + } +} diff --git a/uts/index.html b/uts/index.html new file mode 100644 index 000000000..001a2718a --- /dev/null +++ b/uts/index.html @@ -0,0 +1,825 @@ + + + + + +UTS in ably-java — A Human-Readable Guide + + + + + +
UTS in ably-java — Guide
+ +
+ + +
+
+
Ably SDK Testing
+

UTS in ably-java
A Human-Readable Guide

+

A practical, end-to-end explanation of the Universal Test Specification (UTS) and exactly how the Java/Kotlin code under uts/ makes the unit and proxy-integration tests work — written for a developer who has never touched UTS before.

+ + +
+ + + + Ably features spec + features.md + RSC* · RTN* · RTL* + + + distilled + + + UTS test specs + portable .md pseudocode + source of truth: WHAT + + + derived + + + Derived tests + Kotlin in uts/ + ConnectionRecoveryTest.kt + + +
From the one behavioural contract (features spec) → portable specs → per-SDK runnable tests.
+
+
+ + +
+

1 Introduction: What is UTS?

+

UTS (Universal Test Specification) is Ably's language-neutral catalogue of tests for its client SDKs. The problem it solves: Ably ships many SDKs (JavaScript, Dart, Kotlin/Java, Swift, Go…), and every one of them must obey the same behavioural contract — the Ably features spec (specification/specifications/features.md, whose requirements are tagged RSC7, RTN15a, RTL4f…). Without a shared test definition, each SDK would re-invent its own tests, drift apart, and leave gaps.

+

UTS fixes this by separating what to test from how to test it in a given language (see the pipeline above).

+

Three concepts you will see constantly:

+ + + + + + +
TermMeaning
Spec pointA tagged requirement in the features spec, e.g. RTN16g, RTN22, RTL4f. Test names embed these.
UTS specA markdown file of portable pseudocode describing setup, steps, and assertions for one feature. The source of truth for what to test.
Derived testA faithful translation of a UTS spec into a real test in a specific SDK/language. This is what lives in ably-java/uts/.
DeviationA documented case where the SDK's actual behaviour diverges from the spec. Recorded in deviations.md.
+
Golden rule +

From writing-derived-tests.md: translate the UTS spec faithfully — same structure, same assertions, same naming — don't optimise or skip steps. Every derived test carries a // UTS: <id> (here @UTS …) comment linking it back to its spec.

+
+ + +
+

2 The Three Test Tiers

+

UTS divides tests into three tiers by what infrastructure they need and what confidence they give. This split is the key to understanding the whole uts/ module — the two example tests sit in two different tiers.

+
+

Unit mocked

Transport: mocked (MockWebSocket, MockHttpClient). Backend: none.
Client-side logic: state machines, request formation, response parsing, timers. Fast & deterministic.
→ unit/realtime/ConnectionRecoveryTest.kt

+

Direct sandbox real net

Transport: real. Backend: real Ably sandbox.
Happy-path interop: connect, publish, subscribe. No fault injection.
→ integration/standard/<module>/ (tier exists; no tests yet)

+

Proxy integration faults

Transport: real, through a programmable proxy. Backend: real sandbox.
Fault behaviour: dropped connections, injected errors, timeouts, re-auth.
→ integration/proxy/realtime/AuthReauthTest.kt

+
+

Each tier folder is further organised by module (realtime, liveobjects, …): unit/<module>/, integration/standard/<module>/, and integration/proxy/<module>/. So a feature's tests sit together by SDK area — the two example tests live at unit/realtime/ and integration/proxy/realtime/.

+

Key principles (from integration-testing.md):

+
    +
  • Integration tests do not replace unit tests. A spec point covered by a proxy test should also have a unit test. The unit test proves client logic; the proxy test proves client & real server agree.
    • +
  • Proxy tests prefer "late fault injection". Let the real handshake complete against the real server, then inject the fault as the final interaction — maximising how much of the test exercises genuine client-server behaviour.
    • +
  • Proxy tests always use JSON (useBinaryProtocol = false). Two reasons in the spec corpus: the proxy only supports text WebSocket frames so it can't inspect/modify msgpack (integration-testing.md), and the SDK under test doesn't implement msgpack (helpers/proxy.md).
    • +
+
+ + +
+

3 The UTS Documents (the source of truth)

+

These four documents live in the specification repo at uts/docs/. They are the policy/authoring guides; the Kotlin code in this repo is their implementation. Each title below links to the file on GitHub. ↗

+ +

3.1 writing-test-specs.md — how to author a portable UTS spec

+
    +
  • Test types (unit / integration / proxy) and when each applies.
    • +
  • Test IDs — format <category>/<spec-point>/<descriptive-name>-<n>, e.g. realtime/proxy/RTN22/server-initiated-reauth-0. These IDs are the @UTS comments in the Kotlin tests.
    • +
  • Mock infrastructure pseudocode interfaces — MockHttpClient, MockWebSocket, PendingConnection, PendingRequest with respond_with_success(), send_to_client(), simulate_disconnect(). The Kotlin classes in uts/infra/unit/ realise these.
    • +
  • Handler vs await patterns for mocks (see §6).
    • +
  • WebSocket closing semantics — the crucial rule: send_to_client_and_close() for DISCONNECTED / connection-level ERROR (server closes the socket); send_to_client() for a channel-level ERROR (connection stays open).
    • +
  • Anti-flake conventions — no fixed WAITs; use polling, AWAIT_STATE, fake timers, and the record-and-verify pattern (CONTAINS_IN_ORDER) for transient states.
    • +
+ +

3.2 writing-derived-tests.md — how to translate a spec into a real SDK test

+

Two phases: Translation (always) — faithfully render the spec into the target language, map pseudocode to the SDK's API & test framework, flag ambiguities, ensure it compiles. Evaluation (when an implementation exists) — run the test and, if it fails, work the decision tree:

+
+ + Test fails + + Does UTS spec matchthe features spec? + + NO + fix test +record UTS error + + YES + Does test accuratelytranslate the spec? + NO + fix the test + YES + SDK deviation → adapt + record in deviations.md + + + +
The three-branch decision tree from writing-derived-tests.md.
+
+

It also defines the env-gated skip pattern (RUN_DEVIATIONS) — the test holds the spec-correct assertion but only runs it when the env var is set, so normal runs stay green while each deviation stays individually reproducible. This is exactly what ConnectionRecoveryTest uses for RTN16f.

+ +

3.3 integration-testing.md — the policy for integration & proxy tests

+

Defines what deserves an integration test (request/response interop, error interop, data round-trips, stateful protocol sequences), the directory layout, sandbox provisioning, proxy session lifecycle, timeout strategy, and the late-fault-injection philosophy. The integration/proxy/ segregation exists because proxy tests have different infra needs, CI cadence, and failure modes.

+ +

3.4 completion-status.md — the coverage matrix

+

A big table mapping every features-spec group (RSC, RTN, RTL, RTP…) to the UTS specs that cover it, with a per-tier summary (unit:✓ proxy:✓). The two example tests correspond to these rows:

+
    +
  • RTN16 (connection recovery) → unit spec connection_recovery_test.md → ConnectionRecoveryTest.kt.
    • +
  • RTN22 / RTC8a (server-initiated re-auth) → proxy spec realtime/integration/proxy/auth_reauth.md → AuthReauthTest.kt.
    • +
+
Fifth, referenced doc +

There is also realtime/integration/helpers/proxy.md. It defines the proxy's control API, rule format, action types, and the protocol message action-number table (CONNECTED=4, ATTACH=10, AUTH=17…). The Kotlin ProxySession is the client for exactly that API.

+
+ + +
+

4 The Java Setup: the uts/ module

+

The uts/ directory is a standalone Gradle module (include("uts") in settings.gradle.kts) whose only job is to host UTS-derived tests. It contains no production code — everything lives under uts/src/test/.

+ +

4.1 uts/build.gradle.kts

+
plugins { alias(libs.plugins.kotlin.jvm) }
+
+dependencies {
+    testImplementation(project(":java"))                 // the SDK under test
+    testImplementation(project(":network-client-core"))  // HttpEngine / WebSocketEngine interfaces
+    testImplementation(kotlin("test"))
+    testImplementation(libs.mockk)
+    testImplementation(libs.coroutine.core)              // kotlinx.coroutines
+    testImplementation(libs.coroutine.test)              // runTest, virtual time
+    testImplementation(libs.ktor.client.core)            // HTTP client for proxy/sandbox control
+    testImplementation(libs.ktor.client.cio)
+}
+
+tasks.withType<Test>().configureEach {
+    useJUnitPlatform()                                   // JUnit 5
+    jvmArgs("--add-opens", "java.base/java.time=ALL-UNNAMED")
+    jvmArgs("--add-opens", "java.base/java.lang=ALL-UNNAMED")
+    // Propagate a local proxy build override (see ProxyManager):
+    systemProperty("uts.proxy.localPath", /* -Duts.proxy.localPath=… or $UTS_PROXY_LOCAL_PATH */ …)
+}
+

Takeaways:

+
    +
  • Tests are Kotlin + JUnit 5, using kotlinx.coroutines for async control and Ktor as the HTTP client that talks to the sandbox REST API and the proxy control API.
    • +
  • Depends on :java (the SDK) and :network-client-core (the pluggable transport interfaces the mocks implement).
    • +
  • The --add-opens java.base/java.time and java.base/java.lang flags grant reflective access for the test runtime. They mirror the same flags in java/build.gradle.kts (which additionally opens java.net and java.lang.reflect).
    • +
  • A system property carries an optional path to a locally built proxy binary.
    • +
+ +

4.2 Directory layout

+

Everything lives under the io.ably.lib.uts package, split cleanly into infrastructure (infra/, no @Tests) and the tests themselves. Tests are organised by tier, then by module: unit/ for mocked-transport tests, and integration/ for real-backend tests — the latter splitting again into standard/ (direct sandbox, happy-path) and proxy/ (sandbox through the fault-injecting proxy). Under each, a per-module folder (realtime, liveobjects, …) holds the actual test classes:

+
uts/src/test/kotlin/io/ably/lib/uts/
+├── deviations.md                        # the catalogue of SDK-vs-spec divergences
+│
+├── infra/                               # ── TEST INFRASTRUCTURE (no @Test methods) ──
+│   ├── Utils.kt                         #   awaitState / awaitChannelState / pollUntil (shared)
+│   │
+│   ├── unit/                            #   UNIT infra (mocked transports)
+│   │   ├── ClientFactories.kt           #     TestRealtimeClient / TestRestClient / ClientOptionsBuilder
+│   │   ├── MockWebSocket.kt             #     fake WS transport + WebSocketMockConfig + CONNECTED_MESSAGE
+│   │   ├── MockWebSocketEngineFactory.kt#     plugs the mock into the SDK's WebSocketEngine SPI
+│   │   ├── MockHttpClient.kt            #     fake HTTP engine + HttpMockConfig
+│   │   ├── MockHttpEngine.kt            #     plugs the mock into the SDK's HttpEngine SPI
+│   │   ├── MockEvent.kt                 #     sealed log of everything on a mock transport
+│   │   ├── PendingConnection.kt         #     interface: a connection attempt awaiting a response
+│   │   ├── DefaultPendingConnection.kt  #     WS implementation of PendingConnection
+│   │   ├── PendingRequest.kt            #     interface: an in-flight HTTP request awaiting a response
+│   │   ├── DefaultPendingRequest.kt     #     HTTP implementation of PendingRequest
+│   │   ├── FakeClock.kt                 #     virtual clock + virtual timers (deterministic time)
+│   │   └── Utils.kt                     #     ConnectionDetails { } builder (reflective constructor)
+│   │
+│   └── integration/                     #   INTEGRATION infra (real backend)
+│       ├── SandboxApp.kt                #     provisions/deletes a sandbox app
+│       └── proxy/
+│           ├── ProxyManager.kt          #       downloads/launches the uts-proxy binary
+│           └── ProxySession.kt          #       proxy session: rules, actions, log + connectThroughProxy
+│
+├── unit/                                # ── UNIT TESTS (mock transport) ── · per module
+│   ├── realtime/
+│   │   └── ConnectionRecoveryTest.kt    #   ← the UNIT test (RTN16*)
+│   └── liveobjects/                     #   (further modules as coverage grows)
+│
+└── integration/                         # ── INTEGRATION TESTS (real backend) ── · per module
+    ├── standard/                        #   direct sandbox: happy-path, no fault injection
+    │   ├── realtime/
+    │   └── liveobjects/
+    └── proxy/                           #   sandbox through the fault-injecting uts-proxy
+        ├── realtime/
+        │   └── AuthReauthTest.kt        #   ← the PROXY test (RTN22, RTC8a)
+        └── liveobjects/
+
Mental model

infra/unit/ powers the unit tests · infra/integration/ powers both integration kinds (standard + proxy) · infra/Utils.kt serves all of them. Every tier is sub-divided by module (realtime, liveobjects, …). The top-level unit/ ↔ infra/unit/ and integration/ ↔ infra/integration/ pairing is what the runUtsUnitTests / runUtsIntegrationTests Gradle tasks key off (§12) — runUtsIntegrationTests covers both integration/standard/ and integration/proxy/.

+
+ + +
+

5 How a Test Reaches the SDK: the hook points

+

A test can only mock transports because the SDK was designed with pluggable seams. They live on io.ably.lib.debug.DebugOptions (a subclass of ClientOptions):

+
public class DebugOptions extends ClientOptions {
+    public HttpEngine httpEngine;                         // ← MockHttpClient installs here
+    public WebSocketEngineFactory webSocketEngineFactory; // ← MockWebSocket installs here
+    public Clock clock;                                   // ← FakeClock installs here
+    …
+}
+

and the Clock interface:

+
public interface Clock {
+    long currentTimeMillis();
+    long nanoTime();
+    AblyTimer newTimer(String name);                      // every SDK timer is created through this
+    void waitOn(Object target, long timeout) throws InterruptedException; // every blocking wait
+}
+

So the recipe is:

+
    +
  • Fake the WebSocket? Set webSocketEngineFactory to a factory that produces a mock engine.
    • +
  • Fake HTTP? Set httpEngine to a mock engine.
    • +
  • Control time (timeouts, retries, TTL expiry) deterministically? Set clock to a FakeClock.
    • +
+

The ClientOptionsBuilder wraps all three so tests never touch DebugOptions directly.

+
+ + +
+

6 Unit-Test Infrastructure (mocked transports)

+ +

6.1 The client builder — ClientFactories.kt

+
class ClientOptionsBuilder : DebugOptions("appId.keyId:keySecret") {
+    init { useBinaryProtocol = false }                  // JSON so mocks can decode frames
+    fun install(mock: MockWebSocket) = mock.installOn(this)
+    fun install(mock: MockHttpClient) = mock.installOn(this)
+    fun enableFakeTimers(fakeClock: FakeClock) { clock = fakeClock }
+}
+
+fun TestRealtimeClient(block: ClientOptionsBuilder.() -> Unit): AblyRealtime =
+    AblyRealtime(ClientOptionsBuilder().apply(block))
+fun TestRestClient(block: ClientOptionsBuilder.() -> Unit): AblyRest =
+    AblyRest(ClientOptionsBuilder().apply(block))
+
    +
  • Seeds a dummy API key (appId.keyId:keySecret) — fine, because unit tests never hit a real server and tokens are opaque.
    • +
  • Forces JSON so the mock can parse protocol frames.
    • +
  • install(mock) / enableFakeTimers(clock) wire the seams from §5.
    • +
+
val mock = MockWebSocket { onConnectionAttempt = { it.respondWithSuccess(CONNECTED_MESSAGE) } }
+val client = TestRealtimeClient {
+    autoConnect = false
+    install(mock)
+}
+ +

6.2 MockWebSocket — the fake realtime transport

+

The heart of realtime unit testing. It plugs into the SDK via MockWebSocketEngineFactory (which implements the SDK's WebSocketEngineFactory SPI) and exposes two complementary control styles:

+

(a) Callback style — handle inline, synchronously on the SDK thread

+
val mock = MockWebSocket {
+    onConnectionAttempt = { conn -> conn.respondWithSuccess(CONNECTED_MESSAGE) }
+    onMessageFromClient  = { msg -> /* inspect frames the SDK sent */ }
+}
+

Best when every connection attempt should behave the same way.

+

(b) Await style — suspend until the SDK triggers something, then respond

+
val pending = mock.awaitConnectionAttempt()      // suspend until SDK opens a socket
+pending.respondWithRefused()                     // …then decide how to answer
+val frame = mock.awaitNextMessageFromClient()    // suspend until SDK sends a frame
+

Required when the first connection and a reconnection need different answers (e.g. "connect succeeds, then all retries are refused" — the SUSPENDED scenario in the unit test).

+
⚠ Cannot mix styles

You cannot mix callback and await for the same event type — a callback consumes the event before the queue ever sees it.

+

Server → client direction (driving the SDK), matching the spec's closing semantics:

+ + + + + +
MethodWhat it doesUse for
sendToClient(msg)deliver a frame, connection stays openCONNECTED, ATTACHED, channel-level ERROR, normal messages
sendToClientAndClose(msg)deliver a frame then close (code 1000)DISCONNECTED, connection-level ERROR (fatal)
simulateDisconnect()close with code 1006, no messageunexpected network drop → triggers DISCONNECTED/resume
+

Everything is logged. mock.events is an ordered List<MockEvent> (a sealed class: ConnectionAttempt, ConnectionEstablished, ConnectionRefused, SentToClient, MessageFromClient, ClientClose, Disconnected…). Tests assert against it.

+

CONNECTED_MESSAGE is a ready-to-use CONNECTED ProtocolMessage (connectionId test-connection-id, a connection key, TTL 120 s, max-idle 15 s). It is a val with a custom getter, so each access returns a fresh instance — not a shared singleton; safe to mutate per test via CONNECTED_MESSAGE.apply { … }.

+
Subtlety

DefaultPendingConnection.respondWithSuccess(message) delivers the CONNECTED frame asynchronously on a separate mock-ws-delivery thread. That mirrors reality — the SDK must store the WebSocket reference before it processes CONNECTED.

+ +

6.3 MockHttpClient — the fake REST transport

+

The HTTP analogue, plugged in via MockHttpEngine (implements the SDK's HttpEngine SPI). Same two styles. A request flows in two phases inside MockHttpCall.execute():

+
    +
  1. Connect phase → produces a PendingConnection (respondWithSuccess/Refused/Timeout/DnsError).
    2. +
  2. Request phase → produces a PendingRequest exposing url, method, headers, body, and respondWith(status, body, headers) / respondWithDelay(...) / respondWithTimeout().
    3. +
+

This lets REST unit tests assert on outgoing request shape and feed canned responses back — without a socket.

+ +

6.4 FakeClock — deterministic time

+

Implements the SDK's Clock. Time is frozen until you call advance(ms); on each advance it fires any due virtual timers synchronously and wakes any waitOn sleepers. This drives reconnection backoff and connectionStateTtl expiry without real sleeping:

+
val fakeClock = FakeClock()
+val client = TestRealtimeClient { enableFakeTimers(fakeClock); … }
+…
+fakeClock.advance(2.seconds)      // jump forward; due timers fire now
+

pendingTaskCount(timerName) lets you assert how many tasks are scheduled — useful for verifying retry state.

+
+ + +
+

7 Proxy-Integration Infrastructure (real backend + fault injection)

+

Proxy tests connect the real SDK to the real Ably sandbox, but route traffic through a small Go program — ably/uts-proxy — that can be told to inject faults. Three Kotlin helpers make this work.

+ +

7.1 ProxyManager — gets the proxy binary running

+
    +
  • Pins a proxy version (v0.3.0) and knows the SHA-256 checksums for each OS/arch archive.
    • +
  • ensureProxy() (in @BeforeAll) is idempotent: if a proxy is healthy on the control port (10100) it's a no-op; otherwise it downloads the right archive from GitHub releases, verifies the checksum, extracts the binary with a hand-rolled tar/gzip reader (JDK-only), caches it under ~/.cache/uts-proxy/<version>/, and launches it with --port 10100.
    • +
  • The download is serialised across JVMs by a FileLock and within a JVM by a Mutex.
    • +
  • A JVM shutdown hook force-kills the spawned process on exit.
    • +
  • Override knob: set -Duts.proxy.localPath=… or $UTS_PROXY_LOCAL_PATH to use a locally built proxy binary or .tar.gz (skips download + checksum).
    • +
+
⚠ Single-fork advisory

Because process startup shares the control port, ProxyManager's KDoc advises running proxy suites single-fork (maxParallelForks = 1). Note: this is currently only a documented recommendation — it is not set in uts/build.gradle.kts. With a single proxy test class today the race is not yet triggered, but it should be configured before a second proxy suite is added.

+ +

7.2 ProxySession — one test's window into the proxy

+

The proxy exposes a control REST API on the control port; ProxySession is the typed Kotlin client for it (via Ktor). One session per test.

+ + + + + + + +
MethodControl endpointPurpose
ProxySession.create(rules, …)POST /sessionswith a target (sandbox hosts) + initial rule list; proxy assigns a sessionId & fresh listening port.
addRules(rules, position)POST /sessions/{id}/rulesadd rules mid-test.
triggerAction(action)POST /sessions/{id}/actionsfire an imperative action right now (inject a frame / drop connection).
getLog()GET /sessions/{id}/logreturns a typed List<Event>.
close()DELETE /sessions/{id}always in a finally.
+

Each Event carries type (ws_connect, ws_frame, http_request…), direction, queryParams, and the parsed protocol message (a JsonObject, introspected via message?.get("action")?.asInt).

+

Rules = match + action (+ optional times). Builder helpers keep tests readable: wsConnectRule, wsFrameToClientRule, wsFrameToServerRule, httpRequestRule. Rules evaluate in order, first match wins, unmatched traffic passes through, and times: N auto-removes a rule after N firings. Common actions: refuse_connection, suppress, replace, inject_to_client[_and_close], disconnect, http_respond.

+

Wiring the client to the proxy — connectThroughProxy(session) does exactly what the proxy spec prescribes:

+
fun ClientOptionsBuilder.connectThroughProxy(session: ProxySession) {
+    realtimeHost = session.proxyHost   // "localhost"
+    restHost     = session.proxyHost
+    port         = session.proxyPort   // the session's assigned port
+    tls          = false               // proxy serves plain HTTP/WS; TLS is only upstream
+}
+

Explicit hosts auto-disable fallback hosts (REC2c2), so no fallbackHosts juggling is needed.

+ +

7.3 SandboxApp — a throwaway app on the real sandbox

+
    +
  • SandboxApp.create() fetches the canonical test-app-setup.json from ably-common (specifically its post_apps sub-object), POSTs it to https://sandbox.realtime.ably-nonprod.net/apps, and exposes appId, defaultKey (full-capability appId.keyId:keySecret, from the keyStr field), and the full keys list.
    • +
  • delete() removes the app in teardown (best-effort — errors swallowed, sandbox apps auto-expire).
    • +
  • The Ktor client retries only idempotent GETs (never re-POSTs /apps, to avoid duplicate apps).
    • +
  • SandboxApp is the shared backbone of both integration kinds: proxy tests pair it with a ProxySession, while direct sandbox tests (integration/standard/<module>/) use it alone — connecting straight to ProxyManager.sandboxRealtimeHost / sandboxRestHost with no proxy and no fault rules, for happy-path interop.
    • +
+

The app is provisioned directly (not through the proxy), so it's independent of the fault rules under test.

+
+ + +
+

8 Shared Async Helpers

+

Utils.kt provides the coroutine glue every tier relies on (unit, direct sandbox, and proxy). All three run on a single-thread real dispatcher so their timeouts measure wall-clock time (not the virtual time of kotlinx.coroutines.test). The two state-waiters register their listener before checking current state, to avoid a check-then-register race; pollUntil has no listener — it re-evaluates the predicate every interval.

+ + + + + +
HelperSignaturePurpose
awaitState(client, target, timeout=5s)suspend until connection.state == target (or already there)
awaitChannelState(channel, target, timeout=5s)same, for a channel's state
pollUntil(timeout=15s, interval=100ms) { cond }suspend until a boolean predicate holds — used in proxy tests to wait on real network/proxy state
+

A second Utils.kt under infra/unit/ adds the ConnectionDetails { … } builder DSL so tests can write ConnectionDetails { connectionKey = "key-1"; connectionStateTtl = 120000L }. Since this file no longer sits in the io.ably.lib.types package, it can't call ConnectionDetails's package-private constructor directly — it obtains an instance reflectively (the same package-private-access technique used by liveobjects/.../TestUtils.kt). See Appendix B.1.

+
+ + +
+

9 Walkthrough: the Unit Test (ConnectionRecoveryTest)

+

Tier: Unit (mocked WebSocket, no network). Spec area: RTN16 — connection recovery via the recover option and createRecoveryKey(). Six tests, each tagged @UTS realtime/unit/RTN16…/….

+ +

9.1 RTN16g, RTN16g1 — recovery-key structure (incl. Unicode)

+

Connects, attaches two channels — one ASCII, one Unicode (channel-éàü-世界) — feeding each an ATTACHED with a channelSerial. Calls createRecoveryKey(), decodes it, asserts the connection key, msgSerial == 0, and both channel serials survive — including a full encode→decode round-trip proving the Unicode name isn't corrupted (RTN16g1). Technique: callback-style onConnectionAttempt, sendToClient for ATTACHED, awaitChannelState.

+ +

9.2 RTN16g2 — createRecoveryKey() returns null in inactive states

+

The most elaborate test — walks the connection through five lifecycle states, asserting the key is null in each inactive one:

+
    +
  • INITIALIZED (before connect) → null.
    • +
  • CONNECTED → non-null (sanity).
    • +
  • CLOSING / CLOSED → null (close nulls the key immediately).
    • +
  • FAILED → null. (Contains a documented deviation — see §11: the spec's fatal error code 50000/500 isn't treated as fatal by the SDK, and send_to_client_and_close races the FAILED transition; the test uses code 40000/400 and plain sendToClient.)
    • +
  • SUSPENDED → null. Built with a FakeClock: connect succeeds, then simulateDisconnect(), then a coroutine refuses every reconnection attempt while fakeClock.advance(2.seconds) loops until the short connectionStateTtl (800 ms) expires and the client gives up to SUSPENDED.
    • +
+

Technique: the textbook await-style example — the first connection succeeds via awaitConnectionAttempt(), but reconnections need the refused response, so a separate refuseJob coroutine drives them; combined with fake timers this gives a deterministic SUSPENDED.

+ +

9.3 RTN16k — recover adds the recover query param

+

Constructs the client with recover = <recoveryKey>, captures conn.queryParams on each attempt, then simulateDisconnect() + reconnect. Asserts the first attempt carries recover=<key> (no resume), the second carries resume=<new key> (no recover) — recover is a one-shot bootstrap; subsequent reconnections use resume.

+ +

9.4 RTN16f — recover initialises msgSerial env-gated deviation

+

Asserts the recovered msgSerial (42) is preserved. The SDK resets it to 0, so the spec-correct assertion assertEquals(42L, …) runs only under RUN_DEVIATIONS; otherwise a regression-guard assertEquals(0L, …) runs. (See §11.)

+ +

9.5 RTN16f1 — malformed recover key degrades gracefully

+

recover = "this-is-not-valid-json!!!". Asserts the client still connects normally with a fresh identity, no recover/resume params, and exactly one connection attempt — a bad key is logged and ignored, not fatal.

+ +

9.6 RTN16j — recover instantiates channels with serials (RTN16i too)

+

Recovery key carries three channels (incl. Unicode). Asserts each channels.get(name).properties.channelSerial matches the key, that channels are NOT auto-attached (state INITIALIZED — RTN16i), and that a manual attach() sends an ATTACH frame carrying the recovered serial (verified via awaitNextMessageFromClient()).

+
What this teaches

Callback vs await styles side by side, FakeClock-driven SUSPENDED, sendToClient for server frames, events/awaitNextMessageFromClient for inspecting client output, and the env-gated deviation pattern.

+
+ + +
+

10 Walkthrough: the Proxy Test (AuthReauthTest)

+

Tier: Proxy integration (real sandbox + uts-proxy). Spec points: RTN22 (server-initiated re-authentication) and RTC8a (client sends an AUTH frame with renewed auth details). Unit counterparts: server_initiated_reauth_test.md, realtime_authorize.md.

+ +

10.1 Suite setup/teardown

+
@TestInstance(TestInstance.Lifecycle.PER_CLASS)   // one instance, so @BeforeAll can be non-static
+class AuthReauthTest {
+    @BeforeAll fun setUpAll() = runBlocking {
+        ProxyManager.ensureProxy()                // download+launch proxy if needed
+        app = SandboxApp.create()                 // provision a real sandbox app
+    }
+    @AfterAll fun tearDownAll() = runBlocking { if (::app.isInitialized) app.delete() }
+}
+ +

10.2 The test, step by step

+
    +
  1. Create a session with no rules — the fault is injected imperatively later (late injection — the handshake runs against the real server unmodified): val session = ProxySession.create(rules = emptyList())
    2. +
  2. Auth via authCallback — a locally-signed TokenRequest from the same sandbox key (no external JWT library). A counter records how many times the callback runs: +
    val tokenSigner = AblyRest(app.defaultKey)
+val authCallback = Auth.TokenCallback { params ->
+    authCallbackCount.incrementAndGet()
+    tokenSigner.auth.createTokenRequest(params, null)
+}
    3. +
  3. Build the client through the proxy and connect (JSON stays on so the proxy can inspect frames): +
    val client = TestRealtimeClient {
+    this.authCallback = authCallback
+    connectThroughProxy(session)
+    autoConnect = false
+}
+client.connect()
+awaitState(client, ConnectionState.connected, 15.seconds)
    4. +
  4. Snapshot identity — connection.id and the callback count; assert the callback already ran ≥ 1 (initial auth).
    5. +
  5. Start recording state changes, then inject a server-initiated AUTH (protocol action 17) imperatively: +
    session.triggerAction(mapOf("type" to "inject_to_client",
+                            "message" to mapOf("action" to 17)))
    6. +
  6. Wait for the re-auth round-trip with pollUntil { stateChanges.size > 1 } (real network, so poll — don't sleep).
    7. +
  7. Assertions prove RTN22 + RTC8a: +
      +
    • authCallback was invoked again (count incremented) → re-auth triggered.
      • +
    • Connection is still CONNECTED and connection.id is unchanged → re-auth does not reconnect.
      • +
    • No transitions away from CONNECTED were recorded.
      • +
    • The proxy event log contains a client→server AUTH frame (action 17) carrying non-null auth details (RTC8a) — verified by filtering session.getLog().
      • +
    +
    8. +
  8. Nested teardown in finally: close the client and wait for CLOSED, then always close the session and the token signer.
    9. +
+
What this teaches

ProxyManager.ensureProxy + SandboxApp setup, connectThroughProxy, late imperative fault injection via triggerAction, real-network waiting with pollUntil, and proxy-log assertions as the primary verification.

+
+ + +
+

11 Deviations: when the SDK disagrees with the spec

+

uts/.../deviations.md is the single catalogue of every place the ably-java SDK behaves differently from the features spec, discovered during translation. Each entry records: the spec point, what the spec requires, what the SDK does, the root cause (file/function, where known), the workaround in tests, and the affected tests.

+

The mechanism: the test keeps the spec-correct assertion but gates it behind the RUN_DEVIATIONS env var, with a regression-guard assertion for the SDK's actual behaviour running by default. Normal runs stay green; RUN_DEVIATIONS=1 turns the failing assertions on, and the test flips automatically once the SDK is fixed.

+ + + + + + +
Spec pointGistTouches
RTN16fSDK resets msgSerial to 0 on connect even with recover; spec says preserve it (42).ConnectionRecoveryTest (§9.4) — assertEquals(42L,…) gated, assertEquals(0L,…) default guard.
RTN16g2Spec's fatal error 50000/500 isn't fatal to the SDK (isFatalError() needs code 40000–49999 or status < 500); also send_to_client_and_close races the FAILED transition.ConnectionRecoveryTest (§9.2) — uses 40000/400 + plain sendToClient.
RTL13bATTACHING → SUSPENDED via realtimeRequestTimeout not implemented for channel attach.various channel tests (not the two here).
RTL13cchannelRetryTimeout not cancelled when the connection leaves CONNECTED.various channel tests; assertions gated behind RUN_DEVIATIONS.
+
Why this matters

These deviations are valuable output, not failures — each one is a precise, reproducible bug report the SDK team can act on, and the gated test becomes the acceptance test for the fix.

+
+ + +
+

12 How to Run the Tests

+

Two custom Gradle tasks (registered in uts/build.gradle.kts), filtered by package — they mirror runLiveObjectsUnitTests / runLiveObjectsIntegrationTests in the liveobjects module:

+
# Unit tests only — io.ably.lib.uts.unit.*  (fast, no network). This is the PR gate.
+./gradlew :uts:runUtsUnitTests
+
+# Integration tests only — io.ably.lib.uts.integration.*  (real sandbox; covers both
+# integration/standard/ and integration/proxy/ — proxy tests also download/launch the uts-proxy).
+./gradlew :uts:runUtsIntegrationTests
+
+# Everything (the default Test task still runs both):
+./gradlew :uts:test
+
+# Just one test class (works with any of the tasks above):
+./gradlew :uts:runUtsUnitTests --tests "io.ably.lib.uts.unit.realtime.ConnectionRecoveryTest"
+./gradlew :uts:runUtsIntegrationTests --tests "io.ably.lib.uts.integration.proxy.realtime.AuthReauthTest"
+
+# Turn on the spec-correct (currently failing) deviation assertions:
+RUN_DEVIATIONS=1 ./gradlew :uts:runUtsUnitTests --tests "*ConnectionRecoveryTest*"
+
+# Run proxy tests against a locally built proxy instead of a GitHub release:
+./gradlew :uts:runUtsIntegrationTests -Duts.proxy.localPath=/path/to/uts-proxy   # or .tar.gz
+#   (equivalently: export UTS_PROXY_LOCAL_PATH=/path/to/uts-proxy)
+

Where CI runs them: runUtsUnitTests is part of the check.yml gate (alongside runLiveObjectsUnitTests); runUtsIntegrationTests runs in the check-uts job of integration-test.yml (alongside check-liveobjects).

+

Notes:

+
    +
  • ProxyManager advises running proxy suites single-fork (maxParallelForks = 1) because they share the control port (10100). Not currently set in uts/build.gradle.kts; not yet exercised because there is only one proxy test class.
    • +
  • Proxy/sandbox tests need outbound network (sandbox + GitHub releases on first run; the binary is then cached under ~/.cache/uts-proxy/).
    • +
  • Before pushing, run the static-analysis gate (from CLAUDE.md): ./gradlew checkWithCodenarc checkstyleMain checkstyleTest — Checkstyle is Java-only and easy to miss; remember no star imports. (CI's full line additionally runs runUnitTests runLiveObjectsUnitTests :uts:runUtsUnitTests.)
    • +
+
+ + +
+

13 Quick Reference / Cheat-Sheet

+

The three seams that make unit tests possible (DebugOptions): webSocketEngineFactory httpEngine clock

+

Build a unit-test client

+
val mock = MockWebSocket { onConnectionAttempt = { it.respondWithSuccess(CONNECTED_MESSAGE) } }
+val client = TestRealtimeClient { autoConnect = false; install(mock) }
+client.connect(); awaitState(client, ConnectionState.connected)
+

Build a proxy-test client

+
ProxyManager.ensureProxy(); val app = SandboxApp.create()
+val session = ProxySession.create(rules = emptyList())
+val client = TestRealtimeClient { authCallback = …; connectThroughProxy(session); autoConnect = false }
+

Server→client (mock): sendToClient (stays open) · sendToClientAndClose (DISCONNECTED / fatal ERROR) · simulateDisconnect (1006 drop).

+

Inspect what the SDK did: mock.events (unit) · session.getLog() (proxy).

+

Wait (never sleep): awaitState · awaitChannelState · pollUntil { … } · FakeClock.advance(…).

+

Protocol action numbers: CONNECTED=4 DISCONNECTED=6 ERROR=9 ATTACH=10 ATTACHED=11 DETACH=12 DETACHED=13 AUTH=17

+

Test ID format: <category>/<spec-point>/<descriptive-name>-<n> → @UTS realtime/proxy/RTN22/server-initiated-reauth-0

+

Decision tree on failure: spec wrong → fix test + record UTS spec error · translation wrong → fix test · SDK non-compliant → gate spec-correct assertion behind RUN_DEVIATIONS + record in deviations.md.

+
+ + +
+

A Appendix A: Request-Flow Diagrams

+ +

A.1 Unit test — mocked WebSocket (no network)

+

A unit test installs MockWebSocket into DebugOptions.webSocketEngineFactory. The SDK believes it's talking to a real socket; in fact every byte is intercepted by the mock and surfaced to the test.

+
+ + + TEST (Kotlin coroutine) — in-process, no TCP / DNS / real time + + + AblyRealtime + SDK (:java) + ConnectionManager + + + MockWebSocketEngineFactory + + + MockWebSocket + records MockEvent[] + onConnectionAttempt + onMessageFromClient + sendToClient / … + + + TEST drives both sides + respondWithSuccess(msg) + sendToClient(CONNECTED) + simulateDisconnect() + mock.events (assert) + + + factory + + send(frame) + + onMessage + + + + FakeClock.advance(2.s) + + + +
Everything is in-process and deterministic. The HTTP path is identical in shape: MockHttpClient → DebugOptions.httpEngine → MockHttpEngine → PendingConnection then PendingRequest.
+
+ +

A.2 Proxy integration test — real backend through the fault-injecting proxy

+

A proxy test uses the real SDK transport but points its host/port at the local uts-proxy process, which forwards to the Ably sandbox and can inject faults on command.

+
+ + + + AblyRealtime + REAL transport + + + uts-proxy + forwards traffic + applies rules + records event log + + + Ably sandbox + sandbox.realtime. + ably-nonprod.net + TLS + + + + ws/http (plain, tls=false) + + + upstream TLS + + + TEST controls the proxy (CONTROL plane → :10100) + triggerAction({inject_to_client, action:17}) · addRules(...) · getLog() + + + SandboxApp.create() + POST /apps (direct, TLS) + + + + +
Two channels to the proxy: the data plane (SDK ws/http on session.proxyPort) is separate from the control plane (test REST on CONTROL_PORT=10100). The SDK never sees the control plane; the test never speaks the data plane directly. Everything before the injected fault is real client↔server traffic.
+
+
+ + +
+

B Appendix B: Per-File API Reference

+ +

B.1 Unit-test infrastructure — io.ably.lib.uts.infra.unit

+ + + + + + + + + + + + + + +
FileKey public surfaceRole
ClientFactories.ktClientOptionsBuilder (extends DebugOptions), TestRealtimeClient { }, TestRestClient { }, install(mock), enableFakeTimers(clock)Entry point for building a mocked SDK client; seeds dummy key, forces JSON.
MockWebSocket.ktMockWebSocket, WebSocketMockConfig (onConnectionAttempt, onMessageFromClient, onTextDataFrame, onBinaryDataFrame), events, installOn, awaitConnectionAttempt, awaitNextMessageFromClient, awaitClientClose, sendToClient, sendToClientAndClose, simulateDisconnect, reset; top-level MockWebSocket { }, CONNECTED_MESSAGEFake realtime transport (callback + await styles).
MockWebSocketEngineFactory.ktMockWebSocketEngineFactory, MockWebSocketEngine, MockWebSocketClient (implement WebSocketEngineFactory/Engine/Client)Adapts the mock to the SDK's WebSocket SPI; parses URL → host/port/tls/query.
MockHttpClient.ktMockHttpClient, HttpMockConfig (onConnectionAttempt, onRequest), engine, installOn, awaitConnectionAttempt, awaitRequest, reset; top-level MockHttpClient { }Fake REST transport.
MockHttpEngine.ktMockHttpEngine, MockHttpCall, DefaultHttpPendingConnection (implement HttpEngine/HttpCall)Adapts the mock to the SDK's HTTP SPI; two-phase connect→request in execute().
PendingConnection.ktinterface PendingConnection (host,port,tls,queryParams, respondWithSuccess[ (message) ], respondWithRefused/Timeout/DnsError); plus top-level helper parseQueryString() (not an interface member)Abstract connection attempt awaiting a verdict (shared WS + HTTP).
DefaultPendingConnection.ktDefaultPendingConnection : PendingConnectionWS impl; async CONNECTED delivery on mock-ws-delivery thread.
PendingRequest.ktinterface PendingRequest (url,method,headers,body, respondWith, respondWithDelay, respondWithTimeout)Abstract in-flight HTTP request awaiting a response.
DefaultPendingRequest.ktDefaultPendingRequest : PendingRequestHTTP impl backed by a CompletableDeferred<HttpResponse>.
MockEvent.ktsealed class MockEvent: ConnectionAttempt, ConnectionEstablished, ConnectionRefused, ConnectionTimeout, DnsError, HttpRequest, SentToClient, Disconnected, ClientClose, MessageFromClientOrdered, typed log of everything on a mock transport.
FakeClock.ktFakeClock : Clock (advance(ms|Duration), pendingTaskCount(name), currentTimeMillis, nanoTime, newTimer, waitOn)Virtual clock + virtual timers; deterministic time.
Utils.ktConnectionDetails { } builderTest-only ConnectionDetails DSL; instantiates the type via its package-private constructor reflectively (see §8).
+ +

B.2 Integration infrastructure — io.ably.lib.uts.infra.integration (and …integration.proxy)

+ + + + + +
FileKey public surfaceRole
proxy/ProxyManager.ktobject ProxyManager: ensureProxy(timeoutMs), stopProxy(), CONTROL_PORT=10100, sandboxRealtimeHost, sandboxRestHost; pinned PROXY_VERSION=v0.3.0 + per-arch checksums; uts.proxy.localPath overrideDownloads/verifies/launches the uts-proxy binary; one shared process per run. (package …integration.proxy)
proxy/ProxySession.ktclass ProxySession (create(rules,port,timeoutMs,realtimeHost,restHost), addRules, triggerAction, getLog(): List<Event>, close, sessionId, proxyPort, proxyHost); data class Event; typealias ProxyRule; rule builders wsConnectRule/wsFrameToClientRule/wsFrameToServerRule/httpRequestRule; ClientOptionsBuilder.connectThroughProxy(session)Typed client for the proxy control REST API + client wiring. (package …integration.proxy)
SandboxApp.ktclass SandboxApp (create(), delete(), appId, defaultKey, keys)Provisions/tears down a throwaway sandbox app from ably-common's test-app-setup.json. (package …integration)
+ +

B.3 Shared helpers & tests

+ + + + + + +
FileKey public surfaceRole
infra/Utils.ktawaitState(client,target,timeout=5s), awaitChannelState(channel,target,timeout=5s), pollUntil(timeout=15s,interval=100ms){ }Shared wall-clock coroutine waits (package io.ably.lib.uts.infra); listener registered before state check.
unit/realtime/ConnectionRecoveryTest.kt6 @Tests: RTN16g/g1, RTN16g2, RTN16k, RTN16f, RTN16f1, RTN16jUnit tier (io.ably.lib.uts.unit.realtime) — connection recovery (mocked WS, FakeClock, env-gated deviations).
integration/proxy/realtime/AuthReauthTest.kt1 @Test (two @UTS: RTN22, RTC8a)Integration tier (io.ably.lib.uts.integration.proxy.realtime) — server-initiated re-authentication.
deviations.mdRTN16f, RTN16g2, RTL13b, RTL13cCatalogue of SDK-vs-spec divergences.
+
Coverage note

At the time of writing, the uts/ module contains exactly two test classes (7 @Test methods total: 6 in ConnectionRecoveryTest + 1 in AuthReauthTest). The infrastructure under infra/unit/ and infra/integration/ is built out far beyond what these two tests exercise (full HTTP mock, all four rule builders, REST proxy wiring), anticipating the broader UTS coverage catalogued in completion-status.md.

+
+ +
+

Source map — where each fact comes from

+ + + + + + + + + + + + + + +
TopicFile
Authoring portable specs, test IDs, mock pseudocodeuts/docs/writing-test-specs.md
Translating specs, deviation patterns, decision treeuts/docs/writing-derived-tests.md
Integration/proxy policy, late fault injection, tiersuts/docs/integration-testing.md
Coverage matrixuts/docs/completion-status.md
Proxy control API, rule format, action numbersuts/realtime/integration/helpers/proxy.md
SDK seamslib/.../debug/DebugOptions.java, lib/.../util/Clock.java
Module wiringuts/build.gradle.kts, settings.gradle.kts
Unit mocksuts/.../uts/infra/unit/*
Integration helpersuts/.../uts/infra/integration/* (+ …/integration/proxy/*)
Async helpersuts/.../uts/infra/Utils.kt (awaits), …/uts/infra/unit/Utils.kt (ConnectionDetails builder)
The two example tests…/uts/unit/realtime/ConnectionRecoveryTest.kt, …/uts/integration/proxy/realtime/AuthReauthTest.kt
Deviationsuts/.../io/ably/lib/uts/deviations.md
+

Generated from README.md (in this uts/ directory). Single self-contained HTML file — no external assets.

+
+
+
+ + + + diff --git a/uts/src/test/kotlin/io/ably/lib/types/Utils.kt b/uts/src/test/kotlin/io/ably/lib/types/Utils.kt deleted file mode 100644 index 15c11d557..000000000 --- a/uts/src/test/kotlin/io/ably/lib/types/Utils.kt +++ /dev/null @@ -1,3 +0,0 @@ -package io.ably.lib.types - -fun ConnectionDetails(init: ConnectionDetails.() -> Unit) = ConnectionDetails().apply(init) diff --git a/uts/src/test/kotlin/io/ably/lib/deviations.md b/uts/src/test/kotlin/io/ably/lib/uts/deviations.md similarity index 100% rename from uts/src/test/kotlin/io/ably/lib/deviations.md rename to uts/src/test/kotlin/io/ably/lib/uts/deviations.md diff --git a/uts/src/test/kotlin/io/ably/lib/Utils.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/Utils.kt similarity index 98% rename from uts/src/test/kotlin/io/ably/lib/Utils.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/Utils.kt index 5debec7e7..24180d8f6 100644 --- a/uts/src/test/kotlin/io/ably/lib/Utils.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/Utils.kt @@ -1,4 +1,4 @@ -package io.ably.lib +package io.ably.lib.uts.infra import io.ably.lib.realtime.AblyRealtime import io.ably.lib.realtime.Channel diff --git a/uts/src/test/kotlin/io/ably/lib/test/helper/SandboxApp.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/integration/SandboxApp.kt similarity index 97% rename from uts/src/test/kotlin/io/ably/lib/test/helper/SandboxApp.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/integration/SandboxApp.kt index 901ac5902..a697618a0 100644 --- a/uts/src/test/kotlin/io/ably/lib/test/helper/SandboxApp.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/integration/SandboxApp.kt @@ -1,7 +1,8 @@ -package io.ably.lib.test.helper +package io.ably.lib.uts.infra.integration import com.google.gson.JsonElement import com.google.gson.JsonParser +import io.ably.lib.uts.infra.integration.proxy.ProxyManager import io.ktor.client.* import io.ktor.client.engine.cio.* import io.ktor.client.network.sockets.* diff --git a/uts/src/test/kotlin/io/ably/lib/test/helper/ProxyManager.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/integration/proxy/ProxyManager.kt similarity index 99% rename from uts/src/test/kotlin/io/ably/lib/test/helper/ProxyManager.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/integration/proxy/ProxyManager.kt index eed463ba8..099f404bf 100644 --- a/uts/src/test/kotlin/io/ably/lib/test/helper/ProxyManager.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/integration/proxy/ProxyManager.kt @@ -1,4 +1,4 @@ -package io.ably.lib.test.helper +package io.ably.lib.uts.infra.integration.proxy import io.ktor.client.HttpClient import io.ktor.client.call.body diff --git a/uts/src/test/kotlin/io/ably/lib/test/helper/ProxySession.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/integration/proxy/ProxySession.kt similarity index 99% rename from uts/src/test/kotlin/io/ably/lib/test/helper/ProxySession.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/integration/proxy/ProxySession.kt index 52fe6b94a..cee13b22b 100644 --- a/uts/src/test/kotlin/io/ably/lib/test/helper/ProxySession.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/integration/proxy/ProxySession.kt @@ -1,9 +1,9 @@ -package io.ably.lib.test.helper +package io.ably.lib.uts.infra.integration.proxy import com.google.gson.Gson import com.google.gson.JsonObject import com.google.gson.reflect.TypeToken -import io.ably.lib.uts.infra.ClientOptionsBuilder +import io.ably.lib.uts.infra.unit.ClientOptionsBuilder import io.ktor.client.HttpClient import io.ktor.client.engine.cio.CIO import io.ktor.client.plugins.HttpTimeout diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/ClientFactories.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/ClientFactories.kt similarity index 95% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/ClientFactories.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/ClientFactories.kt index e663bfa27..94a055cdd 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/ClientFactories.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/ClientFactories.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.debug.DebugOptions import io.ably.lib.realtime.AblyRealtime diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/DefaultPendingConnection.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/DefaultPendingConnection.kt similarity index 97% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/DefaultPendingConnection.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/DefaultPendingConnection.kt index 7552371f9..3cfd6df53 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/DefaultPendingConnection.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/DefaultPendingConnection.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.network.WebSocketListener import io.ably.lib.types.ProtocolMessage diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/DefaultPendingRequest.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/DefaultPendingRequest.kt similarity index 97% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/DefaultPendingRequest.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/DefaultPendingRequest.kt index fe92ee5b9..e77ab518d 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/DefaultPendingRequest.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/DefaultPendingRequest.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.network.FailedConnectionException import io.ably.lib.network.HttpBody diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/FakeClock.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/FakeClock.kt similarity index 98% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/FakeClock.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/FakeClock.kt index f17bddf82..0b0c50d2d 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/FakeClock.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/FakeClock.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.util.Clock import io.ably.lib.util.AblyTimer diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockEvent.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockEvent.kt similarity index 97% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/MockEvent.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockEvent.kt index 5d9d78b54..99fea4195 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockEvent.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockEvent.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.types.ProtocolMessage import java.net.URL diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockHttpClient.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockHttpClient.kt similarity index 98% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/MockHttpClient.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockHttpClient.kt index 9e0d142c0..0397cf6a9 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockHttpClient.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockHttpClient.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.debug.DebugOptions import io.ably.lib.network.HttpEngine diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockHttpEngine.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockHttpEngine.kt similarity index 95% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/MockHttpEngine.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockHttpEngine.kt index ed1b45744..dfe159ded 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockHttpEngine.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockHttpEngine.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.network.FailedConnectionException import io.ably.lib.network.HttpCall @@ -40,7 +40,7 @@ internal class MockHttpCall( // Phase 2 — request val rd = CompletableDeferred().also { respDeferred = it } - onRequest(DefaultPendingRequest(request, rd)) + onRequest(io.ably.lib.uts.infra.unit.DefaultPendingRequest(request, rd)) rd.await() } diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockWebSocket.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockWebSocket.kt similarity index 99% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/MockWebSocket.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockWebSocket.kt index 62ef8276a..e57227bf7 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockWebSocket.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockWebSocket.kt @@ -1,9 +1,8 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.debug.DebugOptions import io.ably.lib.network.WebSocketEngineFactory import io.ably.lib.network.WebSocketListener -import io.ably.lib.types.ConnectionDetails import io.ably.lib.types.ProtocolMessage import io.ably.lib.types.ProtocolSerializer import io.ably.lib.util.Serialisation diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockWebSocketEngineFactory.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockWebSocketEngineFactory.kt similarity index 98% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/MockWebSocketEngineFactory.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockWebSocketEngineFactory.kt index 3832d920c..2a41b518c 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/MockWebSocketEngineFactory.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/MockWebSocketEngineFactory.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.network.EngineType import io.ably.lib.network.WebSocketClient diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/PendingConnection.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/PendingConnection.kt similarity index 97% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/PendingConnection.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/PendingConnection.kt index 6092ca894..588f9759d 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/PendingConnection.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/PendingConnection.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import io.ably.lib.types.ProtocolMessage import java.net.URLDecoder diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/PendingRequest.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/PendingRequest.kt similarity index 95% rename from uts/src/test/kotlin/io/ably/lib/uts/infra/PendingRequest.kt rename to uts/src/test/kotlin/io/ably/lib/uts/infra/unit/PendingRequest.kt index baf0920a2..43d28ff4b 100644 --- a/uts/src/test/kotlin/io/ably/lib/uts/infra/PendingRequest.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/PendingRequest.kt @@ -1,4 +1,4 @@ -package io.ably.lib.uts.infra +package io.ably.lib.uts.infra.unit import java.net.URL import kotlin.time.Duration diff --git a/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/Utils.kt b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/Utils.kt new file mode 100644 index 000000000..34527348f --- /dev/null +++ b/uts/src/test/kotlin/io/ably/lib/uts/infra/unit/Utils.kt @@ -0,0 +1,20 @@ +package io.ably.lib.uts.infra.unit + +import io.ably.lib.types.ConnectionDetails + +/** + * Test-only builder DSL for [ConnectionDetails], e.g. + * `ConnectionDetails { connectionKey = "key-1"; connectionStateTtl = 120_000L }`. + * + * [ConnectionDetails]'s no-arg constructor is package-private to `io.ably.lib.types`, so it cannot be + * invoked directly from this package. We obtain an instance reflectively via [newConnectionDetails] — + * the same package-private-access technique used by `liveobjects/.../TestUtils.kt`. + */ +fun ConnectionDetails(init: ConnectionDetails.() -> Unit): ConnectionDetails = + newConnectionDetails().apply(init) + +/** Reflectively invokes [ConnectionDetails]'s package-private no-arg constructor. */ +private fun newConnectionDetails(): ConnectionDetails = + ConnectionDetails::class.java.getDeclaredConstructor() + .apply { isAccessible = true } + .newInstance() diff --git a/uts/src/test/kotlin/io/ably/lib/realtime/integration/proxy/AuthReauthTest.kt b/uts/src/test/kotlin/io/ably/lib/uts/integration/proxy/realtime/AuthReauthTest.kt similarity index 92% rename from uts/src/test/kotlin/io/ably/lib/realtime/integration/proxy/AuthReauthTest.kt rename to uts/src/test/kotlin/io/ably/lib/uts/integration/proxy/realtime/AuthReauthTest.kt index e6df1e4ca..d14e9405c 100644 --- a/uts/src/test/kotlin/io/ably/lib/realtime/integration/proxy/AuthReauthTest.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/integration/proxy/realtime/AuthReauthTest.kt @@ -1,15 +1,15 @@ -package io.ably.lib.realtime.integration.proxy +package io.ably.lib.uts.integration.proxy.realtime -import io.ably.lib.awaitState -import io.ably.lib.pollUntil import io.ably.lib.realtime.ConnectionState import io.ably.lib.rest.AblyRest import io.ably.lib.rest.Auth -import io.ably.lib.test.helper.ProxyManager -import io.ably.lib.test.helper.ProxySession -import io.ably.lib.test.helper.SandboxApp -import io.ably.lib.test.helper.connectThroughProxy -import io.ably.lib.uts.infra.TestRealtimeClient +import io.ably.lib.uts.infra.awaitState +import io.ably.lib.uts.infra.integration.SandboxApp +import io.ably.lib.uts.infra.integration.proxy.ProxyManager +import io.ably.lib.uts.infra.integration.proxy.ProxySession +import io.ably.lib.uts.infra.integration.proxy.connectThroughProxy +import io.ably.lib.uts.infra.pollUntil +import io.ably.lib.uts.infra.unit.TestRealtimeClient import kotlinx.coroutines.runBlocking import kotlinx.coroutines.test.runTest import org.junit.jupiter.api.AfterAll diff --git a/uts/src/test/kotlin/io/ably/lib/uts/unit/liveobjects/helpers.kt b/uts/src/test/kotlin/io/ably/lib/uts/unit/liveobjects/helpers.kt new file mode 100644 index 000000000..0830539e0 --- /dev/null +++ b/uts/src/test/kotlin/io/ably/lib/uts/unit/liveobjects/helpers.kt @@ -0,0 +1,339 @@ +package io.ably.lib.uts.unit.liveobjects + +import com.google.gson.JsonElement +import com.google.gson.JsonObject +import com.google.gson.JsonParser +import io.ably.lib.liveobjects.message.ObjectMessage +import io.ably.lib.liveobjects.path.types.LiveMapPathObject +import io.ably.lib.realtime.AblyRealtime +import io.ably.lib.realtime.Channel +import io.ably.lib.types.ChannelMode +import io.ably.lib.types.ChannelOptions +import io.ably.lib.types.ProtocolMessage +import io.ably.lib.types.PublishResult +import io.ably.lib.uts.infra.unit.ConnectionDetails +import io.ably.lib.uts.infra.unit.MockWebSocket +import io.ably.lib.uts.infra.unit.TestRealtimeClient +import kotlinx.coroutines.future.await + +/** + * LiveObjects unit-test helpers — the ably-java translation of the UTS + * `objects/helpers/standard_test_pool.md` (standard test pool, protocol-message / + * object-message builders, and the synced-channel setup) used by every objects + * unit spec. + * + * Status: + * - The builders construct the **wire JSON** form (Gson [JsonObject]) of object messages and drop them + * into [ProtocolMessage.state] (`Object[]`); the file compiles against `:java` only (no *compile-time* + * `:liveobjects` dependency). + * - [buildPublicObjectMessage] reaches the implemented message/operation layer in `:liveobjects` by + * reflection (`testRuntimeOnly(project(":liveobjects"))` in build.gradle.kts), so PAOM3/PAOOP3 + * construction tests run today. + * - [setupSyncedChannel] drives CONNECTED -> ATTACH/ATTACHED(HAS_OBJECTS) -> OBJECT_SYNC over the existing + * [MockWebSocket], then awaits `channel.object.get()`. That last step needs the SDK's OBJECT_SYNC + * processing + `RealtimeObject.get()`, both still TODO — so the generated tests **compile** now and + * become **runnable** once the SDK lands (translate-only until then). + */ + +// --------------------------------------------------------------------------- +// small Gson DSL +// --------------------------------------------------------------------------- + +private fun json(build: JsonObject.() -> Unit): JsonObject = JsonObject().apply(build) +private fun JsonObject.str(key: String, value: String) = addProperty(key, value) +private fun JsonObject.num(key: String, value: Number) = addProperty(key, value) +private fun JsonObject.bool(key: String, value: Boolean) = addProperty(key, value) + +// --------------------------------------------------------------------------- +// ObjectData (leaf value) wire builders — the `data` of a map entry / mapSet +// --------------------------------------------------------------------------- + +fun dataString(value: String): JsonObject = json { str("string", value) } +fun dataNumber(value: Number): JsonObject = json { num("number", value) } +fun dataBoolean(value: Boolean): JsonObject = json { bool("boolean", value) } +fun dataObjectId(objectId: String): JsonObject = json { str("objectId", objectId) } +fun dataBytes(base64: String): JsonObject = json { str("bytes", base64) } +fun dataJson(element: JsonElement): JsonObject = json { add("json", element) } + +// --------------------------------------------------------------------------- +// map / counter state + createOp fragments +// --------------------------------------------------------------------------- + +fun mapEntry(data: JsonObject, timeserial: String = "t:0", tombstone: Boolean? = null): JsonObject = json { + add("data", data) + str("timeserial", timeserial) + tombstone?.let { bool("tombstone", it) } +} + +/** + * Wire enum codes — the objects JSON protocol carries `action` / `semantics` as integer codes + * (`WireObjectOperationAction` / `WireObjectsMapSemantics`), not strings. The SDK's Gson decodes them by + * code, so the builders must emit the code for messages to deserialize. + */ +private object Action { + const val MAP_CREATE = 0 + const val MAP_SET = 1 + const val MAP_REMOVE = 2 + const val COUNTER_CREATE = 3 + const val COUNTER_INC = 4 + const val OBJECT_DELETE = 5 + const val MAP_CLEAR = 6 +} +private const val SEMANTICS_LWW = 0 + +fun mapState(entries: Map, semantics: Int = SEMANTICS_LWW): JsonObject = json { + num("semantics", semantics) + add("entries", json { entries.forEach { (k, v) -> add(k, v) } }) +} + +fun counterState(count: Number): JsonObject = json { num("count", count) } + +fun mapCreateOp(semantics: Int = SEMANTICS_LWW, entries: Map = emptyMap()): JsonObject = + json { num("action", Action.MAP_CREATE); add("mapCreate", mapState(entries, semantics)) } + +fun counterCreateOp(count: Number): JsonObject = + json { num("action", Action.COUNTER_CREATE); add("counterCreate", json { num("count", count) }) } + +// --------------------------------------------------------------------------- +// ObjectMessage builders — STATE (for OBJECT_SYNC) and OPERATIONS (for OBJECT) +// --------------------------------------------------------------------------- + +/** `build_object_state` — an ObjectMessage wrapping an ObjectState in its `object` field. */ +fun buildObjectState( + objectId: String, + siteTimeserials: Map, + map: JsonObject? = null, + counter: JsonObject? = null, + tombstone: Boolean? = null, + createOp: JsonObject? = null, +): JsonObject = json { + add( + "object", + json { + str("objectId", objectId) + add("siteTimeserials", json { siteTimeserials.forEach { (k, v) -> str(k, v) } }) + map?.let { add("map", it) } + counter?.let { add("counter", it) } + bool("tombstone", tombstone ?: false) // WireObjectState.tombstone is non-nullable + createOp?.let { add("createOp", it) } + }, + ) +} + +/** + * `build_object_message_with_state` — wraps an already-built ObjectState (the inner `object` payload) in + * an ObjectMessage. [buildObjectState] builds the state and wraps it in one step; this is the wrap-only + * form used where a bare ObjectState needs to become an ObjectMessage (e.g. `replaceData`). + */ +fun buildObjectMessageWithState(objectState: JsonObject): JsonObject = json { add("object", objectState) } + +private fun objectMessage( + serial: String?, + siteCode: String?, + serialTimestamp: Long? = null, + operation: JsonObject, +): JsonObject = json { + serial?.let { str("serial", it) } + siteCode?.let { str("siteCode", it) } + serialTimestamp?.let { num("serialTimestamp", it) } + add("operation", operation) +} + +fun buildCounterInc(objectId: String, number: Number, serial: String? = null, siteCode: String? = null): JsonObject = + objectMessage(serial, siteCode, operation = json { + num("action", Action.COUNTER_INC); str("objectId", objectId); add("counterInc", json { num("number", number) }) + }) + +fun buildMapSet(objectId: String, key: String, value: JsonObject, serial: String? = null, siteCode: String? = null): JsonObject = + objectMessage(serial, siteCode, operation = json { + num("action", Action.MAP_SET); str("objectId", objectId) + add("mapSet", json { str("key", key); add("value", value) }) + }) + +fun buildMapRemove(objectId: String, key: String, serial: String? = null, siteCode: String? = null, serialTimestamp: Long? = null): JsonObject = + objectMessage(serial, siteCode, serialTimestamp, operation = json { + num("action", Action.MAP_REMOVE); str("objectId", objectId); add("mapRemove", json { str("key", key) }) + }) + +fun buildMapClear(objectId: String, serial: String? = null, siteCode: String? = null): JsonObject = + objectMessage(serial, siteCode, operation = json { + num("action", Action.MAP_CLEAR); str("objectId", objectId) + }) + +fun buildObjectDelete(objectId: String, serial: String? = null, siteCode: String? = null, serialTimestamp: Long? = null): JsonObject = + objectMessage(serial, siteCode, serialTimestamp, operation = json { + num("action", Action.OBJECT_DELETE); str("objectId", objectId) + }) + +fun buildCounterCreate(objectId: String, counterCreate: JsonObject, serial: String? = null, siteCode: String? = null): JsonObject = + objectMessage(serial, siteCode, operation = json { + num("action", Action.COUNTER_CREATE); str("objectId", objectId); add("counterCreate", counterCreate) + }) + +fun buildMapCreate(objectId: String, mapCreate: JsonObject, serial: String? = null, siteCode: String? = null): JsonObject = + objectMessage(serial, siteCode, operation = json { + num("action", Action.MAP_CREATE); str("objectId", objectId); add("mapCreate", mapCreate) + }) + +// --------------------------------------------------------------------------- +// ProtocolMessage builders +// --------------------------------------------------------------------------- + +private fun List.asState(): Array = Array(size) { this[it] } + +fun buildObjectSyncMessage(channel: String, channelSerial: String, objectMessages: List): ProtocolMessage = + ProtocolMessage(ProtocolMessage.Action.object_sync).apply { + this.channel = channel + this.channelSerial = channelSerial + state = objectMessages.asState() + } + +fun buildObjectMessage(channel: String, objectMessages: List): ProtocolMessage = + ProtocolMessage(ProtocolMessage.Action.`object`).apply { + this.channel = channel + state = objectMessages.asState() + } + +fun buildAckMessage(msgSerial: Long?, serials: List): ProtocolMessage = + ProtocolMessage(ProtocolMessage.Action.ack).apply { + this.msgSerial = msgSerial + res = arrayOf(PublishResult(serials.toTypedArray())) + } + +/** + * `build_public_object_message` — constructs a public [ObjectMessage] (PAOM3) from the wire form of an + * object message (as produced by the operation builders above) and a channel name. + * + * ably-java's public `ObjectMessage` / `ObjectOperation` are getter-only interfaces with no public factory + * — the construction (`WireObjectMessage` -> `DefaultObjectMessage`) lives `internal` in `:liveobjects`. + * We reach it by **reflection**, in the same spirit as `infra/unit/Utils.kt` (which reflectively reaches an + * inaccessible `:java` member) — but here the classes are on the *runtime-only* classpath + * (`testRuntimeOnly(project(":liveobjects"))`), so we load them with `Class.forName` rather than flipping + * `isAccessible`. The targeted members compile to plain `public` on the JVM (Kotlin `internal` is not + * name-mangled here), so they are addressable by their declared names: + * - `JsonSerializationKt.toObjectMessage(JsonObject): WireObjectMessage` (Gson, decodes enum codes) + * - `DefaultObjectMessage(WireObjectMessage, String)` + */ +fun buildPublicObjectMessage(objectMessage: JsonObject, channelName: String): ObjectMessage { + val serializationKt = Class.forName("io.ably.lib.liveobjects.serialization.JsonSerializationKt") + val toWire = serializationKt.getMethod("toObjectMessage", JsonObject::class.java) + val wire = toWire.invoke(null, objectMessage) + + val wireClass = Class.forName("io.ably.lib.liveobjects.message.WireObjectMessage") + val defaultMessage = Class.forName("io.ably.lib.liveobjects.message.DefaultObjectMessage") + .getConstructor(wireClass, String::class.java) + .newInstance(wire, channelName) + return defaultMessage as ObjectMessage +} + +// `provision_objects_via_rest(...)` is intentionally not here — it's REST fixture provisioning for +// *integration* tests and belongs in the integration infra, not this unit helper file. + +// --------------------------------------------------------------------------- +// STANDARD_POOL_OBJECTS — the fixed tree shared by all objects unit specs +// --------------------------------------------------------------------------- + +private val SITE = mapOf("aaa" to "t:0") + +val STANDARD_POOL_OBJECTS: List = listOf( + buildObjectState( + "root", SITE, + map = mapState( + linkedMapOf( + "name" to mapEntry(dataString("Alice")), + "age" to mapEntry(dataNumber(30)), + "active" to mapEntry(dataBoolean(true)), + "score" to mapEntry(dataObjectId("counter:score@1000")), + "profile" to mapEntry(dataObjectId("map:profile@1000")), + "data" to mapEntry(dataJson(JsonParser.parseString("""{"tags":["a","b"]}"""))), + "avatar" to mapEntry(dataBytes("AQID")), + ), + ), + createOp = mapCreateOp(), + ), + buildObjectState("counter:score@1000", SITE, counter = counterState(100), createOp = counterCreateOp(100)), + buildObjectState( + "map:profile@1000", SITE, + map = mapState( + linkedMapOf( + "email" to mapEntry(dataString("alice@example.com")), + "nested_counter" to mapEntry(dataObjectId("counter:nested@1000")), + "prefs" to mapEntry(dataObjectId("map:prefs@1000")), + ), + ), + createOp = mapCreateOp(), + ), + buildObjectState("counter:nested@1000", SITE, counter = counterState(5), createOp = counterCreateOp(5)), + buildObjectState( + "map:prefs@1000", SITE, + map = mapState(linkedMapOf("theme" to mapEntry(dataString("dark")))), + createOp = mapCreateOp(), + ), +) + +// --------------------------------------------------------------------------- +// synced-channel setup +// --------------------------------------------------------------------------- + +/** Result of [setupSyncedChannel] — the spec's `{ client, channel, root, mock_ws }`. */ +data class SyncedChannel( + val client: AblyRealtime, + val channel: Channel, + val root: LiveMapPathObject, + val mockWs: MockWebSocket, +) + +/** `setup_synced_channel` — connected client + channel synced with [STANDARD_POOL_OBJECTS]; auto-ACKs OBJECT publishes. */ +suspend fun setupSyncedChannel(channelName: String): SyncedChannel = setup(channelName, autoAck = true) + +/** `setup_synced_channel_no_ack` — as above but does not ACK OBJECT publishes (for tests that control ACK timing). */ +suspend fun setupSyncedChannelNoAck(channelName: String): SyncedChannel = setup(channelName, autoAck = false) + +private suspend fun setup(channelName: String, autoAck: Boolean): SyncedChannel { + lateinit var mockWs: MockWebSocket + mockWs = MockWebSocket { + onConnectionAttempt = { conn -> + conn.respondWithSuccess( + ProtocolMessage(ProtocolMessage.Action.connected).apply { + connectionId = "conn-1" + connectionDetails = ConnectionDetails { + connectionKey = "conn-key-1" + siteCode = "test-site" + objectsGCGracePeriod = 86_400_000L + } + }, + ) + } + onMessageFromClient = { msg -> + when (msg.action) { + ProtocolMessage.Action.attach -> { + mockWs.sendToClient( + ProtocolMessage(ProtocolMessage.Action.attached).apply { + channel = msg.channel + channelSerial = "sync1:" + setFlag(ProtocolMessage.Flag.has_objects) + }, + ) + mockWs.sendToClient(buildObjectSyncMessage(msg.channel, "sync1:", STANDARD_POOL_OBJECTS)) + } + ProtocolMessage.Action.`object` -> if (autoAck) { + val serials = (msg.state?.indices ?: IntRange.EMPTY).map { "ack-${msg.msgSerial}:$it" } + mockWs.sendToClient(buildAckMessage(msg.msgSerial, serials)) + } + else -> Unit + } + } + } + + val client = TestRealtimeClient { + key = "fake:key" + install(mockWs) + } + val channel = client.channels.get( + channelName, + ChannelOptions().apply { modes = arrayOf(ChannelMode.object_subscribe, ChannelMode.object_publish) }, + ) + // NOTE: throws until :liveobjects implements RealtimeObject.get() + OBJECT_SYNC processing. + val root = channel.`object`.get().await() + return SyncedChannel(client, channel, root, mockWs) +} diff --git a/uts/src/test/kotlin/io/ably/lib/realtime/unit/connection/ConnectionRecoveryTest.kt b/uts/src/test/kotlin/io/ably/lib/uts/unit/realtime/ConnectionRecoveryTest.kt similarity index 95% rename from uts/src/test/kotlin/io/ably/lib/realtime/unit/connection/ConnectionRecoveryTest.kt rename to uts/src/test/kotlin/io/ably/lib/uts/unit/realtime/ConnectionRecoveryTest.kt index b4152d695..9dd55ec44 100644 --- a/uts/src/test/kotlin/io/ably/lib/realtime/unit/connection/ConnectionRecoveryTest.kt +++ b/uts/src/test/kotlin/io/ably/lib/uts/unit/realtime/ConnectionRecoveryTest.kt @@ -1,18 +1,14 @@ -package io.ably.lib.realtime.unit.connection +package io.ably.lib.uts.unit.realtime -import io.ably.lib.uts.infra.TestRealtimeClient -import io.ably.lib.awaitChannelState -import io.ably.lib.awaitState +import io.ably.lib.uts.infra.unit.* import io.ably.lib.realtime.ChannelState import io.ably.lib.realtime.ConnectionState -import io.ably.lib.uts.infra.CONNECTED_MESSAGE -import io.ably.lib.uts.infra.FakeClock -import io.ably.lib.uts.infra.MockEvent -import io.ably.lib.uts.infra.MockWebSocket -import io.ably.lib.types.ConnectionDetails import io.ably.lib.types.ErrorInfo import io.ably.lib.types.ProtocolMessage import io.ably.lib.types.RecoveryKeyContext +import io.ably.lib.uts.infra.awaitChannelState +import io.ably.lib.uts.infra.awaitState +import io.ably.lib.uts.infra.pollUntil import kotlinx.coroutines.launch import kotlinx.coroutines.test.runTest import kotlin.test.* @@ -246,6 +242,11 @@ class ConnectionRecoveryTest { awaitState(client, ConnectionState.connected) mock.simulateDisconnect() + // Wait for the reconnect's connection attempt to be captured, not just for the CONNECTED state: + // right after simulateDisconnect() the state is still `connected`, so awaitState(connected) would + // short-circuit before the second attempt is recorded (the transient-state race called out in + // writing-test-specs.md). Gate on the second attempt actually arriving. + pollUntil { capturedQueryParams.size >= 2 } awaitState(client, ConnectionState.connected) assertEquals("recovered-key-xyz", capturedQueryParams[0]["recover"])