Skip to content

[AIT-1008] Generate LiveObjects tests based on UTS test specs#1221

Draft
sacOO7 wants to merge 8 commits into
feature/path-based-liveobjects-implementationfrom
feature/liveobjects-uts-tests
Draft

[AIT-1008] Generate LiveObjects tests based on UTS test specs#1221
sacOO7 wants to merge 8 commits into
feature/path-based-liveobjects-implementationfrom
feature/liveobjects-uts-tests

Conversation

@sacOO7

@sacOO7 sacOO7 commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator
  • Fixes https://ably.atlassian.net/browse/AIT-1008
  • Generated UTS markdown to have better understanding of existing UTS infra.
  • This is also helpful for a relevant uts-to-kotlin skill to better generate ably-java tests
  • Based on the doc, we have missing sandbox test, need to update the same

📚 Human-readable UTS guide + website

To make the UTS setup approachable, this PR adds a comprehensive guide and a self-contained website, kept in sync:

  • uts/README.md — renders on the module's GitHub page.
  • uts/index.html — a standalone, offline-friendly website (inline SVG flow diagrams, scroll-spy nav, light/dark theme; no external assets).

Both walk through: what UTS is → the three test tiers → the spec docs (linked to GitHub) → the uts/ module layout → the SDK hook points → the unit & proxy infrastructure → the two example tests → deviations → how to run → appendices (flow diagrams + per-file API reference + source map).

🗂️ Test package restructure

Test sources are reorganised under a domain-rooted io.ably.lib.uts package that cleanly separates infrastructure from tests, and unit from integration:

io/ably/lib/uts/
├── infra/                        # test infrastructure (no @Test methods)
│   ├── Utils.kt                  #   shared awaits (awaitState / pollUntil …)
│   ├── unit/                     #   mock transports + ConnectionDetails builder
│   └── integration/              #   SandboxApp
│       └── proxy/                #     ProxyManager, ProxySession
├── unit/realtime/                # UNIT tests          → ConnectionRecoveryTest
└── integration/proxy/realtime/   # INTEGRATION tests   → AuthReauthTest

The unit/infra/unit/ and integration/infra/integration/ pairing is what the new Gradle tasks key off.

Note: the ConnectionDetails test builder no longer lives in io.ably.lib.types, so it now obtains the package-private constructor reflectively (same approach as liveobjects/.../TestUtils.kt).

⚙️ Per-tier Gradle tasks + CI wiring

Added two package-filtered test tasks in uts/build.gradle.kts, mirroring runLiveObjectsUnitTests / runLiveObjectsIntegrationTests:

Task Runs CI
:uts:runUtsUnitTests io.ably.lib.uts.unit.* (mocked, fast) check.yml (PR gate)
:uts:runUtsIntegrationTests io.ably.lib.uts.integration.* (real sandbox + proxy) integration-test.yml → new check-uts job

✅ Validation

  • ./gradlew :uts:runUtsUnitTests → 6/6 pass · :uts:runUtsIntegrationTests → 1/1 pass.
  • Full CI gate checkWithCodenarc checkstyleMain checkstyleTest runUnitTests runLiveObjectsUnitTests :uts:runUtsUnitTestsBUILD SUCCESSFUL.

@coderabbitai

coderabbitai Bot commented Jun 25, 2026

Copy link
Copy Markdown

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 65530f9a-0a70-4efb-8ad3-26fa106d362f

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feature/liveobjects-uts-tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 09:10 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 09:12 Inactive
Move the human-readable UTS guide and its self-contained HTML rendering
into the uts/ module as README.md and index.html. Both cover the UTS
concept, the three test tiers, the spec docs, the uts/ module layout,
mock/proxy infrastructure, the two example tests, deviations, and
appendices. Spec-doc references link to GitHub; paths are fully
qualified; the two artifacts are kept in sync.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 09:54 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 09:55 Inactive
@sacOO7 sacOO7 changed the title Generated UTS doc to better understand the existing UTS infra. [AIT-1008] Generate LiveObjects tests based on UTS test specs Jun 25, 2026
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 11:08 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 11:09 Inactive
…ier Gradle tasks

Reorganise the uts module under a domain-rooted io.ably.lib.uts package that
cleanly separates infrastructure from tests, and unit from integration:

  infra/                         shared awaits (Utils.kt)
  infra/unit/                    mock transports + ConnectionDetails builder
  infra/integration/             SandboxApp
  infra/integration/proxy/       ProxyManager, ProxySession
  unit/realtime/                 ConnectionRecoveryTest        (mocked)
  integration/proxy/realtime/    AuthReauthTest                (sandbox + proxy)

- The ConnectionDetails test builder no longer sits in io.ably.lib.types, so it
  obtains the package-private constructor reflectively (as liveobjects/TestUtils.kt does).
- Add runUtsUnitTests / runUtsIntegrationTests Gradle tasks (filtered by package),
  mirroring runLiveObjectsUnitTests / runLiveObjectsIntegrationTests.
- check.yml now runs :uts:runUtsUnitTests; integration-test.yml gains a check-uts
  job running :uts:runUtsIntegrationTests.
- Bring uts/README.md and uts/index.html in sync with the new structure.
@sacOO7 sacOO7 force-pushed the feature/liveobjects-uts-tests branch from 884ee87 to aa0504e Compare June 25, 2026 11:25
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 11:26 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 11:27 Inactive
Reflect the new test-source structure in the UTS guide, website, and the
uts-to-kotlin skill:

- Add the direct-sandbox (`integration/standard/<module>/`) tier alongside
  the existing unit and proxy tiers, and document that every tier is now
  organised by module (`realtime`, `liveobjects`, …).
- Update §2 tier table, §4.2 directory tree + mental model, §7.3 SandboxApp
  (shared by both integration kinds), and §12 run commands in README.md, and
  mirror all of it in index.html (tags verified balanced, sections intact).
- Generalise the skill's spec→test path mapping to `<module>`, add a
  direct-sandbox row, and split integration specs into fault-injecting (proxy)
  vs happy-path (direct sandbox) flows.
- Correct stale "both tiers" wording now that there are three tiers.
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 11:44 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 11:46 Inactive
…ckage mapping

Rework the uts-to-kotlin skill to translate a whole UTS module at once
instead of a single spec file:

- Take a UTS module directory (e.g. .../specification/uts/objects) and
  validate it sits directly under uts/ with a standard tier structure.
- Resolve the target ably-java package via uts-package-mapping.json (a new
  config file alongside the skill): a shared `testRoot` parent plus a
  `packages` table mapping each source module to its per-tier output dir
  (so objects -> liveobjects is explicit). Offer to create a mapping when
  one is missing.
- Let the user pick a tier (unit / integration / proxy) and then translate
  all specs or a selected subset, looping each through the existing
  per-spec translation steps.

Phase 1 (selection: Steps A-D) is new; Phase 2 (per-spec translation:
Steps 1-7) keeps the existing rules, with Step 1/2 adjusted to consume the
looped spec and the pre-resolved target.
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 12:27 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 12:29 Inactive
…aluate mode

Make the skill's selection phase deterministic and add an explicit
translate-vs-evaluate choice:

- Add scripts/resolve_uts.py — a bundled resolver that validates the module
  directory, reads uts-package-mapping.json, and emits JSON with, per tier,
  the target dir, Kotlin package, and the candidate specs with derived class
  names. This replaces the per-run hand-work (regex validation, path joins,
  snake_case->PascalCase) that the model previously improvised, so Phase 1 is
  byte-for-byte deterministic. Exclusions are checked relative to the tier
  base (robust to the checkout location), and --create guards the target name.
- Rewrite Phase 1 (Steps A-E) around the resolver: resolve, confirm/create
  mapping, choose tier, choose specs, choose translate-only vs evaluate.
- Gate Step 6 (run/fix) behind evaluate mode per writing-derived-tests.md's
  Translation (always) vs Evaluation (only when an implementation exists)
  split; translate-only stops after compile + review.
- Make the reference fetch mandatory (WebFetch added to allowed-tools).
- Fix the file template to use the resolver's package/className (no hardcoded
  realtime, no double Test suffix) and the spec's full @uts id; correct stale
  uts/test/... proxy doc paths.
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 16:58 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 16:59 Inactive
…rence

The objects UTS specs are written in ably-js-style pseudocode, but ably-java
is a typed SDK (RTTS1-10 partition). Add references/objects-mapping.md mapping
each ably-js symbol to its ably-java equivalent: entry point, async
(CompletableFuture/await), the typed PathObject/Instance hierarchies and as*
casts, the LiveMapValue write union, creation value types, subscriptions,
sync-state events, ValueType, message/operation getters, error codes, path
dot-escaping, and the internal-graph caveat for unit specs.

Wire it in deterministically: each module declares its translation reference
via a `notes` field in uts-package-mapping.json; resolve_uts.py resolves it to
`translationNotes` (absolute path, or null), and SKILL.md makes it required
reading before Phase 2 when present.
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 18:50 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 18:52 Inactive
…ap them

Translate objects/helpers/standard_test_pool.md into ably-java test helpers in
uts/.../unit/liveobjects/helpers.kt: the standard object pool, the protocol- and
object-message builders (emitting the integer-coded wire JSON the SDK's Gson
expects), setupSyncedChannel/NoAck over the existing MockWebSocket, and
buildPublicObjectMessage — which reaches the internal PAOM3/PAOOP3 construction
(WireObjectMessage -> DefaultObjectMessage) by reflection, so it runs today even
though the rest of :liveobjects is unimplemented. Add testRuntimeOnly(:liveobjects)
so that reflection resolves while keeping the compile classpath decoupled.

Wire the mapping reference at it: objects-mapping.md gains a "Unit-test helpers"
section mapping each spec helper to its Kotlin name, and §11/§13 are corrected to
note public_object_message.md is translatable via buildPublicObjectMessage rather
than internal-only.
@sacOO7 sacOO7 force-pushed the feature/liveobjects-uts-tests branch from 8276473 to 752b45e Compare June 25, 2026 19:45
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/features June 25, 2026 19:46 Inactive
@github-actions github-actions Bot temporarily deployed to staging/pull/1221/javadoc June 25, 2026 19:48 Inactive
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant