feat(restheart-mongo): keploy compat lane sample + Java line coverage gate#134
Open
AkashKumar7902 wants to merge 7 commits intomainfrom
Open
feat(restheart-mongo): keploy compat lane sample + Java line coverage gate#134AkashKumar7902 wants to merge 7 commits intomainfrom
AkashKumar7902 wants to merge 7 commits intomainfrom
Conversation
Mirrors the doccano-django sample shape: the sample owns orchestration (compose / bootstrap / traffic / coverage), keploy CI lanes consume it as a thin wrapper. This is a SCAFFOLD — the full traffic loop driven by the existing keploy/enterprise lane (`compat_trigger_record_traffic` in .ci/scripts/restheart-linux.sh, ~600 lines covering CRUD on /<db>/<coll> + GraphQL + files + ACL + users + bulk + aggregations) needs to be ported into flow.sh::restheart_record_traffic in a follow-up. The current loop is deliberately minimal (CRUD on a seed collection) which is enough to prove the sample boots end-to-end without keploy. Layout: Dockerfile — pin to softinstigate/restheart:9.2.1 docker-compose.yml — mongo:7 + restheart:9.2.1, env-driven flow.sh — bootstrap | record-traffic | coverage | list-routes keploy.yml.template — globalNoise for _etag/_oid/lastModified/Date README.md — handoff + status notes Signed-off-by: Akash Kumar <meakash7902@gmail.com>
There was a problem hiding this comment.
Pull request overview
Adds a new restheart-mongo/ sample scaffold to serve as a Keploy compat-lane “owned orchestration” fixture (compose + bootstrap + traffic + route coverage), intended to be consumed by CI lanes as a thin wrapper.
Changes:
- Introduces a RESTHeart+MongoDB Docker Compose setup with env-driven names and static network settings.
- Adds
flow.shto bootstrap RESTHeart, drive a minimal REST CRUD traffic loop, and compute route coverage from curated route patterns. - Adds docs and a Keploy noise template for non-deterministic fields.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| restheart-mongo/Dockerfile | Pins the RESTHeart image version used by the sample. |
| restheart-mongo/docker-compose.yml | Defines RESTHeart+Mongo services, networking, and health checks. |
| restheart-mongo/flow.sh | Provides bootstrap/traffic/coverage/list-routes orchestration for the sample. |
| restheart-mongo/keploy.yml.template | Adds a minimal Keploy globalNoise template for RESTHeart responses. |
| restheart-mongo/README.md | Documents the scaffold status, contract, and local run instructions. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+146
to
+156
| local f method route | ||
| local found_keploy=0 | ||
| while IFS= read -r f; do | ||
| found_keploy=1 | ||
| method=$(awk '/^ method:/{print $2; exit}' "$f") | ||
| route=$(awk '/^ url:/{print $2; exit}' "$f") | ||
| route="${route%%\?*}" | ||
| case "$route" in http://*|https://*) route="/${route#*://*/}" ;; esac | ||
| if [ -n "$method" ] && [ -n "$route" ]; then echo "$method $route"; fi | ||
| done < <(find keploy -type f -path '*/tests/*.yaml' 2>/dev/null) | sort -u | ||
| if [ "$found_keploy" = "1" ]; then return 0; fi |
Comment on lines
+151
to
+154
| route=$(awk '/^ url:/{print $2; exit}' "$f") | ||
| route="${route%%\?*}" | ||
| case "$route" in http://*|https://*) route="/${route#*://*/}" ;; esac | ||
| if [ -n "$method" ] && [ -n "$route" ]; then echo "$method $route"; fi |
Comment on lines
+160
to
+163
| method="${line%% *}"; route="${line#* }" | ||
| route="${route%%\?*}" | ||
| case "$route" in http://*|https://*) route="/${route#*://*/}" ;; esac | ||
| [ -n "$method" ] && [ -n "$route" ] && echo "$method $route" |
Comment on lines
+73
to
+79
| restheart_wait_for_app "$timeout" | ||
|
|
||
| # Create the test database. PUT on /<db> is idempotent — | ||
| # 201 first time, 200 on subsequent runs. | ||
| curl -sS -o /dev/null -H "$RESTHEART_ADMIN_AUTH" -X PUT "${base}/${RESTHEART_DB}" || true | ||
| # Seed a collection so reads have something to find. | ||
| curl -sS -o /dev/null -H "$RESTHEART_ADMIN_AUTH" -X PUT "${base}/${RESTHEART_DB}/items" || true |
Comment on lines
+86
to
+104
| log_fired GET "$base/" | ||
| curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/" >/dev/null || true | ||
|
|
||
| log_fired GET "$base/${RESTHEART_DB}" | ||
| curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/${RESTHEART_DB}" >/dev/null || true | ||
|
|
||
| log_fired GET "$base/${RESTHEART_DB}/items" | ||
| curl -sS -H "$RESTHEART_ADMIN_AUTH" "$base/${RESTHEART_DB}/items" >/dev/null || true | ||
|
|
||
| # Insert a document. | ||
| log_fired POST "$base/${RESTHEART_DB}/items" | ||
| curl -fsS -H "$RESTHEART_ADMIN_AUTH" -H "$h_json" -X POST \ | ||
| "$base/${RESTHEART_DB}/items" \ | ||
| -d "{\"_id\":\"keploy-${RESTHEART_PHASE}\",\"name\":\"sample item\",\"score\":42}" >/dev/null || true | ||
|
|
||
| # Read it back. | ||
| log_fired GET "$base/${RESTHEART_DB}/items/keploy-${RESTHEART_PHASE}" | ||
| curl -sS -H "$RESTHEART_ADMIN_AUTH" \ | ||
| "$base/${RESTHEART_DB}/items/keploy-${RESTHEART_PHASE}" >/dev/null || true |
Comment on lines
+39
to
+44
| networks: | ||
| restheart-net: | ||
| driver: bridge | ||
| ipam: | ||
| config: | ||
| - subnet: ${RESTHEART_NETWORK_SUBNET:-172.36.0.0/24} |
Comment on lines
+41
to
+44
| # drive in record-traffic. Override RESTHEART_ADMIN_AUTH to add | ||
| # `Authorization: Basic <b64>` to authenticated calls when porting | ||
| # the full lane traffic. | ||
| RESTHEART_ADMIN_AUTH="${RESTHEART_ADMIN_AUTH:-Basic YWRtaW46c2VjcmV0}" |
Replace the minimal record-traffic stub with the complete loop that the keploy compat lane needs to gate. flow.sh::restheart_record_traffic now drives the full RESTHeart 9.x surface end-to-end against bare RESTHeart, and restheart_list_routes enumerates every (method, route) tuple it fires so coverage stays in lockstep. Covered surfaces: - CRUD on /<db>/<coll> + /<db>/<coll>/<docid> (HAL, _size, _meta, _indexes, ETag conditional flow, writeMode insert/update/upsert, $-operator PATCH variety) - Aggregations via _meta.aggrs with avars variable interpolation (scalars / arrays / nested / missing / malformed) - Bulk writes (POST array body, filter PATCH, filter DELETE, larger 25-doc batches, mixed valid/invalid) - GraphQL apps (gql-apps registration, query / mutation / fragment / alias / multi-op, BSON scalar coercion on outputs and inputs, introspection, error paths) - Files / GridFS (.files buckets, multipart upload, binary download with Range requests, metadata fetch, delete) - ACL rules (predicate evaluator across method / path-prefix / qparams-* / bson-request-* / equals[%U,...] / in[%h,...]) plus the mongo permission interceptors (readFilter, writeFilter, projectResponse, mergeRequest, filterOperatorsBlacklist, propertiesBlacklist, allowBulk*) - Users (/users) with the userPwdHasher bcrypt interceptor; reader / writer roles authenticating via Basic + Bearer; wrong-password deny - Sessions / multi-doc transactions (/_sessions/<id>/_txns/<txnid>) with commit and abort branches - Auth services (/token form grants, JWT, Auth-Token, Digest, OAuth metadata under /.well-known/oauth-*) - Diagnostics (/ping, /metrics in json/prometheus/openmetrics, per-db and per-coll, /health/db, OPTIONS preflight, gzip request encoding, Accept-Encoding negotiation) - MongoMountResolver (multiple databases, encoded collection names, root /_size and /_meta, trailing-slash and double-slash variants) restheart_bootstrap now PUTs every collection record-traffic touches. README.md describes the sample as a complete keploy compat lane sample and lists every surface it exercises. Signed-off-by: Akash Kumar <meakash7902@gmail.com>
AkashKumar7902
changed the title
Adds .github/workflows/restheart-mongo.yml plus the helper .github/workflows/scripts/run-and-measure.sh, modeled on the doccano-django sample's coverage gate. * paths-scoped trigger: pull_request and push-to-main both filter on `restheart-mongo/**` and `.github/workflows/restheart-mongo.yml`, so changes to other samples in this repo do not trigger this workflow (and vice versa). * Three jobs: build-coverage (PR HEAD), release-coverage (PR base with first-PR bootstrap escape hatch), and coverage-gate that fails the PR if coverage drops more than COVERAGE_THRESHOLD percentage points (default 1.0pp, override via repo variable RESTHEART_COVERAGE_THRESHOLD). * Helper script brings the sample up via its own docker-compose.yml, waits for the RESTHeart listener (treating both 200 and 401 as ready since `/` requires auth), runs flow.sh bootstrap → record-traffic → coverage, and emits the parsed percentage onto $GITHUB_OUTPUT for the gate job. * Isolated from the enterprise lane: the enterprise PR pipeline (.woodpecker/restheart-linux.yml) calls `flow.sh coverage` only informationally and does not gate on it. The gate lives only here, on the sample repo, so coverage regressions surface on PRs that touch this sample without coupling enterprise CI to the route table. Signed-off-by: Akash Kumar <meakash7902@gmail.com>
build-coverage on PR #134 hung 8 min when restheart never bound on port 8080 (last_code=000). The helper script silently looped through both the wait and flow.sh bootstrap timers, then the gate job aborted without surfacing why restheart didn't start. Adding an explicit fail-fast + docker logs dump after the 240s wait so a future failure surfaces the restheart Java traceback (or the mongo connection error, or whatever else). Signed-off-by: Akash Kumar <meakash7902@gmail.com>
officialasishkumar
approved these changes
May 1, 2026
Replaces the prior API-route-surface "coverage" (counting fired
routes / curated route table) with actual JaCoCo line coverage
of the RESTHeart 9.x JVM under traffic.
Architecture:
- `Dockerfile.coverage` is a multi-stage build: stage 1 (alpine)
fetches JaCoCo 0.8.13 (jacocoagent.jar + jacococli.jar), stage
2 layers them into the upstream restheart image (which is
distroless — no shell, no curl, so jars must be pulled in a
builder stage and COPY'd over).
- `docker-compose.coverage.yml` is an OVERLAY: applied via `-f
docker-compose.yml -f docker-compose.coverage.yml`. It sets
JAVA_TOOL_OPTIONS=-javaagent:.../jacocoagent.jar=output=tcpserver,...
so JaCoCo attaches at JVM start and listens on port 6300.
The base `Dockerfile` and `docker-compose.yml` are untouched,
so keploy/integrations and keploy/enterprise CI lanes consume
the base compose and pay zero JaCoCo cost (the agent rewrites
bytecode at class-load, adding ~5-10% per-call overhead that
would slow record/replay).
- `flow.sh::restheart_report_coverage` shells into a one-off
coverage container to dump execution data via JaCoCo TCP and
render an XML report against /opt/restheart/restheart.jar.
When called against the base image (no overlay) it prints
"INFO: ... uninstrumented" and exits 0 so enterprise lanes'
`flow.sh coverage || true` informational calls keep working.
Also fixes a pre-existing config bug in the base
docker-compose.yml's RHO env var: the override syntax uses ';'
as a key->value separator (the upstream image's default
RHO uses ';'); the prior YAML-folded version used ',' which
RESTHeart parsed as part of the connection-string value, leading
RESTHeart to ignore the override and bind /http-listener/host to
its localhost default — making the HTTP listener unreachable
from the host port mapping. The base compose now uses ';' AND
explicitly overrides /http-listener/host -> "0.0.0.0".
Removed:
- `restheart_list_routes` (curated route table denominator).
- `restheart_list_recorded_routes` (keploy-tests / fired-routes
reader).
- The legacy route-surface `restheart_report_coverage` body.
- `list-routes` subcommand.
Validated locally: helper produced `coverage=52.3` to
GITHUB_OUTPUT against a clean stack (1663/3182 lines covered
in restheart.jar; INSTRUCTION coverage 50.8%).
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
restheart-mongo sample coverage
Threshold: PR may not drop coverage by more than 1.0pp. Override per-repo via the |
Signed-off-by: Akash Kumar <meakash7902@gmail.com>
…RED_ROUTES refs Signed-off-by: Akash Kumar <meakash7902@gmail.com>
AkashKumar7902
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a new
restheart-mongo/sample that owns end-to-end orchestration (compose / bootstrap / traffic / noise filter / coverage) for the RESTHeart 9.x + MongoDB 7 compat lane. The keploy/enterprise CI lane consumes it as a thin wrapper.The sample drives the full RESTHeart REST surface keploy needs to gate on a record/replay round-trip: REST CRUD on
/{db}/{coll}+_aggrs/{name}+_size+_meta+_indexes+_streams/{name}, GraphQL apps (/graphql,/graphql/{appname}), files binary, ACL, users, OAuth metadata, sessions+transactions, metrics, and/ic//csvservices — 134 distinct (method, route) tuples inrestheart_record_traffic.Layout
Coverage architecture
Real Java line coverage via JaCoCo 0.8.13, validated locally end-to-end at 52.3% (1663/3182 lines) on the existing record-traffic surface (INSTRUCTION coverage 50.8%).
Critical: the base
Dockerfileanddocker-compose.ymlare untouched. Coverage instrumentation lives in a separateDockerfile.coverage+docker-compose.coverage.ymloverlay, applied only by the standalone GH Actions coverage workflow. The keploy/enterprise compat lane consumes the base compose unchanged and pays zero JaCoCo cost (the agent rewrites bytecode at class-load and adds ~5-10% per-call overhead that would slow record/replay).How it works:
Dockerfile.coverageis a multi-stage build: stage 1 (alpine builder) fetches the JaCoCo zip and extractsjacocoagent.jar+jacococli.jar; stage 2 layers them into the upstream restheart image (which is distroless — no shell, so jars must be pulled in a builder stage andCOPY --chown'd).JAVA_TOOL_OPTIONS=-javaagent:/opt/jacoco/jacocoagent.jar=output=tcpserver,address=0.0.0.0,port=6300,sessionid=keploy,append=false. The JVM reads this at startup and prepends the-javaagentflag, so the agent attaches before restheart's classes load.flow.sh coverageshells into a one-off coverage container to dump execution data over the agent's TCP server (no JVM stop needed) and renders an XML report against/opt/restheart/restheart.jarviajacococli report.<counter type="LINE" missed covered/>rows are parsed and emitted asCovered N/M (XX.X%).flow.sh coverageagainst the base compose (no overlay) is a graceful no-op.Bonus: base compose RHO bug fix
This PR also fixes a pre-existing config bug in the base
docker-compose.yml's RHO env var:;as the key->value separator (the upstream image's default). The prior YAML-folded version used,— which RESTHeart parsed as part of the connection-string value, ignoring the override entirely./http-listener/host->"0.0.0.0"; without it, RESTHeart bound to its localhost default and was unreachable from the host port mapping.Both are fixed in this PR.
Coverage gate
.github/workflows/restheart-mongo.ymltriggers ONLY on changes underrestheart-mongo/**. Build (PR HEAD) + release (base ref) coverage compared; PR fails if drop exceeds 1.0pp. Threshold overridable viavars.RESTHEART_COVERAGE_THRESHOLD.Run modes
docker compose up -d && bash flow.sh bootstrap 240 && bash flow.sh record-traffic— exactly what the keploy enterprise lane wraps.flow.sh coverage(no JVM stop needed).keploy/enterprise.woodpecker/restheart-linux.ymllane wrapsdocker compose upinkeploy record/keploy testagainst the base compose.See README for full commands.
Consumers
keploy/enterprise.woodpecker/restheart-linux.yml— three-cell record/replay matrix that delegates compose + bootstrap + traffic to this sample.Test plan
docker compose up -dboots mongo + restheart cleanly; restheart returns 401 on/(auth required, listener up)flow.sh bootstrap 240PUTs the seed db + 11 collections + GraphQL apps + ACL + usersflow.sh record-trafficexercises 134 (method, route) tuples across the full surfaceflow.sh coverageagainst base compose exits 0 cleanly with INFO message