diff --git a/vale_styles/config/vocabularies/Base/accept.txt b/vale_styles/config/vocabularies/Base/accept.txt index 9acf69b28..e4d88a77f 100644 --- a/vale_styles/config/vocabularies/Base/accept.txt +++ b/vale_styles/config/vocabularies/Base/accept.txt @@ -20,10 +20,12 @@ Deduplication distros dockerfile Docusaurus +Dropwizard expected_string Functionize GitHub gjson +Gradle graphql Hacktoberfest HAProxy @@ -40,11 +42,13 @@ Idempotency JaCoCo Jacoco JBehave +Jersey JMeter json_contains json_equal json_path JUnit +[Kk]araf kubectl kubernetes test-gen @@ -68,6 +72,7 @@ Redis [Rr]epo Reqnroll SDK +servlet signin Spotify status_code diff --git a/versioned_docs/version-4.0.0/keploy-cloud/deduplication.md b/versioned_docs/version-4.0.0/keploy-cloud/deduplication.md index 1bbfd6a97..ca24a0321 100644 --- a/versioned_docs/version-4.0.0/keploy-cloud/deduplication.md +++ b/versioned_docs/version-4.0.0/keploy-cloud/deduplication.md @@ -121,52 +121,63 @@ keploy dedup --rm #### 1. Pre-requisite -Add the Keploy Java SDK to your application: +Java dynamic deduplication uses JaCoCo runtime coverage. Attach the Keploy Java agent to emit per-test coverage signals, and attach the JaCoCo runtime agent so the SDK can read the coverage data. The Java agent is framework-agnostic across Spring Boot, Dropwizard/Jersey, plain executable jars, classpath-based apps, servlet/WAR archives, etc. -```xml - - io.keploy - keploy-sdk - 2.0.0 - -``` - -For Spring Boot 2 or other `javax.servlet` applications, register the Keploy middleware in your main class: - -```java -import io.keploy.servlet.KeployMiddleware; -import org.springframework.boot.autoconfigure.SpringBootApplication; -import org.springframework.context.annotation.Import; - -@SpringBootApplication -@Import(KeployMiddleware.class) -public class App { -} -``` - -For Spring Boot 3, Jakarta EE applications, other frameworks, or custom launchers, start the agent during application startup: - -```java -import io.keploy.dedup.KeployDedupAgent; - -KeployDedupAgent.start(); -``` - -Java dynamic deduplication uses JaCoCo runtime coverage. The SDK reads coverage in-process via JaCoCo's runtime API (`org.jacoco.agent.rt.RT.getAgent()`), so attaching the JaCoCo Java agent is enough: no TCP server flags, no `--pass-through-ports`. - -```bash -java -javaagent:/path/to/org.jacoco.agent-runtime.jar -jar target/app.jar -``` +Copy both jars into `target/` during your Maven build (do not add the Keploy SDK as an application dependency, and do not import Keploy classes from your code). -If the in-process API is unavailable for some reason (for example, an isolated class loader), the SDK transparently falls back to JaCoCo's TCP server mode. To force the fallback, launch JaCoCo in `tcpserver` mode and tell Keploy to leave that port alone: - -```bash -java -javaagent:/path/to/org.jacoco.agent-runtime.jar=address=127.0.0.1,port=36320,output=tcpserver \ +```xml + + org.apache.maven.plugins + maven-dependency-plugin + 3.6.1 + + + copy-keploy-java-agent + package + copy + + + + io.keploy + keploy-sdk + 2.0.6 + ${project.build.directory} + keploy-sdk.jar + + + + + + copy-jacoco-agent + package + copy + + + + org.jacoco + org.jacoco.agent + 0.8.12 + runtime + jar + ${project.build.directory} + jacocoagent.jar + + + + + + +``` + +Run the app with both agents attached: + +```bash +java \ + -javaagent:target/keploy-sdk.jar \ + -javaagent:target/jacocoagent.jar \ -jar target/app.jar ``` -The default JaCoCo endpoint for the fallback is `127.0.0.1:36320`. You can override it with `KEPLOY_JACOCO_HOST` and `KEPLOY_JACOCO_PORT`, or with the JVM properties `keploy.jacoco.host` and `keploy.jacoco.port`. When using the fallback, add the JaCoCo port to `--pass-through-ports` so coverage-control traffic is not mocked. - #### 2. Build Configuration Build the application before running Keploy so the Java class files are available for coverage analysis: @@ -175,33 +186,27 @@ Build the application before running Keploy so the Java class files are availabl mvn clean package -DskipTests ``` -By default, the SDK scans `target/classes`, `build/classes/java/main`, and runtime classpath jars. For custom layouts or restricted Docker images, set `KEPLOY_JAVA_CLASS_DIRS` to the class directories or jars that should be analyzed. +By default, the SDK scans Maven `target/classes`, Gradle `build/classes/java/main`, executable jars, Spring Boot `BOOT-INF/classes`, servlet `WEB-INF/classes`, and runtime classpath archives. For custom layouts or restricted Docker images, set `KEPLOY_JAVA_CLASS_DIRS` to the class directories or archives that should be analyzed. For shaded or uber-jar Docker images, copy the compiled application classes into the image and point `KEPLOY_JAVA_CLASS_DIRS` at that directory so dependency classes do not participate in dedup signatures. #### 3. Dockerfile Configuration (Important for Docker Users) -When you use Docker or Docker Compose, make sure the final runtime image contains: - -- the runnable application jar, -- the JaCoCo runtime agent jar, -- the compiled classes or the fat jar that contains the application classes. - -For example: +When you use Docker or Docker Compose, copy four artifacts into the runtime image and attach both agents in the entrypoint: ```dockerfile -COPY target/app.jar /app/app.jar -COPY target/classes /app/target/classes -COPY jacocoagent.jar /app/jacocoagent.jar -``` +COPY target/app.jar /app/app.jar +COPY target/keploy-sdk.jar /app/keploy-sdk.jar +COPY target/jacocoagent.jar /app/jacocoagent.jar +COPY target/classes /app/classes -Then run the app with the JaCoCo agent attached: +ENV KEPLOY_JAVA_CLASS_DIRS=/app/classes -```bash -java -javaagent:/app/jacocoagent.jar -jar /app/app.jar +ENTRYPOINT ["java", \ + "-javaagent:/app/keploy-sdk.jar", \ + "-javaagent:/app/jacocoagent.jar", \ + "-jar", "/app/app.jar"] ``` -Keploy and the Java SDK exchange per-test coverage signals over `/tmp/coverage_control.sock` and `/tmp/coverage_data.sock`. For Docker and Docker Compose, Keploy injects a shared `keploy-sockets-vol:/tmp` mount into the application container and the Keploy agent container so both processes see the same socket paths. - -For hardened Docker runs, the Java dedup sample is validated with a non-root runtime user, a read-only root filesystem, dropped Linux capabilities, `no-new-privileges`, and Keploy's shared `/tmp` named volume for the Keploy control/data sockets and JaCoCo output. Do not add a conflicting `/tmp` bind mount or `tmpfs`; Keploy requires the injected shared `/tmp` volume to reach the Java SDK control socket. +Keploy injects a shared `keploy-sockets-vol:/tmp` mount into both the application container and the Keploy agent container at replay time so the dedup sockets are visible on both sides. Keep `/tmp` writable in the container; do not add a conflicting `/tmp` bind mount or `tmpfs`. Restricted containers (non-root user, read-only root filesystem, dropped capabilities) work as long as `/tmp` stays writable. #### 4. Run Deduplication @@ -214,22 +219,16 @@ keploy test -c "docker compose up" --container-name containerName --dedup --lang For Native, run: ```bash -keploy test -c "java -javaagent:/path/to/org.jacoco.agent-runtime.jar -jar target/app.jar" --dedup --language java +keploy test -c "java -javaagent:target/keploy-sdk.jar -javaagent:target/jacocoagent.jar -jar target/app.jar" --dedup --language java ``` -If the SDK falls back to the JaCoCo TCP server, also pass `--pass-through-ports ` so Keploy does not try to mock the coverage-control connection. - -This will generate a `dedupData.yaml` file. - -After this, run: +This produces `dedupData.yaml`, a per-testcase coverage map (`testSetID/testCaseID` to executed lines per source file) Keploy uses to compute redundancy. ```bash keploy dedup ``` -This command will create a `duplicates.yaml` file containing the test cases that dynamic deduplication marked as redundant. - -To apply the dynamic deduplication cleanup to the local Keploy test set, run: +This reads `dedupData.yaml` and writes `duplicates.yaml`, listing the testcases that dedup marked redundant (grouped by test-set). To remove those testcases from the local Keploy test set: ```bash keploy dedup --rm diff --git a/versioned_docs/version-4.0.0/running-keploy/keploy-karaf.md b/versioned_docs/version-4.0.0/running-keploy/keploy-karaf.md index a5316791a..8df01ecf4 100644 --- a/versioned_docs/version-4.0.0/running-keploy/keploy-karaf.md +++ b/versioned_docs/version-4.0.0/running-keploy/keploy-karaf.md @@ -35,13 +35,13 @@ curl --silent -O -L https://keploy.io/ent/install.sh && source install.sh Use `wget` to download the necessary JAR files: -- [io.keploy.agent-2.0.1.jar](https://keploy-enterprise.s3.us-west-2.amazonaws.com/agent-jars/io.keploy.agent-2.0.1.jar) +- [io.keploy.agent-2.0.2.jar](https://keploy-enterprise.s3.us-west-2.amazonaws.com/agent-jars/io.keploy.agent-2.0.2.jar) - [org.jacoco.agent-0.8.12-runtime.jar](https://keploy-enterprise.s3.us-west-2.amazonaws.com/agent-jars/org.jacoco.agent-0.8.12-runtime.jar) Run the following commands to download the files: ```bash -wget https://keploy-enterprise.s3.us-west-2.amazonaws.com/agent-jars/io.keploy.agent-2.0.1.jar +wget https://keploy-enterprise.s3.us-west-2.amazonaws.com/agent-jars/io.keploy.agent-2.0.2.jar wget https://keploy-enterprise.s3.us-west-2.amazonaws.com/agent-jars/org.jacoco.agent-0.8.12-runtime.jar ``` @@ -54,7 +54,7 @@ wget https://keploy-enterprise.s3.us-west-2.amazonaws.com/agent-jars/org.jacoco. 3. Add the paths of the downloaded agents under the `JAVA_OPTS` section. For example: ```bash - export JAVA_OPTS="-javaagent:/path/to/io.keploy.agent-2.0.1.jar" + export JAVA_OPTS="-javaagent:/path/to/io.keploy.agent-2.0.2.jar" export JAVA_OPTS="$JAVA_OPTS -javaagent:/path/to/org.jacoco.agent-0.8.12-runtime.jar=address=*,port=36320,destfile=jacoco-it.exec,output=tcpserver" ``` diff --git a/versioned_docs/version-4.0.0/server/sdk-installation/java.md b/versioned_docs/version-4.0.0/server/sdk-installation/java.md index 7a75d1ce0..d7da4884f 100644 --- a/versioned_docs/version-4.0.0/server/sdk-installation/java.md +++ b/versioned_docs/version-4.0.0/server/sdk-installation/java.md @@ -1,8 +1,8 @@ --- id: java -title: Java SDK for Dynamic Deduplication +title: Java Agent for Dynamic Deduplication sidebar_label: Java -description: "Configure the Keploy Java SDK for Enterprise dynamic deduplication with in-process JaCoCo coverage." +description: "Configure the Keploy Java agent for Enterprise dynamic deduplication with in-process JaCoCo coverage." tags: - java - coverage @@ -12,6 +12,7 @@ keywords: - JaCoCo - Maven - Spring Boot + - WAR - dynamic deduplication --- @@ -19,106 +20,110 @@ import ProductTier from '@site/src/components/ProductTier'; -The Java SDK is used for Enterprise dynamic deduplication during replay/test mode. It collects per-testcase Java coverage and sends it to Keploy Enterprise so duplicate testcases can be identified. +The Keploy Java SDK is used as a Java agent for Enterprise dynamic deduplication during replay/test mode. It collects per-testcase Java coverage and sends it to Keploy Enterprise so duplicate testcases can be identified. -The Java SDK does not record API traffic or mock dependencies. Record your Keploy tests separately, commit the generated test fixtures when you use them in CI, and run Java dedup during `keploy test --dedup`. +The Java agent does not record API traffic or mock dependencies. Record your Keploy tests separately, commit the generated test fixtures when you use them in CI, and run Java dedup during `keploy test --dedup`. + +Because the SDK is a Java agent, it is framework-agnostic. It can be attached to Spring Boot apps, Dropwizard/Jersey apps, plain executable jars, classpath-based apps, servlet/WAR-style archives, and other JVM frameworks as long as the application JVM also runs the JaCoCo agent. ## Requirements - Java 8, 17, or 21 -- `io.keploy:keploy-sdk` -- JaCoCo Java agent attached to the application JVM +- `io.keploy:keploy-sdk` `2.0.6` (or newer with Java-agent support) +- JaCoCo runtime agent (tested with `0.8.12`) - Keploy Enterprise with dynamic deduplication enabled -## Add the SDK +## Copy the Keploy SDK and JaCoCo Agents -Add the Keploy Java SDK dependency: +Both jars are runtime agents. Copy them into `target/` at build time. Do not add the Keploy SDK under `` and do not import Keploy classes from your code. ```xml - - io.keploy - keploy-sdk - 2.0.0 - -``` - -## Start the Dedup Agent - -For Spring Boot 2 or other `javax.servlet` applications, import the middleware: - -```java -import io.keploy.servlet.KeployMiddleware; -import org.springframework.context.annotation.Import; - -@Import(KeployMiddleware.class) -public class Application { -} + + org.apache.maven.plugins + maven-dependency-plugin + 3.6.1 + + + copy-keploy-java-agent + package + copy + + + + io.keploy + keploy-sdk + 2.0.6 + ${project.build.directory} + keploy-sdk.jar + + + + + + copy-jacoco-agent + package + copy + + + + org.jacoco + org.jacoco.agent + 0.8.12 + runtime + jar + ${project.build.directory} + jacocoagent.jar + + + + + + ``` -For Spring Boot 3, Jakarta EE applications, other frameworks, or custom launchers, start the agent during application startup: +## Run with Both Agents -```java -import io.keploy.dedup.KeployDedupAgent; - -KeployDedupAgent.start(); -``` - -## Run with the JaCoCo Java Agent - -The SDK reads coverage in-process via JaCoCo's runtime API (`org.jacoco.agent.rt.RT.getAgent()`), so attaching the JaCoCo agent is enough: no TCP server flags, no port choice. +Attach both agents in the application JVM. The Keploy agent reads coverage in-process via JaCoCo's runtime API, so order doesn't matter as long as both are present: ```bash -java -javaagent:/path/to/jacocoagent.jar -jar target/app.jar +java \ + -javaagent:target/keploy-sdk.jar \ + -javaagent:target/jacocoagent.jar \ + -jar target/app.jar ``` -If your compiled application classes are not under `target/classes` or `build/classes/java/main`, set `KEPLOY_JAVA_CLASS_DIRS`: +The SDK auto-detects application classes from Maven `target/classes`, Gradle `build/classes/java/main`, executable jars, Spring Boot `BOOT-INF/classes`, servlet `WEB-INF/classes`, and the runtime classpath. For custom layouts, point it at the right directory or archive: ```bash export KEPLOY_JAVA_CLASS_DIRS=/absolute/path/to/target/classes ``` -If the in-process API is unavailable in your environment, the SDK transparently falls back to JaCoCo's TCP server mode. To use the fallback explicitly, launch JaCoCo in `tcpserver` mode and configure `KEPLOY_JACOCO_HOST` / `KEPLOY_JACOCO_PORT` (defaults: `127.0.0.1:36320`): - -```bash -java -javaagent:/path/to/jacocoagent.jar=address=127.0.0.1,port=36320,output=tcpserver \ - -jar target/app.jar -``` - ## Replay with Dedup -Run Keploy in test mode with dynamic deduplication enabled: - ```bash keploy test \ - -c "java -javaagent:/path/to/jacocoagent.jar -jar target/app.jar" \ + -c "java -javaagent:target/keploy-sdk.jar -javaagent:target/jacocoagent.jar -jar target/app.jar" \ --dedup \ --language java ``` -If you are using the JaCoCo TCP fallback, also pass `--pass-through-ports ` so Keploy does not try to mock the coverage-control connection. - -After replay, run: - -```bash -keploy dedup -``` - -To apply the dynamic deduplication cleanup: +This produces `dedupData.yaml` (per-testcase coverage map). Then: ```bash -keploy dedup --rm +keploy dedup # writes duplicates.yaml grouping the redundant testcases per test-set +keploy dedup --rm # removes the redundant testcases from the local Keploy test set ``` -## Docker and Restricted Docker - -Java dedup uses two Unix sockets shared between Keploy Enterprise and the Java process: +## Docker -- `/tmp/coverage_control.sock` -- `/tmp/coverage_data.sock` +Keploy injects a shared `keploy-sockets-vol:/tmp` mount into the application container and the Keploy agent container at replay time, so the dedup sockets are visible on both sides. Keep `/tmp` writable; do not add a conflicting `/tmp` bind mount or `tmpfs`. Restricted containers (non-root user, read-only root filesystem, dropped capabilities) work as long as `/tmp` stays writable. -For Docker or Docker Compose runs, Keploy injects a shared `keploy-sockets-vol:/tmp` mount into the application container and the Keploy agent container so both processes use the same socket paths. Do not add a conflicting `/tmp` bind mount or `tmpfs`; keep `/tmp` writable even when the root filesystem is read-only. +For shaded or uber-jar images, also copy the compiled application classes into the runtime image and set `KEPLOY_JAVA_CLASS_DIRS` so dependency classes do not participate in dedup signatures: -For restricted containers, the application can run as a non-root user with dropped capabilities and `no-new-privileges` as long as the injected `/tmp` volume is shared and writable. If the SDK falls back to JaCoCo TCP mode, the JaCoCo TCP port must also be reachable from the Java process. +```dockerfile +COPY target/classes /app/classes +ENV KEPLOY_JAVA_CLASS_DIRS=/app/classes +``` ## CI Guidance