java: cuopt-java skeleton + CI + LP/MILP/QP public API#1192
Draft
rgsl888prabhu wants to merge 7 commits into
Draft
java: cuopt-java skeleton + CI + LP/MILP/QP public API#1192rgsl888prabhu wants to merge 7 commits into
rgsl888prabhu wants to merge 7 commits into
Conversation
Adds the initial directory structure for a Java interface to cuOpt under java/, modeled after cuvs-java's architecture (Project Panama / FFM, Java 21 base API + Java 22 multi-release JAR for FFM impl). The first end-to-end demo is Solver.getVersion(), which exercises the full FFM bridge: Java 21 public API -> ServiceLoader SPI -> Java 22 FFM implementation -> jextract panama bindings -> libcuopt.so. Full LP/MILP/QP support lands in subsequent PRs. Build setup encodes three lessons from cuvs-java: - Java 21 base API + Java 22 multi-release JAR for FFM (cuvs Issue NVIDIA#1066) - maven-compiler-plugin 3.11.0+ with serial phase bindings (#1293) - Spotless under <plugins> only when added (NVIDIA#1090) Per-folder READMEs explain the role of each layer in the architecture. Adds build_java group and test_java file definition to dependencies.yaml so conda-forge openjdk + maven dependencies flow into developer envs. Toolchain prerequisites for local build (not auto-installed): - conda install -c conda-forge openjdk=22 maven - jextract from https://jdk.java.net/jextract/
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
Builds on the prior commit by: 1. Renaming the dependency group `build_java` -> `java` to match cuvs convention, and adding it to the `all` file's includes so all conda/environments/all_cuda-*_arch-*.yaml files now ship with openjdk=22.* and maven>=3.9.6. 2. Adding ci/build_java.sh and ci/test_java.sh, mirroring cuvs's pattern. ci/test_java.sh is a thin wrapper that delegates to ci/build_java.sh --run-java-tests. Splitting build/test into separate jobs is deferred (matches cuvs Issue NVIDIA#868) — requires shared-workflow surface for cross-job artifact passing. 3. Adding .github/workflows/pr.yaml jobs: - test_java paths group in changed-files - conda-java-build-and-tests-matrix (compute-matrix; amd64-only) - conda-java-build-and-tests (custom-job; gpu-l4-latest-1) - Adds conda-java-build-and-tests to pr-builder.needs 4. Adding java-build job to .github/workflows/build.yaml so main / release-branch / tag pushes produce cuopt-java JAR artifacts. CI will fail on this PR until two follow-up steps: - jextract is added to the rapidsai/ci-conda CI image (or pulled in ci/build_java.sh) - Generated panama bindings under java/cuopt-java/src/main/java22/com/nvidia/cuopt/internal/panama/ are committed (initial bootstrap; runs on a workstation with JDK 22 + jextract installed locally)
Builds out the full public Java API for LP, MILP, and QP on top of the skeleton + CI from the prior commits. Public API surface (Java 21, src/main/java/): - Problem: build problem (addVariable, addConstraint, setObjective), solve(), and post-solve accessors (status, objectiveValue, solveTime, lpStats, milpStats). Owns native handle + all solution arrays. - Variable / Constraint: immutable handles that delegate value() / reducedCost() / dualValue() / slack() back to the owning Problem. - LinearExpr / QuadraticExpr: mutable expression builders supporting both addTerm(coeff, var) (chainable single-term) and addTerms(double[], Variable[]) (bulk). Coefficients accumulate. - SolverSettings: typed setters (setTimeLimit, setOptimalityTolerance, setMethod, setRelativeMipGap, ...) + generic setParameter(name, value) escape hatch for forward compatibility. AutoCloseable. - DataModel: low-level escape hatch matching cuOpt Python's DataModel. CSR setters return this for chaining. - Enums: VType (CONTINUOUS/INTEGER/BINARY), CType (LE/GE/EQ), Sense, SolverMethod, PdlpSolverMode, TerminationStatus, ErrorStatus, ProblemCategory. - Records: LpStats, MilpStats. - CuOpt constants holder: static-importable INF, MINIMIZE, LESS_EQUAL, and parameter-name string constants (TIME_LIMIT, METHOD, etc.). FFM implementation (Java 22, src/main/java22/internal/): - CuOptProviderImpl: marshals Java arrays -> MemorySegment, calls jextract-generated cuopt_c_h methods, extracts solution data into Java arrays before freeing the native handle. - CsrBuilder: translates List<LinearExpr> constraint expressions into CSR (row-offsets, column-indices, values) sorted by column. - Native handles cross the SPI boundary as long (raw addresses); ProblemImpl reconstructs MemorySegment from the address on each call. Tests: - LpSolverTest: simple LP, diet LP, introspection. - MilpSolverTest: 0-1 knapsack. - QpSolverTest: mirrors cuOpt Python test_qp.py::test_solver. - LinearExprTest: addTerm/addTerms interchangeability, coefficient accumulation, length-mismatch and mixed-problem error paths. This commit completes the full LP/MILP/QP public API. Tests cannot run until: (1) JDK 22 + jextract installed on a workstation, (2) generated panama bindings committed under java/cuopt-java/src/main/java22/com/nvidia/cuopt/internal/panama/. See temp/cuopt-java-RESUME.md for the verify-locally checklist.
rgsl888prabhu
changed the title
jextract for JDK 22 is not on conda-forge. Match cuvs's approach of auto-downloading it into java/panama-bindings/jextract-22/ on first run of generate-bindings.sh. - panama-bindings/generate-bindings.sh now downloads openjdk-22-jextract+6-47 from download.java.net if jextract isn't already on PATH. Subsequent builds reuse the local copy. - panama-bindings/.gitignore excludes jextract-22/ and the .tar.gz download artifact from git. - build.sh drops the hard error on missing jextract; the auto-download in generate-bindings.sh handles it. - README and dependencies.yaml comments updated to reflect the auto-download. CI implication: the first 'ci/build_java.sh' run will spend ~10 seconds downloading jextract (62 MB). Subsequent runs cache it in the conda env's workspace if persistent, or re-download otherwise. A follow-up could preinstall jextract in the rapidsai/ci-conda image to skip the download in CI.
Five fixes to get the build green end-to-end:
1. jextract: pass --header-class-name cuopt_c_h so the generated header
class is named what CuOptProviderImpl imports (default would be
headers_h after the umbrella include filename). Also add
--use-system-load-library so dlopen uses java.library.path.
2. Make package-private accessors PUBLIC across the
internal/-vs-linear_programming/ package boundary:
- Problem: linearObjective, quadraticObjective, objectiveSense,
objectiveOffset, constraintExpressions
- QuadraticExpr: quadVar1, quadVar2, quadCoeff
- SolverSettings: nativeHandle
- DataModel: all ~15 internal getters
Java's package-visibility doesn't cross the package boundary the
FFM impl needs to traverse. Marked with javadoc that these are
for internal FFM use only.
3. Multi-Release JAR semantics only apply to classes loaded from an
actual JAR file, NOT exploded target/classes/. So tests that touch
the FFM impl can't run via maven-surefire-plugin against
target/classes/. Moved them to *IT.java (failsafe convention) and
configured maven-failsafe-plugin with classesDirectory pointing at
the packaged JAR. Failsafe runs after package, so the JAR exists.
LinearExprTest stays as a surefire test (pure Java, no FFM impl).
SolverIT / LpSolverIT / MilpSolverIT / QpSolverIT run via failsafe.
4. build.sh: replace CMAKE_PREFIX_PATH with CUOPT_LIB_DIR. The previous
default fell through to the user's conda env path (set by conda
activate), missing the actual libcuopt.so in cpp/build/. Now we
probe cpp/build/ and $CONDA_PREFIX/lib in that order.
5. Commit the jextract-generated panama bindings (cuopt_c_h.java,
__fsid_t.java, cuOptMIPGetSolutionCallback.java,
cuOptMIPSetSolutionCallback.java). These regenerate on every build
and the drift gate enforces consistency from this commit forward.
Verified locally: full pipeline runs clean with
./java/build.sh (no SKIP_DRIFT_CHECK needed after this commit)
Test results:
Surefire: 4/4 LinearExprTest pass
Failsafe: 6/6 integration tests pass against libcuopt.so
(LP, MILP, QP all solve to feasibility/optimality)
Plumbs arm64 through the build and CI pipelines: - supported-platforms.properties: flip linux-aarch64=true. - generate-bindings.sh: detect uname -m and select matching jextract tarball (linux-aarch64 confirmed available on download.java.net) and CUDA include subdir (targets/aarch64-linux/include). - java/build.sh + ci/build_java.sh: add UNIT_TESTS_ONLY mode for the arm64 CPU runner, which has no GPU and so must skip integration tests. - ci/build_java.sh: drop the stale fail-fast jextract-availability check; auto-download in generate-bindings.sh now handles it. - pr.yaml: add conda-java-build-arm64-matrix + conda-java-build-arm64 jobs on linux-arm64-cpu16, building the JAR and running unit tests. amd64 keeps full IT on gpu-l4-latest-1. - build.yaml: add java-build-arm64 sibling job for main/release/tag. IT stays on amd64 until an arm64 GPU runner is wired into rapidsai/shared-workflows; flip --unit-tests-only to --run-java-tests and update the node_type when that lands.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Initial Java interface for cuOpt covering LP, MILP, and QP, modeled after
cuvs-java's architecture (Project Panama / FFM, Java 21 base API + Java 22 multi-release JAR for FFM impl).Three commits build this up incrementally:
java/,pom.xmlwith Java 21 + Java 22 MR-JAR config,Solver.getVersion()end-to-end demo.ci/build_java.sh,ci/test_java.sh, jobs inpr.yamlandbuild.yaml,dependencies.yamlupdates soconda/environments/all_*.yamlship with JDK 22 + Maven.Architecture (5 layers)
Public API compiles to Java 21 release; runtime requires Java 22+.
Public API surface
Problem: build problem,solve(), post-solve accessors (status(),objectiveValue(),solveTime(),lpStats(),milpStats()). Owns all solution arrays.Variable,Constraint: immutable handles that delegatevalue()/dualValue()/slack()back to the owning Problem.LinearExpr,QuadraticExpr: mutable expression builders. BothaddTerm(coeff, var)(chainable single) andaddTerms(double[], Variable[])(bulk). Coefficients accumulate.SolverSettings: typed setters + genericsetParameter(name, value)escape hatch.AutoCloseable.DataModel: low-level escape hatch matching cuOpt Python'sDataModel.VType,CType,Sense,SolverMethod,PdlpSolverMode,TerminationStatus,ErrorStatus,ProblemCategory.LpStats,MilpStats.CuOptconstants holder for static import (INF,MINIMIZE,LESS_EQUAL, etc.).Design inputs
Toolchain prerequisites (not auto-installed)
conda install -c conda-forge openjdk=22 maven # Plus jextract from https://jdk.java.net/jextract/ on PATHTest plan
./java/build.shruns cleanly with JDK 22 + Maven 3.9.6 + jextractSKIP_DRIFT_CHECK=trueto generate panama bindings, then commit themSolver.getVersion()returns version stringLpSolverTestsolves a simple LP to OPTIMALMilpSolverTestsolves a binary knapsack withINTEGERvariablesQpSolverTestsolves a 2-variable QPLinearExprTestvalidates addTerm/addTerms interchangeability + error pathsStatus: DRAFT
Posted as draft. Two follow-ups needed before this can be marked ready:
./java/build.shlocally to generatejava/cuopt-java/src/main/java22/com/nvidia/cuopt/internal/panama/*.java. Commit them so the build doesn't fail oncannot find symbol cuopt_c_h.rapidsai/ci-condarequesting jextract for JDK 22.Out of scope (deferred follow-ups)
java/examples/lp/,qp/,milp/)check-c-abiupstream gate forcuopt_c.hbuild.sh