Skip to content

feat: Add LangChain middleware integration#69

Merged
rapids-bot[bot] merged 110 commits into
NVIDIA:mainfrom
dagardner-nv:david-lcda
May 11, 2026
Merged

feat: Add LangChain middleware integration#69
rapids-bot[bot] merged 110 commits into
NVIDIA:mainfrom
dagardner-nv:david-lcda

Conversation

@dagardner-nv
Copy link
Copy Markdown
Contributor

@dagardner-nv dagardner-nv commented May 7, 2026

Overview

  • Adds a LangChain integration package located in python/nemo_flow/integrations/langchain
  • Adds a nemo_flow.utils.run_sync method, I figured other integrations will likely need this.
  • Defined nemo_flow as known-first party in the ruff settings, this triggered several formatting changes beyond the scope of this PR
  • Work-around for wasm-pack GitHub repo migration causing the package to fail to be installed with taiki-e/install-action
  • I confirm this contribution is my own work, or I have the right to submit it under this project's license.
  • I searched existing issues and open pull requests, and this does not duplicate existing work.

Details

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • Closes #

Summary by CodeRabbit

  • New Features

    • LangChain integration: middleware and callbacks to route model and tool calls through agents
    • Sync-run utility to execute async coroutines from sync code while preserving context
  • Documentation

    • Integration guide and usage examples added; integrations overview restructured
  • Tests

    • New unit and integration tests for LangChain integration and the sync-run utility
  • Chores

    • CI updated to run Python integration tests, cache integration deps, build integration wheels, and ignore editor/third-party paths

Signed-off-by: David Gardner <dagardner@nvidia.com>
…his version requires newer versions of langchain

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…ely this will be needed elsewhere, remove unused code that attempts to lazy-import nemo_flow

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
@dagardner-nv dagardner-nv self-assigned this May 7, 2026
@dagardner-nv dagardner-nv added feat lang:python PR changes/introduces Python code labels May 7, 2026
@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented May 7, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

This pull request introduces a complete LangChain integration for NeMo Flow. It adds a new Python package (langchain-nemo-flow) with serialization utilities, a callback handler bridging LangChain chain lifecycle events to NeMo Flow scopes, a middleware routing model and tool calls through NeMo Flow, a sync/async execution bridge utility, comprehensive test coverage, CI/CD wiring, and documentation updates.

Changes

LangChain NeMo Flow Integration

Layer / File(s) Summary
Package & Build Configuration
integrations/langchain/pyproject.toml
Declares langchain-nemo-flow package with dynamic SCM versioning, runtime dependencies (langchain >=1.2.11,<2.0.0, langchain-core, nemo-flow), test group with pytest, and tooling config (ruff, uv).
Package Initialization
integrations/langchain/src/langchain_nemo_flow/__init__.py
Establishes package surface; exports NemoFlowCallbackHandler and NemoFlowMiddleware via __all__.
Serialization & Codec Inference
integrations/langchain/src/langchain_nemo_flow/_serialization.py
Implements LangChain↔NeMo Flow serialization: model-name extraction, codec inference (Anthropic vs OpenAI with use_responses_api support), ModelRequest↔payload conversion with system-message splitting, ModelResponse JSON wrapping under dedicated key with fallback decoding.
Callback Handler
integrations/langchain/src/langchain_nemo_flow/callbacks.py
Implements NemoFlowCallbackHandler subclassing BaseCallbackHandler to bridge LangChain chain lifecycle (on_chain_start/on_chain_end/on_chain_error) to NeMo Flow scope push/pop with run-id, parent-id, metadata tracking, and error suppression.
Middleware & Call Routing
integrations/langchain/src/langchain_nemo_flow/middleware.py
Implements NemoFlowMiddleware routing sync/async model calls (via wrap_model_call/awrap_model_call) and tool calls (via wrap_tool_call/awrap_tool_call) through NeMo Flow; prepares LLMRequest, infers codec, executes, and converts responses; forwards NeMo-overridden tool arguments via request.override(...).
Sync/Async Bridge Utility
python/nemo_flow/utils.py
Adds run_sync(coro) function to execute awaitables from sync contexts: detects running event loop, uses asyncio.run() when none, offloads to ThreadPoolExecutor with contextvars and NeMo scope-stack propagation when loop is running.
Callback Handler Tests
integrations/langchain/tests/test_callbacks.py
Unit tests for NemoFlowCallbackHandler validating scope lifecycle, metadata/naming, parent-child wiring, graceful no-ops when nemo_flow unavailable, and error suppression.
Middleware Tests
integrations/langchain/tests/test_middleware.py
Tests NemoFlowMiddleware via RecordingMiddleware subclass verifying sync/async model-call routing, request mutation propagation, and correct return values; validates infer_codec_from_model codec selection for Anthropic/OpenAI with use_responses_api support.
Run Sync Concurrency Tests
python/tests/test_utils.py
Verifies run_sync supports concurrent calls from multiple threads with running event loops and preserves NeMo Flow scope-stack identity.
CI/CD Infrastructure
.github/ci-path-filters.yml, .github/workflows/ci_python.yml, justfile
Adds integrations/langchain/** to Python path filter; caches integrations/langchain/uv.lock in Test/Package jobs; adds test-python-integration justfile target; runs integration tests in CI Test job; builds LangChain wheel in Package job (amd64); integrates into test-all.
Type Checking & Documentation
pyproject.toml, .gitignore, integrations/langchain/README.md, AGENTS.md, README.md, python/nemo_flow/__init__.py
Expands type-check allowed-unresolved-imports for langchain/langgraph patterns; adds .gitignore entries for .vscode and third_party/deepagents/; introduces integrations/langchain README with setup and usage examples; documents public API-based integrations in project overview; documents intentional non-import of utils.py to avoid executor creation.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 38.60% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning PR description is incomplete. Required sections 'Details' and 'Where should the reviewer start?' are empty or contain only template comments. Complete the 'Details' section with a comprehensive summary of changes and fill in 'Where should the reviewer start?' to guide reviewers to critical files or decisions.
✅ Passed checks (3 passed)
Check name Status Explanation
Title check ✅ Passed The title follows Conventional Commits format with type 'feat' and provides a concise, imperative summary of the main change under 72 characters.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

  • Generate code and open pull requests
  • Plan features and break down work
  • Investigate incidents and troubleshoot customer tickets together
  • Automate recurring tasks and respond to alerts with triggers
  • Summarize progress and report instantly

Built for teams:

  • Shared memory across your entire org—no repeating context
  • Per-thread sandboxes to safely plan and execute work
  • Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started


Comment @coderabbitai help to get the list of available commands and usage tips.

@dagardner-nv dagardner-nv requested a review from willkill07 May 11, 2026 15:26
willkill07
willkill07 previously approved these changes May 11, 2026
Comment thread pyproject.toml Outdated
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
… on chain end

Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
@dagardner-nv dagardner-nv requested a review from willkill07 May 11, 2026 17:47
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
…/install-action not being updated yet

Signed-off-by: David Gardner <dagardner@nvidia.com>
Salonijain27
Salonijain27 previously approved these changes May 11, 2026
Copy link
Copy Markdown
Collaborator

@Salonijain27 Salonijain27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved from a dependency point of view

willkill07
willkill07 previously approved these changes May 11, 2026
Signed-off-by: David Gardner <dagardner@nvidia.com>
@dagardner-nv dagardner-nv dismissed stale reviews from willkill07 and Salonijain27 via 1155786 May 11, 2026 19:55
willkill07
willkill07 previously approved these changes May 11, 2026
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Signed-off-by: David Gardner <dagardner@nvidia.com>
Copy link
Copy Markdown
Collaborator

@Salonijain27 Salonijain27 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved from a dependency point of view

@dagardner-nv
Copy link
Copy Markdown
Contributor Author

/merge

@rapids-bot rapids-bot Bot merged commit 3bacc4e into NVIDIA:main May 11, 2026
65 checks passed
@willkill07 willkill07 added the Feature a new feature label May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Feature a new feature lang:python PR changes/introduces Python code size:XXL PR is very large

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants