Skip to content

Add unit tests for Gemma 4 LoRA targeting and NNX caching#4265

Open
RexBearIU wants to merge 2 commits into
mainfrom
jackyf/gemma4-lora-tests
Open

Add unit tests for Gemma 4 LoRA targeting and NNX caching#4265
RexBearIU wants to merge 2 commits into
mainfrom
jackyf/gemma4-lora-tests

Conversation

@RexBearIU

@RexBearIU RexBearIU commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Description

This follow-up PR adds robust, CPU-friendly unit tests for the Gemma 4 LoRA support introduced in #3969.

It implements the precise test coverage requested by reviewers to ensure:

  1. LoRA Targeting Integrity: Verifies that the regex pattern defined in lora_module_path.yml for gemma4 correctly resolves and matches the expected module paths (including MoE paths).
  2. MoE NNX Graph Caching Safety: Verifies that the NNX graph caching is dynamically disabled for MoE models (expert count > 1) to allow correct metadata synchronization, and remains enabled for standard Dense architectures (expert count <= 1).

Why this is a good solution:

These lightweight unit tests execute in milliseconds on standard CPUs, fully guarding these critical codepaths against future regressions without adding TPU dependency or CI execution overhead.

Related PR: #3969

Tests

To verify these changes, run the following pytest commands:

# 1. Run the SFT trainer model caching tests
pytest tests/post_training/unit/train_sft_test.py

# 2. Run the LoRA regex path matching tests
pytest tests/post_training/unit/lora_utils_test.py

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@codecov

codecov Bot commented Jun 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant