Skip to content

feat: Add Gemma3 LoRA SFT integration and programmatic weight mapping…#4004

Draft
RexBearIU wants to merge 1 commit into
mainfrom
jackyf/gemma3-lora-e2e
Draft

feat: Add Gemma3 LoRA SFT integration and programmatic weight mapping…#4004
RexBearIU wants to merge 1 commit into
mainfrom
jackyf/gemma3-lora-e2e

Conversation

@RexBearIU
Copy link
Copy Markdown
Collaborator

… end-to-end support

Description

This PR adds end-to-end integration and weight-mapping support for Gemma3 LoRA SFT and vLLM decoding. Previously, standalone offline decoding ( vllm_decode.py ) and the online Tunix server ( adapter.py ) lacked the capability
to
map and load LoRA-adapted weights specifically for Gemma3, causing failures during parameter restoration and serving.

Key Changes:

• Gemma3 Weight Mapping: Added gemma3.py mapping to translate Gemma3 layers programmatically between MaxText and HuggingFace/vLLM structures.
• Generalized Restore API: Updated restore_lora_from_path to handle raw nnx.Module models (used in vLLM adapters) in addition to trainer states.
• vLLM Integration: Hooked both online and offline vLLM decode paths to restore LoRA parameters when lora.enable_lora=True .

Tests

1. Unit Tests

python3 -m pytest tests/post_training/unit/lora_utils_test.py

• Result: 10 passed

2. End-to-End Tests

Tested the end-to-end flow using the newly converted checkpoint:

  1. Convert Checkpoint:
    export HF_TOKEN=
    export RUN_ID=$(date +%Y-%m-%d-%H-%M)
    bash tests/end_to_end/tpu/gemma3/4b/test_gemma3_to_mt.sh $RUN_ID

  2. Run LoRA & Decode (starts from the newly converted checkpoint above):
    bash tests/end_to_end/tpu/gemma3/4b/test_gemma3_lora.sh $RUN_ID

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • I have necessary comments in my code, particularly in hard-to-understand areas.
  • I have run end-to-end tests tests and provided workload links above if applicable.
  • I have made or will make corresponding changes to the doc if needed, including adding new documentation pages to the relevant Table of Contents (toctree directive) as explained in our documentation.

@codecov
Copy link
Copy Markdown

codecov Bot commented May 28, 2026

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant