Skip to content

[codex] Enable Kunlun Qwen2.5 inference support#1369

Draft
zhangyue207 wants to merge 1 commit into
mainfrom
codex/kunlun-qwen25-7b-inference
Draft

[codex] Enable Kunlun Qwen2.5 inference support#1369
zhangyue207 wants to merge 1 commit into
mainfrom
codex/kunlun-qwen25-7b-inference

Conversation

@zhangyue207

Copy link
Copy Markdown
Collaborator

Summary

This PR contains the InfiniCore-side fixes needed to run Qwen2.5-7B-Instruct through InfiniLM on a single Kunlun XPU.

Changes

  • Make Moore mate flash-attn imports optional in python/infinicore/__init__.py so non-Moore backends can import infinicore when the optional Moore Python bridge is unavailable.
  • Update generic linear to materialize the transposed weight as contiguous before GEMM and apply bias after GEMM with add_.
  • Add Kunlun RoPE support for I64 position IDs.

Why

During Kunlun bring-up, importing infinicore could fail because of unrelated optional Moore dependencies. Qwen2.5 inference also needed robust generic linear behavior and position-id dtype compatibility for RoPE on Kunlun.

Validation

Validated together with the matching InfiniLM changes in Docker image fixing:v1:

  • Built and installed infinicore_cpp_api and _infinicore.
  • Ran Qwen2.5-7B-Instruct single-card inference on Kunlun FP32.
  • Verified greedy first token matches HF/CPU reference: token 114602, decoded as 昆仑.
  • Verified max-new-tokens=2 output: 昆仑芯.

Runtime note: the verified path uses DISABLE_XPYTORCH=1 to avoid loading an incompatible XPU runtime from torch_xmlir.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants