Skip to content

Add examples on how to profile a pipeline#13356

Open
sayakpaul wants to merge 17 commits intomainfrom
profiling-workflow
Open

Add examples on how to profile a pipeline#13356
sayakpaul wants to merge 17 commits intomainfrom
profiling-workflow

Conversation

@sayakpaul
Copy link
Copy Markdown
Member

@sayakpaul sayakpaul commented Mar 28, 2026

What does this PR do?

TL;DR: Adds a guide on how to profile a pipeline and fix issues like CPU overhead, CPU<->GPU syncs, etc.

Motivation

Since we provide first-class torch.compile support, it's important that our pipelines are set up for optimal success with it. This includes spotting any obvious issues that plague the torch.compile performance -- CPU overhead, CPU<->GPU syncs, graphbreaks, kernel launch delays, etc.

The best way to spot these bugs is to profile a pipeline, as it gives a granular measurement of where the GPU is spending time and if it is doing so in an expected manner. We can then uncover any unexpected issues and eventually fix them.

Workflow

The README.md added in the PR has all the descriptions, but in summary:

  • take a popular pipeline like Flux/Flux2/QwenImage/Wan/LTX2
  • run the profile with 2 inference steps
  • load the trace on Perfetto
  • spot the potential suspects
  • piggy that back to Claude along with the trace
    • ask it to attempt a fix
    • review the fix
    • compare the results

With this Workflow, I was able to fix some issues in the Flux2 Klein pipeline and the Wan pipeline. All changes look quite harmless to me.

Plan

Not only is it helpful to profile pipelines to get a ceiling on performance, but the community could also help us improve our pipelines should this workflow prove to be useful.

Note to reviewers

Please review the changes in src/diffusers/*. And you can skip straight to the "Afterwards" section in the README.md document.

The tutorial is currently available here. Some inline comments.

@sayakpaul sayakpaul requested review from DN6, dg845 and stevhliu March 28, 2026 04:41
@HuggingFaceDocBuilderDev
Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants