Help us profile important pipelines and improve if needed

[sayakpaul]

In #13356, we added a guide to comprehensively profile our pipelines with Claude. It, in turn, helped us get rid of issues that can get in the way of the benefits provided by torch.compile.

We cannot profile all our important pipelines alone, and this is where the community could really help us! In this thread, we want to gather interest from our community for such an initiative.

How to take part?

• Take a pipeline of your choice (start with T2I or T2V first, as it's simpler than others). Then discuss with us here to gauge feasibility (not all pipelines are equally widely used). Tag dg845 and sayakpaul while discussing.
• If there's mutual agreement, profile the pipeline following the steps outlined in the guide.
• If there are fixes to be made, open a PR to ship the fixes. Also, include the results before and after those fixes.

Cc: @dg845

[akshan-main]

Hey @sayakpaul @dg845, I'd like to help with profiling! Would QwenImage (or QwenImage Edit) be useful? Happy to start with which you think is highest priority.

[akshan-main]

I ran the QwenImage eager profiling on A100 80GB and found 17 cudaStreamSynchronize calls. 3 of them are ~76ms each during the denoising loop (beyond the modulate_index issue in #13404). Happy to dig into those if useful.

[varaprasadtarunkumar]

That's a really valuable finding! 3 cudaStreamSynchronize calls at ~76ms each
during the denoising loop would be a significant bottleneck — especially under
torch.compile where GPU queues are deeper and syncs stall much longer.

It would be great if you could dig into those. The approach that worked well
for Wan (in #13356) was:

1. Search for the sync in Perfetto using /cudaStreamSynchronize to jump to it
2. Click the slice — with with_stack=True, the bottom panel shows the exact
   Python call causing it
3. Common culprits: .item(), nonzero().item(), torch.tensor([scalar]),
   or torch.tensor(list_of_gpu_tensors)

Happy to help with the code analysis / fix once you've identified the source.
Looking forward to seeing what you find!

[akshan-main]

thanks but I'm happy to be working on it myself

[akshan-main]

Opened #13406 for QwenImage. Caches RoPE freq tensors on device to eliminate 3x ~76ms cudaStreamSynchronize per run. Also checked the other known issues from the profiling guide. The modulate_index issue from the profiling guide (#13356 known issue #2) only triggers when zero_cond_t=True, which the standard Qwen/Qwen-Image checkpoint doesn't use (zero_cond_t defaults to False). Confirmed via profiling, it doesn't show up in the trace, and no .item()/.cpu() calls were found in the denoising loop.