Skip to content

[Modular] add explicit workflow support #13028

Merged
yiyixuxu merged 66 commits intomainfrom
modular-workflow
Feb 15, 2026
Merged

[Modular] add explicit workflow support #13028
yiyixuxu merged 66 commits intomainfrom
modular-workflow

Conversation

@yiyixuxu
Copy link
Collaborator

@yiyixuxu yiyixuxu commented Jan 25, 2026

This PR adds support for explicitly defining and retrieving workflows (currently only added to SequentialPipelineBlocks).

Previously, to make sure users understand what workflows a pipeline supports, we:

WIth this PR, workflows are now:

  • explicitly defined via _workflow_map on the auto blocks class, see qwen example
class QwenImageAutoBlocks(SequentialPipelineBlocks):
    ...
    _workflow_map = {
        "text2image": {"prompt": True},
        "image2image": {"prompt": True, "image": True},
        "inpainting": {"prompt": True, "mask_image": True, "image": True},
        "controlnet_text2image": {"prompt": True, "control_image": True},
        "controlnet_image2image": {"prompt": True, "image": True, "control_image": True},
        "controlnet_inpainting": {"prompt": True, "mask_image": True, "image": True, "control_image": True},
    }
   ...
  • Auto-documented in the class docstring (in qwen example with the _workflow_map we defined above, the auto docstring look like this)
# auto_docstring
class QwenImageAutoBlocks(SequentialPipelineBlocks):
    """
    Auto Modular pipeline for text-to-image, image-to-image, inpainting, and controlnet tasks using QwenImage.

      Supported workflows:
        - `text2image`: requires `prompt`
        - `image2image`: requires `prompt`, `image`
        - `inpainting`: requires `prompt`, `mask_image`, `image`
        - `controlnet_text2image`: requires `prompt`, `control_image`
        - `controlnet_image2image`: requires `prompt`, `image`, `control_image`
        - `controlnet_inpainting`: requires `prompt`, `mask_image`, `image`, `control_image`

      Components:
          ...

      Inputs:
          ...

      Outputs:
          ...
    """
  • Programmatically accessible via workflow_names and get_workflow()
    for example, to get the "controlnet_text2image" workflow from qwen
from diffusers import ModularPipeline
pipe = ModularPipeline.from_pretrained("Qwen/Qwen-Image")
text2image_blocks = pipe.blocks.get_workflow("controlnet_image2image"")
print(text2image_blocks.sub_blocks)
InsertableDict([
  0: ('text_encoder', <obj 'diffusers.modular_pipelines.qwenimage.encoders.QwenImageTextEncoderStep'>),
  1: ('controlnet_vae_encoder', <obj 'diffusers.modular_pipelines.qwenimage.encoders.QwenImageControlNetVaeEncoderStep'>),
  2: ('denoise.input', <obj 'diffusers.modular_pipelines.qwenimage.inputs.QwenImageTextInputsStep'>),
  3: ('denoise.controlnet_input', <obj 'diffusers.modular_pipelines.qwenimage.inputs.QwenImageControlNetInputsStep'>),
  4: ('denoise.prepare_latents', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImagePrepareLatentsStep'>),
  5: ('denoise.set_timesteps', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImageSetTimestepsStep'>),
  6: ('denoise.prepare_rope_inputs', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImageRoPEInputsStep'>),
  7: ('denoise.controlnet_before_denoise', <obj 'diffusers.modular_pipelines.qwenimage.before_denoise.QwenImageControlNetBeforeDenoiserStep'>),
  8: ('denoise.controlnet_denoise', <obj 'diffusers.modular_pipelines.qwenimage.denoise.QwenImageControlNetDenoiseStep'>),
  9: ('denoise.after_denoise', <obj 'diffusers.modular_pipelines.qwenimage.decoders.QwenImageAfterDenoiseStep'>),
  10: ('decode.decode', <obj 'diffusers.modular_pipelines.qwenimage.decoders.QwenImageDecoderStep'>),
  11: ('decode.postprocess', <obj 'diffusers.modular_pipelines.qwenimage.decoders.QwenImageProcessImagesOutputStep'>)
])

I will add a test suit for this too!

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.


Inputs:
image (`None`, *optional*):
TODO: Add description.
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: update InputParams/OutputParams to remove the TODO from doc strings

@@ -341,6 +349,34 @@ def test_save_from_pretrained(self):

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: add a test the output from workflow blocks against the auto blocks

block_classes = AUTO_BLOCKS.values()
block_names = AUTO_BLOCKS.keys()

_workflow_map = {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TODO: support set

@yiyixuxu yiyixuxu merged commit c919ec0 into main Feb 15, 2026
16 checks passed
@yiyixuxu yiyixuxu deleted the modular-workflow branch February 15, 2026 02:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants