Skip to content

Move manifest list creation from publish stage to post-build stage #2002

@lbussell

Description

@lbussell

Problem

Manifest lists are currently created and pushed during the Publish stage by the publishManifest command. This causes several problems:

  1. Manifest lists cannot be signed. The Sign stage runs before Publish, so manifest list digests don't exist yet at signing time. See Manifest lists are not signed #2001.
  2. The Publish stage creates new digests. Ideally, the Publish stage should only copy and publish artifacts that were already built and verified - not create new ones.
  3. Signing manifest lists would require the Publish stage to sign. If we tried to sign manifest lists without moving their creation earlier, we'd need signing infrastructure in the Publish stage, adding complexity and deviating from the clean separation of concerns between stages.

Current Pipeline Flow

  1. Build - creates platform images, produces per-job image-info fragments
  2. Post_Build - merges image-info fragments into a single file
  3. Sign - signs all digests in image-info (platform images only, no manifest lists yet)
  4. Test - tests images
  5. Publish - copies images, creates manifest lists, pushes manifests, records manifest digests

Proposed Change

Move manifest list creation from the Publish stage to the Post_Build stage:

  1. Build - creates platform images, produces per-job image-info fragments
  2. Post_Build - merges image-info, creates manifest lists, records manifest digests in image-info
  3. Sign - signs all digests in image-info (platform images AND manifest lists)
  4. Test - tests images
  5. Publish - copies images, pushes already-created manifest lists (no new digests created)

Benefits

  • Enables manifest list signing - manifest list digests are present in image-info.json before the Sign stage
  • Cleaner Publish stage - Publish only copies and pushes existing artifacts, no new content is created
  • Simpler signing model - all signing happens in one place (the Sign stage), no need for signing infrastructure in Publish

Considerations

  • The publishManifest command currently both creates manifest lists and pushes them to the registry. These two operations will need to be separated - creation in Post_Build, pushing in Publish.
  • Manifest lists created in Post_Build will reference platform images in the build registry. During Publish, after images are copied to the publish registry, manifest lists may need to be re-tagged or re-associated with the publish registry.
  • The SaveTagInfoToImageInfoFileAsync logic in PublishManifestCommand (which records manifest digests back to image-info.json) will need to move or be refactored to run at the appropriate stage.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    Status

    Backlog

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions