Skip to content

Add cache validation#625

Open
inteon wants to merge 1 commit into
mainfrom
add_cache_validation
Open

Add cache validation#625
inteon wants to merge 1 commit into
mainfrom
add_cache_validation

Conversation

@inteon
Copy link
Copy Markdown
Member

@inteon inteon commented May 29, 2026

Add verify-cache command that validates the cache, preventing cache poisoning.

Signed-off-by: Tim Ramlot <42113979+inteon@users.noreply.github.com>
@cert-manager-prow cert-manager-prow Bot added dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels May 29, 2026
@cert-manager-prow
Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: inteon

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@cert-manager-prow cert-manager-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label May 29, 2026
@cert-manager cert-manager deleted a comment from cert-manager-prow Bot May 29, 2026
Comment thread modules/tools/00_mod.mk
## restore in CI environments where the cache write-side is not fully trusted
## (e.g. node-local hostPath shared with low-trust presubmit jobs).
## @category [shared] Tools
verify-cache:
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it mean we need to run make verify-cache in every single Prow job? I imagine that's not a big downside, but I wish it was a bit more automatic

Copy link
Copy Markdown
Member Author

@inteon inteon May 29, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I'll update the prowjob definitions to first call make verify-cache.

@maelvls
Copy link
Copy Markdown
Member

maelvls commented May 29, 2026

at first, I didn't believe it as I thought that the inode modification timestamp mismatch would prevent the attack. And I thought that the report was wrong as it mentioned poisonning _bin/downloaded/kind which doesn't actually exists...

I realized that I was wrong.

$ echo $'#!/bin/bash\necho I GOT YOU>&2'> ~/.cache/makefile-modules/downloaded/tools/kind@v0.31.0_darwin_arm64 && chmod +x ~/.cache/makefile-modules/downloaded/tools/kind@v0.31.0_darwin_arm64
$ make _bin/tools/kind
$ _bin/tools/kind
I GOT YOU

I'm surprised that we aren't verifying the hash when creating the symlink. Right now, we verify after downloading and then never verify again.

And since some artifacts are large tarballs, we would need check their tarballs' hash and untar them just before creating the symlink.

My recommendation would be exactly this: instead of uncompressing + checking hash on download, I'd uncompress + check hash upon creating the symlink. This would 100% mitigate the issue IMO.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates that all commits in the pull request have the valid DCO sign-off message. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants