Harden eigh benchmark against content memoization

[nataliakokoromyti]

Summary

• Generate a nonce-derived input batch for warmup and every timed iteration.
• Keep input generation and reference copies outside CUDA event timing.
• Ensure the leaderboard prewarm pass and scored pass cannot share tensor contents.
• Add a regression test with a submission that memoizes outputs by exact input content.
• Cap fresh-input scoring at 50 repeats to bound evaluator wall time.

Why

The current evaluator clones inputs for warmup, then repeatedly times the original tensors with identical contents. Cloning changes object identity but does not prevent content-based memoization, so an output learned during warmup can be returned inside the measured region.

This is stronger than the clone/shuffle mitigation discussed in #102 because every measured call receives data that has not appeared in warmup or an earlier repeat.

Validation

python3 -m unittest problems/linalg/eigh_py/test_eval.py -v

The regression performs two benchmark invocations against one logical case (matching leaderboard prewarm plus scoring), requests 60 repeats each, and verifies the 50-repeat ceiling. It observes 102 unique generated seeds and zero content-cache hits.

A CUDA smoke on an NVIDIA L40S exercised the patched evaluator with 50 inputs per batch:

Stats(runs=4), cache_entries=250, cache_hits=0

Fresh generation remained outside event timing. L40S wall times for one generated benchmark input were:

• mixed 640x512: 0.56 s
• rankdef/clustered/lapack dense 640x512: 1.02-1.03 s
• LAPACK dense geometric 60x1024: 0.23 s

The 50-repeat ceiling bounds this added wall time under the existing 900-second timeout. A B200 smoke was attempted but the configured Modal workspace could not provision because its billing-cycle spend limit was reached.

[nataliakokoromyti]

@msaroufim this affects the currently deployed eigh evaluator. The exact-content memoization regression and an L40S CUDA smoke both pass. After merge, the problem still needs the backend update-problems refresh before the live scorer is fixed.

[msaroufim]

Thanks @nataliakokoromyti i'll take a look! Altho unlikely to merge given its final hours, will just rely on a manual audit to declare the true winner and delete reward hacks in the top 10