MustHave.ai coding fixtures, version 1, November 5, 2026

Purpose: compare code generated for the same small, explicit contracts.
This is a test pack, not a Gemini benchmark result. No Gemini model output
was generated or scored for this release.

Give PROMPT.txt to the model. Save the response as candidate.py. Review it
before running it. Test unfamiliar code only in a disposable environment
without secrets, network access or valuable files. The harness imports the
candidate, so code at the module's top level also runs.

Run: python3 check.py candidate.py
Requirements: Python 3.11 or later, standard library only.

Record model ID, provider, date, thinking setting, output limit, elapsed
time, billed input/output/thinking tokens, price date, retries and raw output.
Use a new run for every repair. Keep the original failure in the record.
Identical inputs and acceptance tests permit a narrow comparison. These
small functions do not measure repository-scale coding or production safety.

The hand-written reference passes all 18 checks. A deliberately broken
baseline fails 9. This validates the fixtures' ability to catch those known
mistakes; it says nothing about Gemini's success rate.

Run the reference: python3 check.py reference.py
The public pack contains acceptance cases, so these are open-book checks.
Do not call them a blind benchmark.
