A fair comparison keeps case identities visible. An omitted failing case is not a repaired case, and a newly added example should not silently change the denominator. Separate fixes, new failures, additions and removals before choosing whether a prompt revision should replace its baseline.
Compare case.baseline and case.candidate by shared case ID. Return sorted fixed IDs, new_failures, added and removed. A case absent from one version is not evidence of a fix.
The notebook executes your Python against an original local mock. Use the supplied case rather than copying visible values. The hidden checks change identities, revisions and boundary inputs; no provider endpoint or external tool is contacted.
An example implementation of the function contract.