Correctness Validation

May 30, 2026 | 5 min read

How Correctness Validation Works

Pyvorin verifies that native-compiled output matches CPython output exactly:

  1. Execute the function with CPython and capture the result.
  2. Execute the native-compiled function with the same inputs.
  3. Compare output hashes (SHA-256 of repr).
  4. If hashes differ, the event is marked correctness_match=False and is non-billable.

Correctness Mismatch Handling

  • The mismatch is logged locally and sent to the server as a non-billable event.
  • You can report the failure manually: pyvorin report-failure failure_event.json
  • Pyvorin engineering investigates mismatches as P0 bugs.

Best Practices

  • Always run benchmarks with correctness validation enabled (default).
  • Report mismatches via pyvorin report-failure.
  • Run pyvorin proof for the most thorough validation.