What went wrong
In its first run, GPT-5 reported $84,000 in operating expenses instead of $96,000. That raised NOI to $162,000 instead of $150,000 and affected two loan limits. Its other two runs got all 27 field values right.
In its second run, Opus 5 reported annual in-place rent of $178,800 instead of $177,600. Its other two runs got all 27 field values right.
Both models flagged historical income differing from current annualized rent as a conflict in every run. These cover different periods; the difference alone is not a discrepancy. This was the remaining financial failure in otherwise correct runs.
Execution and cost
Three repetitions each of openai/gpt-5 and anthropic/claude-opus-5 used the same packet and structured-output schema, with no tools or answer-key access in the request. Provider reasoning defaults were retained and are not equal compute. Gateway-reported inference cost for all six calls: $0.3542. These amounts exclude platform subscriptions, operator time, hosting, and product tooling. Cost is measured for the complete packet; it cannot be allocated to individual task stages from these records.
The grading correction
The original parser rejected nine earlier model responses because of formatting. The corrected scorer can evaluate all nine, while preserving their original responses and execution records. A second correction recognizes equivalent constraint labels such as LTV and ltv_limit. Both corrections apply consistently to every retained answer.
Read the correction and its timing · Inspect the earlier cohort after rescoring
What this pilot cannot establish
The case and answer key are public, practitioner review is pending, and the API inputs are text rather than original PDFs. Repeated calls on one packet do not establish performance on real deals, usable underwriting workbooks or finished OMs. Broader reviewed cases and artifact tests come next.