The thesis
Extraction is not quality control.
Reading a document and checking a loan file are different problems with different failure modes. Most mortgage AI has solved the first well, and reports it as though it solved the second.
The confusion
One question is about a document. The other is about the file.
An extraction system answers: what does this document say? A quality control system answers: is this file sound, under the rules that apply to it?
The first question is about one document. The second is about the relationships between all of them, checked against a body of policy that changes over time.
A system can be excellent at the first and structurally incapable of the second, and from a demo the difference is invisible. Both produce a screen of extracted fields with high confidence numbers beside them. The difference shows up later, in what escapes.
Three approaches
OCR and IDP extraction
Solves turning documents into fields, at volume and low cost. Does not solve whether the fields agree with one another, or with policy.
General-purpose AI
Solves reading unusual documents and answering questions. Does not solve reproducibility: ask twice and the reasoning may differ, and it cannot be pinned to a version.
Sample-based manual review
Solves judgement, context, and the cases no rule anticipates. Bounded by hours available: a sample estimates a rate; it does not find a pocket.
What the problem actually requires
Full-file coverage, cross-document checks as a first-class operation, deterministic disposition, versioned rules, and evidence at value level.
The honest test is files whose outcomes you know.
Comparisons written by vendors are worth very little, including this one. Bring a set of files with known outcomes and see what a full-file, cross-document, versioned check finds that a sample did not.