Case Study · Mortgage Ops

Classify. Extract. File.

A working pipeline that watches an intake folder, classifies each incoming mortgage document, pulls the key fields out as structured data, and routes anything it's not confident about to a human instead of guessing. Built and tested end to end against a fully synthetic 20-document set — no real borrower data touched any part of this.

20/20

Classification
accuracy

85/85

Fields extracted
correctly

5/5

Non-mortgage docs
routed to review

Weekends,
three sessions

The Pipeline

An intake folder is watched for new PDFs. Each one goes to Claude with a single prompt that classifies the document — W-2, paystub, bank statement, or other — and extracts the relevant fields as structured JSON, with a confidence score and any quality flags attached. A router splits on that confidence: high-confidence, correctly-typed documents get logged, renamed, and filed automatically. Everything else — low confidence, or a document type it doesn't recognize — gets logged and routed to a review folder instead of a guess.

Make.com pipeline diagram: Google Drive watch folder, download, Claude classify and extract, JSON parse, router splitting to high-confidence filing, low-confidence review, and income summary logging.

Actual pipeline, built in Make.com

Design Choices

Synthetic data onlyEvery document in the test set is fabricated top to bottom — names, SSNs, account numbers, employers. That's not a limitation of the build; it's what makes this postable at all.

One call, not twoClassification and extraction happen in the same Claude request. Fewer moving parts, fewer failure points, at the cost of a slightly larger prompt.

Confidence-gated routingThe router doesn't just sort by document type — it sorts by whether the model says it's sure. Anything below threshold goes to a person, not a folder.

Make.comClaude APIGoogle DriveGoogle Sheets

What The Test Found

—

Classification landed 20/20 and field extraction 85/85 gradable fields — including three documents built specifically to have unreadable data, where the pipeline correctly returned null instead of inventing a number.

—

The confidence score didn't reliably track difficulty. Documents with real problems still scored 0.95 in most cases, same as a clean document — meaning it isn't yet a trustworthy signal on its own for "this doc had a problem."

—

The "degraded" test documents were clean, machine-readable text with scan issues described as literal markers — not the real visual noise of an actual scanner or phone photo. That's the honest next test, not something this run already proved.

Notes · The Full Story

Twenty for Twenty

Read it →

What's Next

Test against real scanned and photographed documents instead of clean text with described defects
Tune or replace the confidence signal so it actually tracks document quality
Expand the document-type set beyond W-2, paystub, and bank statement
Build out the income calculator's real underwriting logic — averaging periods, variable and overtime pay, when a YTD figure is a fair basis to annualize
Add a notification layer so the right person hears about a document the moment it lands or needs review

Classify, extract, and file is the smallest piece of what actually happens to a document in a mortgage file. But it's the first domino — and the honest miss mattered more here than the perfect score.

← return home