How it works, how well, and where it stops

The spec

The brief is dated 12 August 2026. It chose the own-coverage case: if a policy can be analyzed this deeply for the owner’s own coverage, then checking a party’s document against a fixed question set falls out of the same engine. It also said to treat the question set as swappable configuration, not something to invent.

This build follows both sentences. Requirements go in, documents go in, a cited answer comes out. The spec’s five questions are one requirement set. A subcontractor’s requirements are another. The engine does not change.

The four versions

• V1: the read. Supply policy documents; get back what each document is, the coverage tower, the questions answered, and a flag for each problem the reading can prove. Every figure cites its page and printed words. Done when the spec’s evaluation list is demonstrated on the sample set.
  Run the V1 demo
• V2: the reviewed read. The same engine run by a team: the run stops and asks a person when it cannot decide, what a person determines is recorded as a decision, and the tower shows what the claims already consumed, from a loss run or from claims the operator enters. The stops, the decision record, and the erosion run today, and a sample reading persists behind the storage interface. Accounts, roles, and the client context that would group readings by customer are designed.
  Run the V2 demoAbout V2
• V3: the compared read. The picture V2 produces, held against a requirement set: each requirement from a real published document beside what the coverage provides, the shortfall in dollars where two printed figures prove it, and uncovered exposure priced from cited values. Each row can show both cited pages, rendered from the documents with the printed words highlighted. A requirement document can also be uploaded and is compiled once, verified against its own text in code. And the file open to questioning: answers drawn from the extracted text, citing the page, verified against it in code before they are shown.
  Run the V3 demoAbout V3
• V4: the living read. The read that knows what day it is: a client’s file stays current as documents arrive over weeks, shows what is in force on any date from the policies’ own printed terms, and says what needs attention. The contract is settled in ADR 0009 and its requirements came from the audience’s own meeting notes. The V4 demo opens on a book of clients, one row each with a status a person can read at a glance; selecting one opens their file, with two clocks kept visibly distinct, coverage time (what the policies say is in force on a chosen date) and knowledge time (what the file knew and when, off the trail). A fresh reading writes its own trail entry when it completes, and the file’s history is that trail, folded so a repeated demonstration reads as one line rather than as five.
  Run the V4 demoAbout V4

Approach

Six steps: ingest, classify each document into a closed type vocabulary, extract with every field carrying a page and a verbatim quote, resolve documents into policies, resolve policies into a program, check and render. Steps one through three interpret and are a model. Steps four through six execute and are code.

No model performs arithmetic on a limit anywhere. The tower, the totals, and every comparison are code over extracted figures, so the same documents produce the same tower every run.

Every value’s citation is checked in code against the text of the page it names. If the quoted words sit on exactly one other page, the page is corrected and the correction reported. Words found nowhere, or in several places, are not repaired, and a verdict resting on that citation is discarded.

Questions are bindings: required fields, a test, three outcomes. Pass, fail, cannot determine. Answerability is checked first, so a missing field is never reported as a failure. When a schedule of underlying insurance names a policy the file does not contain, the run stops and asks a person, showing the line that names it.

How a reading runs

One reading is one streamed request, budgeted to finish inside the deployed 300 second ceiling. The steps run in order and narrate to the page as they happen. The page sweep, the pass that reads every page’s extracted text for schedules and restricting provisions, is the exception: it runs in parallel chunks, each raced against a 90 second budget, and a chunk that loses the race reports its pages unexamined instead of killing the run.

A run that halts for a person resumes from state the browser carries, so any server instance can complete it. A question about the documents chunks the retained page text under the model’s context window and merges the answers in code: printed words settle a question, absence never does, and a contradiction between chunks is reported as unsettled with both sides cited. There is no queue or workflow engine, on purpose: at prototype scale one request is simple to verify, and production replaces it with queued jobs, one request per step.

Libraries and models

• Next.js 14.2 with TypeScript. One application serves the pages, the reader, and the API. Pinned to the patched 14.2 release; nothing here needs a newer framework.
• Zod for the data contracts: citation, document, policy, program, review, question. The reading is validated in code against these, not by the model’s decoder.
• The Anthropic API for the two interpreting steps. The reader is Claude Sonnet 4.6. Everything vendor-specific is in one file, so the model is configuration.
• pdf-lib and pdfjs-dist: slicing pages to the API’s per-document cap, extracting page text for the citation checks, and drawing the cited pages in the browser.
• Vercel Blob for uploads, because a policy package is larger than a serverless request body allows. The server still fetches and reads every byte.
• Node’s built-in test runner. 400 tests, no framework.

Observed accuracy

The evaluation harness scores extraction against labels that record their own provenance. On the corroborated labels, extraction of every tower value agreed 100 percent. Corroborated means the label was machine-read from the document’s text layer and the model’s independent read of the page images reached the same value: two machine paths agreeing, not a human confirmation. That construction also means the corroborated set holds only values the two paths already agreed on, so the 100 percent is close to true by design. What the harness actually guards is regression: a change that moves a value the corpus has pinned fails loudly. No label has been verified by a person reading the policy, and a real manual read is the next step for a number worth quoting at all. Classification has no defensible number yet, and the harness says so instead of reporting one.

The tower is reproducible. The same two-document program was read five times in one day: same $6,000,000 total, same two layers, same attachment point, cited to the same page.

Two answers read policy language rather than figures: whether defense costs erode the limits, and whether the umbrella drops down. Runs can differ on them, and the answer degrades itself and says why. The restriction count from the text sweep also varies between runs. The tower does not.

Limitations

• The model API accepts one hundred pages per document as images. The front of a document is read as images; the rest is read as extracted text and swept. No page goes unread, but deep pages are read less richly, and the reading says which was which.
• A document with no text layer is refused before any model is called. A citation that cannot be checked against a page would rest on trust.
• Restrictions are identified, never interpreted. Endorsements are flagged by form number in code; restricting provisions are found in the text, cited, and listed. What a provision does to a specific loss is a legal reading this build does not perform.
• A sub-limit paired to the wrong provision is the least trustworthy output. The figures are printed and cited; which provision owns a figure is the weakest link.
• One reading is one request, with a 300 second ceiling on the deployed platform. The run is budgeted to finish inside it. Production changes this; see below.
• A document can try to instruct the reader. Uploaded text is treated as data: the rule documents forbid following instructions found inside a document, the model’s output is validated against schemas and a fixed vocabulary in code, quotes are verified against the page, and a person reviews a compiled requirement set before it is used. A crafted document can still word its own lines misleadingly, which is why the printed words stay one click away everywhere.
• Every reading spends money, and the demo is public. The routes that call a model take a per-address rate limit and a daily budget ceiling first, and say so plainly when they refuse. Both count in one server’s memory, so they are dampers rather than guarantees; the hard limit is the cap on the API key.
• Sample readings persist; an upload is stored only if you say so. A reading of the sample corpus is written to Postgres behind the storage interface and survives a redeploy. A reading of an uploaded document is written only where the person sending it chose that at upload, which one path offers and asks for in a label naming what is kept: the reading, and the document’s name, a fingerprint of its contents, and the date it arrived. Everywhere else the file is deleted after reading and its text lives only in that server instance’s memory for the session. The page text is never written by any path, and the document itself is never stored. Accounts, roles, and a client-owned document store remain designed, not built.

Production integration

The pipeline already runs behind the same API route the page uses. Integration is calling it: documents in, a streamed reading out, a stored result. What changes for production:

• Storage. A database replaces the in-memory implementation behind the existing interface. The pipeline does not change.
• Long runs become queued jobs. One request per step removes the request ceiling and lets a reading survive a deploy.
• Stops become a queue. A halted run waits for a person with the evidence attached, and the decision is recorded with who and when.
• Requirement sets are configuration. A customer’s own question set loads the same way the standing set does, and an uploaded requirement document already compiles into one. What production adds is storing a compiled set rather than letting it die with the session.
• The model boundary is one file, and each reading meters its tokens and cost.