CASE 05
8 weeks
Associates find precedent in minutes, not hours, with every answer cited.
A mid-size firm had two decades of briefs, memos and executed contracts that only existed as institutional memory. We turned the archive into a searchable, citable knowledge base where every answer resolves to the exact paragraph it came from — and where an uncited answer is structurally impossible.
THE PROBLEM
Associates spent hours grepping through thousands of briefs, memos, and contracts to answer partner questions. Results were inconsistent and hard to audit.
The archive was a document management system with folder names that made sense to whoever created them in 2011. Search was filename and metadata only, so finding the right precedent depended on knowing it existed.
The real cost was seniority-shaped. Partners asked associates; associates asked the one senior associate who had been there longest; that person became a lookup service. When they were on holiday, research slowed to a crawl.
Two partners had independently tried consumer AI tools on redacted documents and got confident, fluent, wrong answers. The firm's appetite for anything that could not show its sources was correctly zero.
CONSTRAINTS WE WORKED UNDER
WHAT WE DID
Built an ingestion pipeline with careful chunking so citations always resolve to exact paragraphs.
Chose Anthropic for long-context synthesis with aggressive groundedness prompts — no confident answers without a cite.
Added a reviewer UI where partners can flag bad answers; flags feed a weekly retrieval-tuning loop.
Locked the whole thing behind SSO with full audit logs for compliance.
ARCHITECTURE
01
Document watcher on the DMS, format detection, OCR for the scanned tail of the corpus, then structural parsing that preserves clause and paragraph boundaries. Every chunk keeps a pointer back to document, page and paragraph so a citation is a location, not a guess.
02
Legal documents have structure worth respecting — numbered clauses, defined terms, schedules. Chunking follows that hierarchy instead of a fixed token window, with parent-clause context attached to each chunk so a subclause never gets read in isolation.
03
pgvector for dense retrieval with metadata filters on matter type, jurisdiction and date, then a long-context synthesis pass that must quote before it concludes. The prompt contract requires a passage ID per assertion; responses failing schema validation are regenerated, not shipped.
04
Answers render with each claim linked to its source passage, expandable inline. Partners flag bad answers with a reason; flags queue into a weekly review where we tuned retrieval rather than arguing with the model.
05
SSO against the firm's identity provider, matter-level permissions mirrored from the DMS, and an append-only audit log of every query, every document surfaced and every answer produced.
KEY DECISIONS
WHY
Legal reasoning depends on surrounding language. Pre-summarizing chunks to save tokens destroys exactly the nuance that makes an answer correct.
TRADE-OFF
Higher token cost per query. Justified in one line of arithmetic: an associate hour costs more than a month of inference at their query volume.
WHY
Prompting a model to 'always cite' is a suggestion. Requiring a schema where each assertion carries a passage ID, and rejecting responses that fail validation, is enforcement.
TRADE-OFF
Occasional regeneration and slightly stiffer prose. The firm considered stiff-and-sourced a strict improvement over fluent-and-unverifiable.
WHY
Matter-level confidentiality already existed and was already audited. Inventing a second permission model would have created two sources of truth about who may see what.
TRADE-OFF
A sync dependency on the DMS, and permission changes propagate on a delay measured in minutes rather than instantly.
THE HARD PARTS
About 15% of the corpus was scanned paper, some of it poorly. Naive OCR produced chunks that retrieved well and cited to garbage. We added a per-document OCR confidence score and excluded low-confidence documents from the index entirely, surfacing them as 'exists, not indexed' rather than silently returning nonsense.
A contract defining 'Territory' in clause 1.4 changes the meaning of every later mention. Flat retrieval got this wrong constantly. Attaching the definitions section to every chunk from the same document fixed a whole class of subtly wrong answers.
The first demo question was deliberately obscure — a jurisdictional carve-out from a 2014 matter. The system found it and cited the clause. That single result did more for adoption than the following two weeks of engineering.
HOW IT RAN
01
Sampled the archive, measured format distribution and OCR quality, and established what fraction of the corpus was realistically indexable before promising anything.
02
Built parsing, OCR, structure-aware chunking and the citation-resolution guarantee. Ran full-corpus ingestion twice as the chunking strategy changed.
03
Evaluated against a partner-written question set with known correct sources, tuning until the right passage appeared in the top results consistently.
04
Reviewer interface, identity integration, audit logging, then a staged rollout starting with the associates who had the most to gain.
OUTCOME
Half a day → <10 min
Typical research task turnaround
40k
Documents indexed with paragraph-level citations
20 yrs
Of archive made searchable, including scanned paper
0
Answers shipped without a resolvable source
Schema-enforced, not prompt-requested.
Research tasks that took half a day now take under ten minutes. Partners report higher confidence because every answer links to the underlying passage.
The senior associate stopped being a human index. That was the stated goal in week one and the thing the firm actually noticed by week ten.
Junior associates benefited most. The system surfaced precedent they did not know existed, which compressed the part of their development that had previously been pure tenure.
STACK
BACKEND
AI
INGESTION
SECURITY
HANDOVER
