Retrieval-augmented generation · Real-time NLP · Capital One
I designed and built a retrieval system that surfaces the right procedure document to an agent as they talk — plus a semantic search over the same corpus — and I measured it honestly.
This is a deliberately high-level account of proprietary work at Capital One — the problem, my role, and the approach, described qualitatively. It contains no confidential figures, internal systems, data, models, code, vendor or team names, or architecture.
The problem
Agents work from a large internal procedure library that is genuinely hard to search, so they survive on bookmarks. New agents — and there are always new agents — routinely can't find the right procedure during a call. That's a direct quality and compliance risk.
What I built
Two surfaces over one retrieval core: a proactive assist that embeds live transcript segments and matches them against a vector database of procedure documents, surfacing clickable suggestions in the agent's UI; and a typo-tolerant semantic search for when the agent wants to look something up. End to end, a transcript segment becomes on-screen suggestions in real time. I also built the extraction pipeline that turns a hard-to-search document corpus into a clean, embeddable one, and I designed the retrieval metrics: per-utterance recall, time-weighted coverage, and off-topic rate.
Measured honestly
I measured call-level document recall against a human-annotated golden set rather than reporting a flattering proxy. When agent feedback said the proactive suggestions changed too quickly, I stood up experiments across dozens of ranking algorithms in a single day (agentic coding). The winner — a similarity ranking with a recency-decay penalty — gave a clear, measured lift in per-utterance recall, coverage, and off-topic rate, keeping the right document on screen even when the immediate transcript window went quiet on the topic.
Why it's a strategic win
The procedure corpus previously had no linkage to call data. Now every surfaced suggestion is tracked against the call it served, and search queries form a demand signal for what calls actually need from policy — enabling QA, policy analytics, and retrieval feedback loops for the first time.