Healthcare (non-clinical documentation)
Healthcare documentation platform
84% top-3 retrieval precision
on the labeled evaluation set, up from an estimated ~40% with the prior keyword search
Adopted by 3 departments
within the first quarter after launch, driven by the visible source citations
Scoped to non-clinical documentation
by design, to keep sensitive patient data entirely out of the system
The Challenge
Staff spent significant time searching scattered policy and procedure documents for answers that existed but were hard to find.
An earlier keyword-search tool had a reputation for surfacing outdated or irrelevant documents, so adoption was low and trust was lower.
This engagement covered internal operational documentation only: no patient data was in scope.
My Approach
Built a labeled evaluation set from real staff questions before writing any retrieval code, so retrieval quality could be measured instead of assumed.
Every answer surfaces the source passage alongside the response, so staff can verify a citation instead of trusting a black box.
Ran the retrieval evaluation harness on every change to chunking or embedding strategy, catching two regressions before they reached staff.