The right circular.
The right reader.
Staff were asking colleagues instead of the policy library, and sometimes citing withdrawn circulars. We built retrieval that filters by permission and effective date inside the query, cites the version, and declines when the documents do not answer.
This is a composite scenario built from how we deliver this kind of work. The client is fictional and the figures are targets, not measured results. We will replace it with a named engagement when a client agrees to be quoted.
The compliance team of a non-bank financial institution in Dhaka with about 1,400 internal policies and regulator circulars, a fifth of them superseded
- The library held 1,412 documents. About a fifth had been superseded, but the newest file was not always the one in force, and department-restricted memos sat beside public circulars. Finding an answer took around forty minutes and often ended in a message to a colleague.
- A pilot with a general-purpose chatbot had quoted a withdrawn circular confidently. Compliance needed answers that carried the version and respected who was allowed to read the source.
How we approached it.
Versions as a first-class field
Every document got an effective date, a supersession link and a status. 23 percent turned out to be superseded, and 31 had no date at all until someone found it.
Permissions inside the query
The reader's groups come from sign-in and filter retrieval before the model sees anything. Restricted memos never reach the prompt.
Hybrid retrieval and rerank
Keyword search catches circular numbers; embeddings catch paraphrase; a reranker orders the short list.
Cited answers or abstention
Each answer quotes its passages with the document version. If the passages do not answer, it says so.
Evaluate every slice
A 2,000-query test set split by department, date and question type, scored on retrieval, abstention, groundedness and completeness.
Try it yourself.
A simplified illustration with made-up data. It plays once when it scrolls into view, and you can pause it or step through.
- 1Contractor leave, circular 2025
Contractors take unpaid leave unless the contract says otherwise.
- 2Employee leave policy
Employees accrue 1.5 days of annual leave per month.
- 3Parental leave policy
Parental leave applies to employees only.
- Contractor leave, circular 2023
- Manager guide: approving contractor time off
- Compliance review: contractor leave terms
- Payroll: contractor invoices
- Disciplinary procedure
Contractors take unpaid leave unless the contract says otherwise.
Source: Contractor leave, circular 2025Step 1 / 3 Reader: Staff. Five chunks are removed before ranking, including the superseded v2 circular.
What we built.
- Ingestion with effective dates, supersession links and a review queue for undated documents
- Permission-filtered hybrid retrieval on pgvector
- Cited answers with version labels and abstention
- Audit log of every question, passage and reader
- Evaluation set and monthly slice report
Outcomes (targets).
These are the results we would aim for. They are not measured results from a named client.
- Target1,412documents ingested, 23% flagged superseded31 undated documents sent for review
- Target0restricted-document leaks in 2,000 test queriesfilters run inside retrieval
- Target96%grounded answers on sampled questionstwo graders, blind
- Target40 min to 2time to a cited answerfrom the pilot survey
Timeline and stack.
9 weeks to pilot with two departments
What we learned.
- The version in force is data, not a filename. Capture it on ingestion.
- Permissions belong in the retrieval query, never in the prompt.
- Abstention is measured. An assistant that answers everything is hiding its failures.
How we would deliver it.
Want this
for your business?
Message an engineer on WhatsApp or send a brief. We agree scope and price before work begins and build in small working steps you can test.