The Unglamorous Work That Decides Answer Quality

Split on meaning, not on character count

Naive splitting every thousand characters cuts tables in half, separates headings from content and leaves passages that make no sense alone.

Split on structure — sections, headings, natural boundaries — and each retrieved passage stands on its own, which is what the model needs.

Attach metadata to everything

• Document title and section heading, carried with every passage
• Last reviewed date, so stale content can be down-weighted
• Owner, so somebody is accountable for accuracy
• Audience or permission level, for filtering
• Product, region or entity, where you have more than one

Metadata is what lets retrieval prefer the current UK policy over the superseded US one. Without it, both look equally relevant.

Delete rather than archive

Superseded documents that remain in the index are the largest single cause of confidently wrong answers. Version 3 and version 5 of a policy look equally authoritative to a retrieval system.

Remove them from the index properly. Keep them in your archive if you must, well away from anything the assistant can see.

Handle the difficult formats deliberately

Give every core document an owner

Twenty documents with named owners and review dates will outperform two hundred orphaned ones within a year, because the twenty stay true.

This is the maintenance that keeps a retrieval system useful, and it is organisational rather than technical.

Frequently asked questions