Retrieval Quality, Not Model Size, Is Now the Main Bottleneck for Enterprise AI Assistants

A consistent finding from companies rolling out internal AI assistants over the past year: swapping to a larger or newer underlying model produces a smaller quality improvement than fixing the retrieval pipeline that feeds it context. The failure pattern is familiar to anyone who has built one of these systems: the assistant confidently answers a question using an outdated policy document because the retrieval system ranked it higher than the current one, or misses the correct answer entirely because it lives in a spreadsheet the indexing pipeline never covered. No amount of model capability fixes a wrong or missing document in the context window. Teams that have gotten this right share a few common practices: aggressive de-duplication and staleness detection on the document corpus, metadata-aware retrieval that can filter by document date and department rather than relying purely on semantic similarity, and a feedback loop where flagged wrong answers trace back to a specific retrieval failure rather than being treated as a generic "model was wrong" incident. The practical takeaway for any organization building this kind of tool: budget the majority of the engineering effort for the data pipeline, not the model selection — the model choice is close to a solved problem at this point, the data plumbing is not.