Building AI Applications with Enterprise Data

Retrieval, not retraining

For most business applications, the practical approach is retrieval: find the relevant material from your own systems at the moment of the question and supply it to the model as context.

This is usually preferable to training a custom model. It is cheaper, updates immediately when information changes, and — importantly — makes it possible to show where an answer came from.

Permission is the hard part

The central risk is straightforward to state: the system must never surface information to someone who is not entitled to see it. That means retrieval has to be filtered by the requesting user's identity before anything reaches the model.

Post-filtering is not sufficient, and instructing the model not to reveal something is not a security control. Access control belongs in the retrieval layer.

• Apply the user's permissions at query time, not afterwards
• Keep an auditable record of what was retrieved for whom
• Exclude sensitive fields from indexing rather than hiding them later
• Set retention and deletion policy for stored conversations

Data quality sets the ceiling

Retrieval surfaces whatever is in your systems, including the out-of-date policy document and the two contradictory versions of the same price list. A grounded system does not resolve contradictions; it repeats them confidently.

Identifying the authoritative source for each category of information, and giving it a human owner, is unglamorous work that determines the quality of everything built on top.

Show the working

Business users trust a system that cites its source and distrust one that does not, with good reason. Returning the underlying document alongside the answer turns an opaque assertion into something a person can verify.

It also makes the system dramatically easier to debug, because a wrong answer can be traced to either bad retrieval or bad information.