A model like the one behind Copilot predicts the next most likely piece of text given everything before it. That's the entire mechanism. It has no separate step where it checks a fact against a source before writing it down, unless you build that step in yourself. Fluent, confident, well-formatted text is what the model is optimised to produce. Accurate text is a byproduct that happens often, not a guarantee.
The failures aren't random. They cluster in predictable places.
The model is most reliable in the opposite situation: handed the actual source material and told explicitly to answer only from what's in front of it. That's a narrower task than open-ended recall, and narrower tasks are where these models are strongest.
Grounding is the industry term for handing an agent the actual source material at answer time instead of relying on what it happened to absorb during training. Point the agent at specific SharePoint sites, files, or Dataverse tables, and its answers get retrieved from those sources rather than generated from general knowledge. The technique is called retrieval-augmented generation: retrieve the relevant documents first, then condition the answer on them (Lewis et al., 2020). It's the concept behind every "chat with your documents" product, and it remains the most reliable answer the field has to hallucination.
You don't need to understand the retrieval plumbing to use it well. You need to know that grounding only works as well as the sources you supply. A grounded agent pointed at the wrong document, or at a folder with gaps, will still answer confidently and still be wrong. Adding a source is not a one-time setup step either. As policies change, contracts renew, and old templates get superseded, a knowledge library that isn't kept current quietly turns from an asset into a liability, because the agent has no way to know a document it's citing has since been replaced.
A detail that matters as much for trust as for accuracy: grounded knowledge in Copilot carries through your existing Microsoft 365 permissions. If a colleague can't open a particular SharePoint site, an agent grounded in that site won't surface its content to them either. The access controls you already have are the first line of defence, before anything else about the agent's design.
Two failure modes survive grounding entirely.
The first is judgement calls: questions where the right answer depends on discretion, policy interpretation, or authority the agent doesn't have, not on a fact sitting in a document. Grounding gives an agent good material to answer from; it doesn't give it the standing to make a decision that belongs to a person.
The second is content that's simply absent. If the answer to a question isn't in any of the agent's sources, grounding doesn't manufacture caution by default. A well-instructed agent says "not specified," but an under-instructed one will reach for the closest analogous fact in its sources and present it as if it answered the question asked. Both look identical in tone. Only one is honest.
Check names, numbers, and dates against the source before forwarding anything an agent produced. It's a small habit that catches the exact failure mode grounding can't fully close: an agent stating something with full confidence that turns out to be adjacent to the truth rather than equal to it. Treat every output as a draft that earns trust after a check, not before.
Ask an ungrounded assistant for the markup rate on a specific category of intercompany service, and it will often produce a number. Not a refusal, not a hedge, a specific figure, phrased the way a correct answer would be phrased. If that category happens to resemble others it has seen described in training data, the number may even be plausible on its face. The only way to know it's wrong is to already know the right answer, which defeats the purpose of asking. Ground the same assistant in the actual rate card and ask the same question, and the answer either matches the document with a citation attached, or the agent says the rate isn't specified. The difference between those two situations is invisible from the outside, since both read as confident, complete sentences, which is exactly why grounding matters more than any amount of politeness or hedging language in the prompt.
Grounding is also the surface an attacker can exploit. See how a document can carry instructions instead of information in the attack that arrives inside your documents. And the same verification instinct belongs inside every prompt you write, not just every agent answer you read. See the [CHECK] convention in five parts of a prompt your whole team can reuse.
The free first module of FluentWorkForce covers grounding hands-on, including a side-by-side of an ungrounded answer and a grounded one with citations. Start Module 1 free → Rolling this out across a team instead? Talk to us →