Originally published on LinkedIn Pulse, May 12, 2026.
In our personal lives, we are increasingly users of an ever-expanding range of artificial intelligence tools through LLMs (large language models). Gemini, ChatGPT, Claude, Copilot — these are propositions differentiated not only by their features, but primarily by the information sources used to underpin their responses.
Drawing on OECD data published in 2025, the picture remains conservative: across OECD member countries, only 20.2% of firms reported using AI — rising to 52% among large enterprises, but falling to just 17.4% among small ones. The gap is not incidental. It reflects exactly what grounding requires and most organisations still lack — governance maturity. To understand why this gap persists, it is worth opening the bonnet and taking a closer look at the mechanics.
At its core, grounding is not intrinsic to LLMs — it is an architectural decision. Information enters through ingestion pipelines; unstructured documents are chunked into smaller units and encoded as embeddings that enable semantic retrieval; these fragments, once tagged with provenance metadata, become the grounding context from which the model draws when responding. Prompts invoke that context — but the quality of grounding is determined upstream: by the rigour of pipeline design, the integrity of data sources, and the controls placed on what counts as authoritative.
In practice, a document repository typically contains multiple versions of the same policy — drafts, superseded versions, working copies, email attachments someone saved to a shared drive. Without explicit controls, the model may retrieve and cite a version that was revoked two years ago. It has no way to know. A well-written prompt cannot compensate for a poorly constructed knowledge base. This is why grounding is not a feature to be switched on — it is infrastructure to be governed. And its absence, or misconfiguration, is arguably the single most underestimated risk in enterprise AI adoption.
What Is Required to Use Corporate LLMs
Every LLM model has a knowledge cut-off date, beyond which its weights freeze and any subsequent evolution of the world ceases to be reflected in its responses. This structural limitation led the industry to seek mechanisms that anchor responses in verifiable external sources. That process is what we call grounding — and in a corporate context, it is not optional.
Grounding capabilities help reduce unsubstantiated inferences — particularly in critical domains such as compliance, finance, legal, and security. In a corporate environment, an LLM must do more than just “respond well” in linguistic terms — it must frame its responses anchored in external facts and verifiable sources. It must:
- Draw on authoritative and accurate data — internal policies, contracts, procedures, knowledge bases, applicable and updated legislation.
- Indicate the origin of its assumptions — references, links, excerpts, records.
- Enable traceability — who asked, what data was used, which model version, which rules were applied.
At this point, grounding is no longer a technical detail — it becomes a pillar of governance, without which there is no solid foundation for establishing operational trust, enabling audit, attributing responsibility, or controlling risk.
Authoritative does not simply mean accurate or current — it means institutionally designated. A document may be factually correct and recently updated, yet carry no formal authority for a given context. Conversely, a board-approved policy may contain errors, yet remain the binding reference until formally revised. A RAG system has no access to the act of designation — it only sees the document. In most enterprise AI implementations, this question is answered implicitly — which means it is answered by whoever configured the pipeline, usually by ingesting whatever was accessible. That is not grounding. That is retrieval with an accountability gap.
External, Volatile and Opaque — All at Once
To push this to its furthest edge: imagine a small company that, feeling lucky, adopts a “plug & play” LLM to evaluate the client-facing behaviour of their customer service employees — with no governance layer, no grounding architecture, no audit trail. The example is deliberately simplistic and extreme. But it illustrates how AI adoption can challenge elementary principles of systems architecture: no external, volatile, or opaque component should be a single point of failure in a system. And an LLM can be all three simultaneously:
⚠ External — because it depends on a vendor, an API, on policies and conditions the organisation does not control. In our example, the small company has no visibility into how the model was trained, what data it was exposed to, or what content policies govern its outputs. If the vendor changes its terms of service, deprecates the model, or simply takes the API offline, the employee evaluation process stops — with no fallback, no continuity, and no recourse. The organisation has outsourced a critical judgement function to an infrastructure it does not own and cannot inspect.
⚠ Volatile — because its behaviour can shift with model updates, changes in prompting, or variations in the underlying data context. The company runs its first batch of evaluations in January. In March, the vendor silently pushes a model update. The same prompts now produce measurably different outputs — not dramatically wrong, but subtly shifted in tone, threshold, and tendency. The evaluations from January and March are no longer comparable. No one inside the organisation knows this has happened, because there is no audit trail, no version pinning, and no monitoring. Decisions about employee performance have been made against an inconsistent and unannounced standard.
⚠ Opaque — because it is not always possible to explain why a particular response was generated, or what “reasoning” led to a given conclusion. One employee receives a negative evaluation. They ask why. The manager cannot explain — not because they are unwilling, but because the model produced a confidence score and a summary, with no traceable chain from input to output. There are no auditable logs, no cited sources, no decision path. When the employee escalates to HR, and HR escalates to legal, the organisation discovers it has been making consequential judgements about people using a system it cannot account for. The risk was never the model getting the answer wrong. It was never being able to prove it got it right.
A Matter of Credibility
The credibility of managers — or any professional in an executive role — rests on their sources, and on the ability to justify, audit, and publicly defend them. Introducing LLMs into processes can, paradoxically, undermine the transparency of the decisions taken. Not because the technology is inherently flawed, but because:
- A response can appear plausible without being true.
- A recommendation may have no grounding in fact.
- An error may go undetected until it is too late.
- When the moment of scrutiny arrives, there may be insufficient auditable logs to justify the decision or the response.
LLMs cannot replace critical thinking or judgement at the moment of decision — simply because it makes no sense to transfer risk to a system that does not share the consequences.
The decision-maker’s credibility is only as solid as the first failure they cannot explain. This is why grounding is not merely good practice — it is the foundation on which accountable AI adoption must be built.
Three Questions Worth Asking First
AI adoption in organisations does not lack practical applications. What it lacks is operational trust. Before deploying an LLM to optimise decision-making processes, three simple questions are worth asking:
1 — What sources are the responses anchored in?
2 — Can I audit and reproduce the path from context to response?
In a RAG system, the documents are transformed into vector representations — embeddings — and stored in a specialised database. When a user submits a question, that question is also converted into a vector and used to locate, by semantic proximity, the most relevant document fragments in the database. Those fragments are then injected into the prompt context, alongside the question, before the model generates its response.
Each of these steps is a potential point of failure — and a potential point of untraceability. Which documents were in the database at the time of the query? Which version of each document? Which fragments were retrieved, and why those rather than others? Was the embedding model the same one used when the documents were originally indexed? Was the retrieval threshold set correctly, or did a relevant fragment fall just below the cutoff and get silently excluded?
3 — If something goes wrong, can I explain publicly why — and correct course quickly?
If you cannot answer these questions after a given occurrence, you cannot audit the response. And if you cannot audit the response, you cannot defend the decision it informed — not to a regulator, not to a board, not to the person who was affected by it. Reproducibility is not a technical nicety. It is the minimum condition for accountability.