From RAG to Enterprise Context Fabric: the next stage of enterprise AI architecture

Over the past few years, discussions about enterprise AI architecture have often revolved around a familiar set of terms: LLMs, RAG, embeddings, vector databases, agents and MCP.

Around 2023, a typical architecture for answering questions over enterprise knowledge looked like this:

Document
   ↓
Chunk
   ↓
Embedding
   ↓
Vector database
   ↓
Top-K retrieval
   ↓
LLM
   ↓
Answer

This architecture addressed a practical problem. A model’s context window is finite, while enterprise knowledge is far larger than a single prompt can hold. The system first has to find a subset of potentially relevant material, then ask the model to answer from it.

That is the most widely recognised form of retrieval-augmented generation, or RAG.

But as agents, longer context windows, tool use, MCP and model reasoning develop, my view is becoming clearer:

Traditional RAG will remain useful, but it should no longer be the centre of enterprise AI architecture.

The more important architectural question is:

How should every AI agent in an enterprise safely, reliably and dynamically obtain the context it needs to complete its task?

The concept I am working towards is an Enterprise Context Fabric. This article sets out that architectural direction, including a possible future role for KuWarden as its governance and control plane.

1. The question is not whether vector search is still needed

Vector retrieval remains valuable. Embeddings are particularly useful for finding synonyms, interpreting loosely phrased requests, searching across languages, discovering similar cases and locating unstructured documents through semantic recall.

The problem is that we have often treated two different things as equivalent:

Finding semantically similar information.

And:

Finding enough evidence to support a correct answer.

Consider the question: Which expenses are explicitly excluded from reimbursement?

Vector search might return an expense reimbursement policy, a travel expense procedure and an expense submission guide. All three are semantically relevant.

Yet the answer may depend on a single provision on one page: “The following expenses are not reimbursable…” The exclusion list, its conditions and any exceptions may all be necessary to interpret that provision correctly.

Semantic similarity
        ≠
Answer relevance
        ≠
Evidence sufficiency

This is why an enterprise AI system cannot assume that one top-K vector search provides sufficient evidence for every question.

2. Agentic retrieval changes the architecture

The basic RAG pattern is straightforward:

Query → Retrieve → Top-K → Generate

In this simplified pattern, the main retrieval decision happens once, before the answer is generated.

An agent makes a different pattern possible:

Question
   ↓
Understand intent
   ↓
Form a retrieval hypothesis
   ↓
Choose a source and tool
   ↓
Retrieve and inspect evidence
   ↓
Is the evidence sufficient?
   ├── Yes → Answer
   └── No
        ↓
     Reformulate
        ↓
     Search again
        ↓
     Cross-check and reassess

Retrieval becomes an iterative reasoning process. The agent can revise its judgement as it discovers evidence.

Suppose someone asks: Why was this customer’s electricity disconnected last month?

The agent might first search the CRM and find a customer complaint. That establishes that a complaint exists; it does not explain the disconnection.

The agent identifies the evidence gap and searches service orders. It finds a disconnection work order. Now it knows an order was issued, but it still has not established what triggered it.

It then checks billing records and meter events, finding an overdue-payment event and evidence of the physical disconnection. Even then, it should not immediately conclude that the overdue payment explains or justified the action.

It must also inspect the applicable disconnection policy and establish whether the event met the relevant conditions.

The result is an evidence chain to examine:

Billing event
      ↓
Disconnection order
      ↓
Meter event
      ↓
Applicable policy rule
      ↓
Supported conclusion

The task is to establish the relationships between those records, rather than simply retrieve text resembling the question. This is a different retrieval pattern from the one-pass pipeline.

3. Agents do not replace retrieval infrastructure

It is easy to draw the wrong conclusion: if agents can search for themselves, perhaps search systems, vector databases and indexes are no longer necessary.

They are still necessary.

A real enterprise’s information environment may include databases, SharePoint, Confluence, Jira, ServiceNow, email, Teams, ERP, CRM, data lakes, object storage, file systems, APIs, scanned documents and knowledge bases. There may be millions or tens of millions of items.

An agent cannot realistically grep everything. It still needs candidate generation and retrieval infrastructure.

The change is that the agent becomes a retrieval orchestrator. Instead of making a single vector_search() call, it can select and combine capabilities according to the question:

• BM25 and full-text search;
• vector search;
• SQL and metadata queries;
• graph traversal;
• file-system search;
• API calls;
• navigation within documents.

Retrieval agent
   ├── SQL
   ├── BM25 / full-text search
   ├── Vector search
   ├── Graph traversal
   ├── File search
   └── APIs / MCP-exposed tools

Vector search remains part of the system, but at a different level of abstraction: one retrieval capability the agent can select, rather than the organising centre of the architecture.

4. Enterprises need an independent retrieval layer

Imagine an enterprise with dozens or hundreds of agents. An initially attractive architecture would let each one connect directly to its own collection of systems:

HR agent
   ├── SharePoint
   ├── HR database
   ├── Email
   └── APIs

Finance agent
   ├── ERP
   ├── Database
   ├── SharePoint
   └── APIs

Developer agent
   ├── Git
   ├── Azure DevOps
   ├── Wiki
   └── Jira

At first, this looks flexible. As it grows, familiar integration problems emerge:

• duplicated tool integrations and search logic;
• inconsistent access-control implementations;
• fragmented auditing;
• schema changes affecting many agents;
• a larger prompt-injection and data-exposure surface;
• retrieval quality that is difficult to evaluate consistently;
• every agent team solving the same infrastructure problems again.

I increasingly favour an independent enterprise retrieval layer:

HR · Finance · Engineering agents
                ↓
     Enterprise Context API
                ↓
     Retrieval intelligence
                ↓
Search tools · Data tools · SaaS tools
                ↓
       Enterprise systems

A business agent should not have to manage the details of where information lives, which search engine is used, whether a query requires vectors or PostgreSQL, or whether access goes through Elasticsearch, a graph or an MCP tool.

The agent expresses what information it needs. The retrieval layer determines where and how to obtain it, within the permitted sources and access boundaries.

5. MCP as a capability interface

This is one of the areas where I see long-term value in the Model Context Protocol, or MCP.

An enterprise context layer could use an expanding capability registry:

Enterprise retrieval agent
   ├── MCP: SharePoint
   ├── MCP: Git
   ├── MCP: PostgreSQL
   ├── MCP: ServiceNow
   ├── MCP: Jira
   ├── MCP: Snowflake
   ├── MCP: ERP
   ├── MCP: CRM
   └── MCP: Internal APIs

The architectural aim is for a new enterprise system to be introduced through a registered capability, without rewriting every consuming business agent. Once the integration is implemented and authorised, its description and schemas help the retrieval agent decide when to use it.

Each capability would declare:

Name
Description
Supported actions
Required permissions
Input schema
Output schema
Data classification
Risk level
Owner
Version

This resembles the enterprise service catalogue we already know. The difference is that its consumers increasingly include AI agents, alongside software developers.

6. MCP is not enterprise governance

Tool discovery and invocation are separate concerns from deciding whether an agent should be permitted to invoke a tool in a particular situation.

MCP is useful at the interface boundary. It does not, by itself, constitute an enterprise’s governance model.

The architecture still needs explicit policy enforcement:

Agent
   ↓
Retrieval / context layer
   ↓
Policy enforcement
   ↓
MCP / tool
   ↓
Enterprise resource

Each tool call should carry the identity and task context needed to answer:

• Who is asking?
• Which agent is acting, and on whose behalf?
• For what purpose?
• Which resource is involved?
• What is its data classification?
• Which tenant and business unit apply?

The policy engine can then determine the appropriate outcome:

ALLOW
DENY
FILTER
MASK
REDACT
REQUIRE APPROVAL

These controls are part of the enterprise architecture, not something the model should be trusted to infer from a tool description.

7. Retrieval should return an evidence package

In a basic RAG implementation, retrieval often returns Chunks[], which are then inserted into a prompt.

For an Enterprise Context Fabric, a more useful return object is an evidence package:

{
  "context": "...",
  "evidence": [
    {
      "source": "SharePoint",
      "document": "Expense Policy v4",
      "section": "4.2",
      "retrieved_at": "...",
      "confidence": 0.96
    }
  ],
  "gaps": [],
  "conflicts": [],
  "retrieval_trace_id": "..."
}

This is an illustrative contract. A confidence field would need a defined meaning and evaluation method; the number itself is not proof that an answer is correct.

The system should also establish explicit claim-to-evidence relationships:

Claim
The customer was disconnected
because payment was overdue.

Supporting evidence to verify
   ├── Billing Event #29382
   ├── Disconnection Order #11283
   └── Disconnection Policy §7.4

The desired output is a conclusion with identifiable sources, supporting evidence and an audit trail. Gaps and conflicts belong in that output too; they should not disappear merely because the model can produce a fluent answer.

8. From retrieval layer to Enterprise Context Fabric

At a higher level of abstraction, “enterprise retrieval layer” is still too narrow a name.

Agents need more than knowledge. Their context can include:

Knowledge
Operational state
User identity
Permissions
Conversation state
Task state
Workflow state
Business rules
Historical actions
Environment state
Evidence

That is why I prefer Enterprise Context Fabric:

Enterprise Context Fabric
   ├── Knowledge
   │     ├── Search
   │     ├── Documents
   │     └── Data
   ├── State
   │     ├── Tasks
   │     ├── Memory
   │     └── Workflows
   ├── Policy context
   │     ├── Identity
   │     ├── Access
   │     └── Governance
   └── Evidence

If a model’s context window is its working memory, the context fabric supplies the infrastructure for longer-term memory, knowledge access, state access and governed use of enterprise information.

The distinction matters. A larger context window does not, by itself, solve the questions of which information is current, which state is authoritative, or which records a particular agent is allowed to access.

9. A future of many enterprise agents

The value of this architecture becomes clearer if we assume that enterprises will operate many agents rather than one general-purpose AI assistant.

Developer, finance, HR, service, security, architecture, procurement, operations and data agents may all coexist.

If each independently connects to databases and SharePoint, builds its own RAG pipeline, manages permissions, records audits, creates indexes and integrates tools, the environment becomes difficult to control.

A more scalable arrangement is:

Enterprise agents
        ↓
Enterprise Context Fabric
   ├── Retrieval
   ├── State
   └── Governance integration
        ↓
Capability registry
   ├── MCP tools
   ├── API tools
   └── Native search
        ↓
Enterprise systems

The shared foundation allows business agents to specialise without rebuilding the same access and retrieval infrastructure for every use case.

10. Where KuWarden could fit

This architecture aligns closely with the direction I see for KuWarden.

Its most useful role would be an agent governance and control plane, rather than another RAG framework or a reimplementation of a vector database.

The context fabric would organise the context an agent can obtain. The control plane would govern under what conditions it may obtain that context, and what it may do with it.

The following capabilities describe a proposed architectural direction. They are not a claim that every capability is available in KuWarden today; its current product focus is governed AI-assisted software delivery.

Enterprise agent
        ↓
Enterprise Context Fabric
   Retrieval · State · Capabilities
        ↕
KuWarden control plane
   ├── Policy and access
   ├── Evidence and provenance
   ├── Approval and human gates
   └── Audit and enforcement

Capability registry

KuWarden could maintain a catalogue of the capabilities agents are permitted to use: MCP servers, APIs, databases, search engines, CLIs, Git, CI/CD, cloud resources and internal tools.

Each capability would carry its owner, version, description, permissions, risk classification, data classification, allowed agents and allowed actions.

Together, these records would form an agent capability catalogue.

Agent identity

Knowing the human user’s identity is no longer enough. An execution may involve a chain:

Human user
    ↓
Business agent
    ↓
Sub-agent
    ↓
Tool

The control plane could maintain human identity, agent identity, delegation chains, execution identity and service identity. This would make it possible to establish who performed an operation on whose behalf.

Policy engine

A policy engine would be central to this role. Policies could express boundaries such as:

Finance agent
   CAN READ Finance SharePoint
   CAN QUERY Invoice Database
   CANNOT READ HR Payroll

Coding agent
   CAN READ Production Logs
   CANNOT MODIFY Production Database

They could also require a human decision before a particular action:

IF action = production deployment
THEN require human approval

The goal is policy enforcement designed for agents, with permissions enforced outside the model’s own reasoning.

Retrieval governance

KuWarden could record what an agent asked, which sources were searched, which tools were called, which records were returned and which evidence was ultimately used.

An answer would then have an inspectable retrieval trace, rather than leaving the information-gathering process opaque.

Evidence and provenance

The control plane could maintain the relationships between:

Claim
  ↓
Evidence
  ↓
Source and version
  ↓
Timestamp
  ↓
Agent
  ↓
Action

This matters particularly for architecture decisions, code changes, security decisions, compliance work and financial decisions. Reviewers need to understand both the conclusion and the material on which it depended.

Approval gates

Higher-risk actions—including production deployments, database updates, infrastructure changes, security-policy changes and financial approvals—could follow an explicit sequence:

Agent proposal
      ↓
Evidence
      ↓
Policy check
      ↓
Human approval
      ↓
Execution

The approval would become part of the execution record, tied to the proposed action and its evidence.

Agent execution audit

Ultimately, an enterprise needs to answer a connected set of questions:

• What happened, and who initiated it?
• Which agent performed the work?
• Which tools were called and what data was accessed?
• What evidence was used?
• Which policy permitted the action?
• Who approved it?
• What changed?

KuWarden could become the audit centre for that execution chain.

11. A clearer long-term position for KuWarden

This suggests a potential progression from AI-assisted software governance towards an enterprise agent governance and control plane.

Software development provides a practical first use case:

Developer → Coding agent → Git / CI / Cloud

The governance concerns are concrete: code changes, architecture rules, review, approval, CI/CD and audit.

A later expansion could cover:

Business agent
      ↓
Enterprise Context Fabric
      ↓
Enterprise tools

The control plane would govern identity, capabilities, policy, evidence, approval, execution and audit across these interactions.

Its enduring value would be agent governance infrastructure, independent of any particular model, RAG framework or agent framework.

12. My view of the next enterprise AI architecture

As a simplified account of the recent architectural conversation, I would describe the progression this way:

Around 2023
LLM + vector database + RAG

2024–2025
LLM + RAG + tools + agents

The more mature architecture I expect to see is:

Models
   ↓
Agent runtime
   ↓
Enterprise Context Fabric
   ↓
Capability layer
   ↓
Enterprise systems

A control plane would operate across those layers, covering identity, policy, approval, evidence, provenance, audit and enforcement:

Enterprise AI
      ↓
Agent runtime
      ↓
Enterprise Context Fabric
   ├── Retrieval → Search / data
   ├── State
   └── Capabilities → MCP / APIs
      ↓
Enterprise systems

──────────────────────────────────
Agent control plane / KuWarden

Identity · Policy · Evidence
Approval · Provenance · Audit
Enforcement · Risk

I increasingly see Enterprise Context Fabric + Agent Control Plane as two important foundations for enterprise AI.

The first answers: How does an agent obtain the context it needs to complete its task?

The second answers: What is the agent permitted to do, and what has it done?

Answering these questions creates a stronger foundation for moving agents from demonstrations into production at scale.

RAG, vector search, MCP, SQL, graphs and search engines remain useful. Within this architecture, they become capabilities that can be selected, combined and extended inside the fabric.

That is the direction in which I believe enterprise AI architecture can mature: context access and accountable execution designed as infrastructure, with individual technologies chosen to serve those responsibilities.

---

Explore KMS ITC’s Intent-to-Evidence architecture method and KuWarden’s current product capabilities.