The question I am asked most often by business leaders evaluating AI agent investment is some version of: "How do we know it will actually work?"
It is a better question than it sounds. Most AI vendor conversations focus on capability — what the model can do, how large its context window is, which benchmark it scores highest on. These are not irrelevant considerations, but they are not the primary determinant of whether an AI agent generates measurable value in a specific business context.
The primary determinant is how the agent's knowledge is structured. And structuring knowledge for reliable professional performance is a discipline that sits at the intersection of business analysis, cognitive science, and domain expertise — not software engineering.
This is why the team that builds an AI agent matters as much as the technology it runs on.
Every AI agent operates on a knowledge architecture — the structured representation of what the agent knows, how it knows it, and how it reasons about it. In a generic deployment, this architecture is implicit: it is whatever the underlying model has learned from its training data, shaped by a brief system prompt that attempts to focus its behaviour.
In an expert-trained agent, the knowledge architecture is explicit, deliberate, and validated. It has been designed by people who understand both the domain and the failure modes of AI reasoning, and it has been tested against the specific queries and contexts in which the agent will operate.
The difference between these two approaches is not primarily a matter of the underlying AI technology. It is a matter of the intellectual work done before the first user interaction takes place.
The MBA component of knowledge engineering is concerned with a question that is deceptively simple: what does this agent need to know in order to generate measurable value for this organisation?
This question requires a structured analysis of the business context in which the agent will operate. What decisions does the agent's output inform? What is the cost of an incorrect decision in this context? What is the value of a correct decision that would otherwise have required significant human time? What are the regulatory, reputational, and operational constraints that bound the agent's behaviour?
These are business analysis questions, not technical ones. Answering them well requires the kind of structured thinking about value creation, risk management, and organisational context that MBA-level training develops — and that most AI implementation projects skip entirely.
The output of this analysis is a value map: a structured representation of the specific use cases the agent will serve, the decisions it will inform, the accuracy requirements for each use case, and the consequences of failure. This map drives every subsequent decision in the knowledge engineering process — what to include, what to exclude, where to invest in depth, and where to draw the agent's knowledge boundaries.
Without this map, knowledge engineering is guesswork. With it, every design decision has a clear rationale grounded in business value.
The PhD component of knowledge engineering is concerned with a different question: how does an expert in this domain actually reason about the problems the agent will encounter?
This is not a question about what an expert knows — it is a question about how they think. The distinction matters because AI agents do not simply retrieve stored information; they reason about it. And the reasoning patterns that produce reliable professional judgement in a domain are not always obvious, even to practitioners in that domain.
PhD-level expertise contributes to knowledge engineering in three specific ways.
First, domain depth: the ability to identify not just the surface-level content of a knowledge domain but the underlying principles, the edge cases, the contested interpretations, and the failure modes that separate expert-level understanding from competent generalism. A system built on surface-level knowledge will perform well on standard queries and fail on the edge cases that arise in real professional practice. A system built on deep domain knowledge handles edge cases reliably because it understands the principles that govern them.
Second, reasoning pathway design: the ability to map the inferential chains that connect domain knowledge to professional conclusions. In many domains, the path from a question to a reliable answer involves multiple steps of reasoning, each of which can introduce error if the underlying knowledge is imprecise. Designing these reasoning pathways explicitly — and testing them against adversarial queries — is a core component of expert knowledge engineering.
Third, uncertainty calibration: the ability to identify the questions to which the honest answer is "it depends" or "I don't know" — and to design the agent's response to those questions in ways that are genuinely useful rather than evasively unhelpful. This requires a level of domain mastery that allows the engineer to distinguish between genuine uncertainty and apparent uncertainty, and to communicate the distinction clearly.
The most valuable knowledge engineering work happens at the intersection of the MBA and PhD contributions — where business logic meets domain expertise.
Consider a compliance agent for a financial services firm. The MBA analysis identifies that the highest-value use case is answering questions about client suitability obligations — a domain where incorrect answers carry significant regulatory and reputational risk. The PhD-level compliance expertise maps the specific regulatory framework, identifies the edge cases that arise most frequently in practice, and designs the reasoning pathways that connect regulatory requirements to client-specific situations.
The integration of these two contributions produces an agent that is not just knowledgeable about compliance — it is knowledgeable about the specific compliance questions that matter most to this organisation, in the specific business context in which they arise, with explicit design for the edge cases where the stakes are highest.
This is what we mean when we say that an agent has been built from the ground up. It is not a generic model that has been pointed at a compliance knowledge base. It is a system whose entire knowledge architecture has been designed around the specific value-generation requirements of a specific business context.
One of the most important outputs of the MBA contribution to knowledge engineering is a measurement framework — a set of metrics that allow the organisation to evaluate whether the agent is generating the value it was designed to generate.
This framework is built around the value map developed in the initial analysis. For each use case, it defines what a successful outcome looks like, how it will be measured, and what the baseline comparison is. This allows the organisation to evaluate AI performance against a business standard — not just a technical benchmark.
The measurement framework also provides the foundation for ongoing knowledge engineering. As the business context evolves — new regulations, new products, new organisational structures — the framework identifies where the agent's knowledge needs to be updated and provides the criteria for evaluating whether the update has been successful.
Without this framework, AI performance management defaults to subjective user feedback and generic satisfaction scores. With it, the organisation has a rigorous, business-grounded basis for evaluating and improving its AI investment over time.
Three practical implications follow for organisations considering AI agent investment.
The first is that the composition of the team building the agent should be a primary evaluation criterion. Ask not just about the technology platform but about who will be doing the knowledge engineering work. What domain expertise do they bring? What business analysis capability? How do they validate that the agent's knowledge is accurate and complete for the specific use cases it will serve?
The second is that the knowledge engineering process should be transparent and documented. You should be able to see the value map, the domain knowledge sources, the reasoning pathway designs, and the adversarial test results. If the vendor cannot provide this documentation, the knowledge engineering has not been done rigorously.
The third is that the measurement framework should be agreed before deployment, not after. The criteria for success should be defined in terms of business outcomes — decision quality, time saved, error rates, user confidence — not in terms of technical metrics that may not correlate with business value.
The question "how do we know it will actually work?" has a specific answer when the knowledge engineering has been done properly: you know because the agent's knowledge architecture has been designed against a clear value map, validated by domain experts, and tested against the specific queries and contexts in which it will operate.
The organisations that are generating measurable, sustained ROI from AI agents are not the ones with access to the most advanced technology. They are the ones that have invested in building knowledge architectures that are genuinely fit for purpose — and that have the measurement frameworks to know the difference.
That investment requires expertise that goes beyond software engineering. It requires the kind of structured thinking about business value, domain knowledge, and reasoning quality that MBA and PhD-level training develops.
The technology is a commodity. The knowledge architecture is the competitive advantage.
View Pricing
Explore our Starter, Professional, and Enterprise tiers and find the right fit for your organisation.
See Pricing