Recently, I had the opportunity to speak at VTI Japan on the theme, “From AI as an Answer Tool to AI as a Knowledge and Decision-Making Infrastructure.”
As generative AI and RAG adoption continue to expand, many companies begin by building AI systems that search internal documents and answer questions. This is a highly effective starting point. Existing manuals, FAQs, specifications, and past reports can be used to prepare initial answers quickly.
However, for AI to remain useful in real business operations, simply producing an answer is not enough.
Which source and which passage supports that answer?
Under what conditions does it apply, and when should an expert be consulted?
Who makes the final decision, carries out the action, and improves the process afterward?
In the session, I shared an approach that treats RAG, LLMs, and agents not merely as technical components, but as elements of a broader architecture for supporting organizational knowledge and decision-making.
Standard RAG Is an Effective Starting Point
Standard RAG is a practical way to make use of documents distributed across an organization.
It enables rapid prototyping.
It makes use of existing materials.
It can speed up the initial response to inquiries and investigations.
However, operational documents often contain conditions, exceptions, revision histories, ownership boundaries, abbreviations, and alternative names. If documents are split and retrieved only by fixed character length, important conditions or exceptions may be omitted from an answer.
The key is not to rebuild everything at once. It is to add the required capabilities step by step.
Five Elements for Evolving Answer AI into Operational AI
In the session, I introduced an architecture that builds on standard RAG by adding the following elements as needed.
First, Semantic Chunking.
Rather than splitting text into fixed-length segments, semantic chunks treat the subject, conditions, procedure, exceptions, and evidence as a meaningful and complete unit. This makes it easier to present the scope of applicability and relevant cautions together with an answer.
Next, Ontology.
Within an organization, the same equipment, component, product, or process may be described differently across departments and documents. By connecting aliases, abbreviations, hierarchies, and attributes to common concepts, companies can improve retrieval accuracy and turn organization-specific knowledge into a reusable asset.
Then, Graph RAG.
In business operations, finding similar documents is not enough. Teams often need to trace relationships among equipment, components, procedures, exceptions, responsible personnel, revision histories, and past incidents. Graph RAG provides a way to explore these relationships and assemble the context needed for an answer.
The next components are DSLs and a Decision Runtime.
Decisions involving safety, quality, approvals, and escalation should not rely solely on the AI’s unconstrained reasoning. Organizations need verifiable rules that define when additional confirmation is required, when a case should be handed over to an expert, and when the AI must stop.
Finally, there is the Decision Trace.
It records what the AI referred to, which candidate it proposed, what a person changed, what action was ultimately taken, and what result followed. This turns the history into more than an audit log. It becomes organizational memory that supports the next improvement.
Defining AI’s Scope Through Act / Ask / Stop
The goal of operational AI is not to automate everything.
There are areas where AI can present an answer candidate.
There are areas where it should request additional information or human review.
There are areas where it should stop and hand the case over to an expert.
Clarifying these boundaries helps create AI systems that people can use with confidence in real work.
This is particularly important in work involving safety, quality, authority, or customer impact. In these contexts, it matters more to define who can make which decision under which conditions than to provide an answer that merely appears plausible.
Fast Development Is Not Just About Writing Code Faster
Another topic of the session was rapid prototyping.
Fast development does not simply mean writing code more quickly. It means quickly creating a state in which customers and operational teams can make the next investment decision.
Instead of aiming immediately for company-wide deployment or large-scale integration, the team narrows the target business process and users, then builds a prototype around representative cases. It then validates value, available data, decision ownership, and operational boundaries using real cases.
The outcome of a prototype can be organized into three categories:
- Act: The necessary conditions are in place, so the team proceeds to a limited production design.
- Ask: Additional data preparation, validation, or stakeholder alignment is needed.
- Stop: The team revises the hypothesis or changes the target because the value or feasibility conditions have not been confirmed.
This approach treats prototyping not as an activity for rushing toward a finished product, but as an activity for making the next decision with greater confidence.
Toward Reusable Operational AI
Each customer has different operational challenges. Yet many components can be reused: knowledge ingestion, semantic structuring, retrieval, terminology integration, rule evaluation, human review, and decision-history recording.
Rather than building one large application for every individual project, organizations can combine the capabilities they need while separating customer-specific knowledge from reusable components. This makes it possible to validate value quickly and develop more reliable operational AI over time.
One point became especially clear through this session: the central question in enterprise AI is not only which model is the most capable.
To use AI effectively in an organization, we need to design how knowledge is structured, which evidence supports a decision, who can decide what, and how the results are fed back into continuous improvement.
Introducing AI as an answer tool is an important first step. The next challenge is how to evolve it into a knowledge and decision-making infrastructure. I will continue to explore this question through both research and implementation.
Chinoba
Intelligence as Relationship
Research Platform
founded by
Masao Watanabe
AI Systems Architecture
Decision Trace
Human–AI Coordination
Algorithmic Governance
Related Research
This topic is part of the Chinoba Knowledge Base.
コメント