What Comes After a Rapid Prototype? Building an Enterprise Inquiry AI into a Production System

🎥 The YouTube version is also available:

Redefining Rapid Prototyping in the Age of Generative AI— From Building Faster to Learning Faster

Books: Rapid Prototyping in the Age of Generative AI: Learning with Streamlit　Building with Docker Al Business Systems

The purpose of rapid prototyping, as discussed in “Redefining Rapid Prototyping in the Generative AI Era: The Difference from Vibe Coding Is Not Speed, but Learning and Decision-Making”, is not to create a working screen as quickly as possible. It is to use the data and real operations that already exist to learn about the most uncertain questions early, and to make the next decision with greater confidence.

So what should happen once a prototype has shown that a business problem has value and clarified the data and human review points that are required?

At that point, the prototype should not simply be turned into a production system. A prototype is an instrument for learning; a production system is infrastructure for operating a business process continuously and safely. Their purposes, design principles, and definitions of completion are different.

This article uses an enterprise inquiry AI—one that supports questions about internal policies, IT, HR, and general affairs—as a concrete example. It explains how to turn what was learned from a rapid prototype into a production implementation, using AI development environments and agents such as Kiro, Claude Code, and Codex.

A Concrete Use Case: Enterprise Inquiry AI

The use case is a service that provides first-line responses and appropriate guidance for employee inquiries, based on internal policies, operating procedures, FAQs, application status, and organizational rules.

Such inquiries can span areas including:

• HR and general-affairs questions about expense reimbursement, business travel, leave, and benefits
• IT questions about PCs, accounts, software, and networks
• Questions about business rules for purchasing requests, contracts, information management, and security
• Questions about application procedures, required documents, responsible contacts, and processing status

In a prototype, the system uses the question text, department, user role, relevant policy, the latest policies and FAQs, and application status to recommend one of the following responses:

• Act: The evidence is sufficient, and the employee can proceed with the appropriate procedure or next action.
• Ask: Information necessary to answer is missing, so the employee must be asked for clarification.
• Stop: The question concerns information outside the user’s authority, personal information, an HR or legal judgment, or an urgent incident. The system stops automated answering and routes the matter to a specialist contact.

The AI’s role is not to replace final decisions by HR, legal, or information-security professionals. Its role is to retrieve evidence-based information, organize answer candidates, identify missing information, direct the employee to the right contact, and create an inquiry record. Exception handling and accountability remain with authorized people.

Making this boundary explicit at the outset clarifies the issues involved in productionization. A production system needs more than more natural answers. It must safely support employee actions by addressing information freshness, access rights, answer provenance, personal information, escalation, auditability, and stop conditions for potentially misleading guidance.

Before Moving from Prototype to Production, Define the Entry Criteria

Before beginning production implementation, convert impressions from the prototype into requirements. If a team proceeds with only the vague sense that “it looks useful,” implementation details will once again get ahead of the real purpose.

First, agree on the following four items in a one-page adoption decision sheet.

Then classify discoveries from the prototype into three categories:

1. What moves into production: effective answer screens, evidence display, follow-up questions, and the Act/Ask/Stop model
2. What must be prepared before implementation: authoritative policies, document update ownership, access rights, inquiry classification, handoff destinations, and SLAs
3. What will not be done in this release: coverage of every policy domain, advanced individualized HR decisions, and automatic updates to external systems

The first production release should not be defined as an internal AI that answers everything. It should be defined as a safe inquiry-support capability with a limited policy scope and a limited set of users.

Phase 0: Investigate the Existing Environment and Specify What the System May and May Not Answer

When connecting to existing internal portals, document platforms, and application systems, investigation matters more than implementation at the beginning. This is where Kiro’s spec-driven workflow, or repository and integration-specification investigation by Claude Code or Codex, can add value.

The deliverable from this investigation should not be merely a list of documents or code. At a minimum, it should record:

• Where policies, procedures, FAQs, application systems, and organizational information reside
• The authoritative source, owning department, revision date, approval status, and update owner for each document
• Viewing permissions for each document and display controls based on user attributes
• Identity verification and authorization required to retrieve individual data such as application status
• Current inquiry channels, team responsibilities, SLAs, and escalation criteria
• Existing audit logs, incident contact procedures, and alternatives when the service is unavailable
• Areas that the system must never answer automatically, and conditions that always require human handoff

At this stage, do not ask the AI to “implement a chatbot.” First, create a map of knowledge, authority, and responsibility: where answer evidence resides, who updates it, and who may see it.

For example, if an old leave policy and a newer one coexist, improving retrieval accuracy alone cannot prevent incorrect guidance. The problem is not model selection; it is the definition of the authoritative source and accountability for updates.

Phase 1: Write the Production Specification from Acceptance Criteria

Next, write acceptance criteria that users and responsible staff can verify before designing the chat screen or choosing the technical architecture. With Kiro, these can be retained as specifications in a Requirements → Design → Tasks sequence. With Claude Code or Codex, the same content can be stored in the repository as issues, Markdown specifications, PR templates, and test cases.

For a first release of an enterprise inquiry AI, acceptance criteria might look like the following.

Functional Acceptance Criteria

• After authentication, employees can search and view only the policies and procedures appropriate to their department and authority.
• Every answer displays the title, revision date, and relevant passage link for its source document.
• When the evidence is insufficient, the AI does not complete the answer through speculation; it asks a follow-up question or guides the employee to the appropriate contact.
• When handling individual application status, only the employee concerned or an authorized manager can view it.
• Users can record whether an answer was helpful or unresolved and, when necessary, hand the inquiry off to the responsible team.
• During handoff, the system provides the responsible team with the question, confirmed information, referenced documents, and the AI’s answer candidate.

Non-Functional and Governance Acceptance Criteria

• The system retains an audit log of every question, referenced document, answer, follow-up question, escalation, and final staff response.
• Unauthorized users cannot access other employees’ application data, restricted documents, or personal information.
• If document synchronization fails, the AI does not claim that it has the latest information; it shows the user the situation.
• For urgent information-security incidents or safety-related inquiries, the system stops automated answering and directs the user to the emergency contact.
• During an outage, employees can return to existing inquiry channels and procedures.

With this approach, completion is not defined as “a conversational screen has been built.” It is defined as satisfying acceptance criteria that include evidence, authority, handoff, and stop conditions.

Phase 2: Design the Production Architecture as a Flow of Knowledge and Responsibility

A production system is not merely a chat screen, an LLM, and vector search placed side by side. For an enterprise inquiry AI, the system must define what information enters, who can see it, what evidence supports an answer, when a matter is handed to a person, and what is recorded.

The minimum production architecture is easier to understand as six layers.

The AI’s role should be contained within the retrieval and answer layer. AI-generated text should not be connected directly to HR or legal decisions, or to the disclosure of personal data. Every action must pass through the decision and authorization layer, which determines Act, Ask, or Stop according to conditions. This separation creates safety and explainability in production operation.

Phase 3: Divide Implementation into Small Milestones

When productionization is delivered as one large effort, integration problems accumulate near the end. Instead, divide the work so that every milestone delivers independent value and can be tested.

Milestone 1: Authentication, Authorization, and Access to Source Documents

The first capability is not conversation. It is the foundation for determining who can see which documents and information.

• Integration with SSO or existing authentication
• Role definitions for general employees, department managers, responsible staff, and administrators
• Access control at the document, organization, and personal-data levels
• An audit-log foundation
• A list and viewing screen for authoritative policies and FAQs

The completion condition for this milestone is that the right users can see only the right information, while tests reject unauthorized access.

Milestone 2: Knowledge Integration and Visibility of Freshness and Authoritativeness

Next, ingest policies, operating procedures, and FAQs. But the ability to search them is not success by itself.

• Store each document’s owning department, version, revision date, and approval status
• Detect expired and duplicate documents
• Apply permission metadata to search results
• Flag ingestion failures and stale information
• Make it possible to see why a document cannot be used as answer evidence

At this stage, it is itself an important outcome if the organization realizes, “The materials existed, but nobody had decided who was responsible for certifying that they were correct.”

Milestone 3: Evidence-Based Retrieval and Answers

Do not begin by allowing the generative AI to answer freely. Start by retrieving approved documents that the authenticated user is authorized to access, then generate answers grounded in those documents.

• Display the source document and relevant passage for every answer
• Do not make definitive statements when evidence is insufficient
• Show a warning or switch to Ask when the revision date is old
• Divide the answer template into “what you can do,” “what needs confirmation,” and “the next procedure”
• Test representative questions, paraphrases, no-match cases, and conflicting policies

For example, if a travel policy contains information only for domestic travel but the user asks about international travel, the system should not produce plausible general advice. It should use Ask to clarify the applicable policy or hand the case off to the responsible contact.

Milestone 4: Follow-Up Questions and Escalation to People

The value of an enterprise inquiry system lies not only in answering questions it can answer, but also in handling questions it cannot answer correctly.

• Follow-up questions by inquiry category
• Stop conditions for labor, legal, personal-information, and security matters
• Ticket creation with responsible team, priority, and SLA
• Handoff data that includes information already provided by the user and the documents referenced
• Emergency contact information shown immediately for urgent cases rather than continuing the AI conversation

Do not hide human handoff as an AI failure. By analyzing the questions that most often become Ask or Stop, teams can improve missing FAQs, ambiguous policies, and procedural friction.

Milestone 5: Individual Status Checks and Procedure Support

Once policy guidance is stable, add individual application-status checks and procedure support in a limited scope.

• Application-status checks by identity-verified employees
• Viewing pending approvals according to authority
• Links to procedures, required documents, and deadlines
• Masking and auditability for answers containing individual status
• Delegating changes and approvals to existing business systems’ authorization and confirmation screens

The AI should not change or approve an application on its own. It supports guidance and confirmation; actual changes must pass through explicit user action and existing approval workflows.

Milestone 6: Audit, Monitoring, and Recovery

A production system is not complete just because its normal answer screen works.

• Who asked what, when, and which documents were referenced
• Which retrieval settings, model version, and prompt version were used
• Which answers were resolved, required follow-up, were handed off, or were stopped
• Whether document synchronization or AI calls failed
• What must be re-ingested after failure and from which state recovery should begin

Establish these items as logs, metrics, alerts, and operating procedures. The system should be designed so employees can return to the internal portal, procedures, and existing contacts even if AI functionality is stopped during an outage.

How to Use Kiro, Claude Code, and Codex

In production implementation, it is useful to assign different roles to different tools.

The important point is not to delegate the definition of completion to a tool. Place at least the following hard gates at every milestone.

1. Type checking passes.
2. Unit, integration, and authorization tests pass.
3. The build passes.
4. Security checks covering inputs, outputs, document access, and personal information pass.
5. Major user flows, handoffs, and error guidance are verified in a real browser.
6. Validation results are recorded against the acceptance criteria.

Kiro makes it easier to organize this process as a coherent spec-driven workflow. With Claude Code and Codex, the same structure can be built through repository rules, CI, tests, PR templates, and reviews by separate agents.

Pre-Production Validation Must Go Beyond Answer Quality

Validation of an enterprise inquiry AI cannot end with a single metric such as answer accuracy. At a minimum, verify the following.

Do not end this validation with ideal test FAQs alone. Conduct user acceptance testing with real but anonymized and masked inquiries, difficult exceptions, and cases immediately after a policy revision. Observe not only whether users say “it was useful,” but also which questions frequently become Ask, why staff revise answers, and where handoffs become delayed.

Phased Rollout: Do Not Start with Every Department and Every Policy

Begin production operation with a limited rollout, not an organization-wide deployment.

Start with one policy domain, one department, published policies and FAQs, and one inquiry scenario. A recommended sequence is as follows.

1. Shadow Mode: The AI produces answer candidates and evidence, but does not answer users directly; responsible staff evaluate the outputs.
2. Advisory Mode: Users access only public FAQs and procedural guidance. Resolution status, unresolved cases, and handoff reasons are recorded.
3. Controlled Action: The system supports low-risk, identity-verified status inquiries and guidance to existing application screens.
4. Scale: Review staff edits, incorrect guidance, handoffs, and audit results, then expand the policy scope gradually.

This sequence allows teams to validate not only answer quality, but also responsibility for knowledge updates, access control, user acceptance, escalation, and operational workload.

Conclusion: Productionization Is Not Making the Prototype Bigger

The production system that should follow a rapid prototype is not simply a prototype with more features.

It is a system that connects authoritative knowledge, information freshness, AI evidence, human handoff, authorization, records, stopping, and recovery in order to support employee inquiries and actions.

A spec-driven environment such as Kiro is valuable when teams need to align requirements, design, tasks, and quality gates for planned implementation. Claude Code and Codex are strong when teams need to investigate existing systems, make cross-cutting changes, and solve problems.

But regardless of which AI development tool is used, the central question of productionization remains the same.

On the basis of which information does this system support whose action, under what conditions does it ask for clarification, and under what conditions does it hand the matter to a person or stop?

When this question is translated into specifications, tests, operations, and audit mechanisms, a rapid prototype can finally grow into a production system that an organization can trust.

Chinoba

Chinoba

Intelligence as Relationship

Research Platform

founded by

Masao Watanabe

AI Systems Architecture
Decision Trace
Human–AI Coordination
Algorithmic Governance