Existing Agent Frameworks Are Not Enough: Building a Custom Framework Optimized for Runtime OS

🎥 The YouTube version is also available:

Runtime OS Architecture: Centralized, Edge & Federated AI Decision Systems

Books: Runtime OS Implementation Guide: A Practical Architecture Integrating AI Agents, Human Gates, and Decision Traces

Introduction

With the emergence of the OpenAI Agents SDK, Claude Agent SDK, AutoGen, LangGraph, and similar platforms, the environment for developing AI agents is rapidly becoming more mature.

Delegation to specialized agents.

Tool calling.

Human-in-the-loop.

State management.

Distributed execution.

The capabilities required to build multi-agent systems for real business use are already widely available.

However, when AI is introduced into enterprises and social systems, a different set of questions arises.

• Who ultimately determines the final decision?
• What is an agent permitted to execute?
• At what point should humans intervene?
• How should the scope of authority be constrained?
• Who is accountable afterward?
• Can we demonstrate why a particular decision was made?

Existing agent frameworks do not sufficiently address these questions.

This is precisely why a custom framework optimized for Runtime OS is needed.

This does not mean creating yet another Agent SDK.

It means building a new execution foundation that governs decisions on top of—and while making use of—existing, capable agent frameworks.

What Agent Frameworks Are Optimized For

Today’s agent frameworks primarily address challenges such as:

• Task decomposition
• Delegation to specialized agents
• Tool execution
• State management
• Conversation management
• Coordination among agents
• Execution monitoring
• Evaluation of execution results

In other words, their purpose is to enable agents to operate efficiently.

But enterprise operations are not simply a matter of task execution.

For example, suppose an AI proposes issuing a refund to a customer.

From the perspective of an agent framework, the flow might be:

1. Retrieve customer information.
2. Verify the contract.
3. Calculate the refund amount.
4. Call the payment API.

From the enterprise perspective, however, this is not enough.

• Is the refund permitted under the contract?
• Does the person in charge have approval authority?
• Does the refund exceed the authorized limit?
• Is there a possibility of fraud?
• Is managerial approval required?
• Has an audit trail been retained?

These are issues separate from the capabilities of the agent itself.

Designing Around Decisions, Not Agents

In Runtime OS, the center of design changes.

The agent is not the center.

The decision is.

Event
 ↓
Structured Signal
 ↓
Context Evaluation
 ↓
Policy Evaluation
 ↓
Decision
 ↓
Boundary Check
 ↓
Human Gate
 ↓
Command Permit
 ↓
Agent Execution
 ↓
Outcome
 ↓
Decision Trace Ledger

Agents perform analysis.

They make proposals.

They formulate plans.

But all of these are merely signals.

It is the Runtime that ultimately determines the decision.

AI produces signals.

Only the Runtime produces decisions.

Agents execute only approved, traceable decisions.

Designing a Framework Specifically for Runtime OS

Rather than using existing agent frameworks as they are, a framework designed specifically for Runtime OS could consist of the following layers.

Layer 1: Interaction Core

This layer standardizes all inputs.

• User input
• Sensor data
• Business events
• System notifications
• Proposals from agents

These inputs are transformed into Structured Signals.

Layer 2: Decision Runtime

This is the core layer of Runtime OS.

{
  "signal": {},
  "context": {},
  "policy": {},
  "risk": {},
  "delegation": {},
  "decision": "Act | Ask | Stop"
}

This layer determines the decision.

Layer 3: Policy Engine

This layer evaluates the validity of a decision.

Examples of evaluation criteria include:

• Organizational policies
• Business rules
• Laws and regulations
• Compliance requirements
• Data access permissions
• Financial limits
• Time constraints

Layer 4: Boundary Engine

This layer constrains the scope of what may be executed.

boundary:
  action:
    - read
    - create
    - update

  prohibited:
    - delete_customer
    - transfer_money

  amount_limit:
    max: 100000

  expiration:
    ttl: 24h

Without boundaries, agents could act without meaningful limits.

Layer 5: Human Gate

This layer manages human approval.

Low Risk
 ↓
Automatic Execution

Medium Risk
 ↓
Manager Approval

High Risk
 ↓
Executive Approval

Human-in-the-loop is not merely an approval function.

It represents the organization’s boundary of responsibility itself.

Layer 6: Command Permit

This layer issues execution authority in the form of a token.

{
  "action": "refund",
  "target": "customer_123",
  "scope": "finance",
  "expires": "2026-08-21T00:00:00Z",
  "max_execution": 1
}

Without a permit, an agent cannot operate external systems.

Layer 7: Agent Runtime Layer

Only at this point are existing agent frameworks used.

Runtime OS
        ↓
OpenAI Agents SDK

Claude Agent SDK

AutoGen

LangGraph
        ↓
Enterprise Gateway

There is no need to replace existing frameworks.

OpenAI can remain OpenAI.

Claude can remain Claude.

AutoGen and LangGraph can continue to be used as they are.

However, none of them should hold Decision Authority.

They are treated solely as implementation components operating within Runtime OS.

New Benchmarks for Runtime OS

Existing benchmarks have evolved around areas such as:

• GAIA
• AgentBench
• OSWorld
• Agent-SafetyBench

Runtime OS, however, requires a different set of evaluation criteria.

These metrics do not measure agent performance.

They measure whether an organization can operate AI safely.

Conclusion

The next generation of AI systems will shift from competing to create “smarter agents” toward competing to make “more trustworthy decisions.”

What matters is not increasing the number of agents.

What matters is safely integrating AI capabilities into organizational decision-making.

For that, agent frameworks alone are not enough.

What is needed is a Runtime OS for governing decisions.

And to implement that Runtime OS, it is not sufficient simply to use existing agent frameworks. A new framework optimized for Runtime OS is required—one centered on Decision, Authority, Human Gate, and Trace.

It is not a new agent runtime.

It is Decision Infrastructure for integrating AI, agents, people, organizations, and physical systems.

Chinoba

Chinoba

Intelligence as Relationship

Research Platform

founded by

Masao Watanabe

AI Systems Architecture
Decision Trace
Human–AI Coordination
Algorithmic Governance