Trust Engine — A Reasoning Engine That Computes Expectations in the Age of AI

🎥 The YouTube version is also available:

Trust Infrastructure: Building the Foundation for Human–AI Collaboration

Books: Trust Infrastructure Practical Guide: AI-Era Trust Infrastructure Design with Decision Trace, Knowledge Flow, and Trust Engine

Introduction

As generative AI and AI agents begin to operate within society, what we require of AI is no longer limited to its ability to return correct answers.

What we truly want to know is:

“Will this AI behave as expected in this situation, right now?”

This question is not unique to AI.

We ask it, often unconsciously, when we delegate work to another person, enter into a contract with a company, begin a joint research project, or coordinate across organizations.

“Can I entrust this to this party?”

“Will they keep their commitments under these circumstances?”

“Will they make appropriate decisions when something unexpected happens?”

We do not trust a person or organization unconditionally as such.

What we actually trust is:

the future in which that party makes decisions and takes actions as expected in the present situation.

Trust, therefore, is not merely an assessment of the past.

It is an expectation about the future.

How, then, are such expectations formed?

And how can we implement, on computers, the process of expectation formation that people perform intuitively?

The core architecture we propose to address this question is the Trust Engine.

What Is a Trust Engine?

A Trust Engine is:

an engine that reasons about the decisions and actions expected in the current situation.

Its purpose is not simply to evaluate AI.

Nor is it to monitor people.

It is not an engine that assigns a fixed Trust Score to another party, either.

Rather than answering the abstract question:

“Is this party trustworthy?”

the Trust Engine addresses the concrete, context-dependent question:

“Within this purpose, these constraints, and this environment, what decisions is this party likely to make, and what actions are they likely to choose?”

Even the same AI may make different decisions when its objective changes.

Even the same person may act differently when their role or constraints change.

Even a party that has succeeded in the past may not produce the same outcome if the present environment has changed significantly.

Trust is therefore not a fixed attribute of a party.

Trust is:

a dynamic expectation that arises from the relationship among a party, its purpose, constraints, environment, knowledge, and role.

The Trust Engine analyzes these relationships and infers expected behavior.

Why Is a Trust Engine Necessary?

Traditionally, trust has often been assessed through past performance and reputation.

“It has not caused problems in the past.”

“It has a high success rate.”

“It holds professional qualifications.”

“It is highly rated by many users.”

Such information is important when judging trust. Yet it is not sufficient.

For example, an AI that has demonstrated high accuracy in the past may make incorrect decisions in an environment it has never encountered. An experienced professional may also make atypical decisions when information is missing or time pressure is severe. Even within the same organization, expected behavior can change when management policy changes.

Past performance, in other words, does not guarantee the future.

What is needed is not to extend past performance directly into the future, but to:

apply past decisions to the current situation and infer future behavior.

This requires, at minimum, the following information:

• The circumstances in which past decisions were made
• The objectives pursued
• The constraints in effect
• The knowledge and rules used
• The decisions selected
• The actions those decisions led to
• The outcomes ultimately produced
• What is the same and what is different between the present and past situations

The Trust Engine integrates these elements to generate expectations appropriate to the current situation.

Inputs to the Trust Engine

The Trust Engine does not calculate trust from a single data source. It performs reasoning by integrating past decisions with current knowledge, goals, constraints, roles, and environmental conditions.

Its five primary inputs are as follows.

1. Decision Trace

The first input is the Decision Trace: a record of past decision-making.

A Decision Trace may include, for example:

• Situation
• Goal
• Constraint
• Context
• Alternatives
• Decision
• Action
• Result
• Feedback

What matters is not merely whether a decision succeeded or failed. What matters is the complete decision-making process: the situation, purpose, constraints, and reasons for selecting a particular decision.

The Trust Engine retrieves Decision Traces that resemble the current situation and refers to past decision patterns.

2. Knowledge

The second input is the Knowledge required for the current decision.

In companies and organizations, knowledge is distributed across sources such as:

• Design documents
• Standards and specifications
• Contracts
• Laws and regulations
• Quality standards
• Operating manuals
• Organizational policies
• Meeting records
• Knowledge Graphs
• Ontologies
• Executable rules and DSLs

This knowledge is needed to determine what is appropriate in the current situation. An action that was allowed in the past may no longer be allowed under current regulations or policies.

The Trust Engine uses current, integrated knowledge provided by Knowledge Flow to recognize differences between the past and the present.

3. Context

The third input is the current Context. Context includes not only environmental information but every state that affects the decision at hand:

• Current operational conditions
• Available resources
• Time constraints
• States of other agents
• Organizational state
• External environment
• Events in progress
• Uncertainty
• Exceptional conditions

Trust changes with Context. A party that can be entrusted with a task under normal conditions may require a different decision in an emergency.

4. Goal

The fourth input is the Goal to be achieved now. Even in the same situation, expected decisions differ when the goal differs.

• Maximize quality
• Prioritize safety above all else
• Reduce cost
• Meet a deadline
• Minimize customer impact

The Trust Engine does not merely predict what action is likely to occur. It evaluates whether that action meets expectations in light of the current Goal.

5. Constraints, Policies, and Roles

The fifth input is the applicable Constraints and Policies, together with the Role assigned to the party.

Constraints may include:

• Laws
• Safety standards
• Budget
• Deadlines
• Authority
• Available data
• Conditions requiring human approval

Policies are decision rules adopted by an organization or system. Roles define what a person or AI is responsible for and the extent of its decision-making authority.

Even AI systems with the same capabilities are expected to act differently when their Roles differ.

What Does the Trust Engine Calculate?

The Trust Engine is not a simple classifier. It performs multi-stage reasoning to generate Expected Behavior in the current situation.

Its basic processing flow is as follows:

Current Context
        ↓
Decision Trace Retrieval
        ↓
Knowledge Integration
        ↓
Goal / Constraint / Policy / Role Evaluation
        ↓
Expectation Graph Generation
        ↓
Expected Decision / Expected Action
        ↓
Risk / Confidence Evaluation
        ↓
Execution Monitoring
        ↓
Feedback and Learning

First, it analyzes the current Context.

Next, it retrieves past decisions similar to the current situation from the Decision Trace Model.

It then obtains currently valid knowledge, rules, and policies from Knowledge Flow.

Taking Goals, Constraints, Policies, and Roles into account, it reasons about:

• What decisions are expected
• What actions are predicted
• What evidence supports those decisions
• How much uncertainty exists
• What risks may arise

However, the Trust Engine does not produce only one kind of output.

It has two broad output modes. One is a graph output that preserves Trust as a structure. The other is a neural outputthat uses a GNN to aggregate that graph into numerical values and vectors.

Two Outputs of the Trust Engine

1. Representing Trust as a Graph

The first approach represents Trust as a graph structure.

Trust cannot be adequately expressed as a single number. If we are told that a “Trust Score is 0.82,” we still do not know:

• Why it is 0.82
• Which past cases influenced it
• Which rules were applied
• Which constraints increased risk
• What behavior is expected

For this reason, the Trust Engine constructs its reasoning result as an Expectation Graph, also called a Trust Graph.

For example, the graph can represent the following elements as nodes:

• Actor
• AI Agent
• Goal
• Role
• Context
• Constraint
• Policy
• Knowledge
• Decision Trace
• Expected Decision
• Expected Action
• Risk
• Result
• Evidence

The relationships among them are represented as edges:

AI Agent
   ├─ hasRole → Quality Inspector
   ├─ pursuesGoal → Defect Prevention
   ├─ constrainedBy → Safety Policy
   ├─ references → Decision Trace #128
   ├─ usesKnowledge → Inspection Standard
   └─ expectedToPerform → Stop Shipment

By representing Trust as a graph, we can make explicit:

who is expected to decide what and take which action, in which situation, and on the basis of which knowledge and rules.

Benefits of Graph Output

The principal advantage of graph output is explainability. The evidence composing Trust can be traced directly.

For example, an auditor can examine:

• The Decision Traces referenced
• The Policies applied
• The Constraints that influenced the decision
• The Knowledge used
• The path leading to the Expected Action

People can also revise part of the graph: correct an incorrectly applied Policy, update a changed Goal, or exclude an inappropriate past Decision Trace.

The Trust Graph is therefore well suited to Human-in-the-Loop validation and correction. It is especially appropriate for:

• High-risk decisions
• Work requiring auditability
• Regulatory explanation
• Human approval
• Analysis of decision rationale
• Structural visualization of Trust
• Root-cause analysis after an incident

Graph output does not simply indicate whether Trust is high or low. It is a representation that shows the relationships through which Trust is formed.

2. Producing Numerical Outputs with a GNN

In contrast, it is not practical for people to inspect an entire graph in every situation. In large-scale multi-agent environments, thousands or even tens of thousands of agents may interact at the same time. Exploring the full graph for each interaction makes real-time decisions difficult.

The Trust Engine therefore inputs the Trust Graph into a Graph Neural Network (GNN), which aggregates the graph structure into numerical representations.

A GNN integrates:

• Node attributes
• Edge relationships
• Influence from neighboring nodes
• Overall graph structure
• Past outcomes
• Current Context

It then generates a Trust Embedding. Depending on the application, this embedding can produce values such as:

• Trust Score
• Expected Behavior Probability
• Risk Score
• Confidence
• Policy Violation Probability
• Failure Probability
• Anomaly Score
• Escalation Probability
• Expected Outcome

For example:

Expected Action: Stop Shipment
Probability: 0.87
Risk Score: 0.21
Policy Compliance: 0.94
Human Escalation Required: 0.18

What Does a GNN Aggregate?

The important point is that a GNN does not simply learn historical data. Its input is the relationships that compose Trust itself, such as:

• Which Role an Agent has
• Which Goal it pursues
• Which Constraints it must follow
• Which Knowledge it can access
• Which Decision Traces it resembles
• What outcomes it has produced in the past
• How it is influenced by other Agents

The GNN aggregates information not only from each node but also from neighboring nodes and relationships.

An AI Agent may be highly capable in isolation, yet be less likely to behave as expected if it has been assigned an inappropriate Role, cannot access the Knowledge it needs, operates under conflicting Goals and Policies, or receives incorrect information from surrounding Agents.

A GNN can infer Trust while incorporating these relationships.

Benefits of Numerical Output

The advantages of GNN-based numerical output are speed and scalability. Once trained, a model can quickly generate Trust Embeddings and related scores from large graphs.

It is consequently suitable for:

• Real-time AI-agent selection
• Multi-agent coordination
• Anomaly detection
• Risk-based access control
• Automatic escalation
• Trust assessment for large volumes of transactions
• Safety checks before execution
• Dynamic Role Assignment
• Continuous Trust Monitoring

Numerical values alone, however, can obscure the grounds for a decision. The Trust Engine therefore retains both graph representations and GNN outputs rather than separating them.

The Trust Engine Has a Dual Representation

The defining feature of the Trust Engine is not choosing either a graph representation or a numerical representation. It is using both according to purpose.

Decision Trace Model
        │
Knowledge Flow
        │
Current Context
        │
        ▼
   Expectation Graph
        │
  ┌─────┴─────────────┐
  │                   │
  ▼                   ▼
Graph Output        GNN Encoder
  │                   │
  │                   ▼
Explanation      Trust Embedding
Evidence              │
Reasoning Path         ▼
  │             Trust Score
  │             Risk
  │             Confidence
  │             Expected Behavior
  └──────────┬────────┘
             ▼
       Trust Decision

Graph output explains the structure of Trust. GNN output aggregates that structure and converts it into numerical values that can be used quickly.

Put differently:

• The graph represents why
• The GNN represents to what degree

Even if a GNN outputs a Trust Score of 0.82, the underlying Trust Graph can show which Decision Traces influenced it, which Policies were evaluated, which Constraints introduced risk, and which Knowledge was missing.

This makes it possible to combine speed with explainability.

Internal Architecture of the Trust Engine

The Trust Engine is not a single, monolithic algorithm. It consists of multiple functional modules that work together to form, infer, evaluate, and learn Trust.

Context Analyzer

The Context Analyzer examines the current situation and constructs the Context needed for reasoning. Its primary targets include:

• Current Goal
• Constraints
• Policies
• Roles
• Events
• Organizational state
• Agent state
• External environment
• Available Knowledge
• Uncertainty

It normalizes input data and transforms it into a form that can be added to the Trust Graph.

Decision Trace Retriever

The Decision Trace Retriever obtains past decisions relevant to the current situation from the Decision Trace Model. It considers more than keyword similarity, including:

• Similarity of Situation
• Similarity of Goal
• Consistency of Constraints
• Consistency of Roles
• Commonality of Knowledge used
• Validity of Results
• Temporal recency

Retrieved Decision Traces are not used as unquestioned correct answers. They are referenced only after their differences from the present situation have been evaluated.

Knowledge Integrator

The Knowledge Integrator combines knowledge provided by Knowledge Flow, including:

• Knowledge extracted from documents
• Ontologies
• Knowledge Graphs
• Policies
• Rules
• Standards
• DSLs
• External data
• Human-validated Knowledge

It connects Decision Traces and the current Context on a Knowledge Graph to form the Expectation Graph.

Expectation Graph Builder

The Expectation Graph Builder is one of the central modules of the Trust Engine. It integrates Decision Traces, Knowledge, Context, Goals, Constraints, Policies, and Roles to generate a graph structure for expected decisions and actions.

This graph is both the explainable output of the Trust Engine and an input to the GNN.

Expectation Reasoner

Using the Expectation Graph, the Expectation Reasoner infers:

• Expected Decision
• Expected Action
• Expected Result
• Alternative Behavior
• Risk
• Uncertainty

It can combine rule-based reasoning, Knowledge Graph reasoning, LLM reasoning, and probabilistic reasoning.

GNN Encoder

The GNN Encoder transforms the Expectation Graph into vector representations. It assigns features to each node and edge and aggregates neighboring information through message passing.

This can generate:

• Agent-level Trust Embeddings
• Decision-level Trust Embeddings
• Context-level Trust Embeddings
• Graph-level Trust Embeddings

Depending on the purpose, these can be connected to tasks such as node classification, link prediction, graph classification, and regression.

Trust Output Layer

The Trust Output Layer produces the outputs needed from embeddings generated by the GNN, such as:

• Trust Score
• Expected Action Probability
• Risk Score
• Confidence
• Compliance Score
• Escalation Required
• Anomaly Score

The numerical values produced here do not fully represent Trust itself. They are values that summarize the Trust Graph for a particular purpose.

Execution Monitor

The Execution Monitor continuously observes actual decisions and actions. It compares:

• Expected Decision with Actual Decision
• Expected Action with Actual Action
• Expected Result with Actual Result

and detects deviations.

Trust Evaluator

The Trust Evaluator assesses the differences between expectation and reality from multiple perspectives, including:

• Goal Alignment
• Constraint Compliance
• Policy Compliance
• Result Quality
• Safety
• Consistency
• Explainability
• Predictability
• Recovery Behavior

Trust is not determined simply by whether an outcome succeeded. Whether the expected process was followed also matters.

Learning & Feedback Manager

The Learning & Feedback Manager feeds execution results into subsequent reasoning. Results are stored as new Decision Traces.

When needed, they are fed back into Knowledge Flow, enabling:

• Addition of Knowledge
• Revision of Rules
• Updates to Ontologies
• Corrections to Knowledge Graphs
• Retraining of the GNN
• Review of features

Trust Is Continuously Updated

Trust is not something calculated once and then finished. The Trust Engine repeats the following loop:

Expectation Graph
        ↓
GNN Inference
        ↓
Expected Behavior
        ↓
Execution
        ↓
Evaluation
        ↓
Decision Trace Update
        ↓
Knowledge Update
        ↓
GNN Learning
        ↓
Next Expectation

Each new execution result updates the Trust Graph. The GNN also learns from new relationships and outcomes.

Trust is therefore not a fixed state. It is a continuously formed, verified, and updated process.

How to Use Graphs and Numerical Values

The Trust Engine’s two outputs do not compete; they are used according to the application.

Graphs are used for high-risk decisions and audits. GNN-based numerical output is used for real-time Agent selection and anomaly detection.

For example, routine cases can be handled automatically using a GNN-based Trust Score. When a Risk Score exceeds a threshold, the Trust Graph can be presented to a human.

Low Risk
   ↓
GNN-based Automatic Decision

Medium Risk
   ↓
GNN Decision + Graph Explanation

High Risk
   ↓
Graph Review + Human Approval

In this way, the balance between explanation and automation can be varied according to the level of Trust and operational risk.

The Trust Engine Is the Core of Trust Infrastructure

Trust Infrastructure consists primarily of three architectures.

Decision Trace Model

Understands past decisions.

Knowledge Flow

Provides the knowledge needed to understand the current situation.

Trust Engine

Integrates the past and present to predict future behavior.

Together, these three architectures create a cycle that learns from the past, understands the present, predicts the future, and learns again from outcomes.

Within that cycle, the Trust Engine integrates Decision Trace and Knowledge into an Expectation Graph, then either outputs the graph directly or aggregates it through a GNN into numerical values as needed.

The Trust Engine is therefore:

a hybrid reasoning engine that integrates symbolic explainability with neural predictive capability.

Conclusion

Trust is not merely a feeling of confidence.

It is:

an expectation about how another party will decide and act in the current situation.

That expectation cannot be represented by a single number alone.

To understand the relationships through which Trust is formed, a graph representation is necessary. At the same time, to use Trust in real time across a large-scale AI society, fast numerical inference using GNNs is also necessary.

The Trust Engine therefore has a dual structure:

• It represents Trust as an Expectation Graph
• It aggregates the Expectation Graph with a GNN
• It outputs Trust Score, Risk, Confidence, and Expected Behavior
• When necessary, it returns to the original graph to explain the grounds for a decision

Trust is not a fixed evaluation. It is a continuous process of understanding, predicting, executing, evaluating, and learning.

The Trust Engine is a new reasoning engine for the AI era: one that implements this process on computers through both graphs and GNNs.