Semantic Digital Twins for Physical AI: How Ontologies and DSLs Transform Real-World Intelligence

🎥 The YouTube version is also available

Semantic Digital Twin for Physical AI

Introduction

With the evolution of generative AI, artificial intelligence is beginning to move beyond merely generating text and toward acting in the physical world.

Robots transport components.

Manufacturing equipment detects anomalies and shuts itself down.

Autonomous mobile machines assess their surroundings.

Drones observe environments and select appropriate actions.

AI systems that perceive, make decisions about, and act within the physical world are generally referred to as Physical AI.

Discussions of Physical AI often focus on technologies such as cameras, sensors, robotic control, simulation, and reinforcement learning.

These technologies are undoubtedly important.

However, for Physical AI to operate safely in real industries and society, another indispensable element is required:

Structured knowledge.

For Physical AI, recognizing an object is not enough.

It must understand:

• What the object is
• Which equipment or system it belongs to
• Which process uses it
• What state it is currently in
• Who is authorized to operate it
• Under which conditions it must stop
• What should be prioritized when an anomaly occurs

In other words, Physical AI must understand meaning, relationships, constraints, and rules.

The Ontology, Knowledge Graph, and Domain-Specific Language, or DSL, within the Knowledge Flow framework we propose provide precisely the foundation required to give Physical AI this understanding of the real world.

What Physical AI Lacks Is Meaning

Today’s AI can identify people, equipment, vehicles, tools, and other objects in images with high accuracy.

For example, it can detect that a camera image contains a worker, a forklift, and a pallet.

However, recognizing these objects is not the same as understanding the situation.

Even if an AI system recognizes a forklift, it does not automatically know:

• Whether the forklift is currently available for use
• Whether the operator holds the required certification
• Whether the area it is entering is access-restricted
• Whether its cargo contains hazardous materials
• Whether its distance from nearby workers meets safety requirements
• Whether emergency shutdown or operational continuity should take priority

Sensors mainly provide information about an observed state.

What is needed in the field, however, is the context that explains what that state means.

Physical AI therefore requires three layers:

Perception

Recognizing what is happening in the physical world.

Semantic Understanding

Understanding the meaning, relationships, states, and roles of recognized entities.

Decision and Action

Selecting an action based on goals, constraints, and rules.

Current Physical AI is often developed by connecting Perception directly to Action.

In industrial environments, however, the layer between them—Semantic Understanding—is critically important.

Knowledge Flow can be understood as a mechanism for constructing this frequently missing intermediate layer.

Ontology Gives Physical AI a Structure of the World

An Ontology is more than a glossary.

It is a knowledge model that defines what kinds of entities exist in the real world, how they relate to one another, and what states and constraints apply to them.

In a manufacturing environment, an Ontology might include concepts such as:

• Factory
• Production line
• Process
• Equipment
• Component
• Product
• Worker
• Work procedure
• Quality standard
• Anomaly
• Maintenance task
• Safety zone
• Authorization
• Responsible person

It also defines the relationships among these concepts:

• Equipment belongs to a production line.
• A process uses specific equipment.
• A component is processed through a particular operation.
• A worker holds specific qualifications.
• An anomaly is detected as a change in equipment state.
• A maintenance task requires approval from a responsible person.

This allows Physical AI to position individual sensor readings within the structure of the entire operational environment.

Suppose, for example, that a temperature sensor detects a reading of 85°C.

The number alone does not indicate whether the condition is normal or abnormal.

Through an Ontology, however, the AI may understand that:

• The sensor is installed on Press Machine A.
• Press Machine A belongs to Production Line 2.
• The material currently being processed is Resin B.
• The permitted processing temperature for Resin B is between 60°C and 80°C.
• A temperature above 85°C may indicate a quality anomaly.

These relationships give meaning to the observed value of 85°C.

Ontology transforms the world perceived by Physical AI from a collection of labeled objects into a world composed of meaningful relationships.

Knowledge Graphs Represent a Changing Operational Environment

If an Ontology defines the fundamental structure of the world, a Knowledge Graph represents the actual entities and relationships that exist in a specific operational environment.

For example, it can connect information such as:

• Production Line 2 is manufacturing Product X.
• Press Machine A is executing Process 3.
• Temperature Sensor S1 has detected 85°C.
• Worker W1 is present in the surrounding area.
• Maintenance Technician M1 is currently engaged in another task.
• A similar temperature increase previously resulted in a defect.

By referring to this Knowledge Graph, Physical AI can obtain a broader context that cannot be derived from an individual sensor alone.

The crucial point is that the physical world is constantly changing.

Equipment states change.

Worker locations change.

Production targets change.

Priorities change.

Available resources change.

A Knowledge Graph for Physical AI must therefore be more than a static database of enterprise information.

It must be a dynamic model of the world, continuously updated through sensors, equipment, operations, decisions, and historical records.

In Physical AI, the Knowledge Graph can function as a Semantic Digital Twin that represents the real world in a semantically meaningful form.

Whereas a conventional Digital Twin primarily reproduces the geometry and numerical states of equipment, a Semantic Digital Twin also represents its roles, relationships, constraints, and decision conditions.

DSL Transforms Knowledge into Executable Rules

Ontology and Knowledge Graphs enable AI to understand the meaning of the operational environment.

Physical AI, however, requires more than understanding. It must act.

This is where a DSL becomes important.

DSL stands for Domain-Specific Language: a language designed to describe rules, constraints, decision conditions, and actions within a particular domain.

A manufacturing safety rule, for example, could be expressed as follows:

WHEN
  equipment.temperature > permitted_temperature
  AND worker.exists_in(safety_zone)
THEN
  stop(equipment)
  notify(line_manager)
  create(incident_record)
REQUIRE
  human_approval BEFORE restart(equipment)

This is more than ordinary program code.

It represents operational knowledge in a form that both AI systems and humans can inspect and verify.

A DSL can explicitly define:

• Under which conditions an action should be taken
• Which actions are permitted
• Which actions are prohibited
• What should be prioritized
• Where human approval is required
• To whom an anomaly should be escalated
• What evidence must be recorded as the basis for a decision

In Physical AI, an AI decision can directly result in a physical action.

An error by a text-generating AI may appear as an incorrect answer.

An error by Physical AI, however, may result in equipment damage, quality incidents, worker injuries, or environmental harm.

Physical AI must therefore manage not only what the AI wants to do, but also what the AI is authorized to do.

A DSL transforms procedures and safety rules written in natural language into verifiable and executable constraints.

Delivering Knowledge from Documents to Physical AI

Most enterprises already possess much of the knowledge required by Physical AI.

The problem is that this knowledge is rarely available in a machine-usable form.

Examples include:

• Standard operating procedures
• Equipment manuals
• Safety regulations
• Quality standards
• Maintenance records
• Incident reports
• Expert knowledge
• Laws and industry standards
• Historical decision records
• Approval workflows

This knowledge is distributed across PDFs, Word documents, spreadsheets, images, paper documents, and human experience.

Even if robots and AI agents are introduced, they will operate without understanding operational rules unless they can access this knowledge.

Knowledge Flow addresses this problem through the following process:

Documents
    ↓
Ingestion
    ↓
Knowledge Extraction
    ↓
Ontology
    ↓
Knowledge Graph
    ↓
DSL
    ↓
Decision Runtime
    ↓
Physical Action

First, documents and data distributed throughout the enterprise are collected.

Next, LLMs and other technologies extract entities, relationships, conditions, constraints, procedures, and exceptions.

The extracted knowledge is integrated into an Ontology and connected to actual equipment, processes, people, and states through a Knowledge Graph.

Knowledge related to action is then expressed as a DSL so that Physical AI can use it at runtime.

Through this process, enterprise documents are transformed from stored information into executable knowledge capable of guiding robots and equipment.

Physical AI Requires More Than Static Knowledge

Structuring operational rules once is not enough.

The environment in which Physical AI operates changes continuously.

An action permitted under normal conditions may be prohibited during an anomaly.

In some situations, productivity should take priority. In others, safety must come first.

Even when the same equipment anomaly occurs, the required response may differ depending on whether people are nearby.

A Physical AI decision should therefore be understood through the following relationship:

Knowledge
+
Current Context
+
Goal
+
Policy
+
Boundary
=
Decision

Context is particularly important.

Ontology defines the structure of the world.

The Knowledge Graph represents current entities and relationships.

The DSL defines behavioral rules and constraints.

The Runtime selects the appropriate knowledge and rules based on the context of that specific moment.

Physical AI does not simply require an enormous knowledge base.

It requires a Knowledge Flow that delivers the right knowledge at the right moment.

Decision Trace Makes Physical Actions Explainable

As Physical AI enters operational environments, the ability to explain why it took a particular action becomes increasingly important.

Why was the equipment stopped?

Why was an alternative route selected instead of the standard route?

Why was the decision escalated to a human?

Which sensor information was referenced?

Which rule was applied?

Which alternative actions were considered?

The mechanism for preserving this information is the Decision Trace.

For an equipment shutdown, for example, the following information could be recorded:

Observation:
  Temperature = 85°C

Context:
  Material = Resin B
  Worker present in Safety Zone

Applicable Knowledge:
  Maximum permitted temperature = 80°C

Applied Rule:
  Safety Rule SR-204

Decision:
  Emergency Stop

Reason:
  Temperature limit exceeded while worker was present

Next Action:
  Notify line manager and require human approval for restart

This allows the behavior of Physical AI to be preserved not merely as a system log, but as a meaningful history of decisions.

Decision Trace is not intended solely for post-incident accountability.

It can also be used to analyze previous decisions and improve the Ontology and DSL.

Physical Action
    ↓
Decision Trace
    ↓
Review
    ↓
Knowledge Update
    ↓
Ontology / DSL Revision
    ↓
Better Physical Action

Through this cycle, Physical AI can learn from experience while allowing humans to review, validate, and govern how the system evolves.

A Hybrid Architecture Connecting Physical AI and Generative AI

In real-world Physical AI systems, not everything can be described through a DSL.

Unpredictable situations and ambiguous natural-language instructions will always exist.

At the same time, delegating every decision to an LLM or learning model would be dangerous.

What is needed is a hybrid architecture combining generative AI with structured knowledge.

Generative AI can be responsible for:

• Interpreting ambiguous instructions
• Extracting knowledge from documents
• Summarizing situations
• Generating multiple action candidates
• Forming hypotheses in unfamiliar situations

Ontology, Knowledge Graphs, and DSLs can be responsible for:

• Defining the meaning of entities and relationships
• Structuring current states
• Applying safety rules
• Managing authority and responsibility
• Restricting the scope of permissible actions
• Validating decision outcomes

In this architecture, generative AI proposes possible actions, while DSLs and Policies validate those proposals.

Perception
    ↓
Context Construction
    ↓
LLM / Planning AI
    ↓
Action Candidate
    ↓
Ontology / DSL / Policy Validation
    ↓
Human Gate if Required
    ↓
Physical Execution
    ↓
Decision Trace

Generative AI provides flexibility, while structured knowledge provides consistency and safety.

Physical AI should not be framed as a choice between generative AI and rule-based systems.

It requires a combination of flexible reasoning and verifiable constraints.

Applications Beyond Manufacturing

Structured knowledge enabled by Knowledge Flow can be applied across many forms of Physical AI.

Logistics

Ontology can define relationships among packages, vehicles, warehouses, hazardous materials, delivery requirements, and workers, while DSLs manage loading restrictions and safety conditions.

Construction

Construction machinery, work zones, processes, qualifications, hazardous areas, and weather conditions can be connected through a Knowledge Graph to enable context-aware operational control.

Healthcare and Long-Term Care

Patients, medical devices, medications, care procedures, contraindications, and staff permissions can be structured to control the scope of actions available to assistive robots.

Agriculture

Crops, soil, weather, agricultural chemicals, machinery, and growth stages can be connected and used to guide autonomous agricultural equipment.

Infrastructure Maintenance

Bridges, roads, electrical equipment, sensors, inspection histories, and failure risks can be integrated to support the action planning of inspection robots and drones.

Disaster Response

Terrain, evacuation routes, hazardous areas, people requiring rescue, and available resources can be dynamically integrated to coordinate multiple robots and drones.

Across all these domains, the common requirement is that Physical AI must understand not only objects, but also their roles, relationships, states, constraints, and associated responsibilities.

Physical AI Is Evolving into Machines with Knowledge

Traditional robots were machines designed to execute predefined movements accurately.

The next generation of Physical AI will perceive its environment and make decisions autonomously.

What is needed beyond that, however, is not merely a machine that can move independently.

It is a machine that can understand operational knowledge, comply with rules, explain its decisions, and coordinate with humans and other AI systems.

Four elements are essential:

• Ontology for giving meaning to the world
• Knowledge Graphs for representing dynamic situations
• DSLs for transforming rules and behavioral constraints into executable forms
• Decision Trace for recording the basis of decisions

Knowledge Flow is the mechanism that continuously connects these elements.

Knowledge Flow is not merely a system for document search or RAG.

It is a knowledge infrastructure for delivering enterprise knowledge to AI systems that act in the physical world.

Conclusion

As Physical AI evolves, the performance of sensors, robots, and learning models will remain important.

However, once Physical AI leaves the laboratory and begins operating in factories, logistics facilities, construction sites, healthcare, agriculture, and social infrastructure, performance alone will no longer be sufficient.

AI must understand the meaning of the real world.

What entities exist?

How are they connected?

What state are they currently in?

What is permitted, and what is prohibited?

When is human judgment required?

Why was a particular action selected?

Ontology provides a structure of the world.

Knowledge Graphs represent changing situations.

DSLs transform knowledge into executable rules.

Decision Trace preserves the reasons behind decisions and actions.

Knowledge Flow connects documents, knowledge, context, decisions, and actions into a continuous process.

The essence of Physical AI is not simply AI with a physical body.

It is AI that understands the meaning of the real world and acts safely based on knowledge.

In the emerging competition around Physical AI, success will depend not only on the physical capabilities of robots or the reasoning capabilities of AI models, but also on how effectively organizations can structure their knowledge and make it available at runtime.

Physical AI is not powered by models alone.

It is powered by structured knowledge that enables AI to understand the real world.