Designing a Semantic Digital Twin for Multiple Robots: Shared Environments, Coordinated Decisions, and Multi-Robot Trace

🎥 The YouTube version is also available:

Semantic Digital Twin for Physical AI

Books: A Practical Guide to Semantic Digital Twin for Physical Al: Building with Chinoba PF Ontology • Knowledge Graph • DSL • Decision Trace

Introduction

When operating a single robot, it is sufficient for that robot to understand its own position, orientation, destination, battery level, and sensor data.

However, when multiple robots operate in the same space—such as a factory, warehouse, hospital, commercial facility, or construction site—the nature of the problem changes significantly.

A route that is optimal for one robot may not be optimal for the fleet as a whole. One robot’s charging request may cause another robot to stop operating. A locally safe avoidance maneuver may create congestion or collision risk in another corridor.

What is needed here is not merely a Digital Twin that tracks the state of each robot individually.

It is a Semantic Digital Twin that semantically connects robots, spaces, resources, tasks, rules, people, and interactions—enabling the fleet’s overall state and decisions to be managed as one system.

This chapter examines the Semantic Digital Twin that supports multi-robot coordination through the lenses of Individual Twins and System Twins, Shared Environments, Resource Competition, Collision and Conflict, Collective Context, Fleet Decisions, and Multi-Robot Trace.

---

51.1 Individual Twin and System Twin

In a multi-robot environment, it is necessary to distinguish between two different kinds of Twins.

One is the Individual Twin, which represents each robot. The other is the System Twin, which treats multiple robots and their shared environment as a single system.

Individual Twin

An Individual Twin represents the current state and capabilities of each robot.

• Robot identity
• Robot type and onboard equipment
• Position, orientation, speed, and reachable range
• Battery level and charging status
• Payload and carrying capacity
• Sensor status and perception confidence
• Current task and progress
• Role, authority, and priority
• Failure and maintenance status
• Available actions and constraints

This Twin provides the foundation for understanding what a particular robot can and cannot do at a given moment.

System Twin

A System Twin, on the other hand, is not simply a registry created by adding together individual robots.

It deals with the relationships between robots and the behavior of the system as a whole.

• Which robots share the same space
• Which tasks depend on one another
• Which routes, chargers, and transport equipment are shared
• How one robot’s delay affects other processes
• Which goals should be prioritized across the fleet
• Whether the current configuration is appropriate in terms of safety, efficiency, and fairness

For example, two transport robots may each select a valid route independently. Yet if they encounter one another in a narrow corridor, the situation becomes a problem from the perspective of the System Twin.

If the Individual Twin represents “the capabilities and state of an individual robot,” the System Twin represents “the overall state and constraints that emerge from relationships.”

Perspective	Individual Twin	System Twin
Primary subject	One robot	Fleet and shared environment
Primary question	What can this robot do?	What should the system do as a whole?
Typical state	Position, power, task, failure	Congestion, conflict, total load, system-wide risk
Typical decisions	Local routing, self-preservation, task execution	Assignment, prioritization, reconfiguration, escalation

The important point is that the System Twin does not override the Individual Twin. The two must continuously inform one another. System-level decisions must respect the constraints of individual robots, while individual actions must refer to the state of the overall system.

---

51.2 Shared Environment

In multi-robot coordination, the environment is not merely a background. The environment itself is a shared resource through which relationships between robots are formed.

A Shared Environment includes not only physical space, but also entities with operational meaning.

• Corridors, intersections, work zones, and restricted areas
• Charging stations, elevators, and transport equipment
• Shelves, worktables, docks, and storage locations
• People, workers, customers, and other vehicles
• Weather, lighting, noise, and network conditions
• Time periods, booking slots, and operating rules

Representing this environment as a simple map is not sufficient.

For example, a corridor may normally allow two-way traffic, but be restricted to one-way traffic during loading operations. During periods of high pedestrian activity, lower speed limits may be required. In an emergency, the same corridor may need to be prioritized as an evacuation route.

In a Semantic Digital Twin, space is treated as an Entity with operational meaning.

• Zone: An area with access permissions, risk level, speed limits, and responsibility scope
• Path: A route with direction, width, occupancy conditions, and priority
• Resource: A shared asset with capacity, reservations, failure status, and access rights
• Event: An occurrence that changes the meaning or operational condition of the environment
• Policy: Conditions of use determined by time, state, and role

This makes it possible to determine not merely “the shortest route,” but “the route that this robot may safely and legitimately take under the current conditions.”

---

51.3 Resource Competition

When multiple robots operate simultaneously, competition for limited resources inevitably arises.

Typical resources include chargers, narrow corridors, elevators, workspaces, network bandwidth, worktables used by robot arms, and even human approval time.

A simple first-come, first-served approach guarantees neither overall optimization nor fairness. A robot nearing battery depletion, a robot assigned to emergency transport, and a robot involved in a process bottleneck should not necessarily be treated with the same priority.

Representing competition semantically

To manage Resource Competition, the Twin must maintain at least the following information:

• Resource type, capacity, and current occupancy
• Start and end times of use requests
• The requesting robot and its role
• Task importance and deadline
• Availability of alternative resources or routes
• Policies that permit or prohibit use
• The impact and risk of waiting

For example, when two robots arrive at a charger, the decision should not simply be “charge the robot that arrived first.” Instead, the system can consider:

• Battery level and predicted energy consumption
• Urgency of the next task
• Distance to another available charger
• The impact of waiting on the overall workflow
• Whether a human has specified a priority

Resource Allocation is part of Fleet Decision

If resource allocation is left entirely to local negotiation among robots, chains of short-term optimization may emerge. When system-wide coordination is required, the System Twin should treat Resource Allocation as part of a Fleet Decision.

It is important not to fix the optimization objective to a single criterion.

• Safety
• Delivery deadlines and service levels
• Energy efficiency
• Overall throughput
• Fairness among robots
• Impact on people
• Resilience during failures

These objectives involve trade-offs. The Twin must preserve not only the decision itself, but also which goals were prioritized, in the Decision Trace.

---

51.4 Collision and Conflict

Collision and Conflict may appear similar, but they are different.

A Collision is a spatial or motion-related problem, such as a physical impact, contact, or dangerous proximity.

A Conflict is a semantic or operational competition involving the same resource, task, authority, or policy.

Two robots facing each other in the same corridor create a collision risk. By contrast, two robots attempting to execute the same transport task, reserve the same charger, or follow contradictory policy-based instructions create conflicts.

Predicting collisions

Avoiding collisions requires handling not only position and velocity, but also intent and uncertainty.

• Current position, orientation, and velocity
• Predicted route and arrival time
• Stopping distance, turning radius, and payload condition
• Sensor blind spots and perception confidence
• Unpredictable movement by people and other vehicles
• Safety policies for each zone

A Semantic Twin represents more than the fact that “two robots are close.” It can represent a future hazardous state: “these two robots are expected to pass through this intersection at the same time.”

Resolving conflicts

Conflict resolution requires explicit priorities and clear boundaries for negotiation.

• Emergency tasks take precedence over normal tasks.
• When human safety is involved, safety takes precedence over efficiency.
• A robot without the required authority cannot secure a resource.
• When conflicts arise, alternative routes and alternative tasks should be explored.
• Conflicts that cannot be resolved should be escalated to a Fleet Manager or Human Gate.

The important point is not to attempt to resolve everything automatically. Conflicts that require judgment among safety standards, business rules, and customer impact should be presented in a form that humans can review.

---

51.5 Collective Context

Simply placing each robot’s context side by side does not create the context of the fleet as a whole.

Collective Context is the shared situational understanding that emerges from the interactions among multiple robots, environments, tasks, and resources.

For example, even if every robot is operating normally on its own, the following conditions become visible only at the fleet level:

• Robots are concentrating in a particular area, increasing congestion.
• Charging queues are cascading and affecting the next shift.
• A failure in one robot is concentrating workload on substitute robots.
• Robots are gathering in human work areas, reducing safety margins.
• Delays in priority tasks are propagating across multiple processes.

Collective Context is not merely an aggregation of local states. It includes relationships, dependencies, and changes over time.

Context that should be shared

• Fleet-wide task queues and deadlines
• Congestion, risk level, and accessibility by zone
• Resource availability, reservations, and failure status
• Capabilities, power levels, locations, and utilization of each robot
• Human activity, priority work, and access restrictions
• External events, weather conditions, and communication failures
• Active policies, emergency modes, and command structures

Sharing Collective Context does not mean distributing all information to every robot. It means sharing the appropriate scope of context according to each robot’s role and operational need.

Given privacy, security, network capacity, and cognitive load, Context itself requires Access Control and Purpose Limitation.

---

51.6 Fleet Decision

A Fleet Decision is a decision made by treating multiple robots as a single operational unit and acting according to system-wide goals and constraints.

Typical decisions include:

• Task Assignment: Which robot should be assigned to which task?
• Route Coordination: Which routes and time windows should robots use?
• Resource Allocation: How should chargers, elevators, and worktables be allocated?
• Role Reconfiguration: How should roles be redistributed during failures or congestion?
• Safety Escalation: Should the system continue automatically, stop, or ask for human confirmation?
• Recovery Planning: In what order should the system recover after a failure?

Evaluation functions for Fleet Decisions

A Fleet Decision cannot be determined solely by “shortest time.” It must account for at least the following objectives:

• Safety: Avoid collisions, dangerous proximity, and unauthorized actions
• Service Level: Meet deadlines, transport volume targets, and customer experience requirements
• Efficiency: Reduce travel distance, waiting time, and energy use
• Robustness: Maintain the ability to recover from failures and environmental changes
• Fairness: Avoid concentrating workload and waiting time on particular robots
• Explainability: Explain why a specific assignment, stop, or priority order was chosen

The important point is not to frame Fleet Decisions as a binary choice between centralization and full decentralization.

Low-risk local avoidance can be decided immediately by each robot. Resource allocation and priority changes can be coordinated by a Fleet Manager. Decisions with major safety or business impact can be escalated to a Human Gate.

This kind of hierarchical autonomy design is more realistic.

Local Decision
  └─ Immediate stop, local avoidance, self-preservation

Fleet Decision
  └─ Assignment, reconfiguration, routing, resource allocation

Human Gate
  └─ Safety exceptions, priority changes, irreversible actions

The Semantic Digital Twin connects the states, authorities, policies, and evidence required for this division of responsibility.

---

51.7 Multi-Robot Trace

In multi-robot operations, it is not enough to record only “which robot did what.”

What is needed is a Multi-Robot Trace that records how a Fleet Decision emerged from shared conditions, conflicts, policies, and robot actions—and what impact it had on the system as a whole.

Information included in a Multi-Robot Trace

• Fleet goals and priorities
• Participating robots, roles, and capabilities
• Shared Environment state
• Contested resources and their reservation and release history
• Candidate actions and local decisions of each robot
• Assignments and replanning by the Fleet Manager
• Applied policies, constraints, and Human Gates
• Detection and resolution paths for collisions and conflicts
• Execution results, delays, anomalies, and recovery actions
• Impacts on other robots, processes, and people

For example, if transport robot R-12 is reassigned to another task, the Trace should not merely state that “R-12 was available.” It should preserve the following causal sequence:

1. An urgent transport task occurred.
2. The usual robot, R-03, was waiting to charge and was predicted to be unable to arrive before the deadline.
3. R-12 was nearby, while the delay impact on its original task was within an acceptable range.
4. The system evaluated the safety policy and priority rules.
5. R-12 was reassigned, and its original task was transferred to R-07.
6. Intersection occupancy reservations were updated, and speed limits were applied in human work areas.
7. As a result, the urgent task was completed within its deadline, while the normal task was completed within the allowable delay.

With this Trace, it becomes possible to review why R-12 was selected, what impact the decision had on its original task, and whether other options were available.

Trace also supports learning

A Multi-Robot Trace is not merely an audit log. It is a learning resource for improving future Fleet Decisions.

• Which congestion predictions were inaccurate?
• Which policies created unnecessary waiting?
• Which task assignments caused workload imbalance?
• Which Human Gates were effective?
• Which Recovery Plans restored safe operations most quickly?

By connecting Decision, Action, Outcome, and Evaluation, the fleet can learn from operational experience and improve how it coordinates.

---

Conclusion

The challenge of multi-robot systems is not simply increasing the number of robots.

It is the challenge of understanding the spaces, resources, tasks, policies, and impacts on others that robots share—and transforming that understanding into coordinated decisions.

For this purpose, Individual Twins alone are not enough. Individual Twins and System Twins must be connected. The system must understand the Shared Environment and Collective Context, manage Resource Competition as well as Collision and Conflict, and execute Fleet Decisions in an explainable manner.

By preserving this coordination process as a Multi-Robot Trace, a fleet can evolve from a group of machines that merely move into a coordinated system that learns safely through experience.

A Semantic Digital Twin is not simply a model for visualizing multiple robots.

It is a decision-making foundation through which people, robots, and environments can share the same reality, respect their roles and boundaries, and coordinate their actions.