DuoMind: Enabling Distributed Multi-Robot Coordination with Semantic Communication

What it claims

DuoMind runs, on each robot, a VLM "orchestrator" that reads the task instruction, the robot's own observations and messages from its peer. From these it emits two things: a short natural-language instruction for that robot's VLA action model, and a structured message for the other robot. The authors argue that both models can be used largely off the shelf, with "only lightweight adaptation when necessary." The action model never has to interpret multi-agent state, which matters because VLAs are mostly trained on single-agent demonstrations. Either component can be swapped independently.

How it works

The orchestrator plans at low frequency and the VLA executes at high frequency. Natural language is the only interface between them, so no joint training or special latent interface is needed. The authors also say this lets robots running different reasoning models still understand each other, and lets a human read or revise the instructions.

flowchart LR
  T[Task instruction] --> O[VLM orchestrator]
  L[Local obs: global + own wrist cam] --> O
  M[Peer message] --> O
  O --> I[Low-level instruction]
  O --> S[Message to peer]
  I --> V[VLA action model]
  V --> A[Fine-grained actions]
  S -.-> P[Peer orchestrator]
  P -.-> M

Each inter-robot message has four fields:

• Intention: identical to the instruction just sent to the robot's own VLA, i.e. its next subtask.
• Subgoals: the task state expected once that subtask finishes.
• Belief of task: facts observed or inferred locally, e.g. "the pot is in the third cabinet."
• Uncertainty of belief (optional): a confidence signal, so the receiving orchestrator can weigh conflicting beliefs.

The RoboPoly benchmark

• Platform: built on ManiSkill3.
• Tasks: seven, namely Food Serve, Prepare Snack, Clean Table, Cook Pot, Put Object Cabinet, Hang Bag and Exchange Bread.
• Robots: two independent 7-DoF Franka Emika Panda arms with grippers, on opposite sides of one table.
• Cameras: one global RGB camera at 512×384 and one wrist camera per robot at 128×128.
• Distributed setting: each robot sees the global camera and its own wrist camera, but never its peer's wrist camera. Each policy controls one arm only.
• Randomisation: key objects are placed randomly within task-specific regions in each episode.
• Demonstration data: generated with ManiSkill3's motion-planning trajectory pipeline.

The authors contrast RoboPoly with RoboTwin. RoboTwin is two arms on one embodiment, and its tasks do not necessarily require explicit cooperation. RoboPoly's tasks are built so that a single robot cannot always finish them, or cannot finish them efficiently.

Evidence

The fetched text stops partway through the dataset section, before any results tables. The claims that DuoMind "consistently outperforms the baselines" on RoboPoly and RoboTwin, and that ablations support the orchestration and messaging design, come only from the abstract and the Figure 1 caption. No success rates, baselines or specific VLM/VLA backbones are visible.

Limits

• Two robots only. All experiments use two agents. Larger teams are explicitly left to future work because of compute and data-collection cost, so scaling to more robots is not demonstrated.
• Simulation only. RoboPoly is a simulated benchmark built on ManiSkill3, and the visible text describes no real-robot deployment.
• No adoption details. The visible text gives no code, weights, licence, model names or inference cost. It only points to a project page.
• Check the full paper first. Before relying on the performance claims, find the result numbers, the baselines used, which backbones were tested and how much adaptation they actually required.