arxiv.orgpaper

Workspace Models: Lightweight Robotic Memory via Saliency-Driven Supervision

This paper introduces "workspace tokens," a lightweight latent memory representation for robotic manipulation policies that replaces expensive in-the-loop VLM queries. During training, a VLM identifies task-salient current and historical information, which is distilled into the workspace token via a set-reconstruction decoder loss; at deployment, the token serves as a drop-in replacement for raw observations. Experiments in simulation and on hardware show the approach improves policy performance on memory-intensive tasks while being far cheaper to run than VLM-in-the-loop methods.