OpenGCUA: Toward a Fully Open Stack for Generalist Computer Use

What it claims

OpenGCUA argues that computer-use agents work better and faster when the model writes programs that mix GUI actions with accessibility-tree, API and shell calls, and when the model makes workflow decisions that the harness would otherwise fix in advance. The headline OSWorld-v2 numbers, +19.6 mean partial-score points and 53.0% fewer reported steps per task, are measured against batched GUI-only execution, not against each model's own native agent. The page also says the full configuration narrows the gap between the strongest and weakest of the eight models by more than half. Gemini 3.8 Flash is among the eight, alongside GPT-6 Astra and Claude Opus 5.

How it works

• Code execution as the action space. GUI actions are written as code and combined with accessibility, API and shell operations in a single program. Loops, branches and intermediate results decide the next operation. This lets the model use its existing programming ability rather than issuing one click per turn.
• Harness-to-model decision transfer. The harness supplies the mechanisms, and the model decides when to use them. Selected observations and task-state decisions move from fixed harness rules to the model. The concrete example given is that the model requests a screenshot only when it needs one, instead of the harness capturing one every step.

Evidence

• OSWorld-v2, all eight models improve. Mean partial score rises by +19.6 points and steps per task fall by 53.0% on average, both relative to batched GUI-only execution. The page gives no per-model breakdown.
• Real-device token gap. Across four real macOS tasks, selected Qwen3.8-Max runs used 19.4× the reported reasoning tokens of GPT-6 Astra. The authors say the gap between models is larger on real devices than on the benchmark.

How to use it

• Code: the harness implementation, prompts and evaluation trajectories from all eight models are at github.com/Tongyi-MAI/OpenGCUA.
• Real-device harness: a macOS menu-bar app that runs the agent on the user's own machine.
• Cross-Device SDK: a single programming interface across computers, browsers and phones. One program can coordinate several devices within one task.
• Interactive lab: tongyi-mai.github.io/OpenGCUA has seven labs, including a side-by-side comparison of four action modes and trajectory replays from five models.
• Report: arxiv.org/abs/2610.22200

This is a harness and SDK release, not a model release. No model weights or datasets are linked, and both citation counts on the page are 0.

Limits

• The real-device harness and the Cross-Device SDK are labelled preliminary or preview.
• The 19.4× token figure covers only four tasks and "selected" runs. Treat it as illustrative, not a benchmark result.
• Because the baseline is batched GUI-only execution, the gains do not show how OpenGCUA compares with closed systems such as GPT-6 Astra's own computer-use stack.
• No licence is stated on the page. Check the repository before adopting.
• The page states the abstract and release notes only. Details such as per-model scores, ablations of code execution versus decision transfer, and the real-device failure cases are in the full report.
• The authors call it an ongoing project. A unified computer- and browser-use model and real-device benchmarks are listed as future work.