Towards Safe Autonomy for Computer Use

What it is

SA-CUA is a dual-LLM information-flow-control design for computer use agents (Seip, Markovinović, Hein; Tübingen). It is a preprint under review (arXiv 2610.18508v1). Its core move is to start with the whole screen masked and unmask only what deterministic rules mark as trusted. UCM, the prior approach, masks untrusted content instead. A missed rule then costs utility, not security.

How it works

flowchart LR
    S[Screen + AT-SPI tree] --> R{Trust rules match?}
    R -- yes --> P[Privileged LLM sees pixels]
    R -- no, masked --> Q[Quarantined LLM reads it]
    Q -- redacted typed variable --> E{Endorse?}
    E -- verified by trusted source --> P
    E -- not verified --> X[Non-consequential use only]

• Offline rules, once per URL template or app. An LLM sees the accessibility tree with every name and text replaced by [name:length:N], keeping role, id and class. It proposes rules, drops ones that are too broad or match nothing, then checks each rule on the unredacted subtree. The page text is cut off before the rest of that check is described.
• Trust definition. A region is user-generated if any party other than the site operator can change what it displays. This is broader than UCM's definition and is decidable. Commit hashes and timestamps count as user-generated. A platform's static "created" label does not.
• Default trusted. Browser chrome, menus, toolbars and the window frame are never masked. The URL bar is treated as trusted.
• Endorsement. The quarantined LLM answers in a type the privileged LLM requests, but the privileged LLM receives only a redacted variable. Reading it or using it in a consequential action requires endorsement against an independent source. Sources are the task text, the LLM's parametric knowledge, trusted contacts (e-mail, GitLab handles) and trusted domains. The value being endorsed cannot influence the check.
• Action gate. A policy forbids, for example, clicks in untrusted regions. A specification classifier sends underspecified tasks back to the user at the start.

Preconditions

• The user builds an allowlist of trusted domains and e-mail contacts up front.
• Adversaries are limited to user-generated content such as posts, e-mails, listings and reviews. Attacks on trusted sources themselves are out of scope.
• Files downloaded from the internet are tracked as untrusted per episode.
• Evaluation covers hybrid web and OS settings through AT-SPI, which UCM's DOM approach cannot do.

Numbers worth knowing

• Untrusted free-text leakage (FNR_free) is 0% for SA-CUA versus 44.1% for UCM, on Qwen3.8-27B.
• Completion on RTC† is 86.2% for SA-CUA versus 18.5% for CaMeL-CUA, on Qwen3.8-27B. Attack success rate is zero on WASP†, RTC† and RTC-value.
• The authors introduce RTC-value injections, where the malicious value has the same type as the benign one. Constraining the output type therefore does not stop them. They also argue that even a single bit can steer the privileged model into an attacker's branch.

Caveats

• The page names three accessibility-tree gaps: aria-hidden removes nodes, aria-owns can make a container's permission cover subtrees outside it, and visible regions may be missing or have no children. The paper's answer is mask-by-default, which trades away utility.
• The zero attack success rate is measured on the authors' own benchmark variants (WASP†, RTC†, RTC-value), not on arbitrary attacks.
• The fetched text ends partway through the rule-validation step. Endorsement-source details and the full results are not covered here.