Jev-Mobile: Jev as an Executor for Mobile GUI Agents

What it claims

Jev-Mobile decouples a mobile GUI agent into low-frequency VLM planning and high-frequency lightweight execution: the VLM sets a local goal and exact input text, then a fast typed decision model ("Jev") repeatedly selects actions from an accessibility-tree-derived action space until it hands control back. On the full AndroidWorld suite it reports 0.79 task success, versus 0.78 for SeeAct-V and 0.84 for a step-wise VLM baseline, while cutting mean end-to-end time by 32.7% and mean per-success model API cost by 73.4% relative to the step-wise VLM. The stated novelty is narrow: not hierarchical control in general, but whether a typed decision model can serve as the action executor over program-generated candidate IDs.

How it works

One VLM call emits delegate/finish/blocked, a local goal g_k, and optional exact text. A deterministic program enumerates executable candidates from the raw accessibility tree (click, long-press, scroll, focus, input, plus Back/Home/Enter/Open-app), each binding a local ID to an action+node. Jev, a remote "Decisions" service, receives only the textual tree, the goal, the current delegation's action history, and candidate descriptions — no screenshot — and returns a candidate ID or DONE/BLOCKED. After each action the executor re-observes and rebuilds candidates, invalidating stale IDs. No post-delegation summary is generated; the VLM sees grouped actual action history at handoff. Multiple actions run under one VLM decision. No new training, hierarchy repair, or semantic field-binding.

flowchart LR
  VLM["VLM (Qwen3.8-Max):\nlocal goal + text"] --> CB["Candidate builder:\nfresh AXTree → IDs"]
  CB --> JEV["Jev: choose ID\nor DONE/BLOCKED"]
  JEV -->|action| EXE["Android executor"]
  EXE -->|new obs| CB
  JEV -->|DONE/BLOCKED| VLM

Evidence

Full AndroidWorld, three systems sharing Qwen3.8-Max as the general VLM (SeeAct-V grounds with UI-TARS-1.5-7B, substituted for UGround). Success rate over all instances; time/cost over successful trajectories only. Mean total time: Jev-Mobile 132.67s, SeeAct-V 162.63s, step-wise 197.21s. Mean executor time: 5.16s vs 12.37s vs 94.30s. Mean model API cost per success: $0.0727 vs $0.193 vs $0.274.

Limits

• AndroidWorld only — no web/DOM testing.
• Tree-dependent: a visually present target absent from the accessibility tree cannot become a candidate; the first version has no coordinate-producing visual fallback (returns BLOCKED).
• Printable ASCII input only; long action histories require compaction.
• The authors flag a missing ablation: they did not test the same workflow with Jev replaced by a small VLM, so it's unproven whether gains come from typed decision-model execution or the delegated workflow itself.
• Timing/cost gains are conditional on successful trajectories, and success (0.79) trails the step-wise baseline (0.84) by 5 points.