InterWebCoder: Learning Interactive Webpage Generation from Executable Tests via Reinforcement Learning

What it claims

InterWebCoder trains a multimodal LLM to turn a sequence of webpage screenshots and user actions into working HTML, CSS and JavaScript. It replaces GUI-agent reward checking with executable Pytest scripts. The authors report an 84.35% synthesis success rate (1,736 verified samples from 2,058 tasks across 275 websites). A 9B backbone outperforms 30B-level models and Gemini-3.7-Flash and is "competitive with Claude-Sonnet-5". End-to-end training runs up to 1.4× faster.

How it works

A GUI agent performs LLM-written tasks on real sites such as Gmail, AliExpress, Coursera, Skyscanner and PayPal. It records actions and the resulting screenshots (states S1). A coding agent then writes a clean, self-contained webpage plus a Pytest script. Replaying the script yields states S2. A sample is kept only if its assertions pass and S2 visually matches S1. Rule-based diagnostics decide whether a failure lies in the page or the test, and only that component is repaired.

flowchart TD
  A[GUI agent records actions + states S1] --> B[Coding agent: webpage + Pytest]
  B --> C[Execute test, replay states S2]
  C --> D{Assertions pass and S1 ≈ S2?}
  D -- yes --> E[Verified training sample]
  D -- no --> F{Failure attributed to}
  F -- webpage code --> G[Repair page]
  F -- test script --> H[Repair test]
  G --> C
  H --> C

Training has two stages. In test-ID-weighted SFT, tokens that form element identifiers referenced by tests, such as data-testid="follow-button", get a higher loss weight λ. This keeps generated elements bound to test actions. In GRPO, the model sees only the interactive prototype, never the test. Candidates are scored on test-step pass rate and state-level visual similarity, and the overview figure also lists code integrity and test-ID coverage as reward terms.

Evidence

The motivating measurement used 102 InfiniteWeb sites with GPT-5.6-Luna on the Browser Use scaffold as the reward verifier. Verification took 39.9% of RL training time, averaging 214.0 seconds per sample. Tasks averaged 16 interaction steps (median 11), and the GUI agent's mean success rate was only 65.8%. The executable-test reward cuts reward latency by over 97% and removes external MLLM verification cost. The final scores are 48.54 on IWR-Bench and 48.42 on Vision2Web. The dataset spans 15 domains, led by Community (15.74%), Media (10.64%) and Technology (10.59%).

Limits

The fetched text ends partway through the data statistics, so the per-baseline benchmark tables, ablations and the authors' stated limitations are not available here. The page names no code, weights, dataset release or licence. Task instructions are LLM-generated and filtered automatically. The website list was hand-curated by four PhD students. Synthesized pages are clean reconstructions, not the sites' real source, so training targets exclude external APIs, session state and third-party scripts.