TaoD2C-Bench: Benchmarking MLLMs for Industrial UI Code Generation Beyond Visual Fidelity

What it claims

TaoD2C pairs 2,861 production UI designs with layer-level expert annotations. Each annotation states what the generated code must do (required) or may do (optional). Scoring code against these requirements, rather than against pixels or a single reference implementation, shows that MLLMs that reconstruct a design visually still often fail to produce code with the right components, nesting, alignment and positioning. The benchmark also separates two failure sources: inferring the requirements and realizing them in code.

How it works

Data.

• The authors began with roughly 30,000 MasterGo design pages from product teams in an Alibaba (Taobao & Tmall) e-commerce ecosystem, used with the owning teams' authorization.
• They filtered and deduplicated these down to 2,861 designs.
• Each design has a rendered image, layer metadata and a reference to its target component library.
• The designs include B-end interfaces (merchant ops, analytics, logistics) and C-end interfaces (shopping, marketing, membership).
• Mean visible nodes per design: 82.5 for B-end and 56.4 for C-end. Overall mean: 64.2 layers.

Annotation.

• Two senior frontend engineers (7 and 5 years of experience) annotated the designs on a custom platform. A third expert reviewed everything iteratively.
• The process took 18 months.
• Annotations follow W3C CSS layout and positioning specs and the target library's APIs. There are four categories:
• Component: library component type plus property configuration, e.g. switch state or select options.
• Group: container regions. Members are compiled deterministically from the layer tree.
• Alignment: main-axis justify-content rule.
• Position: positioning scheme, e.g. absolute relative to a containing block.

Tasks and scoring. All three tasks share the same annotations as reference.

flowchart LR
  I[Image + layer metadata + library docs] --> T1[T1: generate code]
  I --> T2[T2: predict requirements]
  I --> T3[T3: generate code]
  A[Expert annotations] -->|supplied as input| T3
  T2 --> P[micro-F1 vs annotations]
  T1 --> C[Requirement match via source, structure, geometry]
  T3 --> C
  T1 --> V[Visual: CV, VES, MAE]
  T3 --> V
  A --> P
  A --> C

• Code scoring (T1, T3): requirements are matched to the runtime implementation using source-code, structural and geometric evidence, so no reference code is needed. The page reports micro-F1 for Type, Props, Group, Alignment and Position. Props is scored only on components whose type matched.
• Visual scoring: uses 1D-Bench's composite visual score (CV), DINOv2-based visual element similarity (VES) and normalized RGB MAE.

Evidence

• Test set: 421 designs (152 B-end, 269 C-end), all annotation-reviewed and deduplicated.
• Setup: eight MLLMs evaluated with identical context and prompts.
• Comparison with prior benchmarks: the authors' table lists Design2Code, Web2Code, WebUIBench, IW-Bench, WebCode2M, Figma2Code and 1D-Bench. Only Figma2Code (213-design eval split) and 1D-Bench (204-design eval split) include design metadata, and none includes expert implementation requirements.
• Critique of Figma2Code: its heuristic of penalizing absolute positioning is wrong for overlapping elements.

Limits

• The fetched text cuts off at the experimental setup. The model names, per-task scores and the data behind the "visual fidelity ≠ requirement satisfaction" claim are not in this text. Read the full paper before citing specific numbers.
• The annotations encode one ecosystem's component libraries and conventions, from 17 platforms within a single company. Code targeting other libraries will not be scored as-is.
• Only two annotators produced the labels, and the page reports no inter-annotator agreement figure.
• The page does not state a licence for the dataset. It points to https://taod2c-bench.github.io/ for the release.
• The paper is marked "Under review."