GraphForge: Training Working Agents with Graph-Anchored Workspace Synthesis

What it claims

GraphForge synthesizes training tasks for "working agents" (agents that read many files, use tools and produce deliverables) from real crawled files, with rubrics whose criteria point to specific source files. The authors position it against two prior working-agent pipelines. EnvCraft generates files with a model and verifies them with state-check scripts that cannot read file contents. NexForge uses real files but has no task-specific verifier and does not release its tasks. GraphForge releases 2.1K tasks, which is smaller than EnvCraft's 20K and NexForge's 5.6K. The data and models are at https://huggingface.co/collections/groundhogLLM/graphforge.

How it works

1. Seeds. Seeds come from O*NET occupations and their Detailed Work Activities, filtered to tasks AI4Work labels as DIGITAL. That leaves 246 occupations, 891 task types and 3,419 occupation–task pairs. Each seed adds a work demand, one of 16 execution patterns and an input file family. Seeds are chosen greedily by marginal coverage (gain = Σ 1/(1+count) per dimension). The same selection is re-run after files are downloaded. Seeds name no company or dataset.
2. Workspace. A search agent finds a public case and downloads files in their native formats. Each file gets a hidden role: core, supporting, confuser (plausible but inapplicable) or ambient (realistic redundancy). The working agent never sees these roles.
3. Evidence graph. Nodes are a file plus the fact it provides. Edges are cross-file dependencies. The graph is compiled into four parts: a task statement, deliverable requirements, weighted positive criteria and penalty criteria. Each criterion carries anchors to graph nodes and a verification procedure.
4. Revision and admission. The agent sees only the task and files. The judge sees the anchors.

flowchart TD
  A[Task spec C0] --> B[Teacher rollout]
  B --> C[Revision agent fixes task/rubric]
  C --> D{Task statement changed<br/>or trace missing?}
  D -- no --> E[Reuse initial trace]
  D -- yes --> F[Rerun teacher]
  E --> G[Deterministic artifact checks]
  F --> G
  G --> H[Anchored judge score Q]
  H --> I{Q > 0.90 and<br/>no degenerate tool use?}
  I -- yes --> J[Admitted trajectory]

Before judging, deterministic checks confirm filenames, readable formats, and the required sheets and formulas. The score Q is the weighted fraction of criteria met minus binary penalties. Q can be negative. Trajectories are also discarded for degenerate tool use: repeated calls, excessive tool use, high failure rates or repeated truncation.

Evidence

• Funnel. 2,169 trajectories were admitted from 3,638 materialized tasks. They cover 466 O*NET task types, 15 of 16 sectors, all 16 patterns, and 164 of 256 sector×pattern combinations.
• Skew. Data analysis is 14.2% of the corpus and research synthesis is 13.2%. PDF appears in 96.7% of trajectories.
• SFT results on Qwen3.6-27B:

• Second model. The same data also improves Qwen3.6-35B-A3B. The fetched text does not give those numbers.
• RFT. The second stage trains on the SFT model's own rollouts, on queries disjoint from the SFT data, selected by the rubrics. It improves all three benchmarks further. The fetched text does not give those numbers either. The authors claim rubric selection adds signal beyond self-rollouts.

How to use it

The released dataset and trained models are in the Hugging Face collection linked above. The page names no licence, no teacher model and no training compute.

Limits

• The fetched text cuts off in Section 4.1. RFT numbers, Qwen3.6-35B-A3B results, ablations and any stated limitations are not visible here.
• The teacher model behind every trajectory is unnamed. Reproduction cost is therefore unknown.
• The corpus is small (2.1K tasks), heavily PDF-centric and skewed toward analysis and reporting tasks.
• The judge and the revision step are both model-based. The graph is explicitly "not ground truth", so rubric correctness relies on LLM anchoring rather than human checking.
• GDPVal results are on the 220-task subset, not the full 1,320 tasks.