Dayjob: Tax – evaluating AI agents on 50 professional tax assignments

What it claims

DAYJOB: Tax is Surge AI's agent benchmark for tax preparation, compliance and advisory work, following earlier DAYJOB releases for Healthcare and Finance. Its claim is that agents still fail at professional judgment: finding issues the client never raised, weighing conflicting records, and recognizing missing information. The best result is Claude Sonnet 5.5 at 10.8% pass@1. Surge says that even when agents can perform the individual steps, carrying out full tax workflows "remain[s] out of reach."

How it works

Each of the 50 environments gives an agent a client request and a workspace of records. The agent must produce a deliverable: a tax return, a workpaper, a technical review, a strategy letter, or client advice.

Workspaces hold a median of 30 files per task and up to 123. The 1,789 files include 1,487 PDFs, 117 Excel workbooks and 38 Word documents, plus emails, images, CSVs, JSON and text. Some evidence is deliberately non-standard:

• a photo of a handwritten resale receipt that must be reconciled against spending records for cost of goods sold;
• a dispatch log showing a supposedly Alabama-only company working in five states;
• Polish employment records mixed with crypto exchange exports and wallet histories.

Tax experts with backgrounds at PwC, Deloitte, EY, Baker Tilly and Exxon wrote the tasks. Each task passes a three-layer expert review for correctness, completeness and solvability. A rubric grades calculations, supporting evidence, consequential issues and the finished deliverable. An attempt passes only if it meets at least 95% of the criteria.

Evidence

Surge contrasts DAYJOB with three narrower benchmarks:

• TaxBench: 114 defined questions with rubrics.
• TaxCalcBench: accuracy of computed return values.
• TaxLab: questioning a simulated taxpayer to complete a federal individual return.

The page also plots performance against average cost per trial. Sonnet 5.5 and Opus 5.5 sit at the high-performance end of the Pareto frontier. No dollar figures are given.

Two five-attempt case studies show that model rankings flip from task to task:

• Condo sale. The client compared two closing dates. A model called "Astra" never flagged that her two-of-five-year residence eligibility lapses around August 2027. Opus 5.5 flagged it in all five attempts.
• Retirement split. A couple asked how to divide a Roth IRA and a 401(k). Opus 5.5 assumed in all five attempts that the couple was unmarried, despite Texas residence and a Colorado ceremony in their files. Astra asked to verify marital status in four of five attempts.

How to use it

The full benchmark is not public. Access is by request to benchmarks@surgehq.ai. The complete leaderboard is on the DAYJOB: Tax benchmark page.

Limits

• Only the top three scores appear in the post. The remaining models are on the separate leaderboard page.
• The post does not identify "Astra" or give harness details, tool access, or cost figures.
• The post gives no per-practice-area breakdown.
• The 95% threshold makes pass rates low by design. Scores here are not comparable to benchmarks that use partial credit.
• The tasks are written by experts, not drawn from real client files.