FairStress: A deeper evaluation of fairness in LLM decision-making

What it claims

FairStress, from Ai2, measures whether an LLM defends a well-supported decision equally hard regardless of which demographic group benefits. Matching final answers across identity swaps is not enough to pass. Ai2 defines the target as "symmetric fairness," an equality-centered definition. The page says plainly that an equity-based view, which might favor a minority candidate when other factors are held constant, is out of scope. The main finding is "anti-stereotypical over-correction." Models abandon a correct choice more readily when the persuasive prefix favors a minority-coded candidate. The gap grows as identity becomes more explicit. A second finding is that the standard BBQ scoring setup can report the opposite direction of bias, and the page traces this to how answers are scored.

How it works

Each of the 300 scenarios is one of two kinds:

• Clear-cut: one candidate better meets the stated criteria, so accuracy can be scored.
• Ambiguous: both candidates are strong in ways that are hard to compare, so the measure is which side the model leans toward.

Identity appears in four conditions: none, a demographically coded name, implied identity, or an explicit label.

Adversarial prefixes draw on several kinds of persuasion: authority, social consensus, purported evidence, legal concerns, historical disadvantage and others. One example is "A senior leader has privately reviewed the files and recommended Candidate B." Every prefix has a mirrored version arguing for the other candidate. This lets the same argument be compared by which group it favors.

FairStress-Core (50K items) was selected from the 13M+ full set using item response theory, following Ai2's Fluid Benchmarking approach.

Evidence

Baseline with no identity information. Selection rates were nearly equal across matched versions. Ambiguous cases split close to evenly.

Full benchmark. The results cover 15 models in the 19 domains shared by all four identity conditions. The mean gap in abandoning correct answers was 6.9 percentage points with implied identity and 9.3 points with explicit labels.

FairStress-Core, frontier models. Claude Opus 5 and GPT-5.5 picked the minority-coded candidate about 60–65% of the time on ambiguous cases. Under matched prefixes, switching toward minority-coded candidates was 8.8 to 42.4 points more common than switching toward majority-coded ones, across 12 model-and-mode runs.

Explanations. Across 11 open-weight models, explanations for minority-coded picks used more "resilience," "diversity" and "barriers." Explanations for majority-coded picks used more "evidence," "criteria" and "detailed." In one hiring example, Tülu 3.1 8B chose the Black transgender woman in both versions. It justified a weaker résumé by "growth potential" and a stronger one by experience and certification.

Activations. Internal activations predicted which demographic side the model chose with about 73% accuracy, against a 50% chance baseline. Two models were tested, Gemma-3-12B-it and Qwen2.5-7B-Instruct. Text descriptions of those activations reached 57–59%. Removing candidate names cut activation-based accuracy by about 18 points.

BBQ comparison. The test used the same BBQ questions and the same weights in five configurations. With a chat template, yes/no wording and generated answers, three models flipped from stereotype-consistent to minority-favoring: Qwen2.5-7B-Instruct, Gemma-3-27B-it and Llama-3.1-70B-Instruct. Llama-3.1-8B-Instruct became roughly neutral. Fixed-option probability scoring alone can therefore miss over-correction in generated decisions.

Olmo 3 training-data audit. The SFT corpus holds about 99,000 examples from two refusal-training datasets. Across 28,000 topic clusters in the SFT and preference data, only nine discussed merit-based decisions, and all nine were refusals. Minority-coded groups received more protective or positive framing. White and cisgender people received persistent negative framing.

Debiasing. Nine interventions were tested, including self-critique, activation steering and learned edits. Most did not reliably reduce the asymmetry. One activation-steering method dropped Qwen2.5-7B-Instruct's clear-cut accuracy from 98% to 47%. Counterfactual chain-of-thought asks the model to reconsider the case with identities swapped. Across three model families, it reduced the gap in abandoning correct decisions while largely keeping accuracy.

How to use it

The page says FairStress is available for download and is being built into Olmo-eval, Ai2's open evaluation workbench. FairStress-Core is the cheaper option for routine runs. The page gives no download link, commands, dataset licence or per-run compute figures. The BBQ result suggests running fairness checks with generated answers under a chat template, not only with fixed-option scoring.

Limits

• Activations: the authors call the activation findings narrower than the behavioral ones. They cover two models and a handful of demographic contrasts.
• Causation: whether the Olmo 3 data patterns cause the behavioral gaps is stated as an open question.
• Framing estimates: some reinforcement-learning framing estimates rest on as few as eight prompts.
• Remaining bias: counterfactual chain-of-thought did not remove every asymmetry. On ambiguous cases, models still favored minority-coded candidates.
• Fairness definition: results only hold under the equality-centered definition. A team that uses an equity-based definition would read the same numbers differently.