RubricArmor: Adversarial Evolution Improves LLM-Based Rubric Generation

What it claims

RubricArmor hardens LLM-generated rubrics before RL by repeatedly attacking them with responses that score full marks but are wrong, then patching the rubric. The paper reports two results:

• It beats the strongest rubric-generation baseline by 3.12 points of macro accuracy across nine preference evaluation sets.
• It gives the best downstream RL performance in 13 of 15 settings across three policy models.

The fetched text does not name the baselines, the evaluation sets, the policy models or the RL algorithm used.

How it works

The reward is the fraction of rubric criteria an LLM verifier marks Pass. Every criterion has equal weight and gets a binary 0/1 judgment. The system is fully automated and multi-agent. An initializer writes the first rubric R₀ from the query. Each round then runs two modules:

• Attack Discovery. An attacker writes an adversarial response yₜ and names its defect pₜ. The attack counts only if both checks pass: 1. The verifier gives yₜ full reward under the current rubric. 2. A separate quality judge confirms that pₜ is really present and substantially lowers quality.
• Verified Rubric Repair. A repair agent revises the rubric. Each criterion must be atomic (one requirement), self-contained (it states the needed facts or values, e.g. "total = 150 min" rather than "show the calculation"), and non-overlapping. The revision is accepted only if both checks pass: 1. The same yₜ now fails at least one criterion. 2. A review agent confirms the change catches the defect, keeps the other valid criteria, and respects the three properties.

A memory H stores past attacks, their outcomes and accepted fixes. Both the attacker and the repair agent read it, so new fixes do not undo old ones. The loop stops after at most T rounds, or after P consecutive rounds with no accepted revision. The last accepted rubric is used for RL.

flowchart TD
  I[Initialize rubric from query] --> A[Attacker: response + claimed defect]
  A --> V{Verifier: full reward?}
  V -- no --> S[No change; patience +1]
  V -- yes --> Q{Quality judge: defect real?}
  Q -- no --> S
  Q -- yes --> R[Repair agent revises rubric]
  R --> V2{Same response now fails?}
  V2 -- no --> S
  V2 -- yes --> RV{Review agent approves?}
  RV -- no --> S
  RV -- yes --> U[Accept rubric; reset patience]
  U --> C{T rounds or P stalls?}
  S --> C
  C -- no --> A
  C -- yes --> O[Return last accepted rubric]
  M[(Memory H)] -.-> A
  M -.-> R

The paper names two failure modes it targets:

• Omitted criteria. The rubric checks "Paris is the capital" but not a false claim that Paris is in Asia.
• Underspecified criteria. "Provide a definition" passes a definition that is wrong.

How to use it

Code: https://github.com/yhc-666/RubricArmor.git

The fetched text gives no install commands, default T or P values, or recommended backbone models.

Limits

• Truncated page. It cuts off partway through the experiment list. The ablations (response construction, adversarial objective), backbone generalization, sensitivity to the round limit, and the RL experiments were not in the text, so their results cannot be reported here.
• Cost not stated. Each query needs several LLM calls per round (attacker, verifier, judge, repair agent, verifier again, reviewer), repeated up to T rounds. Budget for this before generating rubrics at scale.
• Your judges are part of the method. Robustness rests on the LLM verifier and quality judge being reliable. If they misjudge, the loop can accept bad attacks or bad repairs.
• Equal weights only. All criteria count the same, so weighted rubrics were not shown.
• Preprint. This is arXiv v1, dated 4 Oct 2026.