Artificial Analysis (@ArtificialAnlys) on X

What it claims

Artificial Analysis has added an HLE-Diamond leaderboard to v5 of its Intelligence Index. Only 17 model variants (about 13% of those measured) score above 50%. Gemini 4 Carbon (high) leads at 69%, followed by Claude Opus 5.5 (max) at 68%. HLE-Diamond is the refined version of Humanity's Last Exam, released by CAIS and Scale AI after a year of cleaning the original dataset with input from the research community.

How it works

The benchmark has 1,000 closed-ended questions, split evenly between reasoning and knowledge. 272 of them include an image, so text-only models cannot attempt the full set as posed.

Artificial Analysis runs each question once, without tools, using the official HLE system prompt. GPT-5.6 Luna (medium) grades each answer against the reference answer using the official HLE judge prompt. The reported score is the share judged correct (pass@1).

Evidence

More than 90% of measured models score higher on the reasoning half than on the knowledge half. Here are the scores the post gives on each half:

Gemini 4 Carbon (high) leads both halves. The two models tied on reasoning differ by 8 points on knowledge.

Open weights models trail the leaders by more than 30 points:

• Kimi K3.1 Pro: 37%
• MiniMax-M3.1-Flash: 28%
• Kimi K3 (max): 27%

Limits

Kimi K3.1 Pro is counted as open weights, but its weights are not yet released. The top open model that can actually be downloaded today scores 28% (MiniMax-M3.1-Flash), unless Kimi K3.1 Pro's weights ship.

Each question is run once. The post reports no variance or confidence intervals, so a 1-point gap such as 69% vs 68% cannot be read as a ranking difference.

Grading depends on a single LLM judge, GPT-5.6 Luna (medium). The post gives no judge accuracy figures.

All scores are tool-free, so they do not reflect performance with search or code execution.

The post does not give the total number of model variants evaluated. It does not list the other models above 50%, and it gives no per-split scores for open weights models.