Hype Check

GDPx: SemiAnalysis’s new expert-work benchmark tops out at 12%

By Willa Ravenwire · · 3 min

CHECK

Why it matters. Opus 5.5 leads GDPx, but its 12% score needs context about task selection, safety filters and the benchmark's specialist partners.

Quick Take SemiAnalysis says even the best model scores just 12% on its new GDPx benchmark. The top spot goes to Claude Opus 5.5, which a Neural Brew summary of the leaderboard lists at 12.0. That is a striking result on a deliberately hard test. Read it as a score on chosen tasks, not a measure of how much expert work AI can do across the economy.

What the numbers say

Per Neural Brew’s leaderboard excerpt, Gemini 4 Carbon sits second at 10.9 and GPT-6 Astra third at 9.6. Muse Wonder Full follows at 9.0. The gaps near the top are small, so a single point can reorder the list.

The official GDPx site counts 206 tasks across 11 domains, built with 14 data partners. Neural Brew reports that the headline metric averages three attempts and gives each domain equal weight. It also says each model gets up to 8 hours per task. Calling 12% a simple pass rate would blur those choices.

On cost, SemiAnalysis only describes the shape of the chart. It says Luna and DeepSeek fight for the low end while Opus wins the high end, with OpenAI in the middle. Exact per-task prices sit in an interactive chart, so test your own workload before you compare budgets.

What the test selects

SemiAnalysis argues that the first trillion dollars of AI revenue will cluster in high-ROI sectors rather than spread across the whole economy. So it picked fields such as chip design, biology, finance, legal work and software engineering. It also left out government, real estate and insurance on purpose.

That choice deserves attention. GDPx is a bet on where frontier models will matter most. Judge it against that stated aim before treating its average as a verdict on all work.

The tasks also look different from older tests. Neural Brew says agents are dropped into working setups, such as an M&A data room holding roughly 4,000 files for finance or live CAD sessions for hardware design. That tests gathering context and taking action, not just writing an answer.

Where the headline needs care

Claim: SemiAnalysis calls Opus 5.5 the best all-around model while naming other leaders in particular fields. It credits Astra in recursive self-improvement and materials science, and Carbon in biology and software engineering.

Caveat: The authors say safety classifiers drag down the overall scores of Opus 5.5 and GPT-6 Astra in some areas. They show a second ranking that drops cyber, biology and alignment. Keep those comparisons apart. A score with domains removed answers a different question than the full test.

For finance and legal work, SemiAnalysis says its tasks are so hard that basically every model scores exactly 0 there. Attribute that to the authors. Do not turn it into a claim that models cannot help with any finance or legal job.

Follow the partner incentives

The GDPx site lists its 14 partners, including Living Context for finance, Chakra for legal work and Datacurve for software engineering. Neural Brew notes that the tasks came from firms that build training environments for frontier labs, and that most tasks are private.

That business link warrants scrutiny. It does not prove the tasks leaked into training, and nothing here shows that. Still, ask about task reuse rules, any overlap with training data, and whether outside groups can rerun the results. Neural Brew says the findings have not yet been independently reproduced.

Bandwagon Check

Fairly hyped as a hard, useful comparison; overhyped as a verdict on expert work. SemiAnalysis’s own claim that even the best model scores just 12% supports the narrow headline. The chosen domains and the safety classifier effects should stay attached to that number. Use the results to decide what to test next. Ask for task examples and grading detail before drawing bigger conclusions about replacing experts.

Share on X

Sources

By Willa Ravenwire · The AI Bandwagon

Next from the wireClaude grades its own conversations. That is a screen, not a safety verdict.

More in Hype Check

  1. Hype Check · 3 min

    Humane Bet on a Screenless Future. Its AI Pin Lasted 10 Months

    Humane raised over $230 million to replace screens with a wearable. The $699 AI Pin shipped in April 2024 and was switched off in February 2025.

    Nov 5, 2026

  2. Hype Check · 3 min

    Devin Was Billed as the First AI Software Engineer. Two Years Later

    Cognition launched Devin in 2024 as an autonomous engineer. An outside test found 3 wins in 20 tasks. The business boomed as a helper, not a…

    Nov 5, 2026

  3. Hype Check · 3 min

    GPT-4’s Top 10% Bar Exam Score, Checked Three Years Later

    OpenAI said GPT-4 scored around the top 10% on a simulated bar exam. A peer-reviewed recheck puts it near the 48th percentile among people who passed.

    Nov 5, 2026