The scoreboard
Mean coverage across four multi-part deliverables, 110 rubric criteria in all. Each criterion is scored met (1), partial (½) or missed (0) by an expert-instructed judge, deliberately strict: a criterion that calls for a computed number is missed if the model only discusses it. Multiple choice couldn't separate these models. Real work spreads them 41 points.
On plain multiple choice, these eleven models cluster inside ten points. Grade them on real deliverables against a strict rubric and the field opens to forty-one — Grok and Opus clear 94%, Llama lands at 55%. The frontier is decided on the banking task, where a model has to build the leveraged-buyout returns, not describe them — Llama scores 41% there, and only discovers a deal misses its hurdle if it actually runs the numbers.
Where they break
The same runs, cut by deliverable. Overall scores hide the split personalities: Sonnet writes a flawless legal review (98) then face-plants on the strategy case (81); Gemini 2.5 Pro nails the law (95) but botches the banking math (64). The quantitative, must-be-built tasks do the sorting; the knowledge-heavy legal task barely separates anyone.
| Model | Banking (LBO) | Consulting | Law | Systems |
|---|---|---|---|---|
| Grok 4.5 | 96 | 96 | 95 | 96 |
| Claude Opus 4.8 | 93 | 92 | 98 | 93 |
| Claude Sonnet 4.6 | 93 | 81 | 98 | 88 |
| GPT-5.2 | 89 | 83 | 98 | 80 |
| GPT-5 mini | 84 | 79 | 91 | 89 |
| Claude Haiku 4.5 | 80 | 77 | 96 | 71 |
| Gemini 2.5 Pro | 91 | 73 | 95 | 64 |
| Gemini 2.5 Flash | 80 | 67 | 95 | 73 |
| DeepSeek V3.1 | 57 | 73 | 96 | 73 |
| Qwen3 235B | 45 | 81 | 93 | 71 |
| Llama 4 Maverick | 41 | 60 | 80 | 39 |
What's on the test
Not questions — the multi-part artifacts a junior professional is handed on day one. Every rubric mixes objective catches (a number that must be computed) with judgment catches (a risk that must be flagged, a trap that must be avoided).
Build the LBO model and the IC memo
Sources & uses, a five-year debt-paydown schedule, IRR and MOIC, a downside case, a recommendation.
Take a meal-kit brand into corporate catering
Bottoms-up TAM/SAM/SOM, B2B unit economics, build-vs-partner, a phased go-to-market, a go / no-go.
Find all ten landmines in a founder's story
Ten distinct issues, ranked by severity, each with remediation and a first-week action list.
Serve an 8GB model at 50k rps, p99 < 100ms
Capacity math, batching, a zero-downtime weekly deploy, multi-region failover, an SLO / error budget.
The agentic tier — and the harness that nearly lied
Then we stopped grading prose and made the models work: each is dropped into a
SQLite database it has never seen, given one run_sql tool, and has to investigate — discover the
schema, query, revise, and answer six questions with multi-hop joins and a planted data-quality trap. No
judge: answers are checked against a reference computed live from the data. The board below is the number we
trust — but getting there took building the harness twice.
Our first harness — a text protocol — put Sonnet at 23% and Haiku at 7%. It was lying: the protocol let a model fabricate its own query results (Sonnet invented a region, "West," and answered from it). We rebuilt it on native tool-calling, where the provider's API enforces the turn boundary. That harness had its own bug — it never told thorough models to stop, zeroing Grok and DeepSeek. Only after fixing both did the board converge. The text harness had understated nine of eleven models; the small number under each score above is what it wrongly reported.
Method & honest limits
A benchmark that overstates itself isn't worth publishing. Here is exactly what these runs do and don't establish.
An LLM graded the rubric
The deliverable scores are judge-assigned. A credible board needs expert-authored, expert-validated rubrics — the human-graded work that is Pasiflora's product.
The harness can outweigh the model
On the agentic tier, two harnesses disagreed by up to 93 points on the same model. Only native tool-calling, validated against its own artifacts, is trustworthy.
Too easy at the top
Six models tie at 100% on the fair agentic board. The SQL task separates the mid-tier but not the frontier — it needs harder, longer-horizon tasks.
Averaged, not single-shot
Agentic scores are the mean of multiple runs; GPT-5.2 alone swung 6/6, 2/6, 3/6 before the harness fix. A single agentic run is noise.
The takeaway
The most important result on this page isn't a score. It's the score we almost got wrong.
For frontier models, the hard part of evaluation isn't the models — it's the measurement.
Multiple choice saturates. Strict rubrics separate a little, but the ceiling holds. And the moment you reach for the thing that should be hardest — agentic work with real tools — the harness starts deciding the outcome. We twice built a board that ranked a strong model dead last, and twice it was our wiring, not the model.
Catching that is the whole game. A trustworthy benchmark is native tool-calling per provider, multiple runs, ground-truth grading, and adversarial validation against your own artifacts — before you publish a number. Getting the harness right is unglamorous, and it's exactly where most public benchmarks quietly go wrong.
That's the work Pasiflora does — turning a vetted expert network into evaluations rigorous enough to find the real edge of what these models can do. This bench is the floor of that capability, built in-house to prove the harness. The ceiling is what our experts author.