Overall score
The equal-weight mean of Agent Score and Harness Macro, computed from unrounded run fractions. It summarizes both layers without replacing either one.
TRIAL BY TRACE / TRACETRIAL-BENCH
Leaderboard
Task performance. Harness reliability. A clearer picture of the agent system.
- 95
- Matched tasks
- 4
- Native harnesses
- 9
- Models
- 3,420
- Selected executions
Compare the results
Separate task outcomes from the reliability of the system delivering them.
Harness comparison
Overall score · equal weight for Agent Score and Harness Macro · 0–100 scale
- Claude Code73.5
- OpenClaw71.8
- DeepSeek Harness71.0
- Codex65.2
No matching results. Try a different search or filter.
4 harnesses · 855 runs each
| # | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|
| 01 | Claude Code | All 9 models | 73.5 | 44.0 | 75.9 | 71.2 | 9.1 | $0.436 | 569.4 | 9.5 | 578.8 |
| 02 | OpenClaw | All 9 models | 71.8 | 38.2 | 71.7 | 72.0 | 11.7 | $0.633 | 670.8 | 9.2 | 680.0 |
| 03 | DeepSeek Harness | All 9 models | 71.0 | 40.0 | 73.3 | 68.8 | 9.8 | $0.446 | 361.5 | 11.2 | 372.7 |
| 04 | Codex | All 9 models | 65.2 | 35.8 | 70.7 | 59.8 | 2.8 | $0.595 | 606.9 | 10.2 | 617.2 |
Scores are percentages (higher is better). Resource columns show mean USD and thousands of tokens per selected run. Scroll horizontally for all columns; select a heading to sort.
Read the scores, not just the rank
Task success and harness reliability answer different questions.
Task pass & Agent score
Task pass requires every Agent criterion to pass. Agent Score averages the passed fraction per run, retaining partial credit when the task is not fully completed.
Harness macro & Clean
Harness Macro averages each run’s fraction of passed Harness criteria. Clean is stricter: every criterion in both the Agent and Harness layers must pass.
Evaluation scope
Every model–harness pair covers the same 95 tasks. Harness aggregates pool nine models, or 855 selected runs per harness. The archive retains one selected execution per cell, including recovery replacements; these are descriptive results, not estimates of repeated-run stability.
Cost estimates use recorded token usage and frozen model-specific prices. They exclude Judge calls, discarded runs, storage, and tool charges. Token totals and cost are reported values; rounded input and output counts may not sum exactly to rounded totals.