RS9 Everyday Agent Bench
Best results
Score out of 100, highest first
Best value
Cost per task passed, cheapest first
Subscription agents: what the same work would cost at public API prices. OpenRouter: the real charge.
20 tasks · one attempt each
rs9.com.au/benchmarks
Score is the percentage of tasks passed, one attempt each. Partial credit (in the table below) gives each task the share of its checks that passed, so it can split agents on the same score.
Value is total cost divided by tasks passed, so an agent that's cheap but fails a lot doesn't look good. It sits beside the score and never changes it.
A few of the 20 tasks are “frontier” tasks, added to stretch the top models.
The limit is 60 minutes a task. An attempt that runs out of time with the work still wrong counts as not passed.
View the results as a table
| Model | Agent tool | Score | Tasks passed | Partial credit | Total time |
|---|---|---|---|---|---|
| Opus 5.5 | Claude Code | 95 | 19 of 20 | 99 | 2 h 17 min |
| Sonnet 5.5 | Claude Code | 85 | 17 of 20 | 99 | 1 h 35 min |
| GPT-6 Astra | Codex | 85 | 17 of 20 | 99 | 2 h 24 min |
| Sol 6.1 | Codex | 80 | 16 of 20 | 99 | 4 h 55 min |
| Fable 5.1 | Claude Code | 70 | 14 of 20 | 98 | 2 h 56 min |
| DeepSeek V4.1 Flash | OpenCode (OpenRouter) | 70 | 14 of 20 | 96 | 4 h 7 min |
| Grok 4.7 | Grok Build | 65 | 13 of 20 | 97 | 10 h 19 min |
| Muse Spark 1.3 | Muse Code | 55 | 11 of 20 | 96 | 4 h 54 min |
View the value board as a table
| Model | Agent tool | Cost per task passed | Tasks passed |
|---|---|---|---|
| DeepSeek V4.1 Flash | OpenCode (OpenRouter) | $0.085 per task passed | 14 of 20 |
| Sol 6.1 | Codex | $0.45 per task passed | 16 of 20 |
| Sonnet 5.5 | Claude Code | $1.00 per task passed | 17 of 20 |
| Opus 5.5 | Claude Code | $1.62 per task passed | 19 of 20 |
| GPT-6 Astra | Codex | $1.75 per task passed | 17 of 20 |
| Grok 4.7 | Grok Build | $4.56 per task passed | 13 of 20 |
| Fable 5.1 | Claude Code | $5.91 per task passed | 14 of 20 |
| Muse Spark 1.3 | Muse Code | price unknown | 11 of 20 |
We gave 8 coding agents the same 20 tasks from the kind of work RS9 does, then checked whether each one got the job done. One attempt each, same brief and files, 60 minutes a task.
It's not a research study. Each agent gets one go at each task, so a gap of a task or two means very little. And these are our tasks, so how an agent does here says little about how it'll do on yours. How it's scored, and where it falls short, is on the methodology page.
Run: . These are the versions we tested that day. Update an agent and its numbers can change.
Speed
Passing versus time taken
Score against time taken
Each dot is one model with its agent tool. Higher is a better score. Further left means less total time across its 20 attempts. Top left is the best spot. Hover or tab to a dot to see which is which.
Times include the provider's server speed and queueing, so a slow time can mean a busy server.
View this chart as a table
| Model | Agent tool | Score | Tasks passed | Total time |
|---|---|---|---|---|
| Opus 5.5 | Claude Code | 95 | 19 of 20 | 2 h 17 min |
| Sonnet 5.5 | Claude Code | 85 | 17 of 20 | 1 h 35 min |
| GPT-6 Astra | Codex | 85 | 17 of 20 | 2 h 24 min |
| Sol 6.1 | Codex | 80 | 16 of 20 | 4 h 55 min |
| Fable 5.1 | Claude Code | 70 | 14 of 20 | 2 h 56 min |
| DeepSeek V4.1 Flash | OpenCode (OpenRouter) | 70 | 14 of 20 | 4 h 7 min |
| Grok 4.7 | Grok Build | 65 | 13 of 20 | 10 h 19 min |
| Muse Spark 1.3 | Muse Code | 55 | 11 of 20 | 4 h 54 min |
Cost
What each passed task cost
Subscription agents don't bill per task. So we count the tokens each attempt used and price them at the provider's public API rates. It's not what anyone paid, but it puts every agent on one scale.
The OpenRouter model shows the real charge, which can sit well below list price if the host discounts it. If there's no public price for a model, it says “price unknown”. Never $0.
Cost per task passed
Total cost divided by tasks passed, for each agent. A shorter bar is cheaper. All amounts are US dollars.
View this chart as a table
| Model | Agent tool | Cost per pass | Total cost | Tasks passed | Basis |
|---|---|---|---|---|---|
| DeepSeek V4.1 Flash | OpenCode (OpenRouter) | $0.085 | $1.20 | 14 of 20 | What OpenRouter actually charged |
| Sol 6.1 | Codex | $0.45 | $7.26 | 16 of 20 | Subscription: what the same tokens would cost at the provider's public API price |
| Sonnet 5.5 | Claude Code | $1.00 | $16.93 | 17 of 20 | Subscription: what the same tokens would cost at the provider's public API price |
| Opus 5.5 | Claude Code | $1.62 | $30.77 | 19 of 20 | Subscription: what the same tokens would cost at the provider's public API price |
| GPT-6 Astra | Codex | $1.75 | $29.79 | 17 of 20 | Subscription: what the same tokens would cost at the provider's public API price |
| Grok 4.7 | Grok Build | $4.56 | $59.32 | 13 of 20 | Subscription: what the same tokens would cost at the provider's public API price |
| Fable 5.1 | Claude Code | $5.91 | $82.80 | 14 of 20 | Subscription: what the same tokens would cost at the provider's public API price |
| Muse Spark 1.3 | Muse Code | price unknown | price unknown | 11 of 20 | No public API price, so no cost is shown |
Score against cost per task passed
Each dot is one model with its agent tool. Higher is a better score. Further left means each passed task cost less. Top left is the best spot. Hover or tab to a dot to see which is which.
Not plotted: Muse Spark 1.3 with Muse Code (price unknown).
View this chart as a table
| Model | Agent tool | Score | Tasks passed | Cost per pass |
|---|---|---|---|---|
| Opus 5.5 | Claude Code | 95 | 19 of 20 | $1.62 |
| Sonnet 5.5 | Claude Code | 85 | 17 of 20 | $1.00 |
| GPT-6 Astra | Codex | 85 | 17 of 20 | $1.75 |
| Sol 6.1 | Codex | 80 | 16 of 20 | $0.45 |
| DeepSeek V4.1 Flash | OpenCode (OpenRouter) | 70 | 14 of 20 | $0.085 |
| Fable 5.1 | Claude Code | 70 | 14 of 20 | $5.91 |
| Grok 4.7 | Grok Build | 65 | 13 of 20 | $4.56 |
| Muse Spark 1.3 | Muse Code | 55 | 11 of 20 | price unknown |
The agents
What we tested
Times include the provider's server speed and queueing, so a slow time can mean a busy server.
Fable 5.1
with Claude Code
- Model
- Claude Fable 5.1
- Reasoning
- High
- Client version
- 2.1.285 (Claude Code)
- Score
- 70 · partial credit 98
- Passed
- 14 of 20
- Total time
- 2 h 56 min
- Total cost
- US$82.80
- Cost per pass
- US$5.91
Subscription, at API list price.
Claude Code in headless mode, high reasoning effort, on a Claude subscription.
Opus 5.5
with Claude Code
- Model
- Claude Opus 5.5
- Reasoning
- High
- Client version
- 2.1.285 (Claude Code)
- Score
- 95 · partial credit 99
- Passed
- 19 of 20
- Total time
- 2 h 17 min
- Total cost
- US$30.77
- Cost per pass
- US$1.62
Subscription, at API list price.
Claude Code in headless mode, high reasoning effort, on a Claude subscription.
Sonnet 5.5
with Claude Code
- Model
- Claude Sonnet 5.5
- Reasoning
- High
- Client version
- 2.1.285 (Claude Code)
- Score
- 85 · partial credit 99
- Passed
- 17 of 20
- Total time
- 1 h 35 min
- Total cost
- US$16.93
- Cost per pass
- US$1.00
Subscription, at API list price.
Claude Code in headless mode, high reasoning effort, on a Claude subscription.
GPT-6 Astra
with Codex
- Model
- GPT-6 Astra
- Reasoning
- High
- Client version
- codex-cli 0.159.2
- Score
- 85 · partial credit 99
- Passed
- 17 of 20
- Total time
- 2 h 24 min
- Total cost
- US$29.79
- Cost per pass
- US$1.75
Subscription, at API list price.
Codex CLI (exec mode), high reasoning effort, on a ChatGPT subscription.
Sol 6.1
with Codex
- Model
- ChatGPT Sol 6.1
- Reasoning
- High
- Client version
- codex-cli 0.159.2
- Score
- 80 · partial credit 99
- Passed
- 16 of 20
- Total time
- 4 h 55 min
- Total cost
- US$7.26
- Cost per pass
- US$0.45
Subscription, at API list price.
Codex CLI (exec mode), high reasoning effort, on a ChatGPT subscription.
Grok 4.7
with Grok Build
- Model
- Grok 4.7
- Reasoning
- High
- Client version
- grok 1.0.44 (5b807183dd79)
- Score
- 65 · partial credit 97
- Passed
- 13 of 20
- Timed out
- 2 of 20
- Total time
- 10 h 19 min
- Total cost
- US$59.32
- Cost per pass
- US$4.56
Subscription, at API list price. Slight undercount: 4 attempts were cut off before final usage was reported.
Grok Build CLI in headless mode, high reasoning effort, on a SuperGrok subscription.
Muse Spark 1.3
with Muse Code
- Model
- Muse Spark 1.3
- Reasoning
- High
- Client version
- Muse Code 1.4.1 (1.4.1-R4503.1)
- Score
- 55 · partial credit 96
- Passed
- 11 of 20
- Total time
- 4 h 54 min
- Total cost
- price unknown
- Cost per pass
- price unknown
No public API price.
Muse Code CLI (exec mode), high reasoning effort, on a Muse Code subscription. Meta publishes no per-token price, so cost is unknown.
DeepSeek V4.1 Flash
with OpenCode (OpenRouter)
- Model
- DeepSeek V4.1 Flash
- Reasoning
- High
- Client version
- 1.18.33
- Score
- 70 · partial credit 96
- Passed
- 14 of 20
- Total time
- 4 h 7 min
- Total cost
- US$1.20
- Cost per pass
- US$0.085
Real charge via OpenRouter. At list price it would be US$0.20 per pass.
OpenCode via OpenRouter, served by DeepInfra (US), high reasoning effort. Cost is the real charge.
History
Runs
- 2 October 2026
- 30 September 2026 · this page