RS9 Everyday Agent Bench, 30 September 2026

One dated run. These are the results as they stood that day.

RS9 Everyday Agent Bench

RS9

Best results

Score out of 100, highest first

  1. Opus 5.5 with Claude Code

    95

  2. Sonnet 5.5 with Claude Code

    85

  3. GPT-6 Astra with Codex

    85

  4. Sol 6.1 with Codex

    80

  5. Fable 5.1 with Claude Code

    70

  6. DeepSeek V4.1 Flash with OpenCode (OpenRouter)

    70

  7. Grok 4.7 with Grok Build

    65

  8. Muse Spark 1.3 with Muse Code

    55

Best value

Cost per task passed, cheapest first

  1. DeepSeek V4.1 Flash with OpenCode (OpenRouter)

    $0.085per task passed

  2. Sol 6.1 with Codex

    $0.45per task passed

  3. Sonnet 5.5 with Claude Code

    $1.00per task passed

  4. Opus 5.5 with Claude Code

    $1.62per task passed

  5. GPT-6 Astra with Codex

    $1.75per task passed

  6. Grok 4.7 with Grok Build

    $4.56per task passed

  7. Fable 5.1 with Claude Code

    $5.91per task passed

  8. Muse Spark 1.3 with Muse Code

    price unknown

Subscription agents: what the same work would cost at public API prices. OpenRouter: the real charge.

20 tasks · one attempt each

rs9.com.au/benchmarks

Score is the percentage of tasks passed, one attempt each. Partial credit (in the table below) gives each task the share of its checks that passed, so it can split agents on the same score.

Value is total cost divided by tasks passed, so an agent that's cheap but fails a lot doesn't look good. It sits beside the score and never changes it.

A few of the 20 tasks are “frontier” tasks, added to stretch the top models.

The limit is 60 minutes a task. An attempt that runs out of time with the work still wrong counts as not passed.

View the results as a table
Score, tasks passed and partial credit for each agent, highest score first
ModelAgent toolScoreTasks passedPartial creditTotal time
Opus 5.5Claude Code9519 of 20992 h 17 min
Sonnet 5.5Claude Code8517 of 20991 h 35 min
GPT-6 AstraCodex8517 of 20992 h 24 min
Sol 6.1Codex8016 of 20994 h 55 min
Fable 5.1Claude Code7014 of 20982 h 56 min
DeepSeek V4.1 FlashOpenCode (OpenRouter)7014 of 20964 h 7 min
Grok 4.7Grok Build6513 of 209710 h 19 min
Muse Spark 1.3Muse Code5511 of 20964 h 54 min
View the value board as a table
Cost per task passed for each agent, cheapest first. Subscription agents show the public API price equivalent. OpenRouter shows the real charge.
ModelAgent toolCost per task passedTasks passed
DeepSeek V4.1 FlashOpenCode (OpenRouter)$0.085 per task passed14 of 20
Sol 6.1Codex$0.45 per task passed16 of 20
Sonnet 5.5Claude Code$1.00 per task passed17 of 20
Opus 5.5Claude Code$1.62 per task passed19 of 20
GPT-6 AstraCodex$1.75 per task passed17 of 20
Grok 4.7Grok Build$4.56 per task passed13 of 20
Fable 5.1Claude Code$5.91 per task passed14 of 20
Muse Spark 1.3Muse Codeprice unknown11 of 20

We gave 8 coding agents the same 20 tasks from the kind of work RS9 does, then checked whether each one got the job done. One attempt each, same brief and files, 60 minutes a task.

It's not a research study. Each agent gets one go at each task, so a gap of a task or two means very little. And these are our tasks, so how an agent does here says little about how it'll do on yours. How it's scored, and where it falls short, is on the methodology page.

Run: . These are the versions we tested that day. Update an agent and its numbers can change.

Speed

Passing versus time taken

Score against time taken

Each dot is one model with its agent tool. Higher is a better score. Further left means less total time across its 20 attempts. Top left is the best spot. Hover or tab to a dot to see which is which.

Score against total timeScatter chart. Each dot is one model with its agent tool. Higher is a better score; the score axis starts at 40. Fable 5.1 with Claude Code: score 70 (14 of 20 tasks), 2 h 56 m total; Opus 5.5 with Claude Code: score 95 (19 of 20 tasks), 2 h 17 m total; Sonnet 5.5 with Claude Code: score 85 (17 of 20 tasks), 1 h 35 m total; GPT-6 Astra with Codex: score 85 (17 of 20 tasks), 2 h 24 m total; Sol 6.1 with Codex: score 80 (16 of 20 tasks), 4 h 55 m total; Grok 4.7 with Grok Build: score 65 (13 of 20 tasks), 10 h 19 m total; Muse Spark 1.3 with Muse Code: score 55 (11 of 20 tasks), 4 h 54 m total; DeepSeek V4.1 Flash with OpenCode (OpenRouter): score 70 (14 of 20 tasks), 4 h 7 m total.4050607080901000 min3 h6 h9 h12 hTotal time across all tasks (shorter is faster)Score (axis starts at 40)Sonnet 5.5Opus 5.5GPT-6 AstraFable 5.1DeepSeek V4.1 FlashMuse Spark 1.3Sol 6.1Grok 4.7

Times include the provider's server speed and queueing, so a slow time can mean a busy server.

View this chart as a table
Score and total time for each agent
ModelAgent toolScoreTasks passedTotal time
Opus 5.5Claude Code9519 of 202 h 17 min
Sonnet 5.5Claude Code8517 of 201 h 35 min
GPT-6 AstraCodex8517 of 202 h 24 min
Sol 6.1Codex8016 of 204 h 55 min
Fable 5.1Claude Code7014 of 202 h 56 min
DeepSeek V4.1 FlashOpenCode (OpenRouter)7014 of 204 h 7 min
Grok 4.7Grok Build6513 of 2010 h 19 min
Muse Spark 1.3Muse Code5511 of 204 h 54 min

Cost

What each passed task cost

Subscription agents don't bill per task. So we count the tokens each attempt used and price them at the provider's public API rates. It's not what anyone paid, but it puts every agent on one scale.

The OpenRouter model shows the real charge, which can sit well below list price if the host discounts it. If there's no public price for a model, it says “price unknown”. Never $0.

Cost per task passed

Total cost divided by tasks passed, for each agent. A shorter bar is cheaper. All amounts are US dollars.

  1. DeepSeek V4.1 Flash with OpenCode (OpenRouter)

    $0.085 per pass

    $1.20 total · Real charge via OpenRouter

  2. Sol 6.1 with Codex

    $0.45 per pass

    $7.26 total · Subscription, at API list price

  3. Sonnet 5.5 with Claude Code

    $1.00 per pass

    $16.93 total · Subscription, at API list price

  4. Opus 5.5 with Claude Code

    $1.62 per pass

    $30.77 total · Subscription, at API list price

  5. GPT-6 Astra with Codex

    $1.75 per pass

    $29.79 total · Subscription, at API list price

  6. Grok 4.7 with Grok Build

    $4.56 per pass

    $59.32 total · Subscription, at API list price, a slight undercount: 4 attempts were cut off before final usage was reported

  7. Fable 5.1 with Claude Code

    $5.91 per pass

    $82.80 total · Subscription, at API list price

  8. Muse Spark 1.3 with Muse Code

    price unknown

    No public API price

View this chart as a table
Cost per task passed, cheapest first
ModelAgent toolCost per passTotal costTasks passedBasis
DeepSeek V4.1 FlashOpenCode (OpenRouter)$0.085$1.2014 of 20What OpenRouter actually charged
Sol 6.1Codex$0.45$7.2616 of 20Subscription: what the same tokens would cost at the provider's public API price
Sonnet 5.5Claude Code$1.00$16.9317 of 20Subscription: what the same tokens would cost at the provider's public API price
Opus 5.5Claude Code$1.62$30.7719 of 20Subscription: what the same tokens would cost at the provider's public API price
GPT-6 AstraCodex$1.75$29.7917 of 20Subscription: what the same tokens would cost at the provider's public API price
Grok 4.7Grok Build$4.56$59.3213 of 20Subscription: what the same tokens would cost at the provider's public API price
Fable 5.1Claude Code$5.91$82.8014 of 20Subscription: what the same tokens would cost at the provider's public API price
Muse Spark 1.3Muse Codeprice unknownprice unknown11 of 20No public API price, so no cost is shown

Score against cost per task passed

Each dot is one model with its agent tool. Higher is a better score. Further left means each passed task cost less. Top left is the best spot. Hover or tab to a dot to see which is which.

Score against cost per task passedScatter chart. Each dot is one model with its agent tool. Higher is a better score; the score axis starts at 60. Fable 5.1 with Claude Code: score 70 (14 of 20 tasks), $5.91 per pass; Opus 5.5 with Claude Code: score 95 (19 of 20 tasks), $1.62 per pass; Sonnet 5.5 with Claude Code: score 85 (17 of 20 tasks), $1.00 per pass; GPT-6 Astra with Codex: score 85 (17 of 20 tasks), $1.75 per pass; Sol 6.1 with Codex: score 80 (16 of 20 tasks), $0.45 per pass; Grok 4.7 with Grok Build: score 65 (13 of 20 tasks), $4.56 per pass; DeepSeek V4.1 Flash with OpenCode (OpenRouter): score 70 (14 of 20 tasks), $0.09 per pass.60708090100$0$2$4$6$8Cost per task passed, US dollars (further left is cheaper)Score (axis starts at 60)DeepSeek V4.1 FlashSol 6.1Sonnet 5.5Opus 5.5GPT-6 AstraGrok 4.7Fable 5.1

Not plotted: Muse Spark 1.3 with Muse Code (price unknown).

View this chart as a table
Score and cost per task passed for each agent
ModelAgent toolScoreTasks passedCost per pass
Opus 5.5Claude Code9519 of 20$1.62
Sonnet 5.5Claude Code8517 of 20$1.00
GPT-6 AstraCodex8517 of 20$1.75
Sol 6.1Codex8016 of 20$0.45
DeepSeek V4.1 FlashOpenCode (OpenRouter)7014 of 20$0.085
Fable 5.1Claude Code7014 of 20$5.91
Grok 4.7Grok Build6513 of 20$4.56
Muse Spark 1.3Muse Code5511 of 20price unknown

The agents

What we tested

Times include the provider's server speed and queueing, so a slow time can mean a busy server.

  • Fable 5.1

    with Claude Code

    Model
    Claude Fable 5.1
    Reasoning
    High
    Client version
    2.1.285 (Claude Code)
    Score
    70 · partial credit 98
    Passed
    14 of 20
    Total time
    2 h 56 min
    Total cost
    US$82.80
    Cost per pass
    US$5.91

    Subscription, at API list price.

    Claude Code in headless mode, high reasoning effort, on a Claude subscription.

  • Opus 5.5

    with Claude Code

    Model
    Claude Opus 5.5
    Reasoning
    High
    Client version
    2.1.285 (Claude Code)
    Score
    95 · partial credit 99
    Passed
    19 of 20
    Total time
    2 h 17 min
    Total cost
    US$30.77
    Cost per pass
    US$1.62

    Subscription, at API list price.

    Claude Code in headless mode, high reasoning effort, on a Claude subscription.

  • Sonnet 5.5

    with Claude Code

    Model
    Claude Sonnet 5.5
    Reasoning
    High
    Client version
    2.1.285 (Claude Code)
    Score
    85 · partial credit 99
    Passed
    17 of 20
    Total time
    1 h 35 min
    Total cost
    US$16.93
    Cost per pass
    US$1.00

    Subscription, at API list price.

    Claude Code in headless mode, high reasoning effort, on a Claude subscription.

  • GPT-6 Astra

    with Codex

    Model
    GPT-6 Astra
    Reasoning
    High
    Client version
    codex-cli 0.159.2
    Score
    85 · partial credit 99
    Passed
    17 of 20
    Total time
    2 h 24 min
    Total cost
    US$29.79
    Cost per pass
    US$1.75

    Subscription, at API list price.

    Codex CLI (exec mode), high reasoning effort, on a ChatGPT subscription.

  • Sol 6.1

    with Codex

    Model
    ChatGPT Sol 6.1
    Reasoning
    High
    Client version
    codex-cli 0.159.2
    Score
    80 · partial credit 99
    Passed
    16 of 20
    Total time
    4 h 55 min
    Total cost
    US$7.26
    Cost per pass
    US$0.45

    Subscription, at API list price.

    Codex CLI (exec mode), high reasoning effort, on a ChatGPT subscription.

  • Grok 4.7

    with Grok Build

    Model
    Grok 4.7
    Reasoning
    High
    Client version
    grok 1.0.44 (5b807183dd79)
    Score
    65 · partial credit 97
    Passed
    13 of 20
    Timed out
    2 of 20
    Total time
    10 h 19 min
    Total cost
    US$59.32
    Cost per pass
    US$4.56

    Subscription, at API list price. Slight undercount: 4 attempts were cut off before final usage was reported.

    Grok Build CLI in headless mode, high reasoning effort, on a SuperGrok subscription.

  • Muse Spark 1.3

    with Muse Code

    Model
    Muse Spark 1.3
    Reasoning
    High
    Client version
    Muse Code 1.4.1 (1.4.1-R4503.1)
    Score
    55 · partial credit 96
    Passed
    11 of 20
    Total time
    4 h 54 min
    Total cost
    price unknown
    Cost per pass
    price unknown

    No public API price.

    Muse Code CLI (exec mode), high reasoning effort, on a Muse Code subscription. Meta publishes no per-token price, so cost is unknown.

  • DeepSeek V4.1 Flash

    with OpenCode (OpenRouter)

    Model
    DeepSeek V4.1 Flash
    Reasoning
    High
    Client version
    1.18.33
    Score
    70 · partial credit 96
    Passed
    14 of 20
    Total time
    4 h 7 min
    Total cost
    US$1.20
    Cost per pass
    US$0.085

    Real charge via OpenRouter. At list price it would be US$0.20 per pass.

    OpenCode via OpenRouter, served by DeepInfra (US), high reasoning effort. Cost is the real charge.

History

Runs