FrontierCode
Artificial Analysis leaderboard for models on agentic coding tasks in real open-source repositories: would a maintainer merge the change?
- Models
- 25
- Best score
- 52.2%
- Claude Opus 5.5 (Max, Default Fallback)
- Best open-weights
- 45.4%
- MiMo-V2.6-Pro
- Newest
- Claude Haiku 5.5
- Oct 7, 2026 · 41.8%
Score by cost, time or tokens
Average values per task, computed as on Artificial Analysis. The horizontal axis is reversed: towards the top right, a model scores more for less money, less time or fewer tokens.
Models shown23 of 23
Select a model to show or hide it; hovering it brings it forward on the chart. The axes fit the models shown.
25 / 25 models
Released: Sep 22, 2026Cost / task: $10.15Time / task: 17 min 21 sTokens / task: 157KReasoning
Released: Sep 29, 2026Cost / task: $0.812Time / task: 11 min 34 sTokens / task: 33.2KReasoning
Released: Sep 1, 2026Cost / task: $12.00Time / task: 13 min 52 sTokens / task: 94.2KReasoning
Released: Sep 3, 2026Cost / task: $4.22Time / task: 10 min 15 sTokens / task: 26.9KReasoning
Released: Sep 28, 2026Cost / task: $8.26Time / task: 18 min 44 sTokens / task: 236KReasoning
Released: Sep 21, 2026Cost / task: $0.18Time / task: 29 min 28 sTokens / task: 84.4KReasoningOpen weights
Released: Sep 10, 2026Cost / task: $0.479Time / task: 7 min 51 sTokens / task: 119KReasoningOpen weights
Released: Jul 16, 2026Cost / task: $3.89Time / task: 19 min 09 sTokens / task: 56.5KReasoningOpen weights
Released: Sep 2, 2026Cost / task: $6.75Time / task: 37 min 57 sTokens / task: 120KReasoning
Released: Sep 30, 2026Cost / task: $6.65Time / task: —Tokens / task: 109KReasoning
Released: Sep 22, 2026Cost / task: $0.0922Time / task: 7 min 01 sTokens / task: 48.4KReasoning
Released: Oct 7, 2026Cost / task: $0.329Time / task: 8 min 34 sTokens / task: 197KReasoning
Released: Aug 26, 2026Cost / task: $0.363Time / task: 11 min 27 sTokens / task: 61.4KReasoningOpen weights
Released: Sep 2, 2026Cost / task: $2.81Time / task: 7 min 16 sTokens / task: 56.6KReasoning
Released: Aug 18, 2026Cost / task: $4.32Time / task: 14 min 57 sTokens / task: 83.4KReasoningOpen weights
Released: Sep 18, 2026Cost / task: $1.96Time / task: 12 minTokens / task: 72.5KReasoning
Released: Sep 2, 2026Cost / task: $3.72Time / task: 11 min 32 sTokens / task: 99.3KReasoning
Released: Aug 14, 2026Cost / task: $2.47Time / task: 26 min 41 sTokens / task: 80.1KReasoningOpen weights
Released: Sep 21, 2026Cost / task: $9.31Time / task: 19 min 29 sTokens / task: 82.1KReasoning
Released: Sep 21, 2026Cost / task: $11.31Time / task: 20 min 04 sTokens / task: 94.9KReasoning
Released: Sep 3, 2026Cost / task: —Time / task: 2 min 14 sTokens / task: 62.6KReasoningOpen weights
Released: Jun 1, 2026Cost / task: $1.01Time / task: 7 min 28 sTokens / task: 43KReasoningOpen weights
Released: Jul 15, 2026Cost / task: —Time / task: 3 min 15 sTokens / task: 42KReasoningOpen weights
Released: Aug 10, 2026Cost / task: $0.281Time / task: 3 min 40 sTokens / task: 27.2KReasoningOpen weights
Released: Jun 4, 2026Cost / task: $5.51Time / task: 4 min 43 sTokens / task: 45.1KReasoningOpen weights
| # | Model | Score | Cost / task | Time / task | Tokens / task | Released | Type |
|---|---|---|---|---|---|---|---|
Claude Opus 5.5Max, Default Fallback Anthropic | 52.2% | $10.15 | 17 min 21 s | 157K | Sep 22, 2026 | Reasoning | |
| 2 | GPT-6.1 SolMax OpenAI | 49.1% | $0.812 | 11 min 34 s | 33.2K | Sep 29, 2026 | Reasoning |
| 3 | Claude Fable 5.1Max, Default Fallback Anthropic | 49.0% | $12.00 | 13 min 52 s | 94.2K | Sep 1, 2026 | Reasoning |
| 4 | GPT-6 AstraMax OpenAI | 47.0% | $4.22 | 10 min 15 s | 26.9K | Sep 3, 2026 | Reasoning |
| 5 | Claude Sonnet 5.5Max, Default Fallback Anthropic | 45.8% | $8.26 | 18 min 44 s | 236K | Sep 28, 2026 | Reasoning |
| 6 | MiMo-V2.6-Pro Xiaomi | 45.4% | $0.18 | 29 min 28 s | 84.4K | Sep 21, 2026 | ReasoningOpen weights |
| 7 | DeepSeek V4.1 FlashMax DeepSeek | 44.0% | $0.479 | 7 min 51 s | 119K | Sep 10, 2026 | ReasoningOpen weights |
| 8 | Kimi K3Max Moonshot AI | 43.6% | $3.89 | 19 min 09 s | 56.5K | Jul 16, 2026 | ReasoningOpen weights |
| 9 | Qwen3.8 Max (0902) Alibaba | 43.2% | $6.75 | 37 min 57 s | 120K | Sep 2, 2026 | Reasoning |
| 10 | Gemini 4 ArgonHigh | 42.6% | $6.65 | — | 109K | Sep 30, 2026 | Reasoning |
| 11 | GPT-6 LunaMax OpenAI | 41.9% | $0.0922 | 7 min 01 s | 48.4K | Sep 22, 2026 | Reasoning |
| 12 | Claude Haiku 5.5Max Anthropic | 41.8% | $0.329 | 8 min 34 s | 197K | Oct 7, 2026 | Reasoning |
| 13 | GLM-5.3-Flash Z.AI | 41.0% | $0.363 | 11 min 27 s | 61.4K | Aug 26, 2026 | ReasoningOpen weights |
| 14 | Muse Spark 1.3Max Meta | 40.6% | $2.81 | 7 min 16 s | 56.6K | Sep 2, 2026 | Reasoning |
| 15 | GLM-5.3Max Z.AI | 40.1% | $4.32 | 14 min 57 s | 83.4K | Aug 18, 2026 | ReasoningOpen weights |
| 16 | Step 5 Preview StepFun | 38.4% | $1.96 | 12 min | 72.5K | Sep 18, 2026 | Reasoning |
| 17 | Gemini 3.8 FlashHigh | 36.4% | $3.72 | 11 min 32 s | 99.3K | Sep 2, 2026 | Reasoning |
| 18 | Qwen3.8 27BXhigh Alibaba | 34.2% | $2.47 | 26 min 41 s | 80.1K | Aug 14, 2026 | ReasoningOpen weights |
| 19 | Grok 4.7High SpaceXSI | 34.2% | $9.31 | 19 min 29 s | 82.1K | Sep 21, 2026 | Reasoning |
| 20 | Grok 4.7Xhigh SpaceXSI | 33.8% | $11.31 | 20 min 04 s | 94.9K | Sep 21, 2026 | Reasoning |
| 21 | K2 Horizon 375B A23B Institute of Foundation Models | 32.1% | — | 2 min 14 s | 62.6K | Sep 3, 2026 | ReasoningOpen weights |
| 22 | MiniMax-M3 MiniMax | 23.0% | $1.01 | 7 min 28 s | 43K | Jun 1, 2026 | ReasoningOpen weights |
| 23 | InklingXhigh Thinking Machines | 15.8% | — | 3 min 15 s | 42K | Jul 15, 2026 | ReasoningOpen weights |
| 24 | Muse GlimmerHigh Meta | 13.9% | $0.281 | 3 min 40 s | 27.2K | Aug 10, 2026 | ReasoningOpen weights |
| 25 | Nemotron 3 Ultra 550B A55BReasoning NVIDIA | 10.5% | $5.51 | 4 min 43 s | 45.1K | Jun 4, 2026 | ReasoningOpen weights |
Method and versions
FrontierCode 1.1 is Artificial Analysis' implementation of Cognition's private benchmark. Each task asks an agent to change a real open-source repository, then automated checks decide whether a maintainer would accept the change.
- Artificial Analysis runs the 100-task Main set, drawn from the benchmark's 150 tasks; tasks are weighted equally.
- Each task runs three times; the score is the mean of those attempts.
- Agents run in the Stirrup harness with a 250-turn cap.
- Six check types (classical tests, command checks, reverse-classical tests, adaptive tests, scope checks, LLM prompt review), each blocking or non-blocking.
- The LLM judge is GPT-6 Luna (medium).
- Cost, time and tokens per task use Artificial Analysis' data and rules over the 100 tasks: cache reads at the cache price, other input at the cache-write price, output at the output price; time divides output tokens by output speed, without time to first token. A model without a cache rate or a price has no cost.
Data via Artificial Analysis, FrontierCode 1.1.