DeepSWE

Datacurve leaderboard for coding agents on original, long-horizon software engineering tasks with program-based verifiers.

v1.1 — node-id test scoring, stable June 2026 release.

View on DeepSWE

Configurations: 70
Tasks: 113
Best pass@1: 74.1%; GPT-6 Astra · xhigh
Cheapest in the top 10: $1.97; Gemini 3.8 Flash · 71.0%

Score and cost

Each line joins the effort levels of one configuration; the large point marks the best one. Top right: the best score for the lowest cost. Hover or move through the points with the keyboard to read their values.

Models shown28 of 28

Select a model to show or hide it; hovering it brings it forward on the chart. The axes fit the models shown.

28 / 70 configurations

#1
GPT-6 Astraxhigh
mini-swe-agent
74.1%
pass@4
80.5%
Cost
$4.43
Time
18 min 52 s
Output tokens
29.6K
#2
Gemini 3.8 Flashhigh
mini-swe-agent
73.8%
pass@4
85.8%
Cost
$2.36
Time
11 min 27 s
Output tokens
143K
#3
Claude Opus 5max
mini-swe-agent
73.6%
pass@4
88.5%
Cost
$11.84
Time
31 min 52 s
Output tokens
118K
#9
GPT-5.6 Solmax
mini-swe-agent
72.7%
pass@4
85.8%
Cost
$8.39
Time
18 min 49 s
Output tokens
60K
#12
Claude Fable 5 (Opus 4.8 Fallback)xhigh
mini-swe-agent
69.9%
pass@4
88.5%
Cost
$13.41
Time
23 min 32 s
Output tokens
80.4K
#14
GPT-5.6 Terramax
mini-swe-agent
69.6%
pass@4
88.5%
Cost
$4.95
Time
16 min 57 s
Output tokens
71.9K
#16
GLM-5.3max
mini-swe-agent
69.0%
pass@4
87.6%
Cost
$3.99
Time
35 min 16 s
Output tokens
80.4K
#19
Kimi K3max
mini-swe-agent
68.5%
pass@4
89.4%
Cost
$4.65
Time
1 h 15
Output tokens
81.5K
#20
Grok 4.6medium
mini-swe-agent
67.5%
pass@4
84.1%
Cost
$3.45
Time
14 min 56 s
Output tokens
49.8K
#21
GPT-5.6 Lunamax
mini-swe-agent
67.2%
pass@4
90.3%
Cost
$3.03
Time
18 min 43 s
Output tokens
73.4K
#22
GPT-5.5xhigh
mini-swe-agent
67.0%
pass@4
88.5%
Cost
$7.23
Time
30 min 06 s
Output tokens
46.3K
#25
Gemini 3.7 Flashmedium
mini-swe-agent
65.5%
pass@4
83.2%
Cost
$2.03
Time
20 min 40 s
Output tokens
94K
#30
GLM-5.3-Flashmax
mini-swe-agent
63.4%
pass@4
85.0%
Cost
$0.482
Time
25 min 38 s
Output tokens
72.8K
#31
DeepSeek V4 Pro 0813max
mini-swe-agent
62.8%
pass@4
88.5%
Cost
$0.241
Time
36 min 53 s
Output tokens
106K
#35
Claude Opus 4.8max
mini-swe-agent
59.0%
pass@4
79.3%
Cost
$13.22
Time
58 min 13 s
Output tokens
135K
#37
Qwen3.8 Max (0902)xhigh
mini-swe-agent
57.5%
pass@4
83.2%
Cost
$3.73
Time
42 min 47 s
Output tokens
95.1K
#39
Muse Spark 1.2xhigh
mini-swe-agent
54.9%
pass@4
81.4%
Cost
$3.70
Time
15 min 28 s
Output tokens
99.2K
#42
Claude Sonnet 5max
mini-swe-agent
53.8%
pass@4
78.8%
Cost
$26.40
Time
1 h 20
Output tokens
214K
#45
Grok 4.5high
mini-swe-agent
53.8%
pass@4
77.9%
Cost
$2.42
Time
8 min 05 s
Output tokens
35.5K
#46
DeepSeek V4 Flash 0731max
mini-swe-agent
53.3%
pass@4
80.5%
Cost
$0.10
Time
23 min 59 s
Output tokens
108K
#47
Muse Spark 1.1xhigh
mini-swe-agent
53.3%
pass@4
79.6%
Cost
$2.36
Time
14 min 58 s
Output tokens
74K
#49
GPT-5.4xhigh
mini-swe-agent
51.8%
pass@4
77.9%
Cost
$5.65
Time
22 min 49 s
Output tokens
71.4K
#53
Gemini 3.6 Flashhigh
mini-swe-agent
46.7%
pass@4
75.2%
Cost
$4.42
Time
24 min 38 s
Output tokens
95.8K
#56
GLM-5.2max
mini-swe-agent
43.8%
pass@4
77.0%
Cost
$3.92
Time
43 min 49 s
Output tokens
78.2K
#61
Gemini 3.5 Flashhigh
mini-swe-agent
36.1%
pass@4
63.7%
Cost
$3.45
Time
18 min 45 s
Output tokens
75.7K
#63
Kimi K2.7 Code
mini-swe-agent
30.5%
pass@4
61.1%
Cost
$2.82
Time
36 min 04 s
Output tokens
59.3K
#65
Claude Sonnet 4.6high
mini-swe-agent
29.9%
pass@4
56.6%
Cost
$5.52
Time
46 min 55 s
Output tokens
76.2K
#68
Gemini 3.1 Pro Previewhigh
mini-swe-agent
11.7%
pass@4
28.3%
Cost
$2.14
Time
14 min 28 s
Output tokens
28.4K

DeepSWE
#	Model
	GPT-6 Astraxhigh OpenAI	74.1%71.2%–77.0%	80.5%	$4.43	18 min 52 s	29.6K
2	Gemini 3.8 Flashhigh Google	73.8%72.4%–75.2%	85.8%	$2.36	11 min 27 s	143K
3	Claude Opus 5max Anthropic	73.6%69.8%–77.5%	88.5%	$11.84	31 min 52 s	118K
9	GPT-5.6 Solmax OpenAI	72.7%69.8%–75.5%	85.8%	$8.39	18 min 49 s	60K
12	Claude Fable 5 (Opus 4.8 Fallback)xhigh Anthropic	69.9%66.7%–73.2%	88.5%	$13.41	23 min 32 s	80.4K
14	GPT-5.6 Terramax OpenAI	69.6%67.1%–72.2%	88.5%	$4.95	16 min 57 s	71.9K
16	GLM-5.3max Z.AI	69.0%65.9%–72.0%	87.6%	$3.99	35 min 16 s	80.4K
19	Kimi K3max Moonshot AI	68.5%64.0%–73.1%	89.4%	$4.65	1 h 15	81.5K
20	Grok 4.6medium SpaceXSI	67.5%65.2%–69.8%	84.1%	$3.45	14 min 56 s	49.8K
21	GPT-5.6 Lunamax OpenAI	67.2%63.2%–71.2%	90.3%	$3.03	18 min 43 s	73.4K
22	GPT-5.5xhigh OpenAI	67.0%60.6%–73.5%	88.5%	$7.23	30 min 06 s	46.3K
25	Gemini 3.7 Flashmedium Google	65.5%62.4%–68.6%	83.2%	$2.03	20 min 40 s	94K
30	GLM-5.3-Flashmax Z.AI	63.4%59.0%–67.8%	85.0%	$0.482	25 min 38 s	72.8K
31	DeepSeek V4 Pro 0813max DeepSeek	62.8%56.5%–69.2%	88.5%	$0.241	36 min 53 s	106K
35	Claude Opus 4.8max Anthropic	59.0%57.2%–60.7%	79.3%	$13.22	58 min 13 s	135K
37	Qwen3.8 Max (0902)xhigh Alibaba	57.5%54.8%–60.1%	83.2%	$3.73	42 min 47 s	95.1K
39	Muse Spark 1.2xhigh Meta	54.9%52.7%–57.0%	81.4%	$3.70	15 min 28 s	99.2K
42	Claude Sonnet 5max Anthropic	53.8%49.6%–58.1%	78.8%	$26.40	1 h 20	214K
45	Grok 4.5high SpaceXSI	53.8%51.5%–56.0%	77.9%	$2.42	8 min 05 s	35.5K
46	DeepSeek V4 Flash 0731max DeepSeek	53.3%49.8%–56.9%	80.5%	$0.10	23 min 59 s	108K
47	Muse Spark 1.1xhigh Meta	53.3%50.3%–56.4%	79.6%	$2.36	14 min 58 s	74K
49	GPT-5.4xhigh OpenAI	51.8%50.3%–53.3%	77.9%	$5.65	22 min 49 s	71.4K
53	Gemini 3.6 Flashhigh Google	46.7%43.0%–50.4%	75.2%	$4.42	24 min 38 s	95.8K
56	GLM-5.2max Z.AI	43.8%42.1%–45.5%	77.0%	$3.92	43 min 49 s	78.2K
61	Gemini 3.5 Flashhigh Google	36.1%32.1%–40.0%	63.7%	$3.45	18 min 45 s	75.7K
63	Kimi K2.7 Code Moonshot AI	30.5%30.0%–31.0%	61.1%	$2.82	36 min 04 s	59.3K
65	Claude Sonnet 4.6high Anthropic	29.9%25.8%–34.0%	56.6%	$5.52	46 min 55 s	76.2K
68	Gemini 3.1 Pro Previewhigh Google	11.7%10.2%–13.2%	28.3%	$2.14	14 min 28 s	28.4K

v1.1 vs v1 differences

v1: 53.5%
v1.1: 53.4%
Delta: −0.1 pts
Shared configs: 9

v1.1 vs v1 on the 9 harness+model+reasoning-effort configs run in both versions. Same rollouts, re-graded: v1 is exit-code scoring, v1.1 is node-id scoring. Pass rate is over scored attempts.

Model	v1	v1.1	Delta
GPT 5.5 medium	48.0%	54.0%	+6.0 pts
GPT 5.4 xhigh	55.5%	51.8%	−3.8 pts
Claude Opus 4.8 xhigh	57.7%	54.4%	−3.4 pts
GPT 5.5 xhigh	70.1%	67.0%	−3.0 pts
GPT 5.5 high	62.0%	64.4%	+2.4 pts
Claude Sonnet 4.6 high	31.8%	29.9%	−1.8 pts
Claude Opus 4.8 medium	47.4%	48.7%	+1.3 pts
Claude Opus 4.8 high	50.7%	51.8%	+1.1 pts
Claude Opus 4.8 max	58.2%	59.0%	+0.8 pts

Method and versions

DeepSWE measures model + harness + reasoning-effort configurations. This page keeps those rows separate instead of folding them into the model indices.

pass@1pass@4mini-swe-agent

Updated : Sep 22, 2026

Cost basis

GPT-6 Astra · xhigh : Current pricing at all context lengths: $10/M uncached input, $12.50/M cache writes, $1/M cache reads, $50/M output; no separate compute-unit fee.

Data via Datacurve DeepSWE.

DeepSWE

Score and cost

Preferences and analytics

Allow navigation analytics?