ratelimit now
Get updates

Code scores

Repository bug-fix and code-editing benchmarks (SWE-bench Verified, Aider polyglot).

125 results on 7 boards · checked

How this category is sourced

Repository bug-fix and code-editing benchmarks (SWE-bench Verified, Aider polyglot).

  • Every value is copied as the board publishes it, with its metric, harness, as-of date and a source link. Boards use different metrics, so numbers are never compared across boards.
  • No Ratelimit Now score: we do not normalise, weight or combine results.
  • Stale means the as-of date is more than 90 days old. Frozen means the board stopped scoring new models. Preliminary is the board's own label.
  • A model's score depends on the harness or agent it ran in. It is not a score for any tool on this site.
  • Open and Chinese models sit in the same tables, with the license the board lists. Hosting region is not confirmed for any model.

Category status in our data: frozen/stale · Vals AI stopped scoring new models on SWE-bench Verified (parent brief cites 2026-09-05; the page itself says 'Updated 9/1/2026'). Scores near 95-97% are saturated and don't separate frontier models. LiveCodeBench data file ends 2025-04-07 and was not recorded.

Latest generation on this page
GPT-6 Astra4 resultsMiniMax M3SWE-rebenchClaude Fable 5.14 results

Not on these boards yet: GPT-6.1 Sol, GPT-6 Sol, GPT-6 Luna (see their model pages for other categories).

Older versions = superseded by a newer model of the same family on the same board (the data's own flag).

Terminal-Bench 4.0

Accuracy %Harness: varies by row (2)Newest as-of 2026-09-038 latest-gen · 8 current · 4 older

tbench.ai
Top 8 on Terminal-Bench 4.0Bar height = Accuracy %. Lab logo on top; small logo = harness tool. As of 2026-09-03.
03060GPT-6 Astra (xhigh): 57.88% (Accuracy %) 57.88GPT-6 AstraGPT-6 Astra (high): 57.88% (Accuracy %) 57.88GPT-6 AstraFable 5.1 (xhigh): 57.88% (Accuracy %) 57.88Fable 5.1Fable 5.1 (high): 54.55% (Accuracy %) 54.55Fable 5.1GPT-6 Astra (medium): 54.24% (Accuracy %) 54.24GPT-6 AstraFable 5.1 (medium): 53.94% (Accuracy %) 53.94Fable 5.1GPT-6 Astra (low): 50.61% (Accuracy %) 50.61GPT-6 AstraFable 5.1 (low): 43.33% (Accuracy %) 43.33Fable 5.1
Chart data as a table
Top rows on Terminal-Bench 4.0
ModelLabAccuracy %
GPT-6 Astra (xhigh)OpenAI57.88
GPT-6 Astra (high)OpenAI57.88
Fable 5.1 (xhigh)Anthropic57.88
Fable 5.1 (high)Anthropic54.55
GPT-6 Astra (medium)OpenAI54.24
Fable 5.1 (medium)Anthropic53.94
GPT-6 Astra (low)OpenAI50.61
Fable 5.1 (low)Anthropic43.33

Source: tbench.ai

Share of top-10 rowsShare of top-10 rows on Terminal-Bench 4.0, as of 2026-09-03. Counts rows, not a score.
OpenAI: 4 of 8Anthropic: 4 of 84/8OpenAI
Share of top-10 rows on Terminal-Bench 4.0, as of 2026-09-03. Counts rows, not a score.
LabRows
OpenAI4
Anthropic4

Source: tbench.ai

Terminal-Bench 4.0, Accuracy %, as of 2026-09-03. Values copied as published.
Model (as listed)Accuracy %HarnessAs ofSource
GPT-6 Astra (xhigh)OpenAI · latest gen 57.88% Codex ↗
GPT-6 Astra (high)OpenAI · latest gen 57.88% Codex ↗
Fable 5.1 (xhigh)Anthropic · latest gen 57.88% Claude Code ↗
Fable 5.1 (high)Anthropic · latest gen 54.55% Claude Code ↗
GPT-6 Astra (medium)OpenAI · latest gen 54.24% Codex ↗
Fable 5.1 (medium)Anthropic · latest gen 53.94% Claude Code ↗
Opus 5 (max)Anthropic · older version 51.82% Claude Code Stale↗
GPT-6 Astra (low)OpenAI · latest gen 50.61% Codex ↗
Opus 5 (high)Anthropic · older version 50.3% Claude Code Stale↗
Opus 5 (medium)Anthropic · older version 44.85% Claude Code Stale↗
Fable 5.1 (low)Anthropic · latest gen 43.33% Claude Code ↗
Opus 5 (low)Anthropic · older version 34.85% Claude Code Stale↗

Copied from tbench.ai/leaderboard · checked 2026-10-16 · Board # is the board's own rank · License: not listed on this board

SWE-rebench

Resolved rate % (fresh GitHub tasks)Harness: SWE-rebench harness (agent rows are full products)Newest as-of 2026-07-011 latest-gen · 9 current · 8 older · 6 open weights

Staleswe-rebench.com
Top 9 on SWE-rebenchBar height = Resolved rate % (fresh GitHub tasks). Lab logo on top; small logo = harness tool. As of 2026-07-01.
03570GLM-5.2 [high]: 62.9% (Resolved rate % (fresh GitHub tasks)) CN 62.9GLM-5.2Junie Agent: 61.8% (Resolved rate % (fresh GitHub tasks)) · 61.8Junie AgentCodex Agent: 58% (Resolved rate % (fresh GitHub tasks)) · 58Codex AgentCursor Agent: 51.7% (Resolved rate % (fresh GitHub tasks)) · 51.7Cursor AgentMiniMax M3: 47.2% (Resolved rate % (fresh GitHub tasks)) CN 47.2MiniMax M3MiMo V2.5 Pro: 46.5% (Resolved rate % (fresh GitHub tasks)) CN 46.5MiMo V2.5 ProDeepSeek-V4 Pro [high]: 40.2% (Resolved rate % (fresh GitHub tasks)) CN 40.2DeepSeek-V4 ProQwen3.6-27B: 31.2% (Resolved rate % (fresh GitHub tasks)) CN 31.2Qwen3.6-27BQwen3.6-35B-A3B: 24.7% (Resolved rate % (fresh GitHub tasks)) CN 24.7Qwen3.6-35B-A3B
Chart data as a table
Top rows on SWE-rebench
ModelLabResolved rate % (fresh GitHub tasks)
GLM-5.2 [high]Chinese labs62.9
Junie AgentOthers61.8
Codex AgentOthers58
Cursor AgentOthers51.7
MiniMax M3Chinese labs47.2
MiMo V2.5 ProChinese labs46.5
DeepSeek-V4 Pro [high]Chinese labs40.2
Qwen3.6-27BChinese labs31.2
Qwen3.6-35B-A3BChinese labs24.7

Source: swe-rebench.com

Share of top-10 rowsShare of top-10 rows on SWE-rebench, as of 2026-07-01. Counts rows, not a score.
Chinese labs: 6 of 9Others: 3 of 96/9Chinese labs
Share of top-10 rows on SWE-rebench, as of 2026-07-01. Counts rows, not a score.
LabRows
Chinese labs6
Others3

Source: swe-rebench.com

  • Changelog last updated 2026-07-01; newer models (Opus 5.5, GPT-6, Kimi K3, GLM 5.3) absent
  • SWE-rebench last changelog 2026-07-01 - stale
SWE-rebench, Resolved rate % (fresh GitHub tasks), as of 2026-07-01. Values copied as published.
Board #Model (as listed)Resolved rate %DetailLicenseAs ofSource
1Fable 5 [high]Anthropic · older version 64.5% CI ±1.41Not listed Stale↗
2Grok 4.5 [high]SpaceXAI · older version 63.8% CI ±0.60Not listed Stale↗
3Opus 5 [high]Anthropic · older version 63.4% CI ±1.35Not listed Stale↗
4GLM-5.2 [high]Z.ai 62.9% CI ±1.19Open weightsmit Stale↗
5GPT-5.6 Sol [medium]OpenAI · older version 62.3% CI ±1.83Not listed Stale↗
6Junie Agent 61.8% CI ±0.54Not listed Stale↗
7Claude Code AgentAnthropic · older version 60.4% CI ±1.03Not listed Stale↗
8Codex Agent 58% CI ±1.29Not listed Stale↗
9Sonnet 5 [high]Anthropic · older version 56.8% CI ±0.94Not listed Stale↗
10Cursor Agent 51.7% CI ±0.84Not listed Stale↗
11MiniMax M3MiniMax · latest gen 47.2% CI ±1.13Open weightsminimax-community Stale↗
12MiMo V2.5 ProXiaomi 46.5% CI ±0.54Open weightsmit Stale↗
13GPT-5.6 Luna [medium]OpenAI · older version 43.6% CI ±1.47Not listed Stale↗
14DeepSeek-V4 Pro [high]DeepSeek 40.2% CI ±1.29Open weightsmit Stale↗
15Qwen3.6-27BAlibaba 31.2% CI ±1.68Open weightsapache-2.0 Stale↗
16Qwen3.6-35B-A3BAlibaba 24.7% CI ±0.79Open weightsapache-2.0 Stale↗
17Qwen3.5-35B-A3BAlibaba · older version 17.1% CI ±1.07Not listed Stale↗

Copied from swe-rebench.com · checked 2026-10-16 · Board # is the board's own rank

SWE-Bench Pro (Scale, public set)

% resolved (731 public tasks)Harness: varies by row (2)Newest as-of 2026-10-16no latest-gen models · 11 current · 5 older · 3 open weights

scale.com
Top 10 on SWE-Bench Pro (Scale, public set)Bar height = % resolved (731 public tasks). Lab logo on top; small logo = harness tool. As of 2026-10-16.
02040qwen3-coder-480b-a35b: 38.7% (% resolved (731 public tasks)) CN 38.7qwen3-coder-480bminimax-2.1: 36.81% (% resolved (731 public tasks)) CN 36.81minimax-2.1kimi-k2-instruct: 27.67% (% resolved (731 public tasks)) CN 27.67kimi-k2-instructqwen3-235b-a22b: 21.41% (% resolved (731 public tasks)) CN 21.41qwen3-235b-a22bgpt-oss-120b: 16.2% (% resolved (731 public tasks)) 16.2gpt-oss-120bdeepseek-v3p2: 15.56% (% resolved (731 public tasks)) CN 15.56deepseek-v3p2gemma-3-27b-it: 11.38% (% resolved (731 public tasks)) 11.38gemma-3-27b-itllama3-1-405b-instruct: 11.18% (% resolved (731 public tasks)) 11.18llama3-1-405b-inglm-4.6: 9.67% (% resolved (731 public tasks)) CN 9.67glm-4.6llama4-maverick-17b-instruct: 5.24% (% resolved (731 public tasks)) 5.24llama4-maverick-
Chart data as a table
Top rows on SWE-Bench Pro (Scale, public set)
ModelLab% resolved (731 public tasks)
qwen3-coder-480b-a35bChinese labs38.7
minimax-2.1Chinese labs36.81
kimi-k2-instructChinese labs27.67
qwen3-235b-a22bChinese labs21.41
gpt-oss-120bOpenAI16.2
deepseek-v3p2Chinese labs15.56
gemma-3-27b-itGoogle11.38
llama3-1-405b-instructMeta11.18
glm-4.6Chinese labs9.67
llama4-maverick-17b-instructMeta5.24

Source: scale.com

Share of top-10 rowsShare of top-10 rows on SWE-Bench Pro (Scale, public set), as of 2026-10-16. Counts rows, not a score.
OpenAI: 1 of 10Google: 1 of 10Meta: 2 of 10Chinese labs: 6 of 106/10Chinese labs
Share of top-10 rows on SWE-Bench Pro (Scale, public set), as of 2026-10-16. Counts rows, not a score.
LabRows
OpenAI1
Google1
Meta2
Chinese labs6

Source: scale.com

  • Board lists older models alongside Muse Spark 1.1 (marked NEW); no 2026 Anthropic/OpenAI frontier models listed
  • SWE-Bench Pro (Scale) public board is active (Muse Spark 1.1 marked NEW) but lists no 2026 Anthropic/OpenAI frontier models; no as-of date on page
SWE-Bench Pro (Scale, public set), % resolved (731 public tasks), as of 2026-10-16. Values copied as published.
Board #Model (as listed)% resolvedDetailHarnessLicenseAs ofSource
1Muse Spark 1.1Meta · older version 61.5% CI ±3.10mini-swe-agentNot listed ↗
2gpt-5.4 (xHigh)OpenAI · older version 59.1% CI ±3.56mini-swe-agentNot listed ↗
3Muse SparkMeta · older version 55% CI ±3.60mini-swe-agentNot listed ↗
4claude-opus-4-6 (thinking)Anthropic · older version 51.9% CI ±3.61mini-swe-agentNot listed ↗
5gemini-3.1-pro (thinking)Google · older version 46.1% CI ±3.60mini-swe-agentNot listed ↗
13qwen3-coder-480b-a35bAlibaba 38.7% CI ±3.55SWE-AgentNot listed ↗
14minimax-2.1MiniMax 36.81% CI ±3.55SWE-AgentNot listed ↗
17kimi-k2-instructMoonshot 27.67% CI ±3.25SWE-AgentNot listed ↗
18qwen3-235b-a22bAlibaba 21.41% CI ±2.25SWE-AgentNot listed ↗
19gpt-oss-120bOpenAI 16.2% CI ±2.67SWE-AgentOpen weightsapache-2.0 ↗
20deepseek-v3p2DeepSeek 15.56% CI ±2.63SWE-AgentNot listed ↗
21gemma-3-27b-itGoogle 11.38% CI ±2.15SWE-AgentNot listed ↗
22llama3-1-405b-instruct 11.18% CI ±2.15SWE-AgentOpen weightsllama3.1 (custom) ↗
23glm-4.6Z.ai 9.67% CI ±2.15SWE-AgentNot listed ↗
24llama4-maverick-17b-instruct 5.24% CI ±1.24SWE-AgentOpen weightsllama4 (custom) ↗
25codestral-2405 1.51% CI ±1.51SWE-AgentNot listed ↗

Copied from scale.com/leaderboard/swe_bench_pro_public · checked 2026-10-16 · Board # is the board's own rank

Frozen boards

These boards stopped scoring new releases, so the newest models are missing. Kept for reference.

Vals AI — LiveCodeBenchFrozenas of 2026-09-01

Vals AI — LiveCodeBench

Accuracy % (code generation)Harness: Vals AI implementation (no agent)Newest as-of 2026-09-018 latest-gen · 18 current · 7 older · 14 open weights

FrozenStalevals.ai
Top 10 on Vals AI — LiveCodeBenchBar height = Accuracy % (code generation). Lab logo on top; small logo = harness tool. As of 2026-09-01.
050100Claude Fable 5.1: 90.52% (Accuracy % (code generation)) 90.52Claude Fable 5.1Gemini 3.8 Flash: 89.48% (Accuracy % (code generation)) 89.48Gemini 3.8 FlashQwen 3.8 Max: 87.85% (Accuracy % (code generation)) CN 87.85Qwen 3.8 MaxDeepSeek V4 Pro 0813: 87.53% (Accuracy % (code generation)) CN 87.53DeepSeek V4 Pro DeepSeek V4 Flash 0731: 87.26% (Accuracy % (code generation)) CN 87.26DeepSeek V4 FlasKimi K3: 87.19% (Accuracy % (code generation)) CN 87.19Kimi K3Qwen 3.8 27B: 84% (Accuracy % (code generation)) CN 84Qwen 3.8 27BGPT OSS 120B: 83.23% (Accuracy % (code generation)) 83.23GPT OSS 120BMiniMax-M3: 82.15% (Accuracy % (code generation)) CN 82.15MiniMax-M3GLM 5.3: 80.53% (Accuracy % (code generation)) CN 80.53GLM 5.3
Chart data as a table
Top rows on Vals AI — LiveCodeBench
ModelLabAccuracy % (code generation)
Claude Fable 5.1Anthropic90.52
Gemini 3.8 FlashGoogle89.48
Qwen 3.8 MaxChinese labs87.85
DeepSeek V4 Pro 0813Chinese labs87.53
DeepSeek V4 Flash 0731Chinese labs87.26
Kimi K3Chinese labs87.19
Qwen 3.8 27BChinese labs84
GPT OSS 120BOpenAI83.23
MiniMax-M3Chinese labs82.15
GLM 5.3Chinese labs80.53

Source: vals.ai

Share of top-10 rowsShare of top-10 rows on Vals AI — LiveCodeBench, as of 2026-09-01. Counts rows, not a score.
OpenAI: 1 of 10Anthropic: 1 of 10Google: 1 of 10Chinese labs: 7 of 107/10Chinese labs
Share of top-10 rows on Vals AI — LiveCodeBench, as of 2026-09-01. Counts rows, not a score.
LabRows
OpenAI1
Anthropic1
Google1
Chinese labs7

Source: vals.ai

  • SWE-bench Verified: Vals AI no longer scores new releases (page 'Updated 9/1/2026'; parent cites 2026-09-05). Treat as final standings.
Vals AI — LiveCodeBench, Accuracy % (code generation), as of 2026-09-01. Values copied as published.
Board #Model (as listed)Accuracy %LicenseAs ofSource
1Claude Fable 5.1Anthropic · latest gen 90.52% Not listed Stale↗
2Claude Fable 5Anthropic · older version 89.78% Not listed Stale↗
3Gemini 3.8 FlashGoogle · latest gen 89.48% Not listed Stale↗
4Claude Opus 5Anthropic · older version 89.03% Not listed Stale↗
5Gemini 3.7 FlashGoogle · older version 88.65% Not listed Stale↗
7Grok 4.6SpaceXAI · older version 88.22% Not listed Stale↗
10Qwen 3.8 MaxAlibaba · latest gen 87.85% Not listed Stale↗
13DeepSeek V4 Pro 0813DeepSeek 87.53% Open weightsmit Stale↗
15Grok 4.5SpaceXAI · older version 87.35% Not listed Stale↗
17DeepSeek V4 Flash 0731DeepSeek 87.26% Open weightsmit Stale↗
18Kimi K3Moonshot · latest gen 87.19% Open weightskimi-k3 (custom) Stale↗
29Muse Spark 1.1Meta · older version 85.91% Not listed Stale↗
39Grok 4.3SpaceXAI · older version 84.49% Not listed Stale↗
43Qwen 3.8 27BAlibaba · latest gen 84% Open weightsapache-2.0 Stale↗
51GPT OSS 120BOpenAI 83.23% Open weightsapache-2.0 Stale↗
56MiniMax-M3MiniMax · latest gen 82.15% Open weightsminimax-community Stale↗
68GLM 5.3Z.ai · latest gen 80.53% Open weightsglm-5.3 (custom; LMArena l… Stale↗
69GLM 5.3 FlashZ.ai · latest gen 80.51% Open weightsmit Stale↗
70GPT OSS 20BOpenAI 80.39% Open weightsapache-2.0 Stale↗
113Mistral Large 3Mistral 55.34% Open weightsapache-2.0 Stale↗
116Devstral 2Mistral 51.84% Open weightsother (no license name on … Stale↗
119Llama 4 Maverick 47.25% Open weightsllama4 (custom) Stale↗
125Devstral Small 2Mistral 43.18% Open weightsapache-2.0 Stale↗
132Llama 4 Scout 38.54% Open weightsllama4 (custom) Stale↗
135Llama 3.3 Instruct Turbo (70B) 36.34% Not listed Stale↗

Copied from vals.ai/benchmarks/lcb · checked 2026-10-16 · Board # is the board's own rank

Vals AI — SWE-bench VerifiedFrozenas of 2026-09-01

Vals AI — SWE-bench Verified

% resolved (500 tasks)Harness: Mini-SWE-agentNewest as-of 2026-09-017 latest-gen · 14 current · 16 older · 12 open weights

FrozenStalevals.ai
Top 10 on Vals AI — SWE-bench VerifiedBar height = % resolved (500 tasks). Lab logo on top; small logo = harness tool. As of 2026-09-01.
050100DeepSeek V4 Pro 0813: 96.4% (% resolved (500 tasks)) CN 96.4DeepSeek V4 Pro GLM 5.3: 95.4% (% resolved (500 tasks)) CN 95.4GLM 5.3Kimi K3: 93.4% (% resolved (500 tasks)) CN 93.4Kimi K3GLM 5.3 Flash: 92% (% resolved (500 tasks)) CN 92GLM 5.3 FlashDeepSeek V4 Flash 0731: 88.8% (% resolved (500 tasks)) CN 88.8DeepSeek V4 FlasQwen 3.8 27B: 86% (% resolved (500 tasks)) CN 86Qwen 3.8 27BQwen 3.8 Max: 85.6% (% resolved (500 tasks)) CN 85.6Qwen 3.8 MaxGemini 3.8 Flash: 80% (% resolved (500 tasks)) 80Gemini 3.8 FlashMiniMax-M3: 75% (% resolved (500 tasks)) CN 75MiniMax-M3MiMo V2.5 Pro: 74% (% resolved (500 tasks)) CN 74MiMo V2.5 Pro
Chart data as a table
Top rows on Vals AI — SWE-bench Verified
ModelLab% resolved (500 tasks)
DeepSeek V4 Pro 0813Chinese labs96.4
GLM 5.3Chinese labs95.4
Kimi K3Chinese labs93.4
GLM 5.3 FlashChinese labs92
DeepSeek V4 Flash 0731Chinese labs88.8
Qwen 3.8 27BChinese labs86
Qwen 3.8 MaxChinese labs85.6
Gemini 3.8 FlashGoogle80
MiniMax-M3Chinese labs75
MiMo V2.5 ProChinese labs74

Source: vals.ai

Share of top-10 rowsShare of top-10 rows on Vals AI — SWE-bench Verified, as of 2026-09-01. Counts rows, not a score.
Google: 1 of 10Chinese labs: 9 of 109/10Chinese labs
Share of top-10 rows on Vals AI — SWE-bench Verified, as of 2026-09-01. Counts rows, not a score.
LabRows
Google1
Chinese labs9

Source: vals.ai

  • SWE-bench Verified: Vals AI no longer scores new releases (page 'Updated 9/1/2026'; parent cites 2026-09-05). Treat as final standings.
Vals AI — SWE-bench Verified, % resolved (500 tasks), as of 2026-09-01. Values copied as published.
Board #Model (as listed)% resolvedLicenseAs ofSource
1Claude Opus 5Anthropic · older version 97% Not listed Stale↗
2DeepSeek V4 Pro 0813DeepSeek 96.4% Open weightsmit Stale↗
3GPT-5.6 SolOpenAI · older version 96.2% Not listed Stale↗
4Grok 4.6SpaceXAI · older version 95.6% Not listed Stale↗
5GPT-5.6 TerraOpenAI · older version 95.4% Not listed Stale↗
6GLM 5.3Z.ai · latest gen 95.4% Open weightsglm-5.3 (custom; LMArena l… Stale↗
7Claude Fable 5Anthropic · older version 95% Not listed Stale↗
8Kimi K3Moonshot · latest gen 93.4% Open weightskimi-k3 (custom) Stale↗
9GPT-5.6 LunaOpenAI · older version 93% Not listed Stale↗
10GLM 5.3 FlashZ.ai · latest gen 92% Open weightsmit Stale↗
11DeepSeek V4 Flash 0731DeepSeek 88.8% Open weightsmit Stale↗
12Claude Opus 4.8Anthropic · older version 88.6% Not listed Stale↗
13Grok 4.5SpaceXAI · older version 86.6% Not listed Stale↗
14Muse Spark 1.2Meta · older version 86.6% Not listed Stale↗
15Qwen 3.8 27BAlibaba · latest gen 86% Open weightsapache-2.0 Stale↗
17Qwen 3.8 MaxAlibaba · latest gen 85.6% Not listed Stale↗
21Muse Spark 1.1Meta · older version 82% Not listed Stale↗
23Gemini 3.7 FlashGoogle · older version 80.8% Not listed Stale↗
24Gemini 3.8 FlashGoogle · latest gen 80% Not listed Stale↗
27Claude Sonnet 5Anthropic · older version 79.6% Not listed Stale↗
46MiniMax-M3MiniMax · latest gen 75% Open weightsminimax-community Stale↗
48Muse SparkMeta · older version 74.4% Not listed Stale↗
50MiMo V2.5 ProXiaomi 74% Open weightsmit Stale↗
56Grok 4.20 (Reasoning)SpaceXAI · older version 72.2% Not listed Stale↗
57Grok 4.3SpaceXAI · older version 71.4% Not listed Stale↗
63Qwen 3.6 27BAlibaba · older version 70% Open weightsapache-2.0 Stale↗
72Mistral Medium 3.5Mistral 66.4% Not listed Stale↗
76Devstral 2Mistral 62.8% Open weightsother (no license name on … Stale↗
86Mistral Large 3Mistral 41.4% Open weightsapache-2.0 Stale↗
87GPT OSS 120BOpenAI 33.6% Open weightsapache-2.0 Stale↗

Copied from vals.ai/benchmarks/swebench · checked 2026-10-16 · Board # is the board's own rank

SWE-bench Verified (Bash Only)Frozenas of 2026-02-26

SWE-bench Verified (Bash Only)

% Resolved (500 tasks)Harness: mini-SWE-agent 2.0.0 (same environment for all)Newest as-of 2026-02-26no latest-gen models · 4 current · 9 older · 4 open weights

FrozenStaleswebench.com
Top 4 on SWE-bench Verified (Bash Only)Bar height = % Resolved (500 tasks). Lab logo on top; small logo = harness tool. As of 2026-02-26.
04080MiniMax M2.5 (high): 75.8% (% Resolved (500 tasks)) CN 75.8MiniMax M2.5GLM 5 (high): 72.8% (% Resolved (500 tasks)) CN 72.8GLM 5Kimi K2.5 (high): 70.8% (% Resolved (500 tasks)) CN 70.8Kimi K2.5DeepSeek V3.2 (high): 70% (% Resolved (500 tasks)) CN 70DeepSeek V3.2
Chart data as a table
Top rows on SWE-bench Verified (Bash Only)
ModelLab% Resolved (500 tasks)
MiniMax M2.5 (high)Chinese labs75.8
GLM 5 (high)Chinese labs72.8
Kimi K2.5 (high)Chinese labs70.8
DeepSeek V3.2 (high)Chinese labs70

Source: swebench.com

Share of top-10 rowsShare of top-10 rows on SWE-bench Verified (Bash Only), as of 2026-02-26. Counts rows, not a score.
Chinese labs: 4 of 44/4Chinese labs
Share of top-10 rows on SWE-bench Verified (Bash Only), as of 2026-02-26. Counts rows, not a score.
LabRows
Chinese labs4

Source: swebench.com

  • Latest entry in public leaderboard data is 2026-02-26; no 2026 H2 frontier models listed.
  • SWE-bench Verified: Vals AI no longer scores new releases (page 'Updated 9/1/2026'; parent cites 2026-09-05). Treat as final standings.
SWE-bench Verified (Bash Only), % Resolved (500 tasks), as of 2026-02-26. Values copied as published.
Model (as listed)% ResolvedLicenseAs ofSource
Claude 4.5 Opus (high)Anthropic · older version 76.8% Closed weights Stale↗
Gemini 3 Flash (high)Google · older version 75.8% Closed weights Stale↗
MiniMax M2.5 (high)MiniMax 75.8% Open weights Stale↗
Claude 4.6 OpusAnthropic · older version 75.6% Closed weights Stale↗
GLM 5 (high)Z.ai 72.8% Open weights Stale↗
GPT 5.2 (high)OpenAI · older version 72.8% Closed weights Stale↗
GPT 5.2 CodexOpenAI · older version 72.8% Closed weights Stale↗
Claude 4.5 Sonnet (high)Anthropic · older version 71.4% Closed weights Stale↗
Kimi K2.5 (high)Moonshot 70.8% Open weights Stale↗
DeepSeek V3.2 (high)DeepSeek 70% Open weights Stale↗
Gemini 3 ProGoogle · older version 69.6% Closed weights Stale↗
Claude 4.5 Haiku (high)Anthropic · older version 66.6% Closed weights Stale↗
GPT 5 miniOpenAI · older version 56.2% Closed weights Stale↗

Copied from swebench.com · checked 2026-10-16 · Board # is the board's own rank

Aider polyglotFrozenas of 2025-10-03

Aider polyglot

Percent correct (225 exercises)Harness: aiderNewest as-of 2025-10-03no latest-gen models · 1 current · 11 older

FrozenStaleaider.chat
  • Leaderboard last updated 2025-11-20; no 2026 models.
  • Aider polyglot: not updated since 2025-10-03 (last commit to leaderboard YAML 2025-10-04) - frozen; prefer Vals Vibe Code Bench / Terminal-Bench 4.0
Aider polyglot, Percent correct (225 exercises), as of 2025-10-03. Values copied as published.
Model (as listed)Percent correctDetailAs ofSource
gpt-5 (high)OpenAI · older version 88% $29.08 run cost Stale↗
gpt-5 (medium)OpenAI · older version 86.7% $17.69 run cost Stale↗
o3-pro (high) · older version 84.9% $146.32 run cost Stale↗
gemini-2.5-pro-preview-06-05 (32k think)Google · older version 83.1% $49.88 run cost Stale↗
gpt-5 (low)OpenAI · older version 81.3% $10.37 run cost Stale↗
o3 (high) · older version 81.3% $21.23 run cost Stale↗
grok-4 (high)SpaceXAI · older version 79.6% $59.62 run cost Stale↗
gemini-2.5-pro-preview-06-05 (default think)Google · older version 79.1% $45.6 run cost Stale↗
o3 (high) + gpt-4.1 · older version 78.2% $17.55 run cost Stale↗
o3 · older version 76.9% $13.75 run cost Stale↗
Gemini 2.5 Pro Preview 05-06Google · older version 76.9% $37.41 run cost Stale↗
DeepSeek-V3.2-Exp (Reasoner)DeepSeek 74.2% $1.3 run cost Stale↗

Copied from aider.chat/docs/leaderboards · checked 2026-10-16 · Board # is the board's own rank · License: not listed on this board

Browse models → · How scores are handled

The weekly diff

What changed in AI coding plans: prices, limits, resets. Email signups open soon. Until then, updates go out on X.

Follow @QuietlyByG on X RSS feed

No email form yet, so we don't collect any email addresses.