Android scores
Android app tasks run by Google. Vendor-run, small task sets, wide confidence intervals.
42 results on 2 boards · checked
How this category is sourced
Android app tasks run by Google. Vendor-run, small task sets, wide confidence intervals.
- Every value is copied as the board publishes it, with its metric, harness, as-of date and a source link. Boards use different metrics, so numbers are never compared across boards.
- No Ratelimit Now score: we do not normalise, weight or combine results.
- Stale means the as-of date is more than 90 days old. Frozen means the board stopped scoring new models. Preliminary is the board's own label.
- A model's score depends on the harness or agent it ran in. It is not a score for any tool on this site.
- Open and Chinese models sit in the same tables, with the license the board lists. Hosting region is not confirmed for any model.
Latest generation on this page
GPT-6 LunaAndroid Bench 2.0GPT-6 AstraAndroid Bench 2.0Qwen 3.8 Max2 resultsMiniMax M3Android Bench 1.0 tasksKimi K32 resultsGemma 4 31BAndroid Bench 1.0 tasksGemini 3.8 FlashAndroid Bench 2.0Claude Opus 5.5Android Bench 2.0Claude Fable 5.1Android Bench 2.0
Not on these boards yet: GPT-6.1 Sol, GPT-6 Sol (see their model pages for other categories).
Android Bench 2.0 (long-horizon, 30 tasks)
Pass rate (%)Harness: varies by row (5)Newest as-of 2026-10-067 latest-gen · 7 current · 3 older · 1 open weights
Vendor-rundeveloper.android.com
Chart data as a table
| Model | Lab | Pass rate (%) |
|---|---|---|
| Claude Opus 5.5 | Anthropic | 32.7 |
| GPT 6 Astra | OpenAI | 28 |
| Claude Fable 5.1 | Anthropic | 22.7 |
| Qwen3.8 Max | Chinese labs | 14 |
| Kimi K3 | Chinese labs | 12 |
| GPT 6 Luna | OpenAI | 10.7 |
| Gemini 3.8 Flash | 8 |
Source: developer.android.com
| Lab | Rows |
|---|---|
| OpenAI | 2 |
| Anthropic | 2 |
| 1 | |
| Chinese labs | 2 |
Source: developer.android.com
- Run by Google (vendor); 30 tasks, wide confidence intervals.
| Model (as listed) | Pass rate | Detail | Harness | License | As of | Source |
|---|---|---|---|---|---|---|
| Claude Opus 5.5Anthropic · latest gen | 32.7% | CI 18.7-47.3% · $215.4 avg per run | claude-code | Not listed | ↗ | |
| GPT 6 AstraOpenAI · latest gen | 28% | CI 13.3-42.0% · $375.7 avg per run | codex | Not listed | ↗ | |
| Claude Fable 5.1Anthropic · latest gen | 22.7% | CI 10.7-36.0% · $492.6 avg per run | claude-code | Not listed | ↗ | |
| GPT 6 SolOpenAI · older version | 18% | CI 7.3-29.3% · $93.4 avg per run | codex | Not listed | ↗ | |
| Qwen3.8 MaxAlibaba · latest gen | 14% | CI 4.7-24.0% · $260.2 avg per run | qwen-coder | Not listed | ↗ | |
| Kimi K3Moonshot · latest gen | 12% | CI 3.3-22.0% · $418.3 avg per run | kimi-code | Open weightskimi-k3 (custom) | ↗ | |
| GPT 6 LunaOpenAI · latest gen | 10.7% | CI 4.0-17.3% · $5.3 avg per run | codex | Not listed | ↗ | |
| Gemini 3.8 FlashGoogle · latest gen | 8% | CI 3.3-13.3% · $34.5 avg per run | antigravity-sdk | Not listed | ↗ | |
| Gemini 3.7 FlashGoogle · older version | 7.3% | CI 1.3-14.7% · $26 avg per run | antigravity-sdk | Not listed | ↗ | |
| Claude Sonnet 5Anthropic · older version | 6.7% | CI 1.3-13.3% · $283.7 avg per run | claude-code | Not listed | ↗ |
Android Bench 1.0 tasks (100 tasks)
Score % (avg of 100 tasks)Harness: not_statedNewest as-of 2026-08-114 latest-gen · 12 current · 20 older · 10 open weights
Vendor-rundeveloper.android.com
Chart data as a table
| Model | Lab | Score % (avg of 100 tasks) |
|---|---|---|
| Kimi K3 | Chinese labs | 90.2 |
| Qwen3.8 Max | Chinese labs | 87 |
| GLM 5.2 | Chinese labs | 72.2 |
| Kimi K2.7 Code | Chinese labs | 70.4 |
| Minimax M3 | Chinese labs | 63.6 |
| Mimo V2.5 Pro | Chinese labs | 60.8 |
| Deepseek V4 Pro | Chinese labs | 59.5 |
| Qwen3.7 Plus | Chinese labs | 57.7 |
| Deepseek V4 Flash | Chinese labs | 54.7 |
| Qwen3.6 27b | Chinese labs | 45.1 |
Source: developer.android.com
| Lab | Rows |
|---|---|
| Chinese labs | 10 |
Source: developer.android.com
| Model (as listed) | Score % | License | As of | Source |
|---|---|---|---|---|
| Claude Opus 5Anthropic · older version | 91.8% | Not listed | ↗ | |
| Claude Fable 5Anthropic · older version | 90.9% | Not listed | ↗ | |
| GPT 5.6 SolOpenAI · older version | 90.8% | Not listed | ↗ | |
| Kimi K3Moonshot · latest gen | 90.2% | Open weightskimi-k3 (custom) | ↗ | |
| GPT 5.6 LunaOpenAI · older version | 87.6% | Not listed | ↗ | |
| Qwen3.8 MaxAlibaba · latest gen | 87% | Not listed | ↗ | |
| GPT 5.6 TerraOpenAI · older version | 86.8% | Not listed | ↗ | |
| GPT 5.5OpenAI · older version | 80.2% | Not listed | ↗ | |
| Claude Sonnet 5Anthropic · older version | 76.2% | Not listed | ↗ | |
| Gemini 3.6 FlashGoogle · older version | 75.6% | Not listed | ↗ | |
| Gemini 3.1 Pro PreviewGoogle · older version | 74.3% | Not listed | ↗ | |
| GPT 5.4OpenAI · older version | 74.1% | Not listed | ↗ | |
| Claude Opus 4.8Anthropic · older version | 72.4% | Not listed | ↗ | |
| GLM 5.2Z.ai | 72.2% | Open weightsmit | ↗ | |
| Gemini 3.5 FlashGoogle · older version | 71.1% | Not listed | ↗ | |
| Kimi K2.7 CodeMoonshot | 70.4% | Not listed | ↗ | |
| Claude Opus 4.7Anthropic · older version | 68.7% | Not listed | ↗ | |
| Kimi K2.6Moonshot · older version | 67.6% | Open weightsmodified-mit | ↗ | |
| Claude Sonnet 4.6Anthropic · older version | 67% | Not listed | ↗ | |
| Minimax M3MiniMax · latest gen | 63.6% | Open weightsminimax-community | ↗ | |
| GLM 5.1Z.ai · older version | 63.2% | Not listed | ↗ | |
| Gemini 3 Flash PreviewGoogle · older version | 62.5% | Not listed | ↗ | |
| Mimo V2.5 ProXiaomi | 60.8% | Open weightsmit | ↗ | |
| Deepseek V4 ProDeepSeek | 59.5% | Open weightsmit | ↗ | |
| Qwen3.7 PlusAlibaba | 57.7% | Not listed | ↗ | |
| Deepseek V4 FlashDeepSeek | 54.7% | Open weightsmit | ↗ | |
| Qwen3.7 MaxAlibaba · older version | 54.2% | Not listed | ↗ | |
| Gemini 3.5 Flash-LiteGoogle · older version | 50.6% | Not listed | ↗ | |
| Qwen3.6 27bAlibaba | 45.1% | Open weightsapache-2.0 | ↗ | |
| MiniMax M2.7MiniMax · older version | 41.6% | Not listed | ↗ | |
| Gemma 4 31b ITGoogle · latest gen | 37.1% | Open weightsapache-2.0 | ↗ | |
| Qwen3.6 35b A3bAlibaba | 37% | Open weightsapache-2.0 | ↗ |