New models: first look
New models: first look
Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.
Swipe for more columns →
| Setup | Probabilities | Reasoning setting | Median / P90 request s | $ per 1,000 decisions | Invalid share | Error rate (PR), 95% interval | Paired difference from the reference setups |
|---|---|---|---|---|---|---|---|
| Beam 1.0highOpenRouter, pinned to Nebius (fp8) | letter logprobs | reasoning effort high (reasoning is mandatory) | 34.5 / 72.9 | 8.67API price | 0%0 / 50; 0 failed, 0 timed out | 15.05.9 to 26.7; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +1.4 (−8.9 to +12.2), n = 50vs Gemini 3.8 Flashlow: +6.5 (−3.8 to +18.4), n = 50 |
| Beam 1.0lowOpenRouter, pinned to Nebius (fp8) | letter logprobs | reasoning effort low (reasoning is mandatory) | 7.7 / 16.2 | 1.99API price | 0%0 / 50; 0 failed, 0 timed out | 15.17.3 to 24.2; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +1.6 (−9.0 to +10.6), n = 50vs Gemini 3.8 Flashlow: +6.6 (+1.0 to +13.3), n = 50 |
| Claude Fable 5.1highClaude subscriptionalso in League 3 | none (choice only) | effort high (adaptive thinking only) | 6.7 / 30.8 | 78.66API-equivalent (subscription) | 0%0 / 50; 0 failed, 0 timed out | 9.02.2 to 18.5; n = 50 | vs Gemini 3.8 Flashlow · antigravity: −4.5 (−9.2 to −1.0), n = 50vs Gemini 3.8 Flashlow: +0.5 (−6.6 to +9.6), n = 50 |
| Claude Fable 5.1lowClaude subscriptionalso in League 3 | none (choice only) | effort low (adaptive thinking only) | 1.7 / 2.5 | 46.83API-equivalent (subscription) | 0%0 / 50; 0 failed, 0 timed out | 11.74.8 to 20.5; n = 50 | vs Gemini 3.8 Flashlow · antigravity: −1.9 (−12.7 to +7.8), n = 50vs Gemini 3.8 Flashlow: +3.2 (−0.7 to +8.6), n = 50 |
| Claude Haiku 5.5highClaude subscription | none (choice only) | effort high | 3.2 / 6.2 | 0.69API-equivalent (subscription) | 0%0 / 50; 0 failed, 0 timed out | 21.110.1 to 33.9; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +7.5 (−0.6 to +17.1), n = 50vs Gemini 3.8 Flashlow: +12.5 (+3.0 to +24.0), n = 50 |
| Claude Haiku 5.5lowClaude subscription | none (choice only) | effort low | 1.4 / 1.6 | 0.44API-equivalent (subscription) | 0%0 / 50; 0 failed, 0 timed out | 18.09.1 to 28.0; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +4.4 (−7.1 to +15.2), n = 50vs Gemini 3.8 Flashlow: +9.5 (+2.7 to +17.7), n = 50 |
| d3-turbo | decision model | 0.6 / 0.8 | Not measured | 0%0 / 50; 0 failed, 0 timed out | 34.517.8 to 53.7; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +20.9 (+6.3 to +39.2), n = 50vs Gemini 3.8 Flashlow: +26.0 (+10.1 to +45.0), n = 50 | |
| Gemini 3.8 Flashlow · antigravityGoogle Antigravity subscriptionalso in League 2 | none (choice only) | thinking level low | 2.9 / 6.0 | 3.47API-equivalent (subscription) | 0%0 / 50; 0 failed, 0 timed out | 13.65.4 to 24.3; n = 50 | vs Gemini 3.8 Flashlow: +5.0 (−2.9 to +15.1), n = 50 |
| Grok 4.7highxAI subscription (SuperGrok)also in League 3 | none (choice only) | reasoning effort high, the default | 69.2 / 264.6 | 63.74API-equivalent (subscription) | 0%0 / 50; 0 failed, 4 timed out | 6.50.9 to 14.7; n = 46 | vs Gemini 3.8 Flashlow · antigravity: −6.9 (−17.6 to +2.0), n = 46vs Gemini 3.8 Flashlow: −1.4 (−4.1 to +0.7), n = 46 |
| Grok 4.7lowxAI subscription (SuperGrok)also in League 3 | none (choice only) | reasoning effort low | 18.7 / 64.2 | 20.77API-equivalent (subscription) | 0%0 / 50; 0 failed, 0 timed out | 9.52.8 to 18.0; n = 50 | vs Gemini 3.8 Flashlow · antigravity: −4.1 (−14.9 to +5.0), n = 50vs Gemini 3.8 Flashlow: +0.9 (−2.6 to +5.9), n = 50 |
| Hy4 PreviewoffOpenRouter, pinned to Tencent (fp8) | none (choice only) | reasoning off | 2.7 / 4.0 | 1.71API price | 0%0 / 50; 0 failed, 0 timed out | 20.910.8 to 33.5; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +7.3 (−5.0 to +20.5), n = 50vs Gemini 3.8 Flashlow: +12.4 (+4.1 to +23.8), n = 50 |
| Kimi K3lowOpenRouter, pinned to Inference.net (fp4) | none (choice only) | effort low (an effort is required; no off) | 16.5 / 42.2 | 9.35API price | 0%0 / 50; 0 failed, 0 timed out | 16.77.2 to 28.2; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +3.2 (−1.7 to +9.9), n = 50vs Gemini 3.8 Flashlow: +8.2 (+0.6 to +18.4), n = 50 |
| Mercury 3lowOpenRouter, pinned to Inception (its only endpoint; quantization not published) | none (choice only) | reasoning effort low | 23.0 / 78.7 | 4.85API price | 10%5 / 50; 0 failed, 0 timed out | 23.610.5 to 39.0; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +12.2 (+2.0 to +26.0), n = 45vs Gemini 3.8 Flashlow: +17.8 (+5.5 to +33.7), n = 45 |
| Mercury 3offOpenRouter, pinned to Inception (its only endpoint; quantization not published) | none (choice only) | reasoning off | 0.5 / 1.5 | 0.29API price | 2%1 / 50; 0 failed, 0 timed out | 22.912.8 to 34.3; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +9.0 (−3.5 to +21.4), n = 49vs Gemini 3.8 Flashlow: +14.2 (+6.5 to +23.9), n = 49 |
| MiMo-V2.6-FlashlowOpenRouter, pinned to Makora (fp8) | letter logprobs | reasoning effort low | 39.5 / 332.7 | 3.62API price | 28%14 / 50; 0 failed, 0 timed out | 7.71.6 to 15.4; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +4.4 (−0.2 to +11.4), n = 36vs Gemini 3.8 Flashlow: +3.9 (+0.1 to +9.6), n = 36 |
| MiMo-V2.6-FlashoffOpenRouter, pinned to Makora (fp8) | letter logprobs | reasoning off | 0.9 / 2.7 | 0.14API price | 10%5 / 50; 0 failed, 0 timed out | 12.55.7 to 20.6; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +0.8 (−10.8 to +11.1), n = 45vs Gemini 3.8 Flashlow: +6.4 (+0.6 to +13.8), n = 45 |
| MiMo-V2.6-Flashoff, unconstrainedOpenRouter, pinned to Makora (fp8) | letter logprobs | reasoning off | 0.8 / 1.6 | 0.02API price | 4%2 / 50; 0 failed, 0 timed out | 15.97.9 to 25.7; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +5.3 (−7.2 to +17.1), n = 48vs Gemini 3.8 Flashlow: +10.1 (+3.0 to +19.3), n = 48 |
| MiMo V2.6 ProhighOpenRouter, pinned to Xiaomi (fp8) | none (choice only) | reasoning effort high | 23.6 / 346.1 | 4.05API price | 0%0 / 50; 0 failed, 8 timed out | 7.22.0 to 14.0; n = 42 | vs Gemini 3.8 Flashlow · antigravity: −1.7 (−13.3 to +7.5), n = 42vs Gemini 3.8 Flashlow: +2.7 (−1.5 to +7.9), n = 42 |
| MiMo V2.6 ProoffOpenRouter, pinned to Xiaomi (fp8) | none (choice only) | reasoning off | 2.2 / 3.4 | 0.64API price | 0%0 / 50; 0 failed, 0 timed out | 25.911.9 to 42.9; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +12.3 (−2.6 to +29.1), n = 50vs Gemini 3.8 Flashlow: +17.4 (+5.8 to +32.2), n = 50 |
| MiniMax-M3.1-FlashlowOpenRouter, MiniMax (free listing, its only one) | none (choice only) | effort low (reasoning is mandatory) | 4.3 / 10.7 | 0.72API price | 0%0 / 50; 0 failed, 0 timed out | 22.19.4 to 38.3; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +8.6 (−6.4 to +25.9), n = 50vs Gemini 3.8 Flashlow: +13.6 (+2.3 to +29.0), n = 50 |
| Nemotron 3.5 Bolt (off) | letter logprobs | 2.7 / 14.0 | Not measured | 0%0 / 50; 0 failed, 0 timed out | 23.212.5 to 35.0; n = 50 | vs Gemini 3.8 Flashlow · antigravity: +9.6 (−3.4 to +22.7), n = 50vs Gemini 3.8 Flashlow: +14.6 (+5.8 to +25.7), n = 50 | |
| Nemotron 3.5 Bolt (on) | letter logprobs | 218.0 / 303.4 | Not measured | 0%0 / 50; 0 failed, 0 timed out | 11.65.1 to 19.8; n = 50 | vs Gemini 3.8 Flashlow · antigravity: −2.0 (−11.7 to +5.6), n = 50vs Gemini 3.8 Flashlow: +3.1 (+0.6 to +6.4), n = 50 |
50 positions each, outside the shared sets. Error rates are rough (about ±10 to ±25): enough to spot a clearly weak model, not to rank models. This page feeds no ranking, chart or finding. Sorted by name. Every value is shown with its count and interval: the minimum-sample rule of the other pages does not apply here. Rows marked "also in League N" are models that also play a league; their league results are on the Leagues page and are never compared with these 50 positions.