New models: first look

New models: first look

Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.

Swipe for more columns →

Smoke tests: the same 50 positions for every setup, with the v3 prompt and shuffled plays
SetupProbabilitiesReasoning settingMedian / P90 request s$ per 1,000 decisionsInvalid shareError rate (PR), 95% intervalPaired difference from the reference setups
Beam 1.0highOpenRouter, pinned to Nebius (fp8)letter logprobsreasoning effort high (reasoning is mandatory)34.5 / 72.98.67API price0%0 / 50; 0 failed, 0 timed out15.05.9 to 26.7; n = 50vs Gemini 3.8 Flashlow · antigravity: +1.4 (−8.9 to +12.2), n = 50vs Gemini 3.8 Flashlow: +6.5 (−3.8 to +18.4), n = 50
Beam 1.0lowOpenRouter, pinned to Nebius (fp8)letter logprobsreasoning effort low (reasoning is mandatory)7.7 / 16.21.99API price0%0 / 50; 0 failed, 0 timed out15.17.3 to 24.2; n = 50vs Gemini 3.8 Flashlow · antigravity: +1.6 (−9.0 to +10.6), n = 50vs Gemini 3.8 Flashlow: +6.6 (+1.0 to +13.3), n = 50
Claude Fable 5.1highClaude subscriptionalso in League 3none (choice only)effort high (adaptive thinking only)6.7 / 30.878.66API-equivalent (subscription)0%0 / 50; 0 failed, 0 timed out9.02.2 to 18.5; n = 50vs Gemini 3.8 Flashlow · antigravity: −4.5 (−9.2 to −1.0), n = 50vs Gemini 3.8 Flashlow: +0.5 (−6.6 to +9.6), n = 50
Claude Fable 5.1lowClaude subscriptionalso in League 3none (choice only)effort low (adaptive thinking only)1.7 / 2.546.83API-equivalent (subscription)0%0 / 50; 0 failed, 0 timed out11.74.8 to 20.5; n = 50vs Gemini 3.8 Flashlow · antigravity: −1.9 (−12.7 to +7.8), n = 50vs Gemini 3.8 Flashlow: +3.2 (−0.7 to +8.6), n = 50
Claude Haiku 5.5highClaude subscriptionnone (choice only)effort high3.2 / 6.20.69API-equivalent (subscription)0%0 / 50; 0 failed, 0 timed out21.110.1 to 33.9; n = 50vs Gemini 3.8 Flashlow · antigravity: +7.5 (−0.6 to +17.1), n = 50vs Gemini 3.8 Flashlow: +12.5 (+3.0 to +24.0), n = 50
Claude Haiku 5.5lowClaude subscriptionnone (choice only)effort low1.4 / 1.60.44API-equivalent (subscription)0%0 / 50; 0 failed, 0 timed out18.09.1 to 28.0; n = 50vs Gemini 3.8 Flashlow · antigravity: +4.4 (−7.1 to +15.2), n = 50vs Gemini 3.8 Flashlow: +9.5 (+2.7 to +17.7), n = 50
d3-turbodecision model0.6 / 0.8Not measured0%0 / 50; 0 failed, 0 timed out34.517.8 to 53.7; n = 50vs Gemini 3.8 Flashlow · antigravity: +20.9 (+6.3 to +39.2), n = 50vs Gemini 3.8 Flashlow: +26.0 (+10.1 to +45.0), n = 50
Gemini 3.8 Flashlow · antigravityGoogle Antigravity subscriptionalso in League 2none (choice only)thinking level low2.9 / 6.03.47API-equivalent (subscription)0%0 / 50; 0 failed, 0 timed out13.65.4 to 24.3; n = 50vs Gemini 3.8 Flashlow: +5.0 (−2.9 to +15.1), n = 50
Grok 4.7highxAI subscription (SuperGrok)also in League 3none (choice only)reasoning effort high, the default69.2 / 264.663.74API-equivalent (subscription)0%0 / 50; 0 failed, 4 timed out6.50.9 to 14.7; n = 46vs Gemini 3.8 Flashlow · antigravity: −6.9 (−17.6 to +2.0), n = 46vs Gemini 3.8 Flashlow: −1.4 (−4.1 to +0.7), n = 46
Grok 4.7lowxAI subscription (SuperGrok)also in League 3none (choice only)reasoning effort low18.7 / 64.220.77API-equivalent (subscription)0%0 / 50; 0 failed, 0 timed out9.52.8 to 18.0; n = 50vs Gemini 3.8 Flashlow · antigravity: −4.1 (−14.9 to +5.0), n = 50vs Gemini 3.8 Flashlow: +0.9 (−2.6 to +5.9), n = 50
Hy4 PreviewoffOpenRouter, pinned to Tencent (fp8)none (choice only)reasoning off2.7 / 4.01.71API price0%0 / 50; 0 failed, 0 timed out20.910.8 to 33.5; n = 50vs Gemini 3.8 Flashlow · antigravity: +7.3 (−5.0 to +20.5), n = 50vs Gemini 3.8 Flashlow: +12.4 (+4.1 to +23.8), n = 50
Kimi K3lowOpenRouter, pinned to Inference.net (fp4)none (choice only)effort low (an effort is required; no off)16.5 / 42.29.35API price0%0 / 50; 0 failed, 0 timed out16.77.2 to 28.2; n = 50vs Gemini 3.8 Flashlow · antigravity: +3.2 (−1.7 to +9.9), n = 50vs Gemini 3.8 Flashlow: +8.2 (+0.6 to +18.4), n = 50
Mercury 3lowOpenRouter, pinned to Inception (its only endpoint; quantization not published)none (choice only)reasoning effort low23.0 / 78.74.85API price10%5 / 50; 0 failed, 0 timed out23.610.5 to 39.0; n = 50vs Gemini 3.8 Flashlow · antigravity: +12.2 (+2.0 to +26.0), n = 45vs Gemini 3.8 Flashlow: +17.8 (+5.5 to +33.7), n = 45
Mercury 3offOpenRouter, pinned to Inception (its only endpoint; quantization not published)none (choice only)reasoning off0.5 / 1.50.29API price2%1 / 50; 0 failed, 0 timed out22.912.8 to 34.3; n = 50vs Gemini 3.8 Flashlow · antigravity: +9.0 (−3.5 to +21.4), n = 49vs Gemini 3.8 Flashlow: +14.2 (+6.5 to +23.9), n = 49
MiMo-V2.6-FlashlowOpenRouter, pinned to Makora (fp8)letter logprobsreasoning effort low39.5 / 332.73.62API price28%14 / 50; 0 failed, 0 timed out7.71.6 to 15.4; n = 50vs Gemini 3.8 Flashlow · antigravity: +4.4 (−0.2 to +11.4), n = 36vs Gemini 3.8 Flashlow: +3.9 (+0.1 to +9.6), n = 36
MiMo-V2.6-FlashoffOpenRouter, pinned to Makora (fp8)letter logprobsreasoning off0.9 / 2.70.14API price10%5 / 50; 0 failed, 0 timed out12.55.7 to 20.6; n = 50vs Gemini 3.8 Flashlow · antigravity: +0.8 (−10.8 to +11.1), n = 45vs Gemini 3.8 Flashlow: +6.4 (+0.6 to +13.8), n = 45
MiMo-V2.6-Flashoff, unconstrainedOpenRouter, pinned to Makora (fp8)letter logprobsreasoning off0.8 / 1.60.02API price4%2 / 50; 0 failed, 0 timed out15.97.9 to 25.7; n = 50vs Gemini 3.8 Flashlow · antigravity: +5.3 (−7.2 to +17.1), n = 48vs Gemini 3.8 Flashlow: +10.1 (+3.0 to +19.3), n = 48
MiMo V2.6 ProhighOpenRouter, pinned to Xiaomi (fp8)none (choice only)reasoning effort high23.6 / 346.14.05API price0%0 / 50; 0 failed, 8 timed out7.22.0 to 14.0; n = 42vs Gemini 3.8 Flashlow · antigravity: −1.7 (−13.3 to +7.5), n = 42vs Gemini 3.8 Flashlow: +2.7 (−1.5 to +7.9), n = 42
MiMo V2.6 ProoffOpenRouter, pinned to Xiaomi (fp8)none (choice only)reasoning off2.2 / 3.40.64API price0%0 / 50; 0 failed, 0 timed out25.911.9 to 42.9; n = 50vs Gemini 3.8 Flashlow · antigravity: +12.3 (−2.6 to +29.1), n = 50vs Gemini 3.8 Flashlow: +17.4 (+5.8 to +32.2), n = 50
MiniMax-M3.1-FlashlowOpenRouter, MiniMax (free listing, its only one)none (choice only)effort low (reasoning is mandatory)4.3 / 10.70.72API price0%0 / 50; 0 failed, 0 timed out22.19.4 to 38.3; n = 50vs Gemini 3.8 Flashlow · antigravity: +8.6 (−6.4 to +25.9), n = 50vs Gemini 3.8 Flashlow: +13.6 (+2.3 to +29.0), n = 50
Nemotron 3.5 Bolt (off)letter logprobs2.7 / 14.0Not measured0%0 / 50; 0 failed, 0 timed out23.212.5 to 35.0; n = 50vs Gemini 3.8 Flashlow · antigravity: +9.6 (−3.4 to +22.7), n = 50vs Gemini 3.8 Flashlow: +14.6 (+5.8 to +25.7), n = 50
Nemotron 3.5 Bolt (on)letter logprobs218.0 / 303.4Not measured0%0 / 50; 0 failed, 0 timed out11.65.1 to 19.8; n = 50vs Gemini 3.8 Flashlow · antigravity: −2.0 (−11.7 to +5.6), n = 50vs Gemini 3.8 Flashlow: +3.1 (+0.6 to +6.4), n = 50

50 positions each, outside the shared sets. Error rates are rough (about ±10 to ±25): enough to spot a clearly weak model, not to rank models. This page feeds no ranking, chart or finding. Sorted by name. Every value is shown with its count and interval: the minimum-sample rule of the other pages does not apply here. Rows marked "also in League N" are models that also play a league; their league results are on the Leagues page and are never compared with these 50 positions.