All benches

Ranking

Type
Class
Score

Combined 3 benchmark groups Score 0–100, chance = 0 · higher is better ↑

  1. GPT-6 Astra high · chat96.1
  2. GPT-6.1 Sol high · chat95.7
  3. Claude Fable 5.1 high · chat93.7
  4. Claude Opus 5.5 high · chat93.2
  5. Claude Sonnet 5.5 high · chat88.3
  6. Muse Ember max · chat87.7
  7. Gemma 4 31B reasoning off · chat82.1
  8. Qwen3.8-27B low · chat80.2
  9. Qwen3.8 Flash no reasoning · chat74.1
  10. KnowLine-31B-Gen1.5 DGX Spark LoRA, merged73.2
  11. REAX 1.3 blend71.3
  12. Sieve 31B v3 DGX Spark full fine-tune69.4
  13. d1-ultra68.3tied
  14. Turion Jev 31B DGX Spark LoRA, merged68.2tied
  15. Gemini Decision 1 2026111168.0tied
  16. Mammoth-1 post-trained64.8
  17. Drex Max fine-tune63.4
  18. Drex Max v1.163.2tied
  19. GLiDE v1.262.6tied
  20. d1.2-ultra62.4tied
  21. Perplexity Decider 27B v1.2 DGX Spark61.2
  22. SubQ Snap 1.158.5
  23. celeris-1-decision58.0tied
  24. Jev 1.14 (reference)57.1tied
  25. d1.256.3tied
  26. Jev 1.1355.3tied
  27. Drex 2.051.4
  28. Clef post-trained50.0
  29. Elephant-1 post-trained49.7tied
  30. Mercury Decide48.8tied
  31. Cerno 9B v1.2 full fine-tune47.5
  32. StartLux-Decision-v1.1-4B-Instant fine-tune47.4tied
  33. Cygnet unchanged base47.0tied
  34. Winnow-12B LoRA, merged44.6
  35. KnowLine 4B Gen5.5 LoRA, merged43.5tied
  36. Open-Jev-27B-v1.1 LoRA + head42.8tied
  37. Perplexity Decider Flash v1.241.4
  38. StartLux-Decision-v1.1-2B-Instant fine-tune32.1
  39. KnowLine-2B-Gen1 LoRA, merged31.2tied
  40. d1-2B30.5tied
  41. GLiDE 2B27.5
  42. StartLux-Decision-v1.1-0.8B-Instant fine-tune22.4
  43. KnowLine-0.8B-Gen1 LoRA, merged19.7
  44. GLiDE 0.8B19.7tied
  45. OneJev1.6-0.8B full fine-tune17.7
  46. GLiDE 0.4B3.7
  47. GLiClass v4 Edge fine-tune−0.8

OpenAIAnthropicMetaGoogleAlibabaPelaAIREAXSthānika AILiquid AITurion.AIBase LabsNace.AIFastinoPerplexitySubquadraticCelerisTypeSafeCloudflareInceptionApertix AIStartLuxBlockbrain LabsEldanRingZefan CaiOmniJevKnowledgator

No combined score for GLiDE · API: Fastino's API served a changed model between our benches' runs (Reading 6 Oct, Logic & Math 7 Nov, Backgammon 20 Oct to 7 Nov), so its bench scores are not one model's. (the rule)

Every model in view on every bench · Score 0–100, chance = 0 · higher is better ↑
Rank
GPT-6 Astra high · chat96.1TR 97.4 L&M 98.3 BG 92.5Not applicable96.497.995.2Not run95.999.592.5Not run
GPT-6.1 Sol high · chat95.7TR 97.2 L&M 98.2 BG 91.6Not applicable95.697.994.5Not run95.799.591.6Not run
Claude Fable 5.1 high · chat93.7TR 95.2 L&M 97.7 BG 88.1Not applicable94.595.595.0Not run94.099.788.1Not run
Claude Opus 5.5 high · chat93.2TR 93.7 L&M 98.2 BG 87.7Not applicable94.593.493.1Not run95.299.787.7Not run
Claude Sonnet 5.5 high · chat88.3TR 81.3 L&M 97.8 BG 85.8Not applicable81.581.282.6Not run94.399.685.8Not run
Muse Ember max · chat87.7TR 82.0 L&M 97.7 BG 83.4Not applicable87.879.687.082.894.399.583.4Not run
Gemma 4 31B reasoning off · chatOpen · 31B82.1TR 66.4 L&M 96.7 BG 83.4Not applicable88.657.088.1Not run91.799.283.4Not run
Qwen3.8-27B low · chatOpen · 27B80.2TR 68.3 L&M 96.8 BG 75.4Not applicable86.760.586.3Not run92.699.075.4Not run
Qwen3.8 Flash no reasoning · chat74.1TR 53.8 L&M 97.0 BG 71.5Not applicable84.041.083.8Not run92.499.471.5Not run
1KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedOpen · 31B73.2TR 77.3 L&M 63.9 BG 78.375.3EN 92.0 Logic 48.8 Math 71.7 AC 88.891.071.492.037.348.871.778.388.8
2REAX 1.3 blendOpen · 57B71.3TR 72.4 L&M 64.5 BG 77.174.8EN 90.6 Logic 49.1 Math 72.5 AC 87.090.564.790.6Not run49.172.577.187.0
3Sieve 31B v3 DGX Spark full fine-tuneOpen · 31B69.4TR 67.7 L&M 63.2 BG 77.373.8EN 88.3 Logic 47.3 Math 71.4 AC 88.189.758.388.3Not run47.371.477.388.1
4d1-ultra68.3TR 71.3 L&M 59.4 BG 74.371.1EN 88.3 Logic 43.7 Math 67.6 AC 84.790.163.488.337.643.767.674.384.7
5Turion Jev 31B DGX Spark LoRA, mergedOpen · 31B68.2TR 75.1 L&M 54.1 BG 75.569.1EN 88.3 Logic 40.3 Math 61.3 AC 86.490.968.588.3Not run40.361.375.586.4
6Gemini Decision 1 2026111168.0TR 71.9 L&M 55.1 BG 76.966.9EN 88.8 Logic 37.6 Math 64.2 AC 77.090.164.288.858.137.664.276.977.0
7Mammoth-1 post-trainedOpen · 27B64.8TR 58.9 L&M 59.1 BG 76.470.2EN 87.7 Logic 43.1 Math 67.5 AC 82.488.046.587.7Not run43.167.576.482.4
8Drex Max fine-tuneOpen · 27B63.4TR 59.7 L&M 55.5 BG 74.9Not applicable87.048.087.033.541.362.974.9Not run
9Drex Max v1.163.2TR 59.2 L&M 56.1 BG 74.5Not applicable85.748.085.631.443.462.674.5Not run
10GLiDE v1.2Open · 27B62.6TR 60.6 L&M 56.1 BG 71.169.8EN 87.9 Logic 44.0 Math 62.3 AC 84.987.049.387.9Not run44.062.371.184.9
11d1.2-ultrasetups differ62.4TR 60.1 L&M 54.5 BG 72.769.0EN 87.9 Logic 39.4 Math 62.3 AC 86.487.248.687.931.739.462.372.786.4
12Perplexity Decider 27B v1.2 DGX Sparksetups differOpen · 27B61.2TR 58.8 L&M 55.2 BG 69.5Not applicable86.547.185.8Not run41.762.269.5Not run
13SubQ Snap 1.158.5TR 54.8 L&M 53.8 BG 66.867.1EN 82.4 Logic 40.4 Math 60.7 AC 84.983.842.582.432.240.460.766.884.9
14celeris-1-decision58.0TR 50.6 L&M 52.8 BG 70.6Not applicable81.537.480.1Not run34.162.570.6Not run
15Jev 1.1457.1TR 49.0 L&M 49.2 BG 73.164.5EN 82.6 Logic 35.2 Math 56.4 AC 83.681.335.282.629.135.256.473.183.6
16d1.2setups differ56.3TR 52.2 L&M 49.9 BG 66.965.3EN 86.3 Logic 36.4 Math 56.9 AC 81.582.539.386.333.736.456.966.981.5
17Jev 1.1355.3TR 46.9 L&M 47.5 BG 71.663.3EN 82.4 Logic 32.9 Math 55.1 AC 83.079.633.082.4Not run32.955.171.683.0
18Drex 2.0setups differOpen · 35B MoE · 3.0B active51.4TR 44.7 L&M 41.4 BG 68.358.1EN 82.4 Logic 27.3 Math 48.7 AC 73.876.831.082.4Not run27.348.768.373.8
19Clef post-trainedOpen · 27B50.0TR 46.6 L&M 34.5 BG 69.056.2EN 83.3 Logic 26.0 Math 38.9 AC 76.581.331.883.3Not run26.038.969.076.5
20Elephant-1 post-trainedOpen · 9.0B49.7TR 40.5 L&M 45.3 BG 63.260.8EN 81.7 Logic 33.0 Math 51.7 AC 76.975.625.581.7Not run33.051.763.276.9
21Mercury Decide48.8TR 47.9 L&M 38.3 BG 60.4Not applicable80.633.982.2Not run31.341.960.4Not run
22Cerno 9B v1.2 full fine-tuneOpen · 9.0B47.5TR 43.1 L&M 39.6 BG 60.059.6EN 82.4 Logic 29.1 Math 45.0 AC 81.974.729.682.4Not run29.145.060.081.9
23StartLux-Decision-v1.1-4B-Instant fine-tuneOpen · 4.0B47.4TR 34.8 L&M 40.5 BG 66.859.3EN 78.5 Logic 32.3 Math 44.8 AC 81.767.620.978.5Not run32.344.866.881.7
24Cygnet unchanged basesetups differOpen · 12B47.0TR 44.6 L&M 28.3 BG 68.251.8EN 79.9 Logic 24.1 Math 30.4 AC 72.978.530.279.9Not run24.130.468.272.9
25Winnow-12B LoRA, mergedOpen · 12B44.6TR 44.3 L&M 30.5 BG 58.953.7EN 79.9 Logic 24.7 Math 33.6 AC 76.677.530.279.9Not run24.733.658.976.6
26KnowLine 4B Gen5.5 LoRA, mergedOpen · 4.0B43.5TR 35.6 L&M 39.1 BG 55.857.4EN 76.0 Logic 27.9 Math 45.0 AC 80.668.621.676.012.727.945.055.880.6
27Open-Jev-27B-v1.1 LoRA + headOpen · 27B42.8TR 44.6 L&M 38.1 BG 45.9Not applicable78.730.181.7Not run28.243.245.9Not run
28Perplexity Decider Flash v1.2Open · 4.0B41.4TR 34.7 L&M 38.9 BG 50.856.9EN 77.4 Logic 28.9 Math 44.1 AC 77.265.921.477.4Not run28.944.150.877.2
29StartLux-Decision-v1.1-2B-Instant fine-tuneOpen · 2.0B32.1TR 17.8 L&M 19.9 BG 58.741.2EN 62.1 Logic 13.4 Math 23.3 AC 66.041.17.962.14.313.423.358.766.0
30KnowLine-2B-Gen1 LoRA, mergedOpen · 2.0B31.2TR 21.0 L&M 17.6 BG 55.039.7EN 64.1 Logic 14.7 Math 19.1 AC 61.142.811.764.1Not run14.719.155.061.1
31d1-2BOpen · 1.9B30.5TR 11.7 L&M 18.6 BG 61.135.3EN 57.0 Logic 13.3 Math 21.3 AC 49.626.15.657.04.213.321.361.149.6
32GLiDE 2BOpen · 1.9B27.5TR 19.1 L&M 19.0 BG 44.441.1EN 62.5 Logic 15.3 Math 20.9 AC 65.639.010.762.5Not run15.320.944.465.6
33StartLux-Decision-v1.1-0.8B-Instant fine-tuneOpen · 0.8B22.4TR 10.7 L&M 10.0 BG 46.629.1EN 49.5 Logic 5.9 Math 12.1 AC 48.918.87.349.55.35.912.146.648.9
34KnowLine-0.8B-Gen1 LoRA, mergedOpen · 0.8B19.7TR 8.9 L&M 9.6 BG 40.626.4EN 48.6 Logic 8.3 Math 10.3 AC 38.522.83.048.64.88.310.340.638.5
35GLiDE 0.8BOpen · 0.8B19.7TR 11.6 L&M 13.9 BG 33.634.5EN 50.9 Logic 11.4 Math 15.1 AC 60.828.04.650.9Not run11.415.133.660.8
36OneJev1.6-0.8B full fine-tuneOpen · 0.8B17.7TR 11.4 L&M 9.8 BG 31.929.6EN 49.3 Logic 8.8 Math 10.3 AC 50.024.55.849.38.18.810.331.950.0
37GLiDE 0.4BOpen · 0.4B3.7TR 2.8 L&M 1.8 BG 6.524.6EN 49.3 Logic 4.0 Math 0.7 AC 44.45.71.649.3Not run4.00.76.544.4
38GLiClass v4 Edge fine-tuneOpen · 0.4B−0.8TR 4.2 L&M 3.7 BG −10.225.0EN 47.9 Logic 4.4 Math 3.3 AC 44.37.42.847.9Not run4.43.3−10.244.3
Adjudex 9B v5Open · 9.0BNot applicable62.1EN 82.2 Logic 33.0 Math 52.6 AC 80.476.026.782.2Not run33.052.6Not run80.4
Chinese Decision Pointer 4B LoRA, mergedOpen · 4.0BNot applicable54.3EN 75.5 Logic 25.4 Math 38.0 AC 78.163.620.575.5Not run25.438.0Not run78.1
Claude Fable 5.2 max · chatNot applicableNot applicable97.197.997.5Not runNot runNot runNot runNot run
Claude Haiku 5.5 max · chatNot applicableNot applicable83.630.490.9Not runNot runNot runNot runNot run
Clef-Flash post-trainedOpen · 9.0BNot applicable52.1EN 77.8 Logic 23.7 Math 37.2 AC 69.866.317.677.8Not run23.737.2Not run69.8
Copernicus-31BOpen · 31BNot applicable75.0EN 91.1 Logic 48.3 Math 71.7 AC 89.091.471.291.1Not run48.371.7Not run89.0
Decider 1Not applicableNot applicable18.34.944.7Not runNot runNot runNot run43.3
DeepSeek-V4-Flash chat285B MoE · 13B activeNot applicableNot applicableNot runNot runNot runNot runNot run99.1Not runNot run
DeepSeek-V4.1-Flash reasoning on · chat · first releaseOpen · 552B MoE · 16B activeNot applicableNot applicable91.889.3Not runNot runNot runNot runNot runNot run
DeepSeek-V4.1-Flash reasoning on · chat · 1017setups differOpen · 552B MoE · 16B activeNot applicableNot applicable92.688.590.6Not run95.299.5Not runNot run
DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseNot applicableNot applicableNot runNot runNot runNot runNot runNot run83.9Not run
Dexy-Decision-9B-V2 fine-tuneOpen · 9.0BNot applicable60.2EN 79.4 Logic 31.7 Math 48.5 AC 81.174.524.679.4Not run31.748.5Not run81.1
Erk-14B reasoning off · chatOpen · 14BNot applicableNot applicable65.316.765.0Not run42.888.4Not runNot run
GLM-5.3 low · chatNot applicableNot applicable84.661.284.7Not runNot runNot run73.8Not run
GLM-5.3-Flash reasoning on · chatsetups differOpen · 320B MoE · 18B activeNot applicableNot applicable86.767.987.0Not run91.498.5Not runNot run
GLiDE APINot applicableNot applicable84.849.3Not runNot run43.163.571.985.1
GLiDE 27B DGX SparkOpen · 27BNot applicable71.9EN 88.6 Logic 48.0 Math 65.0 AC 86.188.653.488.637.448.065.0Not run86.1
GLiDE 27B RTX PRO 6000Open · 27BNot applicableNot applicableNot runNot run87.9Not runNot runNot runNot runNot run
GLiDE 9BOpen · 7.9BNot applicable61.8EN 82.9 Logic 34.8 Math 47.4 AC 82.275.225.182.9Not run34.847.4Not run82.2
GLiNER2.5 MultiOpen · 0.3BNot applicableNot applicable−3.5−3.0−4.7Not runNot runNot runNot runNot run
GLiNER2.5-multi-Decide fine-tuneOpen · 0.3BNot applicableNot applicable1.3−0.21.5Not runNot runNot runNot runNot run
GPT-6 Luna low · chatNot applicableNot applicable84.250.282.9Not run47.492.1Not runNot run
GPT-6 Luna Decisions APINot applicableNot applicable81.942.980.115.8Not runNot run64.2Not run
GPT-6.1 Terra high · chatNot applicableNot applicable95.291.895.2Not run92.499.0Not runNot run
Gemini 3.8 Flash high · chatNot applicableNot applicable96.498.195.0Not runNot runNot run87.3Not run
Gemini 4 Argon low · chatNot applicableNot applicable96.295.195.0Not runNot runNot run89.3Not run
Gemini 4 Carbon high · chatNot applicableNot applicable97.098.592.9Not run95.999.5Not runNot run
Gemini 4 Hydrogen high · chatNot applicable93.5EN 95.9 Logic 95.1 Math 99.5 AC 83.796.698.195.9Not run95.199.5Not run83.7
Gemma 4 12B Unified default reasoning · chatOpen · 12BNot applicable85.9EN 83.5 Logic 84.7 Math 98.6 AC 77.0Not runNot run83.5Not run84.798.6Not run77.0
Gemma 4 12B Unified thinking on · chatOpen · 12BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run67.7
Gemma 4 E2B reasoning on · chatOpen · 5.1BNot applicable75.9EN 70.3 Logic 66.1 Math 96.0 AC 71.1Not runNot run70.3Not run66.196.0Not run71.1
Gemma 4 E2B default reasoning · chatOpen · 5.1BNot applicableNot applicableNot runNot run68.2Not runNot runNot runNot run55.2
Gemma 4 E2B thinking off · chatOpen · 5.1BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run54.8
Gemma 4 E4B default reasoning · chatOpen · 8.0BNot applicable83.0EN 78.1 Logic 79.0 Math 97.6 AC 77.272.036.378.128.679.097.6Not run77.2
Gemma 4 E4B thinking on · sampled (temperature 1.0) · chatOpen · 8.0BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run75.1
Granite-4.2-3B reasoning on · chatOpen · 3.0BNot applicable83.1EN 68.0 Logic 89.6 Math 99.0 AC 75.9Not runNot run68.0Not run89.699.0Not run75.9
Granite-4.2-3B reasoning off · chatOpen · 3.0BNot applicableNot applicableNot runNot run65.3Not runNot runNot runNot run52.8
Grok 4.7 high · chatNot applicableNot applicable93.789.791.8Not runNot run99.484.0Not run
Hy4 flash high · chatOpen · 220B MoE · 12B activeNot applicableNot applicable91.238.392.0Not run94.3Not runNot runNot run
Inkling 2 Small high · chatOpen · 594B MoE · 17B activeNot applicable93.0EN 91.8 Logic 94.6 Math 99.4 AC 86.290.143.691.8Not run94.699.4Not run86.2
Inkling-Small high · chat · Jul 2026Open · 276B MoE · 12B activeNot applicableNot applicable84.071.185.4Not run94.299.5Not runNot run
Jeffan-9B-1211Open · 9.0BNot applicable63.1EN 84.5 Logic 35.2 Math 49.8 AC 83.078.732.584.5Not run35.249.8Not run83.0
Karpo Decision 31BOpen · 31BNot applicable77.0EN 90.2 Logic 50.9 Math 76.9 AC 90.091.863.990.2Not run50.976.9Not run90.0
Kimi K3.1 Pro low · chatOpen · 2.8T MoE · 104B activeNot applicableNot applicable92.272.690.4Not runNot runNot run76.2Not run
KnowLine-12B-Gen1Open · 12BNot applicable65.1EN 83.3 Logic 38.2 Math 57.8 AC 80.982.540.683.3Not run38.257.8Not run80.9
KnowLine-31B-Gen1.5 RTX PRO 6000 LoRA, mergedOpen · 31BNot applicableNot applicableNot runNot run91.8Not runNot runNot runNot runNot run
KnowLine-31B-Gen2Open · 31BNot applicable75.5EN 90.4 Logic 48.7 Math 75.3 AC 87.791.870.090.4Not run48.775.3Not run87.7
KnowLine-9B-Gen1Open · 9.0BNot applicable59.7EN 82.4 Logic 30.3 Math 49.7 AC 76.174.829.782.4Not run30.349.7Not run76.1
KnowLine-9B-Gen2Open · 9.0BNot applicable59.2EN 81.5 Logic 29.9 Math 50.1 AC 75.374.527.681.5Not run29.950.1Not run75.3
LFM2.5-2.6B default reasoning · chatOpen · 2.7BNot applicableNot applicable31.811.267.3Not run69.292.5Not runNot run
LFM2.5-24B-A2B default reasoning · chatOpen · 24B MoE · 2.5B activeNot applicableNot applicableNot runNot run78.3Not run83.398.860.6Not run
LFM2.5-8B-A1B default reasoning · chatOpen · 8.3B MoE · 1.5B activeNot applicable66.6EN 63.0 Logic 50.2 Math 95.3 AC 57.77.0−1.263.0Not run50.295.3Not run57.7
LFM3-1.7B reasoning on · chatOpen · 1.7BNot applicable80.1EN 63.9 Logic 81.9 Math 98.7 AC 75.830.39.263.9Not run81.998.7Not run75.8
LFM3-1.7B reasoning off · chatOpen · 1.7BNot applicableNot applicable21.55.764.7Not runNot runNot runNot run74.9
LayaOpen · 0.4BNot applicableNot applicable1.1−2.23.6Not runNot runNot runNot runNot run
Laya MultilingualOpen · 0.3BNot applicableNot applicable−3.7−2.2−5.8Not runNot runNot runNot runNot run
Logits Sigma-1 Duo blendOpen · 57BNot applicable72.5EN 88.8 Logic 47.2 Math 69.0 AC 85.189.959.888.8Not run47.269.0Not run85.1
MAI Code 2 Flash high · chatNot applicableNot applicable84.670.084.7Not runNot runNot runNot runNot run
Metask-Jev-4B LoRA, mergedOpen · 4.5BNot applicable43.1EN 71.4 Logic 15.1 Math 23.9 AC 61.753.514.271.4Not run15.123.9Not run61.7
MiMo-V2.6-Flash high · chatOpen · 309B MoE · 15B activeNot applicableNot applicable83.257.486.3Not run93.999.3Not runNot run
Microsoft-Decision-1Not applicable61.5EN 82.2 Logic 30.6 Math 49.2 AC 83.966.818.782.2Not run30.649.2Not run83.9
Microsoft-Decision-1.1Not applicable66.8EN 81.9 Logic 38.3 Math 61.3 AC 85.485.945.881.9Not run38.361.3Not run85.4
MiniCPM5-2B reasoning off · chatOpen · 2.5BNot applicable81.0EN 59.7 Logic 86.1 Math 99.0 AC 79.2−16.8−18.559.7Not run86.199.0Not run79.2
MiniCPM5-2B reasoning on · chatOpen · 2.5BNot applicableNot applicableNot runNot run72.7Not runNot runNot runNot run67.0
MiniMax-M3.1-Flash high · chatOpen · 428B MoE · 23B activeNot applicableNot applicable78.749.380.6Not run92.497.8Not runNot run
Nemotron-Labs-Sigma-3B-instruct default reasoning · chatOpen · 3.0BNot applicableNot applicableNot runNot run44.7Not runNot runNot runNot run4.3
Nemotron-Labs-Sigma-8B-instruct default reasoning · chatOpen · 8.0BNot applicableNot applicableNot runNot run45.8Not runNot runNot runNot runNot run
OneJev-27B full fine-tuneOpen · 27BNot applicableNot applicable81.535.982.926.625.842.2Not runNot run
Perplexity Decider 27B v1Open · 27BNot applicableNot applicable82.536.985.6Not runNot runNot runNot runNot run
Perplexity Decider 27B v1.1setups differOpen · 27BNot applicableNot applicable86.346.8Not runNot runNot runNot run68.0Not run
Perplexity Decider 27B v1.2 API · with webOpen · 27BNot applicableNot applicable86.150.288.1Not run40.761.6Not runNot run
Perplexity Decider 27B v1.3Open · 27BNot applicable70.6EN 87.0 Logic 43.4 Math 65.6 AC 86.286.349.687.0Not run43.465.6Not run86.2
Perplexity Decider Flash v1.3Open · 4.0BNot applicable58.6EN 79.7 Logic 30.9 Math 44.9 AC 78.868.024.079.7Not run30.944.9Not run78.8
Qwen3-14B reasoning off · chatOpen · 15BNot applicableNot applicable65.118.174.6Not run56.794.7Not runNot run
Qwen3.5-2B reasoning off · chatOpen · 2.0BNot applicable69.7EN 59.1 Logic 56.0 Math 93.3 AC 70.317.92.059.1Not run56.093.3Not run70.3
Qwen3.5-2B reasoning on · chatOpen · 2.0BNot applicableNot applicableNot runNot run69.8Not runNot runNot runNot run50.2
Qwen3.5-397B-A17B high · chatOpen · 397B MoE · 17B activeNot applicableNot applicable88.874.089.3Not run93.9Not runNot runNot run
Qwen3.5-4B reasoning on · chatOpen · 4.0BNot applicable84.9EN 83.8 Logic 90.4 Math 99.3 AC 66.273.937.983.8Not run90.499.3Not run66.2
Qwen3.5-4B reasoning off · chatOpen · 4.0BNot applicableNot applicable60.620.975.5Not runNot runNot runNot run64.6
Qwen4-Max high · chatNot applicableNot applicable93.388.193.1Not run95.399.7Not runNot run
Raon-1 nano reasoning on · chatOpen · 3.1BNot applicable67.1EN 61.2 Logic 56.3 Math 86.9 AC 63.8Not runNot run61.2Not run56.386.9Not run63.8
Raon-1 nano reasoning off · chatOpen · 3.1BNot applicableNot applicableNot runNot run42.2Not runNot runNot runNot run40.6
SenseNova-Loop-9B on · chatOpen · 9.0BNot applicableNot applicableNot runNot runNot runNot run89.298.9Not runNot run
Sieve 31B v3 RTX PRO 6000 full fine-tuneOpen · 31BNot applicableNot applicableNot runNot run88.8Not runNot runNot runNot runNot run
StartLux-Decision-v1.1-9B-InstantOpen · 9.0BNot applicable60.9EN 83.5 Logic 33.2 Math 48.0 AC 78.873.524.483.5Not run33.248.0Not run78.8
Strands Decider 2B LoRA + headOpen · 2.0BNot applicable26.5EN 31.9 Logic 3.0 Math 10.8 AC 60.321.14.331.9Not run3.010.8Not run60.3
SubQ Savant 1.4 high · chatNot applicableNot applicable91.455.491.5Not runNot runNot runNot runNot run
SubQ Snap 1.2Not applicable68.5EN 85.1 Logic 43.1 Math 60.9 AC 84.984.446.685.1Not run43.160.9Not run84.9
SubQ Snap PreviewNot applicableNot applicable84.644.682.4Not runNot runNot run63.8Not run
Turion Jev 31B RTX PRO 6000 LoRA, mergedOpen · 31BNot applicableNot applicableNot runNot run88.8Not runNot runNot runNot runNot run
Valen-Decision-1.5-9BOpen · 9.0BNot applicable57.1EN 79.7 Logic 30.4 Math 44.2 AC 74.368.818.879.7Not run30.444.2Not run74.3
Verdin-4B-v1.1 LoRA, mergedOpen · 4.0BNot applicable54.3EN 77.8 Logic 24.7 Math 39.4 AC 75.464.020.577.8Not run24.739.4Not run75.4
celeris-1.1-decisionNot applicableNot applicableNot runNot run79.7Not run34.162.3Not runNot run
d1Not applicable58.3EN 83.5 Logic 26.0 Math 44.5 AC 79.278.532.683.5Not run26.044.5Not run79.2
d1-Omni-2B NVIDIA RTX PRO 6000 BlackwellOpen · 2.2BNot applicableNot applicableNot runNot runNot runNot runNot runNot run58.9Not run
d1.1Not applicable64.2EN 85.4 Logic 32.6 Math 56.5 AC 82.580.839.685.4Not run32.656.5Not run82.5
d3 fine-tuneOpen · 26BNot applicableNot applicable86.548.487.235.643.163.0Not runNot run
d3-ultra fine-tuneOpen · 26BNot applicableNot applicable87.248.086.731.843.264.4Not runNot run
vera-medium-omniOpen · 4.2BNot applicable59.8EN 79.4 Logic 30.1 Math 46.0 AC 83.769.322.379.4Not run30.146.0Not run83.7

d1.2-ultra, different setups of one model: Backgammon: our hardware; the other benches: a hosted API.

Perplexity Decider 27B v1.2 · DGX Spark, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

d1.2, different setups of one model: Backgammon: our hardware; the other benches: a hosted API.

Drex 2.0, different setups of one model: Answer Check: our hardware; the other benches: a hosted API.

Cygnet, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

DeepSeek-V4.1-Flash · reasoning on · chat · 1017, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

GLM-5.3-Flash · reasoning on · chat, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

Perplexity Decider 27B v1.1, different setups of one model: Backgammon: our hardware; the other benches: a hosted API.

How it's scored

Methodology: every rule, the sources and how models are matched across benches.

Head to head

Score 0–100, chance = 0 · higher is better ↑ · the difference in points; "tied" where the two are too close to tell apart (the rule)
BenchKnowLine-31B-Gen1.5 · DGX SparkREAX 1.3Difference
Combined ↑73.271.3KnowLine-31B-Gen1.5 · DGX Spark ahead by 1.8
Decision Index ↑75.374.8tied (0.5)
Turkish Reading ↑91.090.5tied (0.6)
Turkish Grammar ↑71.464.7KnowLine-31B-Gen1.5 · DGX Spark ahead by 6.6
English Reading ↑92.090.6tied (1.4)
Vision ↑37.3Not runonly KnowLine-31B-Gen1.5 · DGX Spark ran it
Logic ↑48.849.1tied (−0.3)
Math ↑71.772.5tied (−0.8)
Backgammon ↑78.377.1tied (1.3)
Answer Check ↑88.887.0tied (1.9)

Full results

Decision models · highest first by Combined · Score 0–100, chance = 0 · ECE ↓ under each score
ModelSizeCombined ↑Decision Index ↑Turkish Reading ↑Turkish Grammar ↑English Reading ↑Vision ↑Logic ↑Math ↑Backgammon ↑Answer Check ↑Learning from experience ↑points gainedLatency ↓median, p95
KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B73.2TR 77.3 L&M 63.9 BG 78.375.3EN 92.0 Logic 48.8 Math 71.7 AC 88.891.0ECE 3.4 ↓71.4ECE 3.9 ↓92.0ECE 5.0 ↓37.3ECE 6.1 ↓48.8ECE 2.6 ↓71.7ECE 3.6 ↓78.388.8ECE 5.7 ↓–1,051 msp95 1,533 ms
REAX 1.3 blendREAX57.4B71.3TR 72.4 L&M 64.5 BG 77.174.8EN 90.6 Logic 49.1 Math 72.5 AC 87.090.5ECE 2.1 ↓64.7ECE 5.1 ↓90.6ECE 3.6 ↓Not run49.1ECE 3.7 ↓72.5ECE 7.1 ↓77.187.0ECE 4.8 ↓–not timed yet
Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B69.4TR 67.7 L&M 63.2 BG 77.373.8EN 88.3 Logic 47.3 Math 71.4 AC 88.189.7ECE 3.5 ↓58.3ECE 3.3 ↓88.3ECE 4.3 ↓Not run47.3ECE 2.1 ↓71.4ECE 4.3 ↓77.388.1ECE 7.0 ↓–not timed yet
d1-ultraLiquid AINot disclosed68.3TR 71.3 L&M 59.4 BG 74.371.1EN 88.3 Logic 43.7 Math 67.6 AC 84.790.1ECE 3.6 ↓63.4ECE 4.2 ↓88.3ECE 4.4 ↓37.6ECE 3.5 ↓43.7ECE 3.2 ↓67.6ECE 2.3 ↓74.384.7ECE 11.4 ↓–not timed yet
Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B68.2TR 75.1 L&M 54.1 BG 75.569.1EN 88.3 Logic 40.3 Math 61.3 AC 86.490.9ECE 9.5 ↓68.5ECE 12.0 ↓88.3ECE 11.6 ↓Not run40.3ECE 2.9 ↓61.3ECE 4.7 ↓75.586.4ECE 6.7 ↓–not timed yet
Gemini Decision 1 20261111GoogleNot disclosed68.0TR 71.9 L&M 55.1 BG 76.966.9EN 88.8 Logic 37.6 Math 64.2 AC 77.090.1ECE 2.8 ↓64.2ECE 2.9 ↓88.8ECE 4.5 ↓58.1ECE 5.2 ↓37.6ECE 3.1 ↓64.2ECE 2.5 ↓76.977.0ECE 3.9 ↓–not timed yet
Mammoth-1 post-trainedBase Labs27.4B64.8TR 58.9 L&M 59.1 BG 76.470.2EN 87.7 Logic 43.1 Math 67.5 AC 82.488.0ECE 2.0 ↓46.5ECE 7.9 ↓87.7ECE 2.4 ↓Not run43.1ECE 5.3 ↓67.5ECE 3.0 ↓76.482.4ECE 1.9 ↓–not timed yet
Drex Max fine-tuneNace.AI26.1B63.4TR 59.7 L&M 55.5 BG 74.9Not applicable87.0ECE 3.4 ↓48.0ECE 8.0 ↓87.0ECE 2.6 ↓33.5ECE 9.3 ↓41.3ECE 16.2 ↓62.9ECE 11.8 ↓74.9Not run–not timed yet
Drex Max v1.1Nace.AINot disclosed63.2TR 59.2 L&M 56.1 BG 74.5Not applicable85.7ECE 3.4 ↓48.0ECE 2.7 ↓85.6ECE 3.8 ↓31.4ECE 5.2 ↓43.4ECE 6.1 ↓62.6ECE 5.7 ↓74.5Not run–not timed yet
GLiDE v1.2Fastino26.1B62.6TR 60.6 L&M 56.1 BG 71.169.8EN 87.9 Logic 44.0 Math 62.3 AC 84.987.0ECE 3.5 ↓49.3ECE 3.0 ↓87.9ECE 2.7 ↓Not run44.0ECE 5.9 ↓62.3ECE 5.2 ↓71.184.9ECE 4.7 ↓–not timed yet
d1.2-ultraLiquid AINot disclosed62.4TR 60.1 L&M 54.5 BG 72.769.0EN 87.9 Logic 39.4 Math 62.3 AC 86.487.2ECE 5.6 ↓48.6ECE 3.1 ↓87.9ECE 8.5 ↓31.7ECE 2.8 ↓39.4ECE 3.0 ↓62.3ECE 4.6 ↓72.786.4ECE 8.8 ↓–not timed yet
Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B61.2TR 58.8 L&M 55.2 BG 69.5Not applicable86.5ECE 3.1 ↓47.1ECE 6.2 ↓85.8ECE 3.7 ↓Not run41.7ECE 3.4 ↓62.2ECE 3.2 ↓69.5Not run–not timed yet
SubQ Snap 1.1SubquadraticNot disclosed58.5TR 54.8 L&M 53.8 BG 66.867.1EN 82.4 Logic 40.4 Math 60.7 AC 84.983.8ECE 3.6 ↓42.5ECE 14.5 ↓82.4ECE 3.2 ↓32.2ECE 11.7 ↓40.4ECE 12.9 ↓60.7ECE 9.5 ↓66.884.9ECE 2.3 ↓–not timed yet
celeris-1-decisionCelerisNot disclosed58.0TR 50.6 L&M 52.8 BG 70.6Not applicable81.5ECE 6.2 ↓37.4ECE 5.3 ↓80.1ECE 5.2 ↓Not run34.1ECE 10.7 ↓62.5ECE 4.3 ↓70.6Not run–not timed yet
Jev 1.14TypeSafeNot disclosed57.1TR 49.0 L&M 49.2 BG 73.164.5EN 82.6 Logic 35.2 Math 56.4 AC 83.681.3ECE 2.2 ↓35.2ECE 4.0 ↓82.6ECE 2.2 ↓29.1ECE 3.7 ↓35.2ECE 3.3 ↓56.4ECE 2.8 ↓73.183.6ECE 2.8 ↓–not timed yet
d1.2Liquid AINot disclosed56.3TR 52.2 L&M 49.9 BG 66.965.3EN 86.3 Logic 36.4 Math 56.9 AC 81.582.5ECE 6.0 ↓39.3ECE 2.5 ↓86.3ECE 8.3 ↓33.7ECE 5.6 ↓36.4ECE 2.6 ↓56.9ECE 3.2 ↓66.981.5ECE 8.5 ↓–not timed yet
Jev 1.13TypeSafeNot disclosed55.3TR 46.9 L&M 47.5 BG 71.663.3EN 82.4 Logic 32.9 Math 55.1 AC 83.079.6ECE 3.8 ↓33.0ECE 6.5 ↓82.4ECE 3.1 ↓Not run32.9ECE 5.9 ↓55.1ECE 2.6 ↓71.683.0ECE 1.9 ↓–not timed yet
Drex 2.0Nace.AI35.1B3B active51.4TR 44.7 L&M 41.4 BG 68.358.1EN 82.4 Logic 27.3 Math 48.7 AC 73.876.8ECE 10.1 ↓31.0ECE 2.0 ↓82.4ECE 6.9 ↓Not run27.3ECE 4.3 ↓48.7ECE 1.9 ↓68.373.8ECE 11.1 ↓–not timed yet
Clef post-trainedCloudflare27.5B50.0TR 46.6 L&M 34.5 BG 69.056.2EN 83.3 Logic 26.0 Math 38.9 AC 76.581.3ECE 4.0 ↓31.8ECE 15.1 ↓83.3ECE 3.1 ↓Not run26.0ECE 15.7 ↓38.9ECE 11.0 ↓69.076.5ECE 4.0 ↓–not timed yet
Elephant-1 post-trainedBase Labs9.4B49.7TR 40.5 L&M 45.3 BG 63.260.8EN 81.7 Logic 33.0 Math 51.7 AC 76.975.6ECE 4.3 ↓25.5ECE 15.4 ↓81.7ECE 2.7 ↓Not run33.0ECE 8.6 ↓51.7ECE 4.3 ↓63.276.9ECE 3.7 ↓–not timed yet
Mercury DecideInceptionNot disclosed48.8TR 47.9 L&M 38.3 BG 60.4Not applicable80.6ECE 7.5 ↓33.9ECE 19.0 ↓82.2ECE 6.6 ↓Not run31.3ECE 15.4 ↓41.9ECE 15.8 ↓60.4Not run–not timed yet
Cerno 9B v1.2 full fine-tuneApertix AI8.9B47.5TR 43.1 L&M 39.6 BG 60.059.6EN 82.4 Logic 29.1 Math 45.0 AC 81.974.7ECE 3.0 ↓29.6ECE 14.2 ↓82.4ECE 4.2 ↓Not run29.1ECE 17.0 ↓45.0ECE 15.7 ↓60.081.9ECE 2.9 ↓–not timed yet
StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B47.4TR 34.8 L&M 40.5 BG 66.859.3EN 78.5 Logic 32.3 Math 44.8 AC 81.767.6ECE 2.6 ↓20.9ECE 10.3 ↓78.5ECE 3.2 ↓Not run32.3ECE 3.3 ↓44.8ECE 6.7 ↓66.881.7ECE 8.3 ↓–not timed yet
Cygnet unchanged baseBlockbrain Labs12B47.0TR 44.6 L&M 28.3 BG 68.251.8EN 79.9 Logic 24.1 Math 30.4 AC 72.978.5ECE 4.1 ↓30.2ECE 18.9 ↓79.9ECE 4.3 ↓Not run24.1ECE 11.2 ↓30.4ECE 5.9 ↓68.272.9ECE 6.5 ↓–not timed yet
Winnow-12B LoRA, mergedEldanRing11.9B44.6TR 44.3 L&M 30.5 BG 58.953.7EN 79.9 Logic 24.7 Math 33.6 AC 76.677.5ECE 9.3 ↓30.2ECE 24.1 ↓79.9ECE 8.7 ↓Not run24.7ECE 14.4 ↓33.6ECE 8.4 ↓58.976.6ECE 8.3 ↓–not timed yet
KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B43.5TR 35.6 L&M 39.1 BG 55.857.4EN 76.0 Logic 27.9 Math 45.0 AC 80.668.6ECE 6.2 ↓21.6ECE 16.6 ↓76.0ECE 5.8 ↓12.7ECE 11.0 ↓27.9ECE 13.7 ↓45.0ECE 8.6 ↓55.880.6ECE 4.5 ↓–99 msp95 149 ms
Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B42.8TR 44.6 L&M 38.1 BG 45.9Not applicable78.7ECE 43.6 ↓30.1ECE 16.4 ↓81.7ECE 45.3 ↓Not run28.2ECE 12.9 ↓43.2ECE 21.1 ↓45.9Not run–not timed yet
Perplexity Decider Flash v1.2Perplexity4.5B41.4TR 34.7 L&M 38.9 BG 50.856.9EN 77.4 Logic 28.9 Math 44.1 AC 77.265.9ECE 3.7 ↓21.4ECE 11.0 ↓77.4ECE 4.1 ↓Not run28.9ECE 6.0 ↓44.1ECE 10.4 ↓50.877.2ECE 2.8 ↓–not timed yet
StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B32.1TR 17.8 L&M 19.9 BG 58.741.2EN 62.1 Logic 13.4 Math 23.3 AC 66.041.1ECE 3.3 ↓7.9ECE 9.4 ↓62.1ECE 8.2 ↓4.3ECE 3.6 ↓13.4ECE 5.3 ↓23.3ECE 2.6 ↓58.766.0ECE 7.2 ↓–40 msp95 48 ms
KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B31.2TR 21.0 L&M 17.6 BG 55.039.7EN 64.1 Logic 14.7 Math 19.1 AC 61.142.8ECE 1.9 ↓11.7ECE 11.4 ↓64.1ECE 5.1 ↓Not run14.7ECE 8.3 ↓19.1ECE 9.4 ↓55.061.1ECE 2.7 ↓–62 msp95 87 ms
d1-2BLiquid AI1.9B30.5TR 11.7 L&M 18.6 BG 61.135.3EN 57.0 Logic 13.3 Math 21.3 AC 49.626.1ECE 4.4 ↓5.6ECE 6.7 ↓57.0ECE 7.9 ↓4.2ECE 1.1 ↓13.3ECE 6.9 ↓21.3ECE 5.6 ↓61.149.6ECE 7.2 ↓–not timed yet
GLiDE 2BFastino1.9B27.5TR 19.1 L&M 19.0 BG 44.441.1EN 62.5 Logic 15.3 Math 20.9 AC 65.639.0ECE 16.5 ↓10.7ECE 22.5 ↓62.5ECE 8.2 ↓Not run15.3ECE 16.2 ↓20.9ECE 17.2 ↓44.465.6ECE 4.0 ↓–34 msp95 47 ms
StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M22.4TR 10.7 L&M 10.0 BG 46.629.1EN 49.5 Logic 5.9 Math 12.1 AC 48.918.8ECE 5.0 ↓7.3ECE 4.6 ↓49.5ECE 8.7 ↓5.3ECE 1.7 ↓5.9ECE 7.8 ↓12.1ECE 4.8 ↓46.648.9ECE 5.6 ↓–28 msp95 33 ms
KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M19.7TR 8.9 L&M 9.6 BG 40.626.4EN 48.6 Logic 8.3 Math 10.3 AC 38.522.8ECE 5.5 ↓3.0ECE 10.8 ↓48.6ECE 8.1 ↓4.8ECE 1.5 ↓8.3ECE 8.4 ↓10.3ECE 10.5 ↓40.638.5ECE 14.6 ↓–33 msp95 50 ms
GLiDE 0.8BFastino750M19.7TR 11.6 L&M 13.9 BG 33.634.5EN 50.9 Logic 11.4 Math 15.1 AC 60.828.0ECE 21.8 ↓4.6ECE 26.1 ↓50.9ECE 12.5 ↓Not run11.4ECE 14.0 ↓15.1ECE 18.8 ↓33.660.8ECE 3.8 ↓–22 msp95 31 ms
OneJev1.6-0.8B full fine-tuneOmniJev850M17.7TR 11.4 L&M 9.8 BG 31.929.6EN 49.3 Logic 8.8 Math 10.3 AC 50.024.5ECE 7.3 ↓5.8ECE 10.3 ↓49.3ECE 3.7 ↓8.1ECE 4.2 ↓8.8ECE 11.2 ↓10.3ECE 10.0 ↓31.950.0ECE 4.1 ↓–17 msp95 23 ms
GLiDE 0.4BFastino400M3.7TR 2.8 L&M 1.8 BG 6.524.6EN 49.3 Logic 4.0 Math 0.7 AC 44.45.7ECE 7.2 ↓1.6ECE 8.3 ↓49.3ECE 4.8 ↓Not run4.0ECE 8.8 ↓0.7ECE 12.2 ↓6.544.4ECE 7.7 ↓–16 msp95 24 ms
GLiClass v4 Edge fine-tuneKnowledgator440M−0.8TR 4.2 L&M 3.7 BG −10.225.0EN 47.9 Logic 4.4 Math 3.3 AC 44.37.4ECE 11.9 ↓2.8ECE 10.0 ↓47.9ECE 3.8 ↓Not run4.4ECE 8.7 ↓3.3ECE 7.0 ↓−10.244.3ECE 9.8 ↓–25 msp95 53 ms
Adjudex 9B v5newsbang8.4BNot applicable62.1EN 82.2 Logic 33.0 Math 52.6 AC 80.476.0ECE 2.5 ↓26.7ECE 12.2 ↓82.2ECE 3.4 ↓Not run33.0ECE 7.2 ↓52.6ECE 4.8 ↓Not run80.4ECE 4.9 ↓–not timed yet
Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2BNot applicable54.3EN 75.5 Logic 25.4 Math 38.0 AC 78.163.6ECE 3.3 ↓20.5ECE 15.4 ↓75.5ECE 3.5 ↓Not run25.4ECE 15.5 ↓38.0ECE 13.7 ↓Not run78.1ECE 6.7 ↓–not timed yet
Clef-Flash post-trainedCloudflare9.5BNot applicable52.1EN 77.8 Logic 23.7 Math 37.2 AC 69.866.3ECE 5.6 ↓17.6ECE 20.3 ↓77.8ECE 4.0 ↓Not run23.7ECE 11.5 ↓37.2ECE 2.7 ↓Not run69.8ECE 5.9 ↓–not timed yet
Copernicus-31BPoindexter Labs31.3BNot applicable75.0EN 91.1 Logic 48.3 Math 71.7 AC 89.091.4ECE 6.3 ↓71.2ECE 10.5 ↓91.1ECE 7.8 ↓Not run48.3ECE 3.9 ↓71.7ECE 6.5 ↓Not run89.0ECE 11.0 ↓–not timed yet
Decider 1meraGPTNot disclosedNot applicableNot applicable18.3ECE 11.8 ↓4.9ECE 15.4 ↓44.7ECE 5.3 ↓Not runNot runNot runNot run43.3ECE 5.9 ↓–not timed yet
Dexy-Decision-9B-V2 fine-tunePatrick-188.4BNot applicable60.2EN 79.4 Logic 31.7 Math 48.5 AC 81.174.5ECE 3.2 ↓24.6ECE 12.7 ↓79.4ECE 4.2 ↓Not run31.7ECE 14.9 ↓48.5ECE 12.1 ↓Not run81.1ECE 2.5 ↓–not timed yet
GLiDE APIFastinoNot disclosedNot applicablenot one modelNot applicable84.8ECE 5.8 ↓49.3ECE 4.1 ↓Not runNot run43.1ECE 6.6 ↓63.5ECE 5.0 ↓71.985.1ECE 3.0 ↓–not timed yet
GLiDE 27B DGX SparkFastino26.1BNot applicable71.9EN 88.6 Logic 48.0 Math 65.0 AC 86.188.6ECE 2.4 ↓53.4ECE 2.7 ↓88.6ECE 1.7 ↓37.4ECE 4.3 ↓48.0ECE 4.4 ↓65.0ECE 4.1 ↓Not run86.1ECE 3.6 ↓–542 msp95 739 ms
GLiDE 27B RTX PRO 6000Fastino26.1BNot applicableNot applicableNot runNot run87.9ECE 1.9 ↓Not runNot runNot runNot runNot run–not timed yet
GLiDE 9BFastino7.9BNot applicable61.8EN 82.9 Logic 34.8 Math 47.4 AC 82.275.2ECE 3.7 ↓25.1ECE 12.0 ↓82.9ECE 5.6 ↓Not run34.8ECE 6.7 ↓47.4ECE 7.1 ↓Not run82.2ECE 7.2 ↓–207 msp95 259 ms
GLiNER2.5-multi-Decide fine-tuneFastino290MNot applicableNot applicable1.3ECE 24.1 ↓−0.2ECE 22.7 ↓1.5ECE 27.2 ↓Not runNot runNot runNot runNot run–not timed yet
GPT-6 Luna Decisions APIOpenAINot disclosedNot applicableNot applicable81.9ECE 3.8 ↓42.9ECE 3.0 ↓80.1ECE 3.2 ↓15.8ECE 5.7 ↓Not runNot run64.2Not run–not timed yet
Jeffan-9B-1211tattrongvu8.9BNot applicable63.1EN 84.5 Logic 35.2 Math 49.8 AC 83.078.7ECE 11.4 ↓32.5ECE 3.0 ↓84.5ECE 8.9 ↓Not run35.2ECE 4.0 ↓49.8ECE 3.8 ↓Not run83.0ECE 3.3 ↓–not timed yet
Karpo Decision 31BMachinePulse31.3BNot applicable77.0EN 90.2 Logic 50.9 Math 76.9 AC 90.091.8ECE 1.2 ↓63.9ECE 3.4 ↓90.2ECE 2.4 ↓Not run50.9ECE 5.7 ↓76.9ECE 1.2 ↓Not run90.0ECE 2.8 ↓–not timed yet
KnowLine-12B-Gen1PelaAI12BNot applicable65.1EN 83.3 Logic 38.2 Math 57.8 AC 80.982.5ECE 11.4 ↓40.6ECE 4.3 ↓83.3ECE 11.1 ↓Not run38.2ECE 4.8 ↓57.8ECE 4.0 ↓Not run80.9ECE 6.8 ↓–not timed yet
KnowLine-31B-Gen1.5 RTX PRO 6000 LoRA, mergedPelaAI31.3BNot applicableNot applicableNot runNot run91.8ECE 5.3 ↓Not runNot runNot runNot runNot run–not timed yet
KnowLine-31B-Gen2PelaAI31.3BNot applicable75.5EN 90.4 Logic 48.7 Math 75.3 AC 87.791.8ECE 6.3 ↓70.0ECE 5.7 ↓90.4ECE 7.7 ↓Not run48.7ECE 3.7 ↓75.3ECE 3.6 ↓Not run87.7ECE 7.2 ↓–not timed yet
KnowLine-9B-Gen1PelaAI9.7BNot applicable59.7EN 82.4 Logic 30.3 Math 49.7 AC 76.174.8ECE 1.7 ↓29.7ECE 6.8 ↓82.4ECE 3.5 ↓Not run30.3ECE 3.5 ↓49.7ECE 3.1 ↓Not run76.1ECE 3.6 ↓–not timed yet
KnowLine-9B-Gen2PelaAI9.7BNot applicable59.2EN 81.5 Logic 29.9 Math 50.1 AC 75.374.5ECE 8.2 ↓27.6ECE 5.1 ↓81.5ECE 8.7 ↓Not run29.9ECE 3.2 ↓50.1ECE 4.8 ↓Not run75.3ECE 4.5 ↓–not timed yet
LayaConvai Innovations420MNot applicableNot applicable1.1ECE 9.2 ↓−2.2ECE 14.9 ↓3.6ECE 12.1 ↓Not runNot runNot runNot runNot run–not timed yet
Laya MultilingualConvai Innovations320MNot applicableNot applicable−3.7ECE 33.1 ↓−2.2ECE 26.0 ↓−5.8ECE 44.4 ↓Not runNot runNot runNot runNot run–not timed yet
Logits Sigma-1 Duo blendLogits57.4BNot applicable72.5EN 88.8 Logic 47.2 Math 69.0 AC 85.189.9ECE 5.3 ↓59.8ECE 5.9 ↓88.8ECE 8.0 ↓Not run47.2ECE 5.0 ↓69.0ECE 7.5 ↓Not run85.1ECE 7.0 ↓–not timed yet
Metask-Jev-4B LoRA, mergedMetask Lab4.5BNot applicable43.1EN 71.4 Logic 15.1 Math 23.9 AC 61.753.5ECE 5.3 ↓14.2ECE 6.5 ↓71.4ECE 4.4 ↓Not run15.1ECE 2.5 ↓23.9ECE 3.3 ↓Not run61.7ECE 6.8 ↓–not timed yet
Microsoft-Decision-1MicrosoftNot disclosedNot applicable61.5EN 82.2 Logic 30.6 Math 49.2 AC 83.966.8ECE 8.9 ↓18.7ECE 24.4 ↓82.2ECE 2.8 ↓Not run30.6ECE 8.0 ↓49.2ECE 4.1 ↓Not run83.9ECE 2.4 ↓–not timed yet
Microsoft-Decision-1.1MicrosoftNot disclosedNot applicable66.8EN 81.9 Logic 38.3 Math 61.3 AC 85.485.9ECE 9.6 ↓45.8ECE 4.5 ↓81.9ECE 8.3 ↓Not run38.3ECE 8.7 ↓61.3ECE 14.3 ↓Not run85.4ECE 3.0 ↓–not timed yet
OneJev-27B full fine-tuneOmniJev27.4BNot applicableNot applicable81.5ECE 9.9 ↓35.9ECE 3.7 ↓82.9ECE 9.7 ↓26.6ECE 4.9 ↓25.8ECE 4.6 ↓42.2ECE 3.6 ↓Not runNot run–not timed yet
Perplexity Decider 27B v1Perplexity26.1BNot applicableNot applicable82.5ECE 5.0 ↓36.9ECE 5.0 ↓85.6ECE 2.6 ↓Not runNot runNot runNot runNot run–not timed yet
Perplexity Decider 27B v1.1Perplexity26.1BNot applicableNot applicable86.3ECE 3.0 ↓46.8ECE 5.5 ↓Not runNot runNot runNot run68.0Not run–not timed yet
Perplexity Decider 27B v1.2 API · with webPerplexity26.1BNot applicableNot applicable86.1ECE 2.7 ↓50.2ECE 4.9 ↓88.1ECE 3.6 ↓Not run40.7ECE 4.2 ↓61.6ECE 3.7 ↓Not runNot run–not timed yet
Perplexity Decider 27B v1.3Perplexity26.1BNot applicable70.6EN 87.0 Logic 43.4 Math 65.6 AC 86.286.3ECE 2.4 ↓49.6ECE 6.1 ↓87.0ECE 3.4 ↓Not run43.4ECE 9.0 ↓65.6ECE 4.5 ↓Not run86.2ECE 2.8 ↓–not timed yet
Perplexity Decider Flash v1.3Perplexity4.5BNot applicable58.6EN 79.7 Logic 30.9 Math 44.9 AC 78.868.0ECE 6.2 ↓24.0ECE 15.3 ↓79.7ECE 2.7 ↓Not run30.9ECE 10.1 ↓44.9ECE 11.7 ↓Not run78.8ECE 4.2 ↓–not timed yet
Sieve 31B v3 RTX PRO 6000 full fine-tuneSthānika AI31.3BNot applicableNot applicableNot runNot run88.8ECE 4.7 ↓Not runNot runNot runNot runNot run–not timed yet
StartLux-Decision-v1.1-9B-InstantStartLux9.7BNot applicable60.9EN 83.5 Logic 33.2 Math 48.0 AC 78.873.5ECE 3.6 ↓24.4ECE 11.3 ↓83.5ECE 6.6 ↓Not run33.2ECE 3.8 ↓48.0ECE 2.7 ↓Not run78.8ECE 8.2 ↓–not timed yet
Strands Decider 2B LoRA + headStrands Agents2.3BNot applicable26.5EN 31.9 Logic 3.0 Math 10.8 AC 60.321.1ECE 10.2 ↓4.3ECE 13.2 ↓31.9ECE 10.5 ↓Not run3.0ECE 12.0 ↓10.8ECE 8.8 ↓Not run60.3ECE 5.0 ↓–not timed yet
SubQ Snap 1.2SubquadraticNot disclosedNot applicable68.5EN 85.1 Logic 43.1 Math 60.9 AC 84.984.4ECE 1.4 ↓46.6ECE 9.0 ↓85.1ECE 1.8 ↓Not run43.1ECE 8.7 ↓60.9ECE 6.9 ↓Not run84.9ECE 2.3 ↓–not timed yet
SubQ Snap PreviewSubquadraticNot disclosedNot applicableNot applicable84.6ECE 4.2 ↓44.6ECE 6.0 ↓82.4ECE 3.2 ↓Not runNot runNot run63.8Not run–not timed yet
Turion Jev 31B RTX PRO 6000 LoRA, mergedTurion.AI31.3BNot applicableNot applicableNot runNot run88.8ECE 11.6 ↓Not runNot runNot runNot runNot run–not timed yet
Valen-Decision-1.5-9BValen-Team8.4BNot applicable57.1EN 79.7 Logic 30.4 Math 44.2 AC 74.368.8ECE 9.7 ↓18.8ECE 6.9 ↓79.7ECE 8.4 ↓Not run30.4ECE 5.8 ↓44.2ECE 4.1 ↓Not run74.3ECE 3.2 ↓–not timed yet
Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2BNot applicable54.3EN 77.8 Logic 24.7 Math 39.4 AC 75.464.0ECE 3.9 ↓20.5ECE 12.8 ↓77.8ECE 3.5 ↓Not run24.7ECE 10.8 ↓39.4ECE 5.7 ↓Not run75.4ECE 3.4 ↓–not timed yet
celeris-1.1-decisionCelerisNot disclosedNot applicableNot applicableNot runNot run79.7ECE 5.0 ↓Not run34.1ECE 9.8 ↓62.3ECE 4.5 ↓Not runNot run–not timed yet
d1Liquid AINot disclosedNot applicable58.3EN 83.5 Logic 26.0 Math 44.5 AC 79.278.5ECE 2.8 ↓32.6ECE 13.5 ↓83.5ECE 2.6 ↓Not run26.0ECE 10.9 ↓44.5ECE 3.3 ↓Not run79.2ECE 3.1 ↓–not timed yet
d1-Omni-2B NVIDIA RTX PRO 6000 BlackwellLiquid AI2.2BNot applicableNot applicableNot runNot runNot runNot runNot runNot run58.9Not run–not timed yet
d1.1Liquid AINot disclosedNot applicable64.2EN 85.4 Logic 32.6 Math 56.5 AC 82.580.8ECE 4.2 ↓39.6ECE 2.2 ↓85.4ECE 4.2 ↓Not run32.6ECE 4.4 ↓56.5ECE 2.2 ↓Not run82.5ECE 8.7 ↓–not timed yet
d3 fine-tunevLLM Semantic Router26.1BNot applicableNot applicable86.5ECE 3.5 ↓48.4ECE 2.4 ↓87.2ECE 3.6 ↓35.6ECE 3.7 ↓43.1ECE 4.6 ↓63.0ECE 4.5 ↓Not runNot run–not timed yet
d3-ultra fine-tunevLLM Semantic Router26.1BNot applicableNot applicable87.2ECE 3.8 ↓48.0ECE 16.3 ↓86.7ECE 2.8 ↓31.8ECE 18.8 ↓43.2ECE 15.1 ↓64.4ECE 8.6 ↓Not runNot run–not timed yet
vera-medium-omniscar-ai4.2BNot applicable59.8EN 79.4 Logic 30.1 Math 46.0 AC 83.769.3ECE 9.1 ↓22.3ECE 8.0 ↓79.4ECE 9.9 ↓Not run30.1ECE 4.7 ↓46.0ECE 3.0 ↓Not run83.7ECE 12.4 ↓–not timed yet
Chat models · highest first by Combined · Score 0–100, chance = 0 · ECE ↓ under each score
ModelSizeCombined ↑Decision Index ↑Turkish Reading ↑Turkish Grammar ↑English Reading ↑Vision ↑Logic ↑Math ↑Backgammon ↑Answer Check ↑Learning from experience ↑points gainedLatency ↓median, p95
GPT-6 Astra high · chatOpenAINot disclosed96.1TR 97.4 L&M 98.3 BG 92.5Not applicable96.4ECE 1.0 ↓97.9ECE 0.7 ↓95.2ECE 1.8 ↓Not run95.999.592.5Not run–not timed yet
GPT-6.1 Sol high · chatOpenAINot disclosed95.7TR 97.2 L&M 98.2 BG 91.6Not applicable95.6ECE 1.8 ↓97.9ECE 0.6 ↓94.5ECE 2.4 ↓Not run95.799.591.6Not run–not timed yet
Claude Fable 5.1 high · chatAnthropicNot disclosed93.7TR 95.2 L&M 97.7 BG 88.1Not applicable94.5ECE 10.9 ↓95.5ECE 12.5 ↓95.0ECE 11.8 ↓Not run94.099.788.1Not runBackgammon 9.4not timed yet
Claude Opus 5.5 high · chatAnthropicNot disclosed93.2TR 93.7 L&M 98.2 BG 87.7Not applicable94.5ECE 14.3 ↓93.4ECE 14.9 ↓93.1ECE 13.3 ↓Not run95.299.787.7Not runBackgammon 6.7not timed yet
Claude Sonnet 5.5 high · chatAnthropicNot disclosed88.3TR 81.3 L&M 97.8 BG 85.8Not applicable81.5ECE 13.8 ↓81.2ECE 16.9 ↓82.6ECE 13.9 ↓Not run94.399.685.8Not runBackgammon 7.0not timed yet
Muse Ember max · chatMetaNot disclosed87.7TR 82.0 L&M 97.7 BG 83.4Not applicable87.8ECE 7.5 ↓79.6ECE 6.3 ↓87.0ECE 7.0 ↓82.894.399.583.4Not run–not timed yet
Gemma 4 31B reasoning off · chatGoogle31.3B82.1TR 66.4 L&M 96.7 BG 83.4Not applicable88.6ECE 8.1 ↓57.0ECE 31.0 ↓88.1ECE 8.5 ↓Not run91.799.283.4Not runBackgammon 8.5not timed yet
Qwen3.8-27B low · chatAlibaba27.8B80.2TR 68.3 L&M 96.8 BG 75.4Not applicable86.7ECE 10.7 ↓60.5ECE 29.4 ↓86.3ECE 11.0 ↓Not run92.699.075.4Not runBackgammon 19.0not timed yet
Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed74.1TR 53.8 L&M 97.0 BG 71.5Not applicable84.0ECE 5.7 ↓41.0ECE 38.5 ↓83.8ECE 5.5 ↓Not run92.499.471.5Not runBackgammon 2.7not timed yet
Claude Fable 5.2 max · chatAnthropicNot disclosedNot applicableNot applicable97.1ECE 8.2 ↓97.9ECE 7.2 ↓97.5ECE 10.7 ↓Not runNot runNot runNot runNot run–not timed yet
Claude Haiku 5.5 max · chatAnthropicNot disclosedNot applicableNot applicable83.6ECE 18.9 ↓30.4ECE 16.4 ↓90.9ECE 18.4 ↓Not runNot runNot runNot runNot run–not timed yet
DeepSeek-V4-Flash chatDeepSeek285B13B activeNot applicableNot applicableNot runNot runNot runNot runNot run99.1Not runNot run–not timed yet
DeepSeek-V4.1-Flash reasoning on · chat · first releaseDeepSeek552B16B activeNot applicableNot applicable91.8ECE 6.6 ↓89.3ECE 8.6 ↓Not runNot runNot runNot runNot runNot run–not timed yet
DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B activeNot applicableNot applicable92.6ECE 6.0 ↓88.5ECE 7.3 ↓90.6ECE 7.5 ↓Not run95.299.5Not runNot run–not timed yet
DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseDeepSeekNot disclosedNot applicableNot applicableNot runNot runNot runNot runNot runNot run83.9Not runBackgammon 8.2not timed yet
Erk-14B reasoning off · chateCloud Technology14.8BNot applicableNot applicable65.3ECE 6.7 ↓16.7ECE 16.3 ↓65.0ECE 10.9 ↓Not run42.888.4Not runNot run–not timed yet
GLM-5.3 low · chatZ.ai753.3BNot applicableNot applicable84.6ECE 8.6 ↓61.2ECE 5.5 ↓84.7ECE 3.6 ↓Not runNot runNot run73.8Not runBackgammon 17.6not timed yet
GLM-5.3-Flash reasoning on · chatZ.ai320B18B activeNot applicableNot applicable86.7ECE 5.5 ↓67.9ECE 4.4 ↓87.0ECE 7.0 ↓Not run91.498.5Not runNot run–not timed yet
GLiNER2.5 MultiFastino290MNot applicableNot applicable−3.5ECE 35.1 ↓−3.0ECE 35.8 ↓−4.7ECE 36.4 ↓Not runNot runNot runNot runNot run–not timed yet
GPT-6 Luna low · chatOpenAINot disclosedNot applicableNot applicable84.2ECE 9.0 ↓50.2ECE 32.7 ↓82.9ECE 10.1 ↓Not run47.492.1Not runNot run–not timed yet
GPT-6.1 Terra high · chatOpenAINot disclosedNot applicableNot applicable95.2ECE 1.3 ↓91.8ECE 3.5 ↓95.2ECE 1.3 ↓Not run92.499.0Not runNot run–not timed yet
Gemini 3.8 Flash high · chatGoogleNot disclosedNot applicableNot applicable96.4ECE 1.3 ↓98.1ECE 1.0 ↓95.0ECE 1.7 ↓Not runNot runNot run87.3Not runBackgammon 8.2not timed yet
Gemini 4 Argon low · chatGoogleNot disclosedNot applicableNot applicable96.2ECE 1.1 ↓95.1ECE 2.7 ↓95.0ECE 1.5 ↓Not runNot runNot run89.3Not runBackgammon 8.5not timed yet
Gemini 4 Carbon high · chatGoogleNot disclosedNot applicableNot applicable97.0ECE 0.7 ↓98.5ECE 0.3 ↓92.9ECE 3.1 ↓Not run95.999.5Not runNot run–not timed yet
Gemini 4 Hydrogen high · chatGoogleNot disclosedNot applicable93.5EN 95.9 Logic 95.1 Math 99.5 AC 83.796.6ECE 2.3 ↓98.1ECE 0.9 ↓95.9ECE 3.1 ↓Not run95.199.5Not run83.7–not timed yet
Gemma 4 12B Unified default reasoning · chatGoogle12BNot applicable85.9EN 83.5 Logic 84.7 Math 98.6 AC 77.0Not runNot run83.5Not run84.798.6Not run77.0–not timed yet
Gemma 4 12B Unified thinking on · chatGoogle11.9BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run67.7–not timed yet
Gemma 4 E2B reasoning on · chatGoogle5.1BNot applicable75.9EN 70.3 Logic 66.1 Math 96.0 AC 71.1Not runNot run70.3Not run66.196.0Not run71.1–not timed yet
Gemma 4 E2B default reasoning · chatGoogle5.1BNot applicableNot applicableNot runNot run68.2Not runNot runNot runNot run55.2–not timed yet
Gemma 4 E2B thinking off · chatGoogle5.1BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run54.8–not timed yet
Gemma 4 E4B default reasoning · chatGoogle8BNot applicable83.0EN 78.1 Logic 79.0 Math 97.6 AC 77.272.036.378.128.679.097.6Not run77.2–not timed yet
Gemma 4 E4B thinking on · sampled (temperature 1.0) · chatGoogle8BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run75.1–not timed yet
Granite-4.2-3B reasoning on · chatIBM3.7BNot applicable83.1EN 68.0 Logic 89.6 Math 99.0 AC 75.9Not runNot run68.0Not run89.699.0Not run75.9–not timed yet
Granite-4.2-3B reasoning off · chatIBM3.7BNot applicableNot applicableNot runNot run65.3Not runNot runNot runNot run52.8–not timed yet
Grok 4.7 high · chatxAINot disclosedNot applicableNot applicable93.7ECE 10.5 ↓89.7ECE 12.4 ↓91.8ECE 11.1 ↓Not runNot run99.484.0Not run–not timed yet
Hy4 flash high · chatTencent220.2B11.7B activeNot applicableNot applicable91.2ECE 4.0 ↓38.3ECE 3.0 ↓92.0ECE 3.9 ↓Not run94.3Not runNot runNot run–not timed yet
Inkling 2 Small high · chatThinking Machines594B17B activeNot applicable93.0EN 91.8 Logic 94.6 Math 99.4 AC 86.290.1ECE 8.7 ↓43.6ECE 7.8 ↓91.8ECE 6.8 ↓Not run94.699.4Not run86.2–not timed yet
Inkling-Small high · chat · Jul 2026Thinking Machines266B12B activeNot applicableNot applicable84.0ECE 3.2 ↓71.1ECE 9.9 ↓85.4ECE 2.6 ↓Not run94.299.5Not runNot run–not timed yet
Kimi K3.1 Pro low · chatMoonshot2.8T104B activeNot applicableNot applicable92.2ECE 4.3 ↓72.6ECE 16.2 ↓90.4ECE 5.7 ↓Not runNot runNot run76.2Not runBackgammon 4.8not timed yet
LFM2.5-2.6B default reasoning · chatLiquid AI2.7BNot applicableNot applicable31.811.267.3Not run69.292.5Not runNot run–not timed yet
LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B activeNot applicableNot applicableNot runNot run78.3Not run83.398.860.6Not run–not timed yet
LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B activeNot applicable66.6EN 63.0 Logic 50.2 Math 95.3 AC 57.77.0−1.263.0Not run50.295.3Not run57.7–not timed yet
LFM3-1.7B reasoning on · chatLiquid AI1.7BNot applicable80.1EN 63.9 Logic 81.9 Math 98.7 AC 75.830.39.263.9Not run81.998.7Not run75.8–not timed yet
LFM3-1.7B reasoning off · chatLiquid AI1.7BNot applicableNot applicable21.55.764.7Not runNot runNot runNot run74.9–not timed yet
MAI Code 2 Flash high · chatMicrosoftNot disclosedNot applicableNot applicable84.6ECE 2.7 ↓70.0ECE 6.9 ↓84.7ECE 4.5 ↓Not runNot runNot runNot runNot run–not timed yet
MiMo-V2.6-Flash high · chatXiaomi309B15B activeNot applicableNot applicable83.2ECE 3.7 ↓57.4ECE 10.3 ↓86.3ECE 6.5 ↓Not run93.999.3Not runNot run–not timed yet
MiniCPM5-2B reasoning off · chatOpenBMB2.5BNot applicable81.0EN 59.7 Logic 86.1 Math 99.0 AC 79.2−16.8−18.559.7Not run86.199.0Not run79.2–not timed yet
MiniCPM5-2B reasoning on · chatOpenBMB2.5BNot applicableNot applicableNot runNot run72.7Not runNot runNot runNot run67.0–not timed yet
MiniMax-M3.1-Flash high · chatMiniMax428B23B activeNot applicableNot applicable78.7ECE 9.5 ↓49.3ECE 26.1 ↓80.6ECE 9.0 ↓Not run92.497.8Not runNot run–not timed yet
Nemotron-Labs-Sigma-3B-instruct default reasoning · chatNVIDIA3.8BNot applicableNot applicableNot runNot run44.7Not runNot runNot runNot run4.3–not timed yet
Nemotron-Labs-Sigma-8B-instruct default reasoning · chatNVIDIA8.5BNot applicableNot applicableNot runNot run45.8Not runNot runNot runNot runNot run–not timed yet
Qwen3-14B reasoning off · chatAlibaba14.8BNot applicableNot applicable65.1ECE 23.2 ↓18.1ECE 48.6 ↓74.6ECE 17.8 ↓Not run56.794.7Not runNot run–not timed yet
Qwen3.5-2B reasoning off · chatAlibaba2.3BNot applicable69.7EN 59.1 Logic 56.0 Math 93.3 AC 70.317.92.059.1Not run56.093.3Not run70.3–not timed yet
Qwen3.5-2B reasoning on · chatAlibaba2.3BNot applicableNot applicableNot runNot run69.8Not runNot runNot runNot run50.2–not timed yet
Qwen3.5-397B-A17B high · chatAlibaba397B17B activeNot applicableNot applicable88.8ECE 2.2 ↓74.0ECE 9.1 ↓89.3ECE 3.3 ↓Not run93.9Not runNot runNot run–not timed yet
Qwen3.5-4B reasoning on · chatAlibaba4.7BNot applicable84.9EN 83.8 Logic 90.4 Math 99.3 AC 66.273.937.983.8Not run90.499.3Not run66.2–not timed yet
Qwen3.5-4B reasoning off · chatAlibaba4.7BNot applicableNot applicable60.620.975.5Not runNot runNot runNot run64.6–not timed yet
Qwen4-Max high · chatAlibabaNot disclosedNot applicableNot applicable93.3ECE 7.0 ↓88.1ECE 8.3 ↓93.1ECE 8.5 ↓Not run95.399.7Not runNot run–not timed yet
Raon-1 nano reasoning on · chatKRAFTON3.5BNot applicable67.1EN 61.2 Logic 56.3 Math 86.9 AC 63.8Not runNot run61.2Not run56.386.9Not run63.8–not timed yet
Raon-1 nano reasoning off · chatKRAFTON3.5BNot applicableNot applicableNot runNot run42.2Not runNot runNot runNot run40.6–not timed yet
SenseNova-Loop-9B on · chatno maker9BNot applicableNot applicableNot runNot runNot runNot run89.298.9Not runNot run–not timed yet
SubQ Savant 1.4 high · chatSubquadraticNot disclosedNot applicableNot applicable91.4ECE 6.5 ↓55.4ECE 6.8 ↓91.5ECE 7.5 ↓Not runNot runNot runNot runNot run–not timed yet

Size, calibration and latency against score

Size against score

Combined: 28 decision models; 10 left out (models whose maker discloses no size are left out)

0204060801000.10.3131030100Parameters, billions (log scale)Score 0–100, chance = 0 ↑KnowLine-31B-Gen1.5 · DGX Spark: 73.2, Parameters, billions (log scale) 31.3REAX 1.3: 71.3, Parameters, billions (log scale) 57.4Sieve 31B v3 · DGX Spark: 69.4, Parameters, billions (log scale) 31.3Turion Jev 31B · DGX Spark: 68.2, Parameters, billions (log scale) 31.3Mammoth-1: 64.8, Parameters, billions (log scale) 27.4Drex Max: 63.4, Parameters, billions (log scale) 26.1GLiDE · v1.2: 62.6, Parameters, billions (log scale) 26.1Perplexity Decider 27B v1.2 · DGX Spark: 61.2, Parameters, billions (log scale) 26.1Drex 2.0: 51.4, Parameters, billions (log scale) 35.1Clef: 50.0, Parameters, billions (log scale) 27.5Elephant-1: 49.7, Parameters, billions (log scale) 9.4Cerno 9B v1.2: 47.5, Parameters, billions (log scale) 8.9StartLux-Decision-v1.1-4B-Instant: 47.4, Parameters, billions (log scale) 4.7Cygnet: 47.0, Parameters, billions (log scale) 12Winnow-12B: 44.6, Parameters, billions (log scale) 11.9KnowLine 4B Gen5.5: 43.5, Parameters, billions (log scale) 4.7Open-Jev-27B-v1.1: 42.8, Parameters, billions (log scale) 27.8Perplexity Decider Flash v1.2: 41.4, Parameters, billions (log scale) 4.5StartLux-Decision-v1.1-2B-Instant: 32.1, Parameters, billions (log scale) 2.3KnowLine-2B-Gen1: 31.2, Parameters, billions (log scale) 2.3d1-2B: 30.5, Parameters, billions (log scale) 1.9GLiDE 2B: 27.5, Parameters, billions (log scale) 1.9StartLux-Decision-v1.1-0.8B-Instant: 22.4, Parameters, billions (log scale) 0.9KnowLine-0.8B-Gen1: 19.7, Parameters, billions (log scale) 0.9GLiDE 0.8B: 19.7, Parameters, billions (log scale) 0.8OneJev1.6-0.8B: 17.7, Parameters, billions (log scale) 0.8GLiDE 0.4B: 3.7, Parameters, billions (log scale) 0.4GLiClass v4 Edge: −0.8, Parameters, billions (log scale) 0.4Gemma 4 31B · reasoning off · chat: 82.1, Parameters, billions (log scale) 31.3Qwen3.8-27B · low · chat: 80.2, Parameters, billions (log scale) 27.8KnowLine-31B-Gen1.5 · DGX SparkREAX 1.3Turion Jev 31B · DGX SparkMammoth-1Perplexity Decider 27B v1.2 · DGX SparkDrex 2.0Elephant-1Winnow-12BStartLux-Decision-v1.1-2B-InstantGLiDE 2BStartLux-Decision-v1.1-0.8B-InstantOneJev1.6-0.8BGLiDE 0.4BGLiClass v4 EdgeGemma 4 31B · reasoning off · chat

Calibration error ↓ against score

Combined: 38 decision models

0204060801000510152025Calibration error, ECE in points: mean of the benches' calibration errorsScore 0–100, chance = 0 ↑KnowLine-31B-Gen1.5 · DGX Spark: 73.2, Calibration error, ECE in points: mean of the benches' calibration errors 3REAX 1.3: 71.3, Calibration error, ECE in points: mean of the benches' calibration errors 4.4Sieve 31B v3 · DGX Spark: 69.4, Calibration error, ECE in points: mean of the benches' calibration errors 2.7d1-ultra: 68.3, Calibration error, ECE in points: mean of the benches' calibration errors 2.8Turion Jev 31B · DGX Spark: 68.2, Calibration error, ECE in points: mean of the benches' calibration errors 7.3Gemini Decision 1 · 20261111: 68.0, Calibration error, ECE in points: mean of the benches' calibration errors 2Mammoth-1: 64.8, Calibration error, ECE in points: mean of the benches' calibration errors 4.4Drex Max: 63.4, Calibration error, ECE in points: mean of the benches' calibration errors 9.8Drex Max v1.1: 63.2, Calibration error, ECE in points: mean of the benches' calibration errors 4.3GLiDE · v1.2: 62.6, Calibration error, ECE in points: mean of the benches' calibration errors 4d1.2-ultra: 62.4, Calibration error, ECE in points: mean of the benches' calibration errors 3.3Perplexity Decider 27B v1.2 · DGX Spark: 61.2, Calibration error, ECE in points: mean of the benches' calibration errors 3.8SubQ Snap 1.1: 58.5, Calibration error, ECE in points: mean of the benches' calibration errors 10.8celeris-1-decision: 58.0, Calibration error, ECE in points: mean of the benches' calibration errors 5Jev 1.14: 57.1, Calibration error, ECE in points: mean of the benches' calibration errors 2.2d1.2: 56.3, Calibration error, ECE in points: mean of the benches' calibration errors 2.9Jev 1.13: 55.3, Calibration error, ECE in points: mean of the benches' calibration errors 4.2Drex 2.0: 51.4, Calibration error, ECE in points: mean of the benches' calibration errors 2.7Clef: 50.0, Calibration error, ECE in points: mean of the benches' calibration errors 11.6Elephant-1: 49.7, Calibration error, ECE in points: mean of the benches' calibration errors 8.7Mercury Decide: 48.8, Calibration error, ECE in points: mean of the benches' calibration errors 15.6Cerno 9B v1.2: 47.5, Calibration error, ECE in points: mean of the benches' calibration errors 13StartLux-Decision-v1.1-4B-Instant: 47.4, Calibration error, ECE in points: mean of the benches' calibration errors 6.2Cygnet: 47.0, Calibration error, ECE in points: mean of the benches' calibration errors 10.6Winnow-12B: 44.6, Calibration error, ECE in points: mean of the benches' calibration errors 15.1KnowLine 4B Gen5.5: 43.5, Calibration error, ECE in points: mean of the benches' calibration errors 11.9Open-Jev-27B-v1.1: 42.8, Calibration error, ECE in points: mean of the benches' calibration errors 21.4Perplexity Decider Flash v1.2: 41.4, Calibration error, ECE in points: mean of the benches' calibration errors 8.6StartLux-Decision-v1.1-2B-Instant: 32.1, Calibration error, ECE in points: mean of the benches' calibration errors 4.8KnowLine-2B-Gen1: 31.2, Calibration error, ECE in points: mean of the benches' calibration errors 8.7d1-2B: 30.5, Calibration error, ECE in points: mean of the benches' calibration errors 5.9GLiDE 2B: 27.5, Calibration error, ECE in points: mean of the benches' calibration errors 18.8StartLux-Decision-v1.1-0.8B-Instant: 22.4, Calibration error, ECE in points: mean of the benches' calibration errors 5.1KnowLine-0.8B-Gen1: 19.7, Calibration error, ECE in points: mean of the benches' calibration errors 9GLiDE 0.8B: 19.7, Calibration error, ECE in points: mean of the benches' calibration errors 21OneJev1.6-0.8B: 17.7, Calibration error, ECE in points: mean of the benches' calibration errors 9.9GLiDE 0.4B: 3.7, Calibration error, ECE in points: mean of the benches' calibration errors 9.5GLiClass v4 Edge: −0.8, Calibration error, ECE in points: mean of the benches' calibration errors 9.1KnowLine-31B-Gen1.5 · DGX Sparkd1-ultraTurion Jev 31B · DGX SparkDrex MaxDrex Max v1.1SubQ Snap 1.1celeris-1-decisionJev 1.14Drex 2.0ClefElephant-1Mercury DecideKnowLine 4B Gen5.5StartLux-Decision-v1.1-2B-InstantGLiDE 2BStartLux-Decision-v1.1-0.8B-InstantGLiDE 0.8BOneJev1.6-0.8BGLiDE 0.4BGLiClass v4 Edge

Latency ↓ against score

Combined: 11 decision models; 27 left out (latency: Reading · Turkish requests on spark2, a DGX Spark (GB10))

02040608010010301003001,0003,00010,000Median time per request, ms (log scale)Score 0–100, chance = 0 ↑KnowLine-31B-Gen1.5 · DGX Spark: 73.2, Median time per request, ms (log scale) 1,051KnowLine 4B Gen5.5: 43.5, Median time per request, ms (log scale) 99StartLux-Decision-v1.1-2B-Instant: 32.1, Median time per request, ms (log scale) 40KnowLine-2B-Gen1: 31.2, Median time per request, ms (log scale) 62GLiDE 2B: 27.5, Median time per request, ms (log scale) 34StartLux-Decision-v1.1-0.8B-Instant: 22.4, Median time per request, ms (log scale) 28KnowLine-0.8B-Gen1: 19.7, Median time per request, ms (log scale) 33GLiDE 0.8B: 19.7, Median time per request, ms (log scale) 22OneJev1.6-0.8B: 17.7, Median time per request, ms (log scale) 17GLiDE 0.4B: 3.7, Median time per request, ms (log scale) 16GLiClass v4 Edge: −0.8, Median time per request, ms (log scale) 25KnowLine-31B-Gen1.5 · DGX SparkKnowLine 4B Gen5.5StartLux-Decision-v1.1-2B-InstantGLiDE 2BStartLux-Decision-v1.1-0.8B-InstantOneJev1.6-0.8BGLiDE 0.4BGLiClass v4 Edge

Their score against ours

Jev Decision Index 0.4.1, Full score (6 November 2026, source): a sanity check, never our score; its 15 open decision models joined to ours by Hugging Face repo.

9 of the index's 15 models have a score here. Of their 36 pairs, 33 keep the index's order, 2 are reversed and 1 is tied on our score.

Index rankModelTheir score ↑Jev Decision Index 0.4.1, Full scoreOur score ↑Combined, score 0–100, chance = 0Our rankof these
1REAX 1.3decisionmodels/REAX-1.374.071.31
2GLiDE 27Bfastino/GLiDE-27B73.4not on this bench yet–
3KnowLine-31B-Gen1.5PelaAI/KnowLine-31B-Gen1.571.9not on this bench yet–
4Sieve 31B v3sthanika-ai/Sieve-31B-v371.1not on this bench yet–
5Turion Jev 31Bturion-ai/turion-jev-31b70.5not on this bench yet–
6d3-ultravllm-sr/d3-ultra70.2not on this bench yet–
7GLiDE 9Bfastino/GLiDE-9B62.5not on this bench yet–
8Elephant-1baselabs/elephant-160.349.72
9Cerno 9B v1.2apertixai/cerno-9b-v1.259.047.53
10KnowLine 4B Gen5.5PelaAI/KnowLine-4B-Gen5.556.043.55
11StartLux-Decision-v1.1-4B-Instantstartlux-models/StartLux-Decision-v1.1-4B-Instant54.847.44
12Perplexity Decider Flash v1.2perplexity-ai/pplx-decider-flash-v1.254.341.46
13GLiDE 0.8Bfastino/GLiDE-0.8B36.419.77
14GLiDE 0.4Bfastino/GLiDE-0.4B33.83.79
15OneJev1.6-0.8BOmniJev/OneJev1.6-0.8B29.617.78
ref.Jev 1.14reference, hostedno index score here57.1–

All benches

Turkish Reading

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B91.893.4%656 questions1.2d69cdd5
2KnowLine-31B-Gen2PelaAI31.3B91.8tied with Karpo Decision 31B93.4%656 questions6.3fd99a9e
3Copernicus-31BPoindexter Labs31.3B91.4tied with KnowLine-31B-Gen293.1%656 questions6.3be71525
4KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B91.0tied with Copernicus-31B92.8%656 questions3.48e56aa1
5Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B90.9tied with KnowLine-31B-Gen1.5 · DGX Spark92.7%656 questions9.5866c0fc
6REAX 1.3 blendREAX57.4B90.5tied with Turion Jev 31B · DGX Spark92.4%656 questions2.1fa4bf7e
7Gemini Decision 1 20261111GoogleNot disclosed90.1tied with REAX 1.392.1%656 questions2.8no repo
8d1-ultraLiquid AINot disclosed90.1tied with Gemini Decision 1 · 2026111192.1%656 questions3.6no repo
9Logits Sigma-1 Duo blendLogits57.4B89.9tied with d1-ultra91.9%656 questions5.3a0c1b6e
10Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B89.7tied with Logits Sigma-1 Duo91.8%656 questions3.58fcee85
11GLiDE 27B DGX SparkFastino26.1B88.6tied with Sieve 31B v3 · DGX Spark90.9%656 questions2.44d32743
12Mammoth-1 post-trainedBase Labs27.4B88.0tied with GLiDE 27B · DGX Spark90.4%656 questions2.061cec44
13d1.2-ultraLiquid AINot disclosed87.2tied with Mammoth-189.8%656 questions5.6no repo
14d3-ultra fine-tunevLLM Semantic Router26.1B87.2tied with d1.2-ultra89.8%656 questions3.852ba9e7
15Drex Max fine-tuneNace.AI26.1B87.0tied with d3-ultra89.6%656 questions3.4cac65e1
16GLiDE v1.2Fastino26.1B87.0tied with Drex Max89.6%656 questions3.5a7e4589
17Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B86.5tied with GLiDE · v1.289.2%656 questions3.1225c913
18d3 fine-tunevLLM Semantic Router26.1B86.5tied with Perplexity Decider 27B v1.2 · DGX Spark89.2%656 questions3.520f37b9
19Perplexity Decider 27B v1.1Perplexity26.1B86.3tied with d389.0%656 questions3.0no repo
20Perplexity Decider 27B v1.3Perplexity26.1B86.3tied with Perplexity Decider 27B v1.189.0%656 questions2.49c2ebb2
21Perplexity Decider 27B v1.2 API · with webPerplexity26.1B86.1tied with Perplexity Decider 27B v1.388.9%656 questions2.7no repo
22Microsoft-Decision-1.1MicrosoftNot disclosed85.9tied with Perplexity Decider 27B v1.2 · API · with web88.7%656 questions9.6no repo
23Drex Max v1.1Nace.AINot disclosed85.7tied with Microsoft-Decision-1.188.6%656 questions3.4no repo
24GLiDE APIFastinoNot disclosed84.8tied with Drex Max v1.187.8%656 questions5.8no repo
25SubQ Snap PreviewSubquadraticNot disclosed84.6tied with GLiDE · API87.7%656 questions4.2no repo
26SubQ Snap 1.2SubquadraticNot disclosed84.4tied with SubQ Snap Preview87.5%656 questions1.4no repo
27SubQ Snap 1.1SubquadraticNot disclosed83.8tied with SubQ Snap 1.287.0%656 questions3.6no repo
28KnowLine-12B-Gen1PelaAI12B82.5tied with SubQ Snap 1.186.0%656 questions11.4f26fbc2
29Perplexity Decider 27B v1Perplexity26.1B82.5tied with KnowLine-12B-Gen186.0%656 questions5.0no repo
30d1.2Liquid AINot disclosed82.5tied with Perplexity Decider 27B v186.0%656 questions6.0no repo
31GPT-6 Luna Decisions APIOpenAINot disclosed81.9tied with d1.285.5%656 questions3.8no repo
32OneJev-27B full fine-tuneOmniJev27.4B81.5tied with GPT-6 Luna · Decisions API85.2%656 questions9.9a5e5f55
33celeris-1-decisionCelerisNot disclosed81.5tied with OneJev-27B85.2%656 questions6.2no repo
34Clef post-trainedCloudflare27.5B81.3tied with celeris-1-decision85.1%656 questions4.02f3de3d
35Jev 1.14TypeSafeNot disclosed81.3tied with Clef85.1%656 questions2.2no repo
36d1.1Liquid AINot disclosed80.8tied with Jev 1.1484.6%656 questions4.2no repo
37Mercury DecideInceptionNot disclosed80.6tied with d1.184.5%656 questions7.5no repo
38Jev 1.13TypeSafeNot disclosed79.6tied with Mercury Decide83.7%656 questions3.8no repo
39Jeffan-9B-1211tattrongvu8.9B78.7tied with Jev 1.1382.9%656 questions11.4862d1a7
40Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B78.7tied with Jeffan-9B-121182.9%656 questions43.628cf730
41Cygnet unchanged baseBlockbrain Labs12B78.5tied with Open-Jev-27B-v1.182.8%656 questions4.1no repo
42d1Liquid AINot disclosed78.5tied with Cygnet82.8%656 questions2.8no repo
43Winnow-12B LoRA, mergedEldanRing11.9B77.5tied with d182.0%656 questions9.3b2b1421
44Drex 2.0Nace.AI35.1B3B active76.8tied with Winnow-12B81.4%656 questions10.1no repo
45Adjudex 9B v5newsbang8.4B76.0tied with Drex 2.080.8%656 questions2.5ffdcb82
46Elephant-1 post-trainedBase Labs9.4B75.6tied with Adjudex 9B v580.5%656 questions4.3b3f8a93
47GLiDE 9BFastino7.9B75.2tied with Elephant-180.2%656 questions3.7ab54df5
48KnowLine-9B-Gen1PelaAI9.7B74.8tied with GLiDE 9B79.9%656 questions1.7325378d
49Cerno 9B v1.2 full fine-tuneApertix AI8.9B74.7tied with KnowLine-9B-Gen179.7%656 questions3.0f02a632
50Dexy-Decision-9B-V2 fine-tunePatrick-188.4B74.5tied with Cerno 9B v1.279.6%656 questions3.24f434e2
51KnowLine-9B-Gen2PelaAI9.7B74.5tied with Dexy-Decision-9B-V279.6%656 questions8.28ec8eb6
52StartLux-Decision-v1.1-9B-InstantStartLux9.7B73.5tied with KnowLine-9B-Gen278.8%656 questions3.618a17f7
53vera-medium-omniscar-ai4.2B69.375.5%656 questions9.11f23a6c
54Valen-Decision-1.5-9BValen-Team8.4B68.8tied with vera-medium-omni75.0%656 questions9.75a51211
55KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B68.6tied with Valen-Decision-1.5-9B74.8%656 questions6.2779944d
56Perplexity Decider Flash v1.3Perplexity4.5B68.0tied with KnowLine 4B Gen5.574.4%656 questions6.2d06fb3f
57StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B67.6tied with Perplexity Decider Flash v1.374.1%656 questions2.6416088f
58Microsoft-Decision-1MicrosoftNot disclosed66.8tied with StartLux-Decision-v1.1-4B-Instant73.5%656 questions8.9no repo
59Clef-Flash post-trainedCloudflare9.5B66.3tied with Microsoft-Decision-173.0%656 questions5.6fde727a
60Perplexity Decider Flash v1.2Perplexity4.5B65.9tied with Clef-Flash72.7%656 questions3.76d771ad
61Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B64.0tied with Perplexity Decider Flash v1.271.2%656 questions3.945830d2
62Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B63.6tied with Verdin-4B-v1.170.9%656 questions3.3889f23e
63Metask-Jev-4B LoRA, mergedMetask Lab4.5B53.562.8%656 questions5.3ea20fe8
64KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B42.854.3%656 questions1.953245d4
65StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B41.1tied with KnowLine-2B-Gen152.9%656 questions3.3f1e6cd1
66GLiDE 2BFastino1.9B39.0tied with StartLux-Decision-v1.1-2B-Instant51.2%656 questions16.500f122c
67GLiDE 0.8BFastino750M28.042.4%656 questions21.89eacdee
68d1-2BLiquid AI1.9B26.1tied with GLiDE 0.8B40.9%656 questions4.410e7671
69OneJev1.6-0.8B full fine-tuneOmniJev850M24.5tied with d1-2B39.6%656 questions7.3bcd88a3
70KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M22.8tied with OneJev1.6-0.8B38.3%656 questions5.56b27831
71Strands Decider 2B LoRA + headStrands Agents2.3B21.1tied with KnowLine-0.8B-Gen136.9%656 questions10.22b52a62
72StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M18.8tied with Strands Decider 2B35.1%656 questions5.06496ae6
73Decider 1meraGPTNot disclosed18.3tied with StartLux-Decision-v1.1-0.8B-Instant34.6%656 questions11.8no repo
74GLiClass v4 Edge fine-tuneKnowledgator440M7.425.9%656 questions11.9885af1a
75GLiDE 0.4BFastino400M5.7tied with GLiClass v4 Edge24.5%656 questions7.26bda7ab
76GLiNER2.5-multi-Decide fine-tuneFastino290M1.321.0%656 questions24.1a35a0cd
77LayaConvai Innovations420M1.1tied with GLiNER2.5-multi-Decide20.9%656 questions9.2c5d7873
78Laya MultilingualConvai Innovations320M−3.717.1%656 questions33.1c5d7873

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Claude Fable 5.2 max · chatAnthropicNot disclosed97.197.7%656 questions8.2no repo
2Gemini 4 Carbon high · chatGoogleNot disclosed97.0tied with Claude Fable 5.2 · max · chat97.6%656 questions0.7no repo
3Gemini 4 Hydrogen high · chatGoogleNot disclosed96.6tied with Gemini 4 Carbon · high · chat97.3%656 questions2.3no repo
4GPT-6 Astra high · chatOpenAINot disclosed96.4tied with Gemini 4 Hydrogen · high · chat97.1%656 questions1.0no repo
5Gemini 3.8 Flash high · chatGoogleNot disclosed96.4tied with GPT-6 Astra · high · chat97.1%656 questions1.3no repo
6Gemini 4 Argon low · chatGoogleNot disclosed96.2tied with Gemini 3.8 Flash · high · chat97.0%656 questions1.1no repo
7GPT-6.1 Sol high · chatOpenAINot disclosed95.6tied with Gemini 4 Argon · low · chat96.5%656 questions1.8no repo
8GPT-6.1 Terra high · chatOpenAINot disclosed95.2tied with GPT-6.1 Sol · high · chat96.2%656 questions1.3no repo
9Claude Fable 5.1 high · chatAnthropicNot disclosed94.5tied with GPT-6.1 Terra · high · chat95.6%656 questions10.9no repo
10Claude Opus 5.5 high · chatAnthropicNot disclosed94.5tied with Claude Fable 5.1 · high · chat95.6%656 questions14.3no repo
11Grok 4.7 high · chatxAINot disclosed93.7tied with Claude Opus 5.5 · high · chat95.0%656 questions10.5no repo
12Qwen4-Max high · chatAlibabaNot disclosed93.3tied with Grok 4.7 · high · chat94.7%656 questions7.0no repo
13DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active92.6tied with Qwen4-Max · high · chat94.1%656 questions6.0no repo
14Kimi K3.1 Pro low · chatMoonshot2.8T104B active92.2tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101793.8%656 questions4.3no repo
15DeepSeek-V4.1-Flash reasoning on · chat · first releaseDeepSeek552B16B active91.8tied with Kimi K3.1 Pro · low · chat93.4%656 questions6.6no repo
16SubQ Savant 1.4 high · chatSubquadraticNot disclosed91.4tied with DeepSeek-V4.1-Flash · reasoning on · chat · first release93.1%656 questions6.5no repo
17Hy4 flash high · chatTencent220.2B11.7B active91.2tied with SubQ Savant 1.4 · high · chat93.0%656 questions4.0no repo
18Inkling 2 Small high · chatThinking Machines594B17B active90.1tied with Hy4 flash · high · chat92.1%656 questions8.7no repo
19Qwen3.5-397B-A17B high · chatAlibaba397B17B active88.8tied with Inkling 2 Small · high · chat91.0%656 questions2.2no repo
20Gemma 4 31B reasoning off · chatGoogle31.3B88.6tied with Qwen3.5-397B-A17B · high · chat90.9%656 questions8.1842da37
21Muse Ember max · chatMetaNot disclosed87.8tied with Gemma 4 31B · reasoning off · chat90.2%656 questions7.5no repo
22GLM-5.3-Flash reasoning on · chatZ.ai320B18B active86.7tied with Muse Ember · max · chat89.3%656 questions5.5no repo
23Qwen3.8-27B low · chatAlibaba27.8B86.7tied with GLM-5.3-Flash · reasoning on · chat89.3%656 questions10.71d4bf0f
24GLM-5.3 low · chatZ.ai753.3B84.6tied with Qwen3.8-27B · low · chat87.7%656 questions8.6no repo
25MAI Code 2 Flash high · chatMicrosoftNot disclosed84.6tied with GLM-5.3 · low · chat87.7%656 questions2.7no repo
26GPT-6 Luna low · chatOpenAINot disclosed84.2tied with MAI Code 2 Flash · high · chat87.3%656 questions9.0no repo
27Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active84.0tied with GPT-6 Luna · low · chat87.2%656 questions3.2no repo
28Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed84.0tied with Inkling-Small · high · chat · Jul 202687.2%656 questions5.7no repo
29Claude Haiku 5.5 max · chatAnthropicNot disclosed83.6tied with Qwen3.8 Flash · no reasoning · chat86.9%656 questions18.9no repo
30MiMo-V2.6-Flash high · chatXiaomi309B15B active83.2tied with Claude Haiku 5.5 · max · chat86.6%656 questions3.7no repo
31Claude Sonnet 5.5 high · chatAnthropicNot disclosed81.5tied with MiMo-V2.6-Flash · high · chat85.2%656 questions13.8no repo
32MiniMax-M3.1-Flash high · chatMiniMax428B23B active78.7tied with Claude Sonnet 5.5 · high · chat82.9%656 questions9.5no repo
33Qwen3.5-4B reasoning on · chatAlibaba4.7B73.979.1%656 questions–851bf6e
34Gemma 4 E4B default reasoning · chatGoogle8B72.0tied with Qwen3.5-4B · reasoning on · chat77.6%656 questions–4ecc2ba
35Erk-14B reasoning off · chateCloud Technology14.8B65.372.3%656 questions6.78584b14
36Qwen3-14B reasoning off · chatAlibaba14.8B65.1tied with Erk-14B · reasoning off · chat72.1%656 questions23.240c0698
37Qwen3.5-4B reasoning off · chatAlibaba4.7B60.668.4%656 questions–851bf6e
38LFM2.5-2.6B default reasoning · chatLiquid AI2.7B31.845.4%656 questions–7141b56
39LFM3-1.7B reasoning on · chatLiquid AI1.7B30.3tied with LFM2.5-2.6B · default reasoning · chat44.2%656 questions–0dc4268
40LFM3-1.7B reasoning off · chatLiquid AI1.7B21.537.2%656 questions–0dc4268
41Qwen3.5-2B reasoning off · chatAlibaba2.3B17.9tied with LFM3-1.7B · reasoning off · chat34.3%656 questions–15852e8
42LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active7.025.6%656 questions–dd83f5c
43GLiNER2.5 MultiFastino290M−3.517.2%656 questions35.12ca71aa
44MiniCPM5-2B reasoning off · chatOpenBMB2.5B−16.86.6%656 questions–42433e9

656 questions per model · updated 19 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 656 Turkish reading questions (units 1 to 6) right (accuracy 100%) · feed: bench-scores.json

Turkish Grammar

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B71.477.1%1,542 questions3.98e56aa1
2Copernicus-31BPoindexter Labs31.3B71.2tied with KnowLine-31B-Gen1.5 · DGX Spark77.0%1,542 questions10.5be71525
3KnowLine-31B-Gen2PelaAI31.3B70.0tied with Copernicus-31B76.0%1,542 questions5.7fd99a9e
4Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B68.5tied with KnowLine-31B-Gen274.8%1,542 questions12.0866c0fc
5REAX 1.3 blendREAX57.4B64.771.8%1,542 questions5.1fa4bf7e
6Gemini Decision 1 20261111GoogleNot disclosed64.2tied with REAX 1.371.3%1,542 questions2.9no repo
7Karpo Decision 31BMachinePulse31.3B63.9tied with Gemini Decision 1 · 2026111171.1%1,542 questions3.4d69cdd5
8d1-ultraLiquid AINot disclosed63.4tied with Karpo Decision 31B70.7%1,542 questions4.2no repo
9Logits Sigma-1 Duo blendLogits57.4B59.867.8%1,542 questions5.9a0c1b6e
10Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B58.3tied with Logits Sigma-1 Duo66.7%1,542 questions3.38fcee85
11GLiDE 27B DGX SparkFastino26.1B53.462.7%1,542 questions2.74d32743
12Perplexity Decider 27B v1.2 API · with webPerplexity26.1B50.260.2%1,542 questions4.9no repo
13Perplexity Decider 27B v1.3Perplexity26.1B49.6tied with Perplexity Decider 27B v1.2 · API · with web59.7%1,542 questions6.19c2ebb2
14GLiDE v1.2Fastino26.1B49.3tied with Perplexity Decider 27B v1.359.5%1,542 questions3.0a7e4589
15GLiDE APIFastinoNot disclosed49.3tied with GLiDE · v1.259.4%1,542 questions4.1no repo
16d1.2-ultraLiquid AINot disclosed48.6tied with GLiDE · API58.9%1,542 questions3.1no repo
17d3 fine-tunevLLM Semantic Router26.1B48.4tied with d1.2-ultra58.7%1,542 questions2.420f37b9
18Drex Max fine-tuneNace.AI26.1B48.0tied with d358.4%1,542 questions8.0cac65e1
19d3-ultra fine-tunevLLM Semantic Router26.1B48.0tied with Drex Max58.4%1,542 questions16.352ba9e7
20Drex Max v1.1Nace.AINot disclosed48.0tied with d3-ultra58.4%1,542 questions2.7no repo
21Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B47.1tied with Drex Max v1.157.7%1,542 questions6.2225c913
22Perplexity Decider 27B v1.1Perplexity26.1B46.8tied with Perplexity Decider 27B v1.2 · DGX Spark57.5%1,542 questions5.5no repo
23SubQ Snap 1.2SubquadraticNot disclosed46.6tied with Perplexity Decider 27B v1.157.3%1,542 questions9.0no repo
24Mammoth-1 post-trainedBase Labs27.4B46.5tied with SubQ Snap 1.257.2%1,542 questions7.961cec44
25Microsoft-Decision-1.1MicrosoftNot disclosed45.8tied with Mammoth-156.7%1,542 questions4.5no repo
26SubQ Snap PreviewSubquadraticNot disclosed44.6tied with Microsoft-Decision-1.155.7%1,542 questions6.0no repo
27GPT-6 Luna Decisions APIOpenAINot disclosed42.9tied with SubQ Snap Preview54.3%1,542 questions3.0no repo
28SubQ Snap 1.1SubquadraticNot disclosed42.5tied with GPT-6 Luna · Decisions API54.0%1,542 questions14.5no repo
29KnowLine-12B-Gen1PelaAI12B40.6tied with SubQ Snap 1.152.5%1,542 questions4.3f26fbc2
30d1.1Liquid AINot disclosed39.6tied with KnowLine-12B-Gen151.7%1,542 questions2.2no repo
31d1.2Liquid AINot disclosed39.3tied with d1.151.4%1,542 questions2.5no repo
32celeris-1-decisionCelerisNot disclosed37.4tied with d1.249.9%1,542 questions5.3no repo
33Perplexity Decider 27B v1Perplexity26.1B36.9tied with celeris-1-decision49.5%1,542 questions5.0no repo
34OneJev-27B full fine-tuneOmniJev27.4B35.9tied with Perplexity Decider 27B v148.7%1,542 questions3.7a5e5f55
35Jev 1.14TypeSafeNot disclosed35.2tied with OneJev-27B48.2%1,542 questions4.0no repo
36Mercury DecideInceptionNot disclosed33.9tied with Jev 1.1447.1%1,542 questions19.0no repo
37Jev 1.13TypeSafeNot disclosed33.0tied with Mercury Decide46.4%1,542 questions6.5no repo
38d1Liquid AINot disclosed32.6tied with Jev 1.1346.0%1,542 questions13.5no repo
39Jeffan-9B-1211tattrongvu8.9B32.5tied with d146.0%1,542 questions3.0862d1a7
40Clef post-trainedCloudflare27.5B31.8tied with Jeffan-9B-121145.5%1,542 questions15.12f3de3d
41Drex 2.0Nace.AI35.1B3B active31.0tied with Clef44.8%1,542 questions2.0no repo
42Cygnet unchanged baseBlockbrain Labs12B30.2tied with Drex 2.044.2%1,542 questions18.9no repo
43Winnow-12B LoRA, mergedEldanRing11.9B30.2tied with Cygnet44.2%1,542 questions24.1b2b1421
44Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B30.1tied with Winnow-12B44.1%1,542 questions16.428cf730
45KnowLine-9B-Gen1PelaAI9.7B29.7tied with Open-Jev-27B-v1.143.8%1,542 questions6.8325378d
46Cerno 9B v1.2 full fine-tuneApertix AI8.9B29.6tied with KnowLine-9B-Gen143.7%1,542 questions14.2f02a632
47KnowLine-9B-Gen2PelaAI9.7B27.6tied with Cerno 9B v1.242.1%1,542 questions5.18ec8eb6
48Adjudex 9B v5newsbang8.4B26.7tied with KnowLine-9B-Gen241.4%1,542 questions12.2ffdcb82
49Elephant-1 post-trainedBase Labs9.4B25.5tied with Adjudex 9B v540.4%1,542 questions15.4b3f8a93
50GLiDE 9BFastino7.9B25.1tied with Elephant-140.1%1,542 questions12.0ab54df5
51Dexy-Decision-9B-V2 fine-tunePatrick-188.4B24.6tied with GLiDE 9B39.7%1,542 questions12.74f434e2
52StartLux-Decision-v1.1-9B-InstantStartLux9.7B24.4tied with Dexy-Decision-9B-V239.5%1,542 questions11.318a17f7
53Perplexity Decider Flash v1.3Perplexity4.5B24.0tied with StartLux-Decision-v1.1-9B-Instant39.2%1,542 questions15.3d06fb3f
54vera-medium-omniscar-ai4.2B22.3tied with Perplexity Decider Flash v1.337.9%1,542 questions8.01f23a6c
55KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B21.6tied with vera-medium-omni37.3%1,542 questions16.6779944d
56Perplexity Decider Flash v1.2Perplexity4.5B21.4tied with KnowLine 4B Gen5.537.2%1,542 questions11.06d771ad
57StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B20.9tied with Perplexity Decider Flash v1.236.7%1,542 questions10.3416088f
58Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B20.5tied with StartLux-Decision-v1.1-4B-Instant36.4%1,542 questions15.4889f23e
59Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B20.5tied with Chinese Decision Pointer 4B36.4%1,542 questions12.845830d2
60Valen-Decision-1.5-9BValen-Team8.4B18.8tied with Verdin-4B-v1.135.0%1,542 questions6.95a51211
61Microsoft-Decision-1MicrosoftNot disclosed18.7tied with Valen-Decision-1.5-9B35.0%1,542 questions24.4no repo
62Clef-Flash post-trainedCloudflare9.5B17.6tied with Microsoft-Decision-134.1%1,542 questions20.3fde727a
63Metask-Jev-4B LoRA, mergedMetask Lab4.5B14.231.4%1,542 questions6.5ea20fe8
64KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B11.7tied with Metask-Jev-4B29.4%1,542 questions11.453245d4
65GLiDE 2BFastino1.9B10.7tied with KnowLine-2B-Gen128.5%1,542 questions22.500f122c
66StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B7.926.3%1,542 questions9.4f1e6cd1
67StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M7.3tied with StartLux-Decision-v1.1-2B-Instant25.8%1,542 questions4.66496ae6
68OneJev1.6-0.8B full fine-tuneOmniJev850M5.8tied with StartLux-Decision-v1.1-0.8B-Instant24.6%1,542 questions10.3bcd88a3
69d1-2BLiquid AI1.9B5.6tied with OneJev1.6-0.8B24.5%1,542 questions6.710e7671
70Decider 1meraGPTNot disclosed4.9tied with d1-2B23.9%1,542 questions15.4no repo
71GLiDE 0.8BFastino750M4.6tied with Decider 123.7%1,542 questions26.19eacdee
72Strands Decider 2B LoRA + headStrands Agents2.3B4.3tied with GLiDE 0.8B23.5%1,542 questions13.22b52a62
73KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M3.0tied with Strands Decider 2B22.4%1,542 questions10.86b27831
74GLiClass v4 Edge fine-tuneKnowledgator440M2.8tied with KnowLine-0.8B-Gen122.2%1,542 questions10.0885af1a
75GLiDE 0.4BFastino400M1.6tied with GLiClass v4 Edge21.3%1,542 questions8.36bda7ab
76GLiNER2.5-multi-Decide fine-tuneFastino290M−0.2tied with GLiDE 0.4B19.8%1,542 questions22.7a35a0cd
77LayaConvai Innovations420M−2.2tied with GLiNER2.5-multi-Decide18.2%1,542 questions14.9c5d7873
78Laya MultilingualConvai Innovations320M−2.2tied with Laya18.2%1,542 questions26.0c5d7873

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Gemini 4 Carbon high · chatGoogleNot disclosed98.598.8%1,542 questions0.3no repo
2Gemini 3.8 Flash high · chatGoogleNot disclosed98.1tied with Gemini 4 Carbon · high · chat98.5%1,542 questions1.0no repo
3Gemini 4 Hydrogen high · chatGoogleNot disclosed98.1tied with Gemini 3.8 Flash · high · chat98.4%1,542 questions0.9no repo
4Claude Fable 5.2 max · chatAnthropicNot disclosed97.9tied with Gemini 4 Hydrogen · high · chat98.3%1,542 questions7.2no repo
5GPT-6 Astra high · chatOpenAINot disclosed97.9tied with Claude Fable 5.2 · max · chat98.3%1,542 questions0.7no repo
6GPT-6.1 Sol high · chatOpenAINot disclosed97.9tied with GPT-6 Astra · high · chat98.3%1,542 questions0.6no repo
7Claude Fable 5.1 high · chatAnthropicNot disclosed95.5tied with GPT-6.1 Sol · high · chat96.4%1,542 questions12.5no repo
8Gemini 4 Argon low · chatGoogleNot disclosed95.1tied with Claude Fable 5.1 · high · chat96.1%1,542 questions2.7no repo
9Claude Opus 5.5 high · chatAnthropicNot disclosed93.4tied with Gemini 4 Argon · low · chat94.7%1,542 questions14.9no repo
10GPT-6.1 Terra high · chatOpenAINot disclosed91.8tied with Claude Opus 5.5 · high · chat93.5%1,542 questions3.5no repo
11Grok 4.7 high · chatxAINot disclosed89.7tied with GPT-6.1 Terra · high · chat91.8%1,542 questions12.4no repo
12DeepSeek-V4.1-Flash reasoning on · chat · first releaseDeepSeek552B16B active89.3tied with Grok 4.7 · high · chat91.4%1,542 questions8.6no repo
13DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active88.5tied with DeepSeek-V4.1-Flash · reasoning on · chat · first release90.8%1,542 questions7.3no repo
14Qwen4-Max high · chatAlibabaNot disclosed88.1tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101790.5%1,542 questions8.3no repo
15Claude Sonnet 5.5 high · chatAnthropicNot disclosed81.285.0%1,542 questions16.9no repo
16Muse Ember max · chatMetaNot disclosed79.6tied with Claude Sonnet 5.5 · high · chat83.7%1,542 questions6.3no repo
17Qwen3.5-397B-A17B high · chatAlibaba397B17B active74.079.2%1,542 questions9.1no repo
18Kimi K3.1 Pro low · chatMoonshot2.8T104B active72.6tied with Qwen3.5-397B-A17B · high · chat78.1%1,542 questions16.2no repo
19Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active71.1tied with Kimi K3.1 Pro · low · chat76.9%1,542 questions9.9no repo
20MAI Code 2 Flash high · chatMicrosoftNot disclosed70.0tied with Inkling-Small · high · chat · Jul 202676.0%1,542 questions6.9no repo
21GLM-5.3-Flash reasoning on · chatZ.ai320B18B active67.9tied with MAI Code 2 Flash · high · chat74.3%1,542 questions4.4no repo
22GLM-5.3 low · chatZ.ai753.3B61.268.9%1,542 questions5.5no repo
23Qwen3.8-27B low · chatAlibaba27.8B60.5tied with GLM-5.3 · low · chat68.4%1,542 questions29.41d4bf0f
24MiMo-V2.6-Flash high · chatXiaomi309B15B active57.466.0%1,542 questions10.3no repo
25Gemma 4 31B reasoning off · chatGoogle31.3B57.0tied with MiMo-V2.6-Flash · high · chat65.6%1,542 questions31.0842da37
26SubQ Savant 1.4 high · chatSubquadraticNot disclosed55.4tied with Gemma 4 31B · reasoning off · chat64.3%1,542 questions6.8no repo
27GPT-6 Luna low · chatOpenAINot disclosed50.260.2%1,542 questions32.7no repo
28MiniMax-M3.1-Flash high · chatMiniMax428B23B active49.3tied with GPT-6 Luna · low · chat59.5%1,542 questions26.1no repo
29Inkling 2 Small high · chatThinking Machines594B17B active43.654.9%1,542 questions7.8no repo
30Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed41.052.8%1,542 questions38.5no repo
31Hy4 flash high · chatTencent220.2B11.7B active38.350.6%1,542 questions3.0no repo
32Qwen3.5-4B reasoning on · chatAlibaba4.7B37.9tied with Hy4 flash · high · chat50.3%1,542 questions–851bf6e
33Gemma 4 E4B default reasoning · chatGoogle8B36.3tied with Qwen3.5-4B · reasoning on · chat49.0%1,542 questions–4ecc2ba
34Claude Haiku 5.5 max · chatAnthropicNot disclosed30.444.4%1,542 questions16.4no repo
35Qwen3.5-4B reasoning off · chatAlibaba4.7B20.936.7%1,542 questions–851bf6e
36Qwen3-14B reasoning off · chatAlibaba14.8B18.134.5%1,542 questions48.640c0698
37Erk-14B reasoning off · chateCloud Technology14.8B16.7tied with Qwen3-14B · reasoning off · chat33.4%1,542 questions16.38584b14
38LFM2.5-2.6B default reasoning · chatLiquid AI2.7B11.229.0%1,542 questions–7141b56
39LFM3-1.7B reasoning on · chatLiquid AI1.7B9.2tied with LFM2.5-2.6B · default reasoning · chat27.4%1,542 questions–0dc4268
40LFM3-1.7B reasoning off · chatLiquid AI1.7B5.724.6%1,542 questions–0dc4268
41Qwen3.5-2B reasoning off · chatAlibaba2.3B2.021.6%1,542 questions–15852e8
42LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active−1.219.0%1,542 questions–dd83f5c
43GLiNER2.5 MultiFastino290M−3.0tied with LFM2.5-8B-A1B · default reasoning · chat17.6%1,542 questions35.82ca71aa
44MiniCPM5-2B reasoning off · chatOpenBMB2.5B−18.55.2%1,542 questions–42433e9

1,542 questions per model · updated 19 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 1,542 Turkish grammar questions (units 7 to 20) right (accuracy 100%) · feed: bench-scores.json

Reading · English

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B92.093.6%547 questions5.08e56aa1
2KnowLine-31B-Gen1.5 RTX PRO 6000 LoRA, mergedPelaAI31.3B91.8tied with KnowLine-31B-Gen1.5 · DGX Spark93.4%547 questions5.38e56aa1
3Copernicus-31BPoindexter Labs31.3B91.1tied with KnowLine-31B-Gen1.5 · RTX PRO 600092.9%547 questions7.8be71525
4REAX 1.3 blendREAX57.4B90.6tied with Copernicus-31B92.5%547 questions3.6fa4bf7e
5KnowLine-31B-Gen2PelaAI31.3B90.4tied with REAX 1.392.3%547 questions7.7fd99a9e
6Karpo Decision 31BMachinePulse31.3B90.2tied with KnowLine-31B-Gen292.1%547 questions2.4d69cdd5
7Gemini Decision 1 20261111GoogleNot disclosed88.8tied with Karpo Decision 31B91.0%547 questions4.5no repo
8Logits Sigma-1 Duo blendLogits57.4B88.8tied with Gemini Decision 1 · 2026111191.0%547 questions8.0a0c1b6e
9Sieve 31B v3 RTX PRO 6000 full fine-tuneSthānika AI31.3B88.8tied with Logits Sigma-1 Duo91.0%547 questions4.78fcee85
10Turion Jev 31B RTX PRO 6000 LoRA, mergedTurion.AI31.3B88.8tied with Sieve 31B v3 · RTX PRO 600091.0%547 questions11.6866c0fc
11GLiDE 27B DGX SparkFastino26.1B88.6tied with Turion Jev 31B · RTX PRO 600090.9%547 questions1.74d32743
12Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B88.3tied with GLiDE 27B · DGX Spark90.7%547 questions4.38fcee85
13Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B88.3tied with Sieve 31B v3 · DGX Spark90.7%547 questions11.6866c0fc
14d1-ultraLiquid AINot disclosed88.3tied with Turion Jev 31B · DGX Spark90.7%547 questions4.4no repo
15Perplexity Decider 27B v1.2 API · with webPerplexity26.1B88.1tied with d1-ultra90.5%547 questions3.6no repo
16GLiDE v1.2Fastino26.1B87.9tied with Perplexity Decider 27B v1.2 · API · with web90.3%547 questions2.7a7e4589
17GLiDE 27B RTX PRO 6000Fastino26.1B87.9tied with GLiDE · v1.290.3%547 questions1.94d32743
18d1.2-ultraLiquid AINot disclosed87.9tied with GLiDE 27B · RTX PRO 600090.3%547 questions8.5no repo
19Mammoth-1 post-trainedBase Labs27.4B87.7tied with d1.2-ultra90.1%547 questions2.461cec44
20d3 fine-tunevLLM Semantic Router26.1B87.2tied with Mammoth-189.8%547 questions3.620f37b9
21Drex Max fine-tuneNace.AI26.1B87.0tied with d389.6%547 questions2.6cac65e1
22Perplexity Decider 27B v1.3Perplexity26.1B87.0tied with Drex Max89.6%547 questions3.49c2ebb2
23d3-ultra fine-tunevLLM Semantic Router26.1B86.7tied with Perplexity Decider 27B v1.389.4%547 questions2.852ba9e7
24d1.2Liquid AINot disclosed86.3tied with d3-ultra89.0%547 questions8.3no repo
25Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B85.8tied with d1.288.7%547 questions3.7225c913
26Drex Max v1.1Nace.AINot disclosed85.6tied with Perplexity Decider 27B v1.2 · DGX Spark88.5%547 questions3.8no repo
27Perplexity Decider 27B v1Perplexity26.1B85.6tied with Drex Max v1.188.5%547 questions2.6no repo
28d1.1Liquid AINot disclosed85.4tied with Perplexity Decider 27B v188.3%547 questions4.2no repo
29SubQ Snap 1.2SubquadraticNot disclosed85.1tied with d1.188.1%547 questions1.8no repo
30Jeffan-9B-1211tattrongvu8.9B84.5tied with SubQ Snap 1.287.6%547 questions8.9862d1a7
31StartLux-Decision-v1.1-9B-InstantStartLux9.7B83.5tied with Jeffan-9B-121186.8%547 questions6.618a17f7
32d1Liquid AINot disclosed83.5tied with StartLux-Decision-v1.1-9B-Instant86.8%547 questions2.6no repo
33Clef post-trainedCloudflare27.5B83.3tied with d186.7%547 questions3.12f3de3d
34KnowLine-12B-Gen1PelaAI12B83.3tied with Clef86.7%547 questions11.1f26fbc2
35GLiDE 9BFastino7.9B82.9tied with KnowLine-12B-Gen186.3%547 questions5.6ab54df5
36OneJev-27B full fine-tuneOmniJev27.4B82.9tied with GLiDE 9B86.3%547 questions9.7a5e5f55
37Jev 1.14TypeSafeNot disclosed82.6tied with OneJev-27B86.1%547 questions2.2no repo
38Cerno 9B v1.2 full fine-tuneApertix AI8.9B82.4tied with Jev 1.1485.9%547 questions4.2f02a632
39Drex 2.0Nace.AI35.1B3B active82.4tied with Cerno 9B v1.285.9%547 questions6.9no repo
40Jev 1.13TypeSafeNot disclosed82.4tied with Drex 2.085.9%547 questions3.1no repo
41KnowLine-9B-Gen1PelaAI9.7B82.4tied with Jev 1.1385.9%547 questions3.5325378d
42SubQ Snap 1.1SubquadraticNot disclosed82.4tied with KnowLine-9B-Gen185.9%547 questions3.2no repo
43SubQ Snap PreviewSubquadraticNot disclosed82.4tied with SubQ Snap 1.185.9%547 questions3.2no repo
44Adjudex 9B v5newsbang8.4B82.2tied with SubQ Snap Preview85.7%547 questions3.4ffdcb82
45Mercury DecideInceptionNot disclosed82.2tied with Adjudex 9B v585.7%547 questions6.6no repo
46Microsoft-Decision-1MicrosoftNot disclosed82.2tied with Mercury Decide85.7%547 questions2.8no repo
47Microsoft-Decision-1.1MicrosoftNot disclosed81.9tied with Microsoft-Decision-185.6%547 questions8.3no repo
48Elephant-1 post-trainedBase Labs9.4B81.7tied with Microsoft-Decision-1.185.4%547 questions2.7b3f8a93
49Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B81.7tied with Elephant-185.4%547 questions45.328cf730
50KnowLine-9B-Gen2PelaAI9.7B81.5tied with Open-Jev-27B-v1.185.2%547 questions8.78ec8eb6
51GPT-6 Luna Decisions APIOpenAINot disclosed80.1tied with KnowLine-9B-Gen284.1%547 questions3.2no repo
52celeris-1-decisionCelerisNot disclosed80.1tied with GPT-6 Luna · Decisions API84.1%547 questions5.2no repo
53Cygnet unchanged baseBlockbrain Labs12B79.9tied with celeris-1-decision83.9%547 questions4.3no repo
54Winnow-12B LoRA, mergedEldanRing11.9B79.9tied with Cygnet83.9%547 questions8.7b2b1421
55Perplexity Decider Flash v1.3Perplexity4.5B79.7tied with Winnow-12B83.7%547 questions2.7d06fb3f
56Valen-Decision-1.5-9BValen-Team8.4B79.7tied with Perplexity Decider Flash v1.383.7%547 questions8.45a51211
57celeris-1.1-decisionCelerisNot disclosed79.7tied with Valen-Decision-1.5-9B83.7%547 questions5.0no repo
58Dexy-Decision-9B-V2 fine-tunePatrick-188.4B79.4tied with celeris-1.1-decision83.5%547 questions4.24f434e2
59vera-medium-omniscar-ai4.2B79.4tied with Dexy-Decision-9B-V283.5%547 questions9.91f23a6c
60StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B78.5tied with vera-medium-omni82.8%547 questions3.2416088f
61Clef-Flash post-trainedCloudflare9.5B77.8tied with StartLux-Decision-v1.1-4B-Instant82.3%547 questions4.0fde727a
62Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B77.8tied with Clef-Flash82.3%547 questions3.545830d2
63Perplexity Decider Flash v1.2Perplexity4.5B77.4tied with Verdin-4B-v1.181.9%547 questions4.16d771ad
64KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B76.0tied with Perplexity Decider Flash v1.280.8%547 questions5.8779944d
65Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B75.5tied with KnowLine 4B Gen5.580.4%547 questions3.5889f23e
66Metask-Jev-4B LoRA, mergedMetask Lab4.5B71.4tied with Chinese Decision Pointer 4B77.1%547 questions4.4ea20fe8
67KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B64.171.3%547 questions5.153245d4
68GLiDE 2BFastino1.9B62.5tied with KnowLine-2B-Gen170.0%547 questions8.200f122c
69StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B62.1tied with GLiDE 2B69.7%547 questions8.2f1e6cd1
70d1-2BLiquid AI1.9B57.065.6%547 questions7.910e7671
71GLiDE 0.8BFastino750M50.960.7%547 questions12.59eacdee
72StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M49.5tied with GLiDE 0.8B59.6%547 questions8.76496ae6
73GLiDE 0.4BFastino400M49.3tied with StartLux-Decision-v1.1-0.8B-Instant59.4%547 questions4.86bda7ab
74OneJev1.6-0.8B full fine-tuneOmniJev850M49.3tied with GLiDE 0.4B59.4%547 questions3.7bcd88a3
75KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M48.6tied with OneJev1.6-0.8B58.9%547 questions8.16b27831
76GLiClass v4 Edge fine-tuneKnowledgator440M47.9tied with KnowLine-0.8B-Gen158.3%547 questions3.8885af1a
77Decider 1meraGPTNot disclosed44.7tied with GLiClass v4 Edge55.8%547 questions5.3no repo
78Strands Decider 2B LoRA + headStrands Agents2.3B31.945.5%547 questions10.52b52a62
79LayaConvai Innovations420M3.622.9%547 questions12.1c5d7873
80GLiNER2.5-multi-Decide fine-tuneFastino290M1.5tied with Laya21.2%547 questions27.2a35a0cd
81Laya MultilingualConvai Innovations320M−5.815.4%547 questions44.4c5d7873

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Claude Fable 5.2 max · chatAnthropicNot disclosed97.598.0%547 questions10.7no repo
2Gemini 4 Hydrogen high · chatGoogleNot disclosed95.9tied with Claude Fable 5.2 · max · chat96.7%547 questions3.1no repo
3GPT-6 Astra high · chatOpenAINot disclosed95.2tied with Gemini 4 Hydrogen · high · chat96.2%547 questions1.8no repo
4GPT-6.1 Terra high · chatOpenAINot disclosed95.2tied with GPT-6 Astra · high · chat96.2%547 questions1.3no repo
5Claude Fable 5.1 high · chatAnthropicNot disclosed95.0tied with GPT-6.1 Terra · high · chat96.0%547 questions11.8no repo
6Gemini 3.8 Flash high · chatGoogleNot disclosed95.0tied with Claude Fable 5.1 · high · chat96.0%547 questions1.7no repo
7Gemini 4 Argon low · chatGoogleNot disclosed95.0tied with Gemini 3.8 Flash · high · chat96.0%547 questions1.5no repo
8GPT-6.1 Sol high · chatOpenAINot disclosed94.5tied with Gemini 4 Argon · low · chat95.6%547 questions2.4no repo
9Claude Opus 5.5 high · chatAnthropicNot disclosed93.1tied with GPT-6.1 Sol · high · chat94.5%547 questions13.3no repo
10Qwen4-Max high · chatAlibabaNot disclosed93.1tied with Claude Opus 5.5 · high · chat94.5%547 questions8.5no repo
11Gemini 4 Carbon high · chatGoogleNot disclosed92.9tied with Qwen4-Max · high · chat94.3%547 questions3.1no repo
12Hy4 flash high · chatTencent220.2B11.7B active92.0tied with Gemini 4 Carbon · high · chat93.6%547 questions3.9no repo
13Grok 4.7 high · chatxAINot disclosed91.8tied with Hy4 flash · high · chat93.4%547 questions11.1no repo
14Inkling 2 Small high · chatThinking Machines594B17B active91.8tied with Grok 4.7 · high · chat93.4%547 questions6.8no repo
15SubQ Savant 1.4 high · chatSubquadraticNot disclosed91.5tied with Inkling 2 Small · high · chat93.2%547 questions7.5no repo
16Claude Haiku 5.5 max · chatAnthropicNot disclosed90.9tied with SubQ Savant 1.4 · high · chat92.7%547 questions18.4no repo
17DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active90.6tied with Claude Haiku 5.5 · max · chat92.5%547 questions7.5no repo
18Kimi K3.1 Pro low · chatMoonshot2.8T104B active90.4tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101792.3%547 questions5.7no repo
19Qwen3.5-397B-A17B high · chatAlibaba397B17B active89.3tied with Kimi K3.1 Pro · low · chat91.4%547 questions3.3no repo
20Gemma 4 31B reasoning off · chatGoogle31.3B88.1tied with Qwen3.5-397B-A17B · high · chat90.5%547 questions8.5842da37
21GLM-5.3-Flash reasoning on · chatZ.ai320B18B active87.0tied with Gemma 4 31B · reasoning off · chat89.6%547 questions7.0no repo
22Muse Ember max · chatMetaNot disclosed87.0tied with GLM-5.3-Flash · reasoning on · chat89.6%547 questions7.0no repo
23MiMo-V2.6-Flash high · chatXiaomi309B15B active86.3tied with Muse Ember · max · chat89.0%547 questions6.5no repo
24Qwen3.8-27B low · chatAlibaba27.8B86.3tied with MiMo-V2.6-Flash · high · chat89.0%547 questions11.01d4bf0f
25Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active85.4tied with Qwen3.8-27B · low · chat88.3%547 questions2.6no repo
26GLM-5.3 low · chatZ.ai753.3B84.7tied with Inkling-Small · high · chat · Jul 202687.8%547 questions3.6no repo
27MAI Code 2 Flash high · chatMicrosoftNot disclosed84.7tied with GLM-5.3 · low · chat87.8%547 questions4.5no repo
28Qwen3.5-4B reasoning on · chatAlibaba4.7B83.8tied with MAI Code 2 Flash · high · chat87.0%547 questions–851bf6e
29Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed83.8tied with Qwen3.5-4B · reasoning on · chat87.0%547 questions5.5no repo
30Gemma 4 12B Unified default reasoning · chatGoogle12B83.5tied with Qwen3.8 Flash · no reasoning · chat86.8%547 questions–43c9d01
31GPT-6 Luna low · chatOpenAINot disclosed82.9tied with Gemma 4 12B Unified · default reasoning · chat86.3%547 questions10.1no repo
32Claude Sonnet 5.5 high · chatAnthropicNot disclosed82.6tied with GPT-6 Luna · low · chat86.1%547 questions13.9no repo
33MiniMax-M3.1-Flash high · chatMiniMax428B23B active80.6tied with Claude Sonnet 5.5 · high · chat84.5%547 questions9.0no repo
34LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active78.3tied with MiniMax-M3.1-Flash · high · chat82.6%547 questions–1769516
35Gemma 4 E4B default reasoning · chatGoogle8B78.1tied with LFM2.5-24B-A2B · default reasoning · chat82.4%547 questions–4ecc2ba
36Qwen3.5-4B reasoning off · chatAlibaba4.7B75.5tied with Gemma 4 E4B · default reasoning · chat80.4%547 questions–851bf6e
37Qwen3-14B reasoning off · chatAlibaba14.8B74.6tied with Qwen3.5-4B · reasoning off · chat79.7%547 questions17.840c0698
38MiniCPM5-2B reasoning on · chatOpenBMB2.5B72.7tied with Qwen3-14B · reasoning off · chat78.1%547 questions–42433e9
39Gemma 4 E2B reasoning on · chatGoogle5.1B70.3tied with MiniCPM5-2B · reasoning on · chat76.2%547 questions–4adf157
40Qwen3.5-2B reasoning on · chatAlibaba2.3B69.8tied with Gemma 4 E2B · reasoning on · chat75.9%547 questions–15852e8
41Gemma 4 E2B default reasoning · chatGoogle5.1B68.2tied with Qwen3.5-2B · reasoning on · chat74.6%547 questions–4adf157
42Granite-4.2-3B reasoning on · chatIBM3.7B68.0tied with Gemma 4 E2B · default reasoning · chat74.4%547 questions–4e9c275
43LFM2.5-2.6B default reasoning · chatLiquid AI2.7B67.3tied with Granite-4.2-3B · reasoning on · chat73.9%547 questions–7141b56
44Granite-4.2-3B reasoning off · chatIBM3.7B65.3tied with LFM2.5-2.6B · default reasoning · chat72.2%547 questions–4e9c275
45Erk-14B reasoning off · chateCloud Technology14.8B65.0tied with Granite-4.2-3B · reasoning off · chat72.0%547 questions10.98584b14
46LFM3-1.7B reasoning off · chatLiquid AI1.7B64.7tied with Erk-14B · reasoning off · chat71.8%547 questions–0dc4268
47LFM3-1.7B reasoning on · chatLiquid AI1.7B63.9tied with LFM3-1.7B · reasoning off · chat71.1%547 questions–0dc4268
48LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active63.0tied with LFM3-1.7B · reasoning on · chat70.4%547 questions–dd83f5c
49Raon-1 nano reasoning on · chatKRAFTON3.5B61.2tied with LFM2.5-8B-A1B · default reasoning · chat68.9%547 questions–fcf2d1d
50MiniCPM5-2B reasoning off · chatOpenBMB2.5B59.7tied with Raon-1 nano · reasoning on · chat67.8%547 questions–42433e9
51Qwen3.5-2B reasoning off · chatAlibaba2.3B59.1tied with MiniCPM5-2B · reasoning off · chat67.3%547 questions–15852e8
52Nemotron-Labs-Sigma-8B-instruct default reasoning · chatNVIDIA8.5B45.856.7%547 questions–e5aab18
53Nemotron-Labs-Sigma-3B-instruct default reasoning · chatNVIDIA3.8B44.7tied with Nemotron-Labs-Sigma-8B-instruct · default reasoning · chat55.8%547 questions–057a084
54Raon-1 nano reasoning off · chatKRAFTON3.5B42.2tied with Nemotron-Labs-Sigma-3B-instruct · default reasoning · chat53.7%547 questions–fcf2d1d
55GLiNER2.5 MultiFastino290M−4.716.3%547 questions36.42ca71aa

547 questions per model · updated 19 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 547 kept reading questions, translated into English, right (accuracy 100%) · feed: bench-scores.json

Reading · Vision

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Gemini Decision 1 20261111GoogleNot disclosed58.166.4%763 questions5.2no repo
2d1-ultraLiquid AINot disclosed37.650.1%763 questions3.5no repo
3GLiDE 27B DGX SparkFastino26.1B37.4tied with d1-ultra49.9%763 questions4.34d32743
4KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B37.3tied with GLiDE 27B · DGX Spark49.8%763 questions6.18e56aa1
5d3 fine-tunevLLM Semantic Router26.1B35.6tied with KnowLine-31B-Gen1.5 · DGX Spark48.5%763 questions3.720f37b9
6d1.2Liquid AINot disclosed33.7tied with d346.9%763 questions5.6no repo
7Drex Max fine-tuneNace.AI26.1B33.5tied with d1.246.8%763 questions9.3cac65e1
8SubQ Snap 1.1SubquadraticNot disclosed32.2tied with Drex Max45.7%763 questions11.7no repo
9d3-ultra fine-tunevLLM Semantic Router26.1B31.8tied with SubQ Snap 1.145.5%763 questions18.852ba9e7
10d1.2-ultraLiquid AINot disclosed31.7tied with d3-ultra45.3%763 questions2.8no repo
11Drex Max v1.1Nace.AINot disclosed31.4tied with d1.2-ultra45.1%763 questions5.2no repo
12Jev 1.14TypeSafeNot disclosed29.1tied with Drex Max v1.143.3%763 questions3.7no repo
13OneJev-27B full fine-tuneOmniJev27.4B26.6tied with Jev 1.1441.3%763 questions4.9a5e5f55
14GPT-6 Luna Decisions APIOpenAINot disclosed15.832.6%763 questions5.7no repo
15KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B12.7tied with GPT-6 Luna · Decisions API30.1%763 questions11.0779944d
16OneJev1.6-0.8B full fine-tuneOmniJev850M8.126.5%763 questions4.2bcd88a3
17StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M5.3tied with OneJev1.6-0.8B24.2%763 questions1.76496ae6
18KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M4.8tied with StartLux-Decision-v1.1-0.8B-Instant23.9%763 questions1.56b27831
19StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B4.3tied with KnowLine-0.8B-Gen123.5%763 questions3.6f1e6cd1
20d1-2BLiquid AI1.9B4.2tied with StartLux-Decision-v1.1-2B-Instant23.3%763 questions1.110e7671

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Muse Ember max · chatMetaNot disclosed82.886.2%763 questions–no repo
2Gemma 4 E4B default reasoning · chatGoogle8B28.642.9%763 questions–4ecc2ba

763 questions per model · updated 19 November 2026 · 0 = chance: 1 of each question's options right on every question (accuracy 20% with 5 options); 100 = every one of the 763 questions that depend on the printed page right, read from the page image alone (accuracy 100%) · feed: bench-scores.json

Logic

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B50.960.7%1,125 questions5.7d69cdd5
2REAX 1.3 blendREAX57.4B49.1tied with Karpo Decision 31B59.3%1,125 questions3.7fa4bf7e
3KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B48.8tied with REAX 1.359.0%1,125 questions2.68e56aa1
4KnowLine-31B-Gen2PelaAI31.3B48.7tied with KnowLine-31B-Gen1.5 · DGX Spark58.9%1,125 questions3.7fd99a9e
5Copernicus-31BPoindexter Labs31.3B48.3tied with KnowLine-31B-Gen258.7%1,125 questions3.9be71525
6GLiDE 27B DGX SparkFastino26.1B48.0tied with Copernicus-31B58.4%1,125 questions4.44d32743
7Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B47.3tied with GLiDE 27B · DGX Spark57.9%1,125 questions2.18fcee85
8Logits Sigma-1 Duo blendLogits57.4B47.2tied with Sieve 31B v3 · DGX Spark57.8%1,125 questions5.0a0c1b6e
9GLiDE v1.2Fastino26.1B44.055.2%1,125 questions5.9a7e4589
10d1-ultraLiquid AINot disclosed43.7tied with GLiDE · v1.254.9%1,125 questions3.2no repo
11Drex Max v1.1Nace.AINot disclosed43.4tied with d1-ultra54.8%1,125 questions6.1no repo
12Perplexity Decider 27B v1.3Perplexity26.1B43.4tied with Drex Max v1.154.8%1,125 questions9.09c2ebb2
13d3-ultra fine-tunevLLM Semantic Router26.1B43.2tied with Perplexity Decider 27B v1.354.6%1,125 questions15.152ba9e7
14GLiDE APIFastinoNot disclosed43.1tied with d3-ultra54.5%1,125 questions6.6no repo
15Mammoth-1 post-trainedBase Labs27.4B43.1tied with GLiDE · API54.5%1,125 questions5.361cec44
16SubQ Snap 1.2SubquadraticNot disclosed43.1tied with Mammoth-154.5%1,125 questions8.7no repo
17d3 fine-tunevLLM Semantic Router26.1B43.1tied with SubQ Snap 1.254.5%1,125 questions4.620f37b9
18Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B41.7tied with d353.3%1,125 questions3.4no repo
19Drex Max fine-tuneNace.AI26.1B41.3tied with Perplexity Decider 27B v1.2 · DGX Spark53.1%1,125 questions16.2cac65e1
20Perplexity Decider 27B v1.2 API · with webPerplexity26.1B40.7tied with Drex Max52.5%1,125 questions4.2no repo
21SubQ Snap 1.1SubquadraticNot disclosed40.4tied with Perplexity Decider 27B v1.2 · API · with web52.4%1,125 questions12.9no repo
22Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B40.3tied with SubQ Snap 1.152.3%1,125 questions2.9866c0fc
23d1.2-ultraLiquid AINot disclosed39.4tied with Turion Jev 31B · DGX Spark51.6%1,125 questions3.0no repo
24Microsoft-Decision-1.1MicrosoftNot disclosed38.3tied with d1.2-ultra50.7%1,125 questions8.7no repo
25KnowLine-12B-Gen1PelaAI12B38.2tied with Microsoft-Decision-1.150.6%1,125 questions4.8f26fbc2
26Gemini Decision 1 20261111GoogleNot disclosed37.6tied with KnowLine-12B-Gen150.0%1,125 questions3.1no repo
27d1.2Liquid AINot disclosed36.4tied with Gemini Decision 1 · 2026111149.2%1,125 questions2.6no repo
28Jeffan-9B-1211tattrongvu8.9B35.2tied with d1.248.2%1,125 questions4.0862d1a7
29Jev 1.14TypeSafeNot disclosed35.2tied with Jeffan-9B-121148.2%1,125 questions3.3no repo
30GLiDE 9BFastino7.9B34.8tied with Jev 1.1447.8%1,125 questions6.7ab54df5
31celeris-1-decisionCelerisNot disclosed34.1tied with GLiDE 9B47.3%1,125 questions10.7no repo
32celeris-1.1-decisionCelerisNot disclosed34.1tied with celeris-1-decision47.3%1,125 questions9.8no repo
33StartLux-Decision-v1.1-9B-InstantStartLux9.7B33.2tied with celeris-1.1-decision46.6%1,125 questions3.818a17f7
34Adjudex 9B v5newsbang8.4B33.0tied with StartLux-Decision-v1.1-9B-Instant46.4%1,125 questions7.2ffdcb82
35Elephant-1 post-trainedBase Labs9.4B33.0tied with Adjudex 9B v546.4%1,125 questions8.6b3f8a93
36Jev 1.13TypeSafeNot disclosed32.9tied with Elephant-146.3%1,125 questions5.9no repo
37d1.1Liquid AINot disclosed32.6tied with Jev 1.1346.0%1,125 questions4.4no repo
38StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B32.3tied with d1.145.9%1,125 questions3.3416088f
39Dexy-Decision-9B-V2 fine-tunePatrick-188.4B31.7tied with StartLux-Decision-v1.1-4B-Instant45.3%1,125 questions14.94f434e2
40Mercury DecideInceptionNot disclosed31.3tied with Dexy-Decision-9B-V245.1%1,125 questions15.4no repo
41Perplexity Decider Flash v1.3Perplexity4.5B30.9tied with Mercury Decide44.7%1,125 questions10.1d06fb3f
42Microsoft-Decision-1MicrosoftNot disclosed30.6tied with Perplexity Decider Flash v1.344.4%1,125 questions8.0no repo
43Valen-Decision-1.5-9BValen-Team8.4B30.4tied with Microsoft-Decision-144.4%1,125 questions5.85a51211
44KnowLine-9B-Gen1PelaAI9.7B30.3tied with Valen-Decision-1.5-9B44.3%1,125 questions3.5325378d
45vera-medium-omniscar-ai4.2B30.1tied with KnowLine-9B-Gen144.1%1,125 questions4.71f23a6c
46KnowLine-9B-Gen2PelaAI9.7B29.9tied with vera-medium-omni43.9%1,125 questions3.28ec8eb6
47Cerno 9B v1.2 full fine-tuneApertix AI8.9B29.1tied with KnowLine-9B-Gen243.3%1,125 questions17.0f02a632
48Perplexity Decider Flash v1.2Perplexity4.5B28.9tied with Cerno 9B v1.243.1%1,125 questions6.06d771ad
49Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B28.2tied with Perplexity Decider Flash v1.242.6%1,125 questions12.928cf730
50KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B27.9tied with Open-Jev-27B-v1.142.3%1,125 questions13.7779944d
51Drex 2.0Nace.AI35.1B3B active27.3tied with KnowLine 4B Gen5.541.9%1,125 questions4.3no repo
52Clef post-trainedCloudflare27.5B26.0tied with Drex 2.040.8%1,125 questions15.72f3de3d
53d1Liquid AINot disclosed26.0tied with Clef40.8%1,125 questions10.9no repo
54OneJev-27B full fine-tuneOmniJev27.4B25.8tied with d140.6%1,125 questions4.6a5e5f55
55Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B25.4tied with OneJev-27B40.4%1,125 questions15.5889f23e
56Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B24.7tied with Chinese Decision Pointer 4B39.7%1,125 questions10.845830d2
57Winnow-12B LoRA, mergedEldanRing11.9B24.7tied with Verdin-4B-v1.139.7%1,125 questions14.4b2b1421
58Cygnet unchanged baseBlockbrain Labs12B24.1tied with Winnow-12B39.3%1,125 questions11.2no repo
59Clef-Flash post-trainedCloudflare9.5B23.7tied with Cygnet38.9%1,125 questions11.5fde727a
60GLiDE 2BFastino1.9B15.332.3%1,125 questions16.200f122c
61Metask-Jev-4B LoRA, mergedMetask Lab4.5B15.1tied with GLiDE 2B32.1%1,125 questions2.5ea20fe8
62KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B14.7tied with Metask-Jev-4B31.7%1,125 questions8.353245d4
63StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B13.4tied with KnowLine-2B-Gen130.8%1,125 questions5.3f1e6cd1
64d1-2BLiquid AI1.9B13.3tied with StartLux-Decision-v1.1-2B-Instant30.7%1,125 questions6.910e7671
65GLiDE 0.8BFastino750M11.4tied with d1-2B29.2%1,125 questions14.09eacdee
66OneJev1.6-0.8B full fine-tuneOmniJev850M8.8tied with GLiDE 0.8B27.0%1,125 questions11.2bcd88a3
67KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M8.3tied with OneJev1.6-0.8B26.7%1,125 questions8.46b27831
68StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M5.9tied with KnowLine-0.8B-Gen124.7%1,125 questions7.86496ae6
69GLiClass v4 Edge fine-tuneKnowledgator440M4.4tied with StartLux-Decision-v1.1-0.8B-Instant23.6%1,125 questions8.7885af1a
70GLiDE 0.4BFastino400M4.0tied with GLiClass v4 Edge23.2%1,125 questions8.86bda7ab
71Strands Decider 2B LoRA + headStrands Agents2.3B3.0tied with GLiDE 0.4B22.4%1,125 questions12.02b52a62

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1GPT-6 Astra high · chatOpenAINot disclosed95.996.7%1,125 questions–no repo
2Gemini 4 Carbon high · chatGoogleNot disclosed95.9tied with GPT-6 Astra · high · chat96.7%1,125 questions–no repo
3GPT-6.1 Sol high · chatOpenAINot disclosed95.7tied with Gemini 4 Carbon · high · chat96.5%1,125 questions–no repo
4Qwen4-Max high · chatAlibabaNot disclosed95.3tied with GPT-6.1 Sol · high · chat96.3%1,125 questions–no repo
5Claude Opus 5.5 high · chatAnthropicNot disclosed95.2tied with Qwen4-Max · high · chat96.2%1,125 questions–no repo
6DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active95.2tied with Claude Opus 5.5 · high · chat96.2%1,125 questions–no repo
7Gemini 4 Hydrogen high · chatGoogleNot disclosed95.1tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101796.1%1,125 questions–no repo
8Inkling 2 Small high · chatThinking Machines594B17B active94.6tied with Gemini 4 Hydrogen · high · chat95.6%1,125 questions–no repo
9Claude Sonnet 5.5 high · chatAnthropicNot disclosed94.3tied with Inkling 2 Small · high · chat95.5%1,125 questions–no repo
10Hy4 flash high · chatTencent220.2B11.7B active94.3tied with Claude Sonnet 5.5 · high · chat95.5%1,125 questions–no repo
11Muse Ember max · chatMetaNot disclosed94.3tied with Hy4 flash · high · chat95.5%1,125 questions–no repo
12Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active94.2tied with Muse Ember · max · chat95.4%1,125 questions–no repo
13Claude Fable 5.1 high · chatAnthropicNot disclosed94.0tied with Inkling-Small · high · chat · Jul 202695.2%1,125 questions–no repo
14MiMo-V2.6-Flash high · chatXiaomi309B15B active93.9tied with Claude Fable 5.1 · high · chat95.1%1,125 questions–no repo
15Qwen3.5-397B-A17B high · chatAlibaba397B17B active93.9tied with MiMo-V2.6-Flash · high · chat95.1%1,125 questions–no repo
16Qwen3.8-27B low · chatAlibaba27.8B92.6tied with Qwen3.5-397B-A17B · high · chat94.0%1,125 questions–1d4bf0f
17GPT-6.1 Terra high · chatOpenAINot disclosed92.4tied with Qwen3.8-27B · low · chat94.0%1,125 questions–no repo
18MiniMax-M3.1-Flash high · chatMiniMax428B23B active92.4tied with GPT-6.1 Terra · high · chat94.0%1,125 questions–no repo
19Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed92.4tied with MiniMax-M3.1-Flash · high · chat94.0%1,125 questions–no repo
20Gemma 4 31B reasoning off · chatGoogle31.3B91.7tied with Qwen3.8 Flash · no reasoning · chat93.3%1,125 questions–842da37
21GLM-5.3-Flash reasoning on · chatZ.ai320B18B active91.4tied with Gemma 4 31B · reasoning off · chat93.2%1,125 questions–no repo
22Qwen3.5-4B reasoning on · chatAlibaba4.7B90.4tied with GLM-5.3-Flash · reasoning on · chat92.4%1,125 questions–851bf6e
23Granite-4.2-3B reasoning on · chatIBM3.7B89.6tied with Qwen3.5-4B · reasoning on · chat91.6%1,125 questions–4e9c275
24SenseNova-Loop-9B on · chatno maker9B89.2tied with Granite-4.2-3B · reasoning on · chat91.4%1,125 questions–3e1ba6e
25MiniCPM5-2B reasoning off · chatOpenBMB2.5B86.188.9%1,125 questions–42433e9
26Gemma 4 12B Unified default reasoning · chatGoogle12B84.7tied with MiniCPM5-2B · reasoning off · chat87.7%1,125 questions–43c9d01
27LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active83.3tied with Gemma 4 12B Unified · default reasoning · chat86.7%1,125 questions–1769516
28LFM3-1.7B reasoning on · chatLiquid AI1.7B81.9tied with LFM2.5-24B-A2B · default reasoning · chat85.5%1,125 questions–0dc4268
29Gemma 4 E4B default reasoning · chatGoogle8B79.0tied with LFM3-1.7B · reasoning on · chat83.2%1,125 questions–4ecc2ba
30LFM2.5-2.6B default reasoning · chatLiquid AI2.7B69.275.4%1,125 questions–7141b56
31Gemma 4 E2B reasoning on · chatGoogle5.1B66.172.9%1,125 questions–4adf157
32Qwen3-14B reasoning off · chatAlibaba14.8B56.765.3%1,125 questions–40c0698
33Raon-1 nano reasoning on · chatKRAFTON3.5B56.3tied with Qwen3-14B · reasoning off · chat65.1%1,125 questions–fcf2d1d
34Qwen3.5-2B reasoning off · chatAlibaba2.3B56.0tied with Raon-1 nano · reasoning on · chat64.8%1,125 questions–15852e8
35LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active50.260.2%1,125 questions–dd83f5c
36GPT-6 Luna low · chatOpenAINot disclosed47.4tied with LFM2.5-8B-A1B · default reasoning · chat58.0%1,125 questions–no repo
37Erk-14B reasoning off · chateCloud Technology14.8B42.854.2%1,125 questions–8584b14

1,125 questions per model · updated 19 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 1,125 Logic questions right (accuracy 100%) · feed: bench-scores.json

Math

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B76.981.6%2,163 questions1.2d69cdd5
2KnowLine-31B-Gen2PelaAI31.3B75.3tied with Karpo Decision 31B80.3%2,163 questions3.6fd99a9e
3REAX 1.3 blendREAX57.4B72.578.0%2,163 questions7.1fa4bf7e
4KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B71.7tied with REAX 1.377.4%2,163 questions3.68e56aa1
5Copernicus-31BPoindexter Labs31.3B71.7tied with KnowLine-31B-Gen1.5 · DGX Spark77.3%2,163 questions6.5be71525
6Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B71.4tied with Copernicus-31B77.1%2,163 questions4.38fcee85
7Logits Sigma-1 Duo blendLogits57.4B69.075.2%2,163 questions7.5a0c1b6e
8d1-ultraLiquid AINot disclosed67.6tied with Logits Sigma-1 Duo74.1%2,163 questions2.3no repo
9Mammoth-1 post-trainedBase Labs27.4B67.5tied with d1-ultra74.0%2,163 questions3.061cec44
10Perplexity Decider 27B v1.3Perplexity26.1B65.6tied with Mammoth-172.5%2,163 questions4.59c2ebb2
11GLiDE 27B DGX SparkFastino26.1B65.0tied with Perplexity Decider 27B v1.372.0%2,163 questions4.14d32743
12d3-ultra fine-tunevLLM Semantic Router26.1B64.4tied with GLiDE 27B · DGX Spark71.5%2,163 questions8.652ba9e7
13Gemini Decision 1 20261111GoogleNot disclosed64.2tied with d3-ultra71.3%2,163 questions2.5no repo
14GLiDE APIFastinoNot disclosed63.5tied with Gemini Decision 1 · 2026111170.8%2,163 questions5.0no repo
15d3 fine-tunevLLM Semantic Router26.1B63.0tied with GLiDE · API70.4%2,163 questions4.520f37b9
16Drex Max fine-tuneNace.AI26.1B62.9tied with d370.3%2,163 questions11.8cac65e1
17Drex Max v1.1Nace.AINot disclosed62.6tied with Drex Max70.1%2,163 questions5.7no repo
18celeris-1-decisionCelerisNot disclosed62.5tied with Drex Max v1.170.0%2,163 questions4.3no repo
19GLiDE v1.2Fastino26.1B62.3tied with celeris-1-decision69.9%2,163 questions5.2a7e4589
20celeris-1.1-decisionCelerisNot disclosed62.3tied with GLiDE · v1.269.9%2,163 questions4.5no repo
21d1.2-ultraLiquid AINot disclosed62.3tied with celeris-1.1-decision69.9%2,163 questions4.6no repo
22Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B62.2tied with d1.2-ultra69.8%2,163 questions3.2no repo
23Perplexity Decider 27B v1.2 API · with webPerplexity26.1B61.6tied with Perplexity Decider 27B v1.2 · DGX Spark69.3%2,163 questions3.7no repo
24Microsoft-Decision-1.1MicrosoftNot disclosed61.3tied with Perplexity Decider 27B v1.2 · API · with web69.1%2,163 questions14.3no repo
25Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B61.3tied with Microsoft-Decision-1.169.0%2,163 questions4.7866c0fc
26SubQ Snap 1.2SubquadraticNot disclosed60.9tied with Turion Jev 31B · DGX Spark68.7%2,163 questions6.9no repo
27SubQ Snap 1.1SubquadraticNot disclosed60.7tied with SubQ Snap 1.268.6%2,163 questions9.5no repo
28KnowLine-12B-Gen1PelaAI12B57.866.3%2,163 questions4.0f26fbc2
29d1.2Liquid AINot disclosed56.9tied with KnowLine-12B-Gen165.5%2,163 questions3.2no repo
30d1.1Liquid AINot disclosed56.5tied with d1.265.2%2,163 questions2.2no repo
31Jev 1.14TypeSafeNot disclosed56.4tied with d1.165.1%2,163 questions2.8no repo
32Jev 1.13TypeSafeNot disclosed55.1tied with Jev 1.1464.1%2,163 questions2.6no repo
33Adjudex 9B v5newsbang8.4B52.662.1%2,163 questions4.8ffdcb82
34Elephant-1 post-trainedBase Labs9.4B51.7tied with Adjudex 9B v561.3%2,163 questions4.3b3f8a93
35KnowLine-9B-Gen2PelaAI9.7B50.1tied with Elephant-160.1%2,163 questions4.88ec8eb6
36Jeffan-9B-1211tattrongvu8.9B49.8tied with KnowLine-9B-Gen259.8%2,163 questions3.8862d1a7
37KnowLine-9B-Gen1PelaAI9.7B49.7tied with Jeffan-9B-121159.8%2,163 questions3.1325378d
38Microsoft-Decision-1MicrosoftNot disclosed49.2tied with KnowLine-9B-Gen159.4%2,163 questions4.1no repo
39Drex 2.0Nace.AI35.1B3B active48.7tied with Microsoft-Decision-159.0%2,163 questions1.9no repo
40Dexy-Decision-9B-V2 fine-tunePatrick-188.4B48.5tied with Drex 2.058.8%2,163 questions12.14f434e2
41StartLux-Decision-v1.1-9B-InstantStartLux9.7B48.0tied with Dexy-Decision-9B-V258.4%2,163 questions2.718a17f7
42GLiDE 9BFastino7.9B47.4tied with StartLux-Decision-v1.1-9B-Instant57.9%2,163 questions7.1ab54df5
43vera-medium-omniscar-ai4.2B46.0tied with GLiDE 9B56.8%2,163 questions3.01f23a6c
44Cerno 9B v1.2 full fine-tuneApertix AI8.9B45.0tied with vera-medium-omni56.0%2,163 questions15.7f02a632
45KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B45.0tied with Cerno 9B v1.256.0%2,163 questions8.6779944d
46Perplexity Decider Flash v1.3Perplexity4.5B44.9tied with KnowLine 4B Gen5.555.9%2,163 questions11.7d06fb3f
47StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B44.8tied with Perplexity Decider Flash v1.355.8%2,163 questions6.7416088f
48d1Liquid AINot disclosed44.5tied with StartLux-Decision-v1.1-4B-Instant55.6%2,163 questions3.3no repo
49Valen-Decision-1.5-9BValen-Team8.4B44.2tied with d155.3%2,163 questions4.15a51211
50Perplexity Decider Flash v1.2Perplexity4.5B44.1tied with Valen-Decision-1.5-9B55.2%2,163 questions10.46d771ad
51Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B43.2tied with Perplexity Decider Flash v1.254.6%2,163 questions21.128cf730
52OneJev-27B full fine-tuneOmniJev27.4B42.2tied with Open-Jev-27B-v1.153.8%2,163 questions3.6a5e5f55
53Mercury DecideInceptionNot disclosed41.9tied with OneJev-27B53.5%2,163 questions15.8no repo
54Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B39.451.5%2,163 questions5.745830d2
55Clef post-trainedCloudflare27.5B38.9tied with Verdin-4B-v1.151.1%2,163 questions11.02f3de3d
56Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B38.0tied with Clef50.4%2,163 questions13.7889f23e
57Clef-Flash post-trainedCloudflare9.5B37.2tied with Chinese Decision Pointer 4B49.8%2,163 questions2.7fde727a
58Winnow-12B LoRA, mergedEldanRing11.9B33.646.9%2,163 questions8.4b2b1421
59Cygnet unchanged baseBlockbrain Labs12B30.444.3%2,163 questions5.9no repo
60Metask-Jev-4B LoRA, mergedMetask Lab4.5B23.939.2%2,163 questions3.3ea20fe8
61StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B23.3tied with Metask-Jev-4B38.7%2,163 questions2.6f1e6cd1
62d1-2BLiquid AI1.9B21.3tied with StartLux-Decision-v1.1-2B-Instant37.1%2,163 questions5.610e7671
63GLiDE 2BFastino1.9B20.9tied with d1-2B36.8%2,163 questions17.200f122c
64KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B19.1tied with GLiDE 2B35.3%2,163 questions9.453245d4
65GLiDE 0.8BFastino750M15.132.1%2,163 questions18.89eacdee
66StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M12.129.7%2,163 questions4.86496ae6
67Strands Decider 2B LoRA + headStrands Agents2.3B10.8tied with StartLux-Decision-v1.1-0.8B-Instant28.7%2,163 questions8.82b52a62
68OneJev1.6-0.8B full fine-tuneOmniJev850M10.3tied with Strands Decider 2B28.2%2,163 questions10.0bcd88a3
69KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M10.3tied with OneJev1.6-0.8B28.2%2,163 questions10.56b27831
70GLiClass v4 Edge fine-tuneKnowledgator440M3.322.7%2,163 questions7.0885af1a
71GLiDE 0.4BFastino400M0.720.6%2,163 questions12.26bda7ab

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Claude Opus 5.5 high · chatAnthropicNot disclosed99.799.8%2,163 questions–no repo
2Qwen4-Max high · chatAlibabaNot disclosed99.7tied with Claude Opus 5.5 · high · chat99.8%2,163 questions–no repo
3Claude Fable 5.1 high · chatAnthropicNot disclosed99.7tied with Qwen4-Max · high · chat99.7%2,163 questions–no repo
4Claude Sonnet 5.5 high · chatAnthropicNot disclosed99.6tied with Claude Fable 5.1 · high · chat99.7%2,163 questions–no repo
5DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active99.5tied with Claude Sonnet 5.5 · high · chat99.6%2,163 questions–no repo
6GPT-6 Astra high · chatOpenAINot disclosed99.5tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101799.6%2,163 questions–no repo
7GPT-6.1 Sol high · chatOpenAINot disclosed99.5tied with GPT-6 Astra · high · chat99.6%2,163 questions–no repo
8Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active99.5tied with GPT-6.1 Sol · high · chat99.6%2,163 questions–no repo
9Gemini 4 Carbon high · chatGoogleNot disclosed99.5tied with Inkling-Small · high · chat · Jul 202699.6%2,163 questions–no repo
10Gemini 4 Hydrogen high · chatGoogleNot disclosed99.5tied with Gemini 4 Carbon · high · chat99.6%2,163 questions–no repo
11Muse Ember max · chatMetaNot disclosed99.5tied with Gemini 4 Hydrogen · high · chat99.6%2,163 questions–no repo
12Grok 4.7 high · chatxAINot disclosed99.4tied with Muse Ember · max · chat99.5%2,163 questions–no repo
13Inkling 2 Small high · chatThinking Machines594B17B active99.4tied with Grok 4.7 · high · chat99.5%2,163 questions–no repo
14Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed99.4tied with Inkling 2 Small · high · chat99.5%2,163 questions–no repo
15MiMo-V2.6-Flash high · chatXiaomi309B15B active99.3tied with Qwen3.8 Flash · no reasoning · chat99.4%2,163 questions–no repo
16Qwen3.5-4B reasoning on · chatAlibaba4.7B99.3tied with MiMo-V2.6-Flash · high · chat99.4%2,163 questions–851bf6e
17Gemma 4 31B reasoning off · chatGoogle31.3B99.2tied with Qwen3.5-4B · reasoning on · chat99.4%2,163 questions–842da37
18DeepSeek-V4-Flash chatDeepSeek285B13B active99.1tied with Gemma 4 31B · reasoning off · chat99.3%2,163 questions–no repo
19MiniCPM5-2B reasoning off · chatOpenBMB2.5B99.0tied with DeepSeek-V4-Flash · chat99.2%2,163 questions–42433e9
20GPT-6.1 Terra high · chatOpenAINot disclosed99.0tied with MiniCPM5-2B · reasoning off · chat99.2%2,163 questions–no repo
21Granite-4.2-3B reasoning on · chatIBM3.7B99.0tied with GPT-6.1 Terra · high · chat99.2%2,163 questions–4e9c275
22Qwen3.8-27B low · chatAlibaba27.8B99.0tied with Granite-4.2-3B · reasoning on · chat99.2%2,163 questions–1d4bf0f
23SenseNova-Loop-9B on · chatno maker9B98.9tied with Qwen3.8-27B · low · chat99.1%2,163 questions–3e1ba6e
24LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active98.8tied with SenseNova-Loop-9B · on · chat99.0%2,163 questions–1769516
25LFM3-1.7B reasoning on · chatLiquid AI1.7B98.7tied with LFM2.5-24B-A2B · default reasoning · chat98.9%2,163 questions–0dc4268
26Gemma 4 12B Unified default reasoning · chatGoogle12B98.6tied with LFM3-1.7B · reasoning on · chat98.8%2,163 questions–43c9d01
27GLM-5.3-Flash reasoning on · chatZ.ai320B18B active98.5tied with Gemma 4 12B Unified · default reasoning · chat98.8%2,163 questions–no repo
28MiniMax-M3.1-Flash high · chatMiniMax428B23B active97.8tied with GLM-5.3-Flash · reasoning on · chat98.2%2,163 questions–no repo
29Gemma 4 E4B default reasoning · chatGoogle8B97.6tied with MiniMax-M3.1-Flash · high · chat98.1%2,163 questions–4ecc2ba
30Gemma 4 E2B reasoning on · chatGoogle5.1B96.0tied with Gemma 4 E4B · default reasoning · chat96.8%2,163 questions–4adf157
31LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active95.3tied with Gemma 4 E2B · reasoning on · chat96.3%2,163 questions–dd83f5c
32Qwen3-14B reasoning off · chatAlibaba14.8B94.7tied with LFM2.5-8B-A1B · default reasoning · chat95.8%2,163 questions–40c0698
33Qwen3.5-2B reasoning off · chatAlibaba2.3B93.3tied with Qwen3-14B · reasoning off · chat94.6%2,163 questions–15852e8
34LFM2.5-2.6B default reasoning · chatLiquid AI2.7B92.5tied with Qwen3.5-2B · reasoning off · chat94.0%2,163 questions–7141b56
35GPT-6 Luna low · chatOpenAINot disclosed92.1tied with LFM2.5-2.6B · default reasoning · chat93.7%2,163 questions–no repo
36Erk-14B reasoning off · chateCloud Technology14.8B88.490.8%2,163 questions–8584b14
37Raon-1 nano reasoning on · chatKRAFTON3.5B86.9tied with Erk-14B · reasoning off · chat89.6%2,163 questions–fcf2d1d

2,163 questions per model · updated 19 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 2,163 Math questions right (accuracy 100%) · feed: bench-scores.json

Backgammon

Decision models

RankModelSizeScore ↑0–100, chance = 0Error rate (pr, lower is better)Calibration error ↓ECE, pointsCommit
1KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B78.326.21521,928 decisions–8e56aa1
2Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B77.3tied with KnowLine-31B-Gen1.5 · DGX Spark27.49141,740 decisions–8fcee85
3REAX 1.3 blendREAX57.4B77.1tied with Sieve 31B v3 · DGX Spark27.77231,998 decisions–fa4bf7e
4Gemini Decision 1 20261111GoogleNot disclosed76.9tied with REAX 1.327.96861,704 decisions–no repo
5Mammoth-1 post-trainedBase Labs27.4B76.4tied with Gemini Decision 1 · 2026111128.54322,021 decisions–61cec44
6Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B75.5tied with Mammoth-129.69571,855 decisions–866c0fc
7Drex Max fine-tuneNace.AI26.1B74.9tied with Turion Jev 31B · DGX Spark30.39942,148 decisions–cac65e1
8Drex Max v1.1Nace.AINot disclosed74.5tied with Drex Max30.93832,154 decisions–no repo
9d1-ultraLiquid AINot disclosed74.3tied with Drex Max v1.130.33742,071 decisions–no repo
10Jev 1.14TypeSafeNot disclosed73.1tied with d1-ultra31.73451,912 decisions–no repo
11d1.2-ultraLiquid AINot disclosed72.7tied with Jev 1.1433.06741,835 decisions–no repo
12GLiDE APIFastinoNot disclosed71.9tied with d1.2-ultra33.18262,159 decisions–no repo
13Jev 1.13TypeSafeNot disclosed71.6tied with GLiDE · API34.40941,911 decisions–no repo
14GLiDE v1.2Fastino26.1B71.1tied with Jev 1.1335.00462,003 decisions–a7e4589
15celeris-1-decisionCelerisNot disclosed70.6tied with GLiDE · v1.235.64281,922 decisions–no repo
16Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B69.5tied with celeris-1-decision36.93342,028 decisions–225c913
17Clef post-trainedCloudflare27.5B69.0tied with Perplexity Decider 27B v1.2 · DGX Spark36.53641,978 decisions–no repo
18Drex 2.0Nace.AI35.1B3B active68.3tied with Clef38.441,937 decisions–no repo
19Cygnet unchanged baseBlockbrain Labs12B68.2tied with Drex 2.038.55032,016 decisions–no repo
20Perplexity Decider 27B v1.1Perplexity26.1B68.0tied with Cygnet38.78292,052 decisions–no repo
21d1.2Liquid AINot disclosed66.9tied with Perplexity Decider 27B v1.140.06761,900 decisions–no repo
22StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B66.8tied with d1.240.14491,989 decisions–416088f
23SubQ Snap 1.1SubquadraticNot disclosed66.8tied with StartLux-Decision-v1.1-4B-Instant40.19972,326 decisions–no repo
24GPT-6 Luna Decisions APIOpenAINot disclosed64.243.31071,962 decisions–no repo
25SubQ Snap PreviewSubquadraticNot disclosed63.8tied with GPT-6 Luna · Decisions API43.87062,200 decisions–no repo
26Elephant-1 post-trainedBase Labs9.4B63.2tied with SubQ Snap Preview44.56162,044 decisions–b3f8a93
27d1-2BLiquid AI1.9B61.1tied with Elephant-147.10251,980 decisions–10e7671
28Mercury DecideInceptionNot disclosed60.4tied with d1-2B47.9481,853 decisions–no repo
29Cerno 9B v1.2 full fine-tuneApertix AI8.9B60.0tied with Mercury Decide48.46732,108 decisions–f02a632
30Winnow-12B LoRA, mergedEldanRing11.9B58.9tied with Cerno 9B v1.249.72772,015 decisions–no repo
31d1-Omni-2B NVIDIA RTX PRO 6000 BlackwellLiquid AI2.2B58.9tied with Winnow-12B49.79611,710 decisions–81222da
32StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B58.7tied with d1-Omni-2B · NVIDIA RTX PRO 6000 Blackwell50.04171,771 decisions–f1e6cd1
33KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B55.853.58962,153 decisions–779944d
34KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B55.0tied with KnowLine 4B Gen5.554.54791,955 decisions–53245d4
35Perplexity Decider Flash v1.2Perplexity4.5B50.859.64222,006 decisions–no repo
36StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M46.664.71331,882 decisions–6496ae6
37Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B45.9tied with StartLux-Decision-v1.1-0.8B-Instant65.55712,002 decisions–no repo
38GLiDE 2BFastino1.9B44.4tied with Open-Jev-27B-v1.167.35151,676 decisions–00f122c
39KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M40.671.94121,728 decisions–6b27831
40GLiDE 0.8BFastino750M33.680.36731,832 decisions–9eacdee
41OneJev1.6-0.8B full fine-tuneOmniJev850M31.9tied with GLiDE 0.8B82.51931,763 decisions–bcd88a3
42GLiDE 0.4BFastino400M6.5113.2191,355 decisions–6bda7ab
43GLiClass v4 Edge fine-tuneKnowledgator440M−10.2133.5221,405 decisions–885af1a

Chat models

RankModelSizeScore ↑0–100, chance = 0Error rate (pr, lower is better)Calibration error ↓ECE, pointsCommit
1GPT-6 Astra high · chatOpenAINot disclosed92.58.828561,489 decisions–no repo
2GPT-6.1 Sol high · chatOpenAINot disclosed91.6tied with GPT-6 Astra · high · chat9.899021,389 decisions–no repo
3Gemini 4 Argon low · chatGoogleNot disclosed89.3tied with GPT-6.1 Sol · high · chat12.57231,332 decisions–no repo
4Claude Fable 5.1 high · chatAnthropicNot disclosed88.1tied with Gemini 4 Argon · low · chat13.9551,286 decisions–no repo
5Claude Opus 5.5 high · chatAnthropicNot disclosed87.7tied with Claude Fable 5.1 · high · chat14.44361,477 decisions–no repo
6Gemini 3.8 Flash high · chatGoogleNot disclosed87.3tied with Claude Opus 5.5 · high · chat14.94642,168 decisions–no repo
7Claude Sonnet 5.5 high · chatAnthropicNot disclosed85.8tied with Gemini 3.8 Flash · high · chat16.71942,164 decisions–no repo
8Grok 4.7 high · chatxAINot disclosed84.0tied with Claude Sonnet 5.5 · high · chat18.78421,337 decisions–no repo
9DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseDeepSeekNot disclosed83.9tied with Grok 4.7 · high · chat18.99182,188 decisions–no repo
10Gemma 4 31B reasoning off · chatGoogle31.3B83.4tied with DeepSeek-V4.1-Flash · low · API + NVIDIA RTX PRO 6000 Blackwell · first release19.57512,122 decisions–no repo
11Muse Ember max · chatMetaNot disclosed83.4tied with Gemma 4 31B · reasoning off · chat20.10641,667 decisions–no repo
12Kimi K3.1 Pro low · chatMoonshot2.8T104B active76.228.04551,167 decisions–no repo
13Qwen3.8-27B low · chatAlibaba27.8B75.4tied with Kimi K3.1 Pro · low · chat29.03262,108 decisions–no repo
14GLM-5.3 low · chatZ.ai753.3B73.8tied with Qwen3.8-27B · low · chat30.86621,989 decisions–no repo
15Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed71.533.57772,027 decisions–no repo
16LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active60.647.68632,193 decisions–no repo

1,167 to 2,326 decisions per model · updated 19 November 2026 · 0 = 0 = a random legal play's error rate in the same league (League 1: 121.1, League 2: 117.9, League 3: 117.6; late entries: the league whose field they played); below 0 = worse than a random legal play. score = 100 × (1 − error rate / Random's error rate in that league); error rate (PR) = 500 × mean GNUbg equity loss per decision; 100 = 100 = error rate 0: the engine's best play on every decision · feed: bench-scores.json

Answer Check · does the text give this answer?

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B90.093.3%2,069 pairs2.8d69cdd5
2Copernicus-31BPoindexter Labs31.3B89.0tied with Karpo Decision 31B92.7%2,069 pairs11.0be71525
3KnowLine-31B-Gen1.5 DGX Spark LoRA, mergedPelaAI31.3B88.8tied with Copernicus-31B92.6%2,069 pairs5.78e56aa1
4Sieve 31B v3 DGX Spark full fine-tuneSthānika AI31.3B88.1tied with KnowLine-31B-Gen1.5 · DGX Spark92.1%2,069 pairs7.08fcee85
5KnowLine-31B-Gen2PelaAI31.3B87.7tied with Sieve 31B v3 · DGX Spark91.8%2,069 pairs7.2fd99a9e
6REAX 1.3 blendREAX57.4B87.0tied with KnowLine-31B-Gen291.3%2,069 pairs4.8fa4bf7e
7Turion Jev 31B DGX Spark LoRA, mergedTurion.AI31.3B86.4tied with REAX 1.391.0%2,069 pairs6.7866c0fc
8d1.2-ultraLiquid AINot disclosed86.4tied with Turion Jev 31B · DGX Spark91.0%2,069 pairs8.8no repo
9Perplexity Decider 27B v1.3Perplexity26.1B86.2tied with d1.2-ultra90.8%2,069 pairs2.89c2ebb2
10GLiDE 27B DGX SparkFastino26.1B86.1tied with Perplexity Decider 27B v1.390.7%2,069 pairs3.64d32743
11Microsoft-Decision-1.1MicrosoftNot disclosed85.4tied with GLiDE 27B · DGX Spark90.3%2,069 pairs3.0no repo
12Logits Sigma-1 Duo blendLogits57.4B85.1tied with Microsoft-Decision-1.190.1%2,069 pairs7.0a0c1b6e
13GLiDE APIFastinoNot disclosed85.1tied with Logits Sigma-1 Duo90.0%2,069 pairs3.0no repo
14GLiDE v1.2Fastino26.1B84.9tied with GLiDE · API89.9%2,069 pairs4.7a7e4589
15SubQ Snap 1.1SubquadraticNot disclosed84.9tied with GLiDE · v1.289.9%2,069 pairs2.3no repo
16SubQ Snap 1.2SubquadraticNot disclosed84.9tied with SubQ Snap 1.189.9%2,069 pairs2.3no repo
17d1-ultraLiquid AINot disclosed84.7tied with SubQ Snap 1.289.8%2,069 pairs11.4no repo
18Microsoft-Decision-1MicrosoftNot disclosed83.9tied with d1-ultra89.3%2,069 pairs2.4no repo
19vera-medium-omniscar-ai4.2B83.7tied with Microsoft-Decision-189.1%2,069 pairs12.41f23a6c
20Jev 1.14TypeSafeNot disclosed83.6tied with vera-medium-omni89.1%2,069 pairs2.8no repo
21Jeffan-9B-1211tattrongvu8.9B83.0tied with Jev 1.1488.6%2,069 pairs3.3862d1a7
22Jev 1.13TypeSafeNot disclosed83.0tied with Jeffan-9B-121188.6%2,069 pairs1.9no repo
23d1.1Liquid AINot disclosed82.5tied with Jev 1.1388.4%2,069 pairs8.7no repo
24Mammoth-1 post-trainedBase Labs27.4B82.4tied with d1.188.3%2,069 pairs1.961cec44
25GLiDE 9BFastino7.9B82.2tied with Mammoth-188.2%2,069 pairs7.2ab54df5
26Cerno 9B v1.2 full fine-tuneApertix AI8.9B81.9tied with GLiDE 9B87.9%2,069 pairs2.9f02a632
27StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B81.7tied with Cerno 9B v1.287.8%2,069 pairs8.3416088f
28d1.2Liquid AINot disclosed81.5tied with StartLux-Decision-v1.1-4B-Instant87.7%2,069 pairs8.5no repo
29Dexy-Decision-9B-V2 fine-tunePatrick-188.4B81.1tied with d1.287.4%2,069 pairs2.54f434e2
30KnowLine-12B-Gen1PelaAI12B80.9tied with Dexy-Decision-9B-V287.2%2,069 pairs6.8f26fbc2
31KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B80.6tied with KnowLine-12B-Gen187.1%2,069 pairs4.5779944d
32Adjudex 9B v5newsbang8.4B80.4tied with KnowLine 4B Gen5.587.0%2,069 pairs4.9ffdcb82
33d1Liquid AINot disclosed79.2tied with Adjudex 9B v586.1%2,069 pairs3.1no repo
34Perplexity Decider Flash v1.3Perplexity4.5B78.8tied with d185.9%2,069 pairs4.2d06fb3f
35StartLux-Decision-v1.1-9B-InstantStartLux9.7B78.8tied with Perplexity Decider Flash v1.385.9%2,069 pairs8.218a17f7
36Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B78.1tied with StartLux-Decision-v1.1-9B-Instant85.4%2,069 pairs6.7889f23e
37Perplexity Decider Flash v1.2Perplexity4.5B77.2tied with Chinese Decision Pointer 4B84.8%2,069 pairs2.86d771ad
38Gemini Decision 1 20261111GoogleNot disclosed77.0tied with Perplexity Decider Flash v1.284.7%2,069 pairs3.9no repo
39Elephant-1 post-trainedBase Labs9.4B76.9tied with Gemini Decision 1 · 2026111184.6%2,069 pairs3.7b3f8a93
40Winnow-12B LoRA, mergedEldanRing11.9B76.6tied with Elephant-184.4%2,069 pairs8.3b2b1421
41Clef post-trainedCloudflare27.5B76.5tied with Winnow-12B84.3%2,069 pairs4.02f3de3d
42KnowLine-9B-Gen1PelaAI9.7B76.1tied with Clef84.1%2,069 pairs3.6325378d
43Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B75.4tied with KnowLine-9B-Gen183.6%2,069 pairs3.445830d2
44KnowLine-9B-Gen2PelaAI9.7B75.3tied with Verdin-4B-v1.183.5%2,069 pairs4.58ec8eb6
45Valen-Decision-1.5-9BValen-Team8.4B74.3tied with KnowLine-9B-Gen282.8%2,069 pairs3.25a51211
46Drex 2.0Nace.AI35.1B3B active73.8tied with Valen-Decision-1.5-9B82.5%2,069 pairs11.1e646674
47Cygnet unchanged baseBlockbrain Labs12B72.9tied with Drex 2.081.9%2,069 pairs6.5no repo
48Clef-Flash post-trainedCloudflare9.5B69.879.8%2,069 pairs5.9fde727a
49StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B66.077.3%2,069 pairs7.2f1e6cd1
50GLiDE 2BFastino1.9B65.6tied with StartLux-Decision-v1.1-2B-Instant77.1%2,069 pairs4.000f122c
51Metask-Jev-4B LoRA, mergedMetask Lab4.5B61.774.5%2,069 pairs6.8ea20fe8
52KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B61.1tied with Metask-Jev-4B74.0%2,069 pairs2.753245d4
53GLiDE 0.8BFastino750M60.8tied with KnowLine-2B-Gen173.9%2,069 pairs3.89eacdee
54Strands Decider 2B LoRA + headStrands Agents2.3B60.3tied with GLiDE 0.8B73.6%2,069 pairs5.02b52a62
55OneJev1.6-0.8B full fine-tuneOmniJev850M50.066.7%2,069 pairs4.1bcd88a3
56d1-2BLiquid AI1.9B49.6tied with OneJev1.6-0.8B66.4%2,069 pairs7.210e7671
57StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M48.9tied with d1-2B65.9%2,069 pairs5.66496ae6
58GLiDE 0.4BFastino400M44.462.9%2,069 pairs7.76bda7ab
59GLiClass v4 Edge fine-tuneKnowledgator440M44.3tied with GLiDE 0.4B62.9%2,069 pairs9.8885af1a
60Decider 1meraGPTNot disclosed43.3tied with GLiClass v4 Edge62.2%2,069 pairs5.9no repo
61KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M38.559.0%2,069 pairs14.66b27831

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Inkling 2 Small high · chatThinking Machines594B17B active86.290.8%2,069 pairs–no repo
2Gemini 4 Hydrogen high · chatGoogleNot disclosed83.789.1%2,069 pairs–no repo
3MiniCPM5-2B reasoning off · chatOpenBMB2.5B79.286.2%2,069 pairs–42433e9
4Gemma 4 E4B default reasoning · chatGoogle8B77.2tied with MiniCPM5-2B · reasoning off · chat84.8%2,069 pairs–4ecc2ba
5Gemma 4 12B Unified default reasoning · chatGoogle12B77.0tied with Gemma 4 E4B · default reasoning · chat84.7%2,069 pairs–43c9d01
6Granite-4.2-3B reasoning on · chatIBM3.7B75.9tied with Gemma 4 12B Unified · default reasoning · chat83.9%2,069 pairs–4e9c275
7LFM3-1.7B reasoning on · chatLiquid AI1.7B75.8tied with Granite-4.2-3B · reasoning on · chat83.9%2,069 pairs–0dc4268
8Gemma 4 E4B thinking on · sampled (temperature 1.0) · chatGoogle8B75.1tied with LFM3-1.7B · reasoning on · chat83.4%2,069 pairs–4ecc2ba
9LFM3-1.7B reasoning off · chatLiquid AI1.7B74.9tied with Gemma 4 E4B · thinking on · sampled (temperature 1.0) · chat83.3%2,069 pairs–0dc4268
10Gemma 4 E2B reasoning on · chatGoogle5.1B71.180.8%2,069 pairs–4adf157
11Qwen3.5-2B reasoning off · chatAlibaba2.3B70.3tied with Gemma 4 E2B · reasoning on · chat80.2%2,069 pairs–15852e8
12Gemma 4 12B Unified thinking on · chatGoogle11.9B67.778.4%2,069 pairs–43c9d01
13MiniCPM5-2B reasoning on · chatOpenBMB2.5B67.0tied with Gemma 4 12B Unified · thinking on · chat78.0%2,069 pairs–42433e9
14Qwen3.5-4B reasoning on · chatAlibaba4.7B66.2tied with MiniCPM5-2B · reasoning on · chat77.5%2,069 pairs–851bf6e
15Qwen3.5-4B reasoning off · chatAlibaba4.7B64.6tied with Qwen3.5-4B · reasoning on · chat76.4%2,069 pairs–851bf6e
16Raon-1 nano reasoning on · chatKRAFTON3.5B63.8tied with Qwen3.5-4B · reasoning off · chat75.9%2,069 pairs–fcf2d1d
17LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active57.771.8%2,069 pairs–dd83f5c
18Gemma 4 E2B default reasoning · chatGoogle5.1B55.270.1%2,069 pairs–4adf157
19Gemma 4 E2B thinking off · chatGoogle5.1B54.8tied with Gemma 4 E2B · default reasoning · chat69.8%2,069 pairs–4adf157
20Granite-4.2-3B reasoning off · chatIBM3.7B52.8tied with Gemma 4 E2B · thinking off · chat68.5%2,069 pairs–4e9c275
21Qwen3.5-2B reasoning on · chatAlibaba2.3B50.266.8%2,069 pairs–15852e8
22Raon-1 nano reasoning off · chatKRAFTON3.5B40.660.4%2,069 pairs–fcf2d1d
23Nemotron-Labs-Sigma-3B-instruct default reasoning · chatNVIDIA3.8B4.336.2%2,069 pairs–057a084

2,069 pairs per model · updated 18 November 2026 · 0 = uniform chance: one of the three labels at random on every pair (accuracy 1/3); 100 = every one of the 2,069 pairs labelled right (accuracy 100%) · feed: bench-scores.json

Learning from experience

ModelPoints gained ↑
Qwen3.8-27B low · chatAlibabaBackgammon 19.0
GLM-5.3 low · chatZ.aiBackgammon 17.6
Claude Fable 5.1 high · chatAnthropicBackgammon 9.4
Gemini 4 Argon low · chatGoogleBackgammon 8.5
Gemma 4 31B reasoning off · chatGoogleBackgammon 8.5
DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseDeepSeekBackgammon 8.2
Gemini 3.8 Flash high · chatGoogleBackgammon 8.2
Claude Sonnet 5.5 high · chatAnthropicBackgammon 7.0
Claude Opus 5.5 high · chatAnthropicBackgammon 6.7
Kimi K3.1 Pro low · chatMoonshotBackgammon 4.8
Qwen3.8 Flash no reasoning · chatAlibabaBackgammon 2.7