All benches

Ranking

Type
Class
Score

Combined 3 benchmark groups Score 0–100, chance = 0 · higher is better ↑

  1. GPT-6 Astra high · chat96.1
  2. GPT-6.1 Sol high · chat95.7
  3. Claude Fable 5.1 high · chat93.7
  4. Claude Opus 5.5 high · chat93.2
  5. Claude Sonnet 5.5 high · chat88.3
  6. Muse Ember max · chat87.7
  7. Gemma 4 31B reasoning off · chat82.1
  8. Qwen3.8-27B low · chat80.2
  9. Qwen3.8 Flash no reasoning · chat74.1
  10. KnowLine-31B-Gen1.5 LoRA, merged73.2
  11. REAX 1.3 blend71.3
  12. Sieve 31B v3 full fine-tune69.4
  13. d1-ultra68.3tied
  14. Turion Jev 31B LoRA, merged68.2tied
  15. Gemini Decision 1 2026111168.0tied
  16. Mammoth-1 post-trained64.8
  17. Drex Max fine-tune63.4
  18. Drex Max v1.163.2tied
  19. GLiDE v1.262.6tied
  20. Perplexity Decider 27B v1.2 DGX Spark61.2
  21. SubQ Snap 1.158.5
  22. celeris-1-decision58.0tied
  23. Jev 1.14 (reference)57.1tied
  24. Jev 1.1355.3
  25. Drex 2.051.4
  26. Clef post-trained50.0
  27. Elephant-1 post-trained49.7tied
  28. Mercury Decide48.8tied
  29. Cerno 9B v1.2 full fine-tune47.5
  30. StartLux-Decision-v1.1-4B-Instant fine-tune47.4tied
  31. Cygnet unchanged base47.0tied
  32. Winnow-12B LoRA, merged44.6
  33. KnowLine 4B Gen5.5 LoRA, merged43.5tied
  34. Open-Jev-27B-v1.1 LoRA + head42.8tied
  35. Perplexity Decider Flash v1.241.4
  36. StartLux-Decision-v1.1-2B-Instant fine-tune32.1
  37. KnowLine-2B-Gen1 LoRA, merged31.2tied
  38. d1-2B30.5tied
  39. GLiDE 2B27.5
  40. StartLux-Decision-v1.1-0.8B-Instant fine-tune22.4
  41. KnowLine-0.8B-Gen1 LoRA, merged19.7
  42. GLiDE 0.8B19.7tied
  43. OneJev1.6-0.8B full fine-tune17.7
  44. GLiDE 0.4B3.7
  45. GLiClass v4 Edge fine-tune−0.8

OpenAIAnthropicMetaGoogleAlibabaPelaAIREAXSthānika AILiquid AITurion.AIBase LabsNace.AIFastinoPerplexitySubquadraticCelerisTypeSafeCloudflareInceptionApertix AIStartLuxBlockbrain LabsEldanRingZefan CaiOmniJevKnowledgator

No combined score for GLiDE · API: Fastino's API served a changed model between our benches' runs (Reading 6 Oct, Logic & Math 7 Nov, Backgammon 20 Oct to 7 Nov), so its bench scores are not one model's. (the rule)

Every model in view on every bench · Score 0–100, chance = 0 · higher is better ↑
Rank
GPT-6 Astra high · chat96.1TR 97.4 L&M 98.3 BG 92.5Not applicable96.497.995.2Not run95.999.592.5Not run
GPT-6.1 Sol high · chat95.7TR 97.2 L&M 98.2 BG 91.6Not applicable95.697.994.5Not run95.799.591.6Not run
Claude Fable 5.1 high · chat93.7TR 95.2 L&M 97.7 BG 88.1Not applicable94.595.595.0Not run94.099.788.1Not run
Claude Opus 5.5 high · chat93.2TR 93.7 L&M 98.2 BG 87.7Not applicable94.593.493.1Not run95.299.787.7Not run
Claude Sonnet 5.5 high · chat88.3TR 81.3 L&M 97.8 BG 85.8Not applicable81.581.282.6Not run94.399.685.8Not run
Muse Ember max · chat87.7TR 82.0 L&M 97.7 BG 83.4Not applicable87.879.687.082.894.399.583.4Not run
Gemma 4 31B reasoning off · chatOpen · 31B82.1TR 66.4 L&M 96.7 BG 83.4Not applicable88.657.088.1Not run91.799.283.4Not run
Qwen3.8-27B low · chatOpen · 27B80.2TR 68.3 L&M 96.8 BG 75.4Not applicable86.760.586.3Not run92.699.075.4Not run
Qwen3.8 Flash no reasoning · chat74.1TR 53.8 L&M 97.0 BG 71.5Not applicable84.041.083.8Not run92.499.471.5Not run
1KnowLine-31B-Gen1.5 LoRA, mergedOpen · 31B73.2TR 77.3 L&M 63.9 BG 78.375.3EN 92.0 Logic 48.8 Math 71.7 AC 88.891.071.492.037.348.871.778.388.8
2REAX 1.3 blendOpen · 57B71.3TR 72.4 L&M 64.5 BG 77.174.8EN 90.6 Logic 49.1 Math 72.5 AC 87.090.564.790.6Not run49.172.577.187.0
3Sieve 31B v3 full fine-tuneOpen · 31B69.4TR 67.7 L&M 63.2 BG 77.373.8EN 88.3 Logic 47.3 Math 71.4 AC 88.189.758.388.3Not run47.371.477.388.1
4d1-ultra68.3TR 71.3 L&M 59.4 BG 74.371.1EN 88.3 Logic 43.7 Math 67.6 AC 84.790.163.488.337.643.767.674.384.7
5Turion Jev 31B LoRA, mergedOpen · 31B68.2TR 75.1 L&M 54.1 BG 75.569.1EN 88.3 Logic 40.3 Math 61.3 AC 86.490.968.588.3Not run40.361.375.586.4
6Gemini Decision 1 2026111168.0TR 71.9 L&M 55.1 BG 76.966.9EN 88.8 Logic 37.6 Math 64.2 AC 77.090.164.288.858.137.664.276.977.0
7Mammoth-1 post-trainedOpen · 27B64.8TR 58.9 L&M 59.1 BG 76.4Not applicable88.046.587.7Not run43.167.576.4Not run
8Drex Max fine-tuneOpen · 27B63.4TR 59.7 L&M 55.5 BG 74.9Not applicable87.048.087.033.541.362.974.9Not run
9Drex Max v1.163.2TR 59.2 L&M 56.1 BG 74.5Not applicable85.748.085.631.443.462.674.5Not run
10GLiDE v1.2Open · 27B62.6TR 60.6 L&M 56.1 BG 71.169.8EN 87.9 Logic 44.0 Math 62.3 AC 84.987.049.387.9Not run44.062.371.184.9
11Perplexity Decider 27B v1.2 DGX Sparksetups differOpen · 27B61.2TR 58.8 L&M 55.2 BG 69.5Not applicable86.547.185.8Not run41.762.269.5Not run
12SubQ Snap 1.158.5TR 54.8 L&M 53.8 BG 66.867.1EN 82.4 Logic 40.4 Math 60.7 AC 84.983.842.582.432.240.460.766.884.9
13celeris-1-decision58.0TR 50.6 L&M 52.8 BG 70.6Not applicable81.537.480.1Not run34.162.570.6Not run
14Jev 1.1457.1TR 49.0 L&M 49.2 BG 73.164.5EN 82.6 Logic 35.2 Math 56.4 AC 83.681.335.282.629.135.256.473.183.6
15Jev 1.1355.3TR 46.9 L&M 47.5 BG 71.663.3EN 82.4 Logic 32.9 Math 55.1 AC 83.079.633.082.4Not run32.955.171.683.0
16Drex 2.0Open · 35B MoE · 3.0B active51.4TR 44.7 L&M 41.4 BG 68.3Not applicable76.831.082.4Not run27.348.768.3Not run
17Clef post-trainedOpen · 27B50.0TR 46.6 L&M 34.5 BG 69.0Not applicable81.331.883.3Not run26.038.969.0Not run
18Elephant-1 post-trainedOpen · 9.0B49.7TR 40.5 L&M 45.3 BG 63.260.8EN 81.7 Logic 33.0 Math 51.7 AC 76.975.625.581.7Not run33.051.763.276.9
19Mercury Decide48.8TR 47.9 L&M 38.3 BG 60.4Not applicable80.633.982.2Not run31.341.960.4Not run
20Cerno 9B v1.2 full fine-tuneOpen · 9.0B47.5TR 43.1 L&M 39.6 BG 60.059.6EN 82.4 Logic 29.1 Math 45.0 AC 81.974.729.682.4Not run29.145.060.081.9
21StartLux-Decision-v1.1-4B-Instant fine-tuneOpen · 4.0B47.4TR 34.8 L&M 40.5 BG 66.859.3EN 78.5 Logic 32.3 Math 44.8 AC 81.767.620.978.5Not run32.344.866.881.7
22Cygnet unchanged basesetups differOpen · 12B47.0TR 44.6 L&M 28.3 BG 68.251.8EN 79.9 Logic 24.1 Math 30.4 AC 72.978.530.279.9Not run24.130.468.272.9
23Winnow-12B LoRA, mergedOpen · 12B44.6TR 44.3 L&M 30.5 BG 58.953.7EN 79.9 Logic 24.7 Math 33.6 AC 76.677.530.279.9Not run24.733.658.976.6
24KnowLine 4B Gen5.5 LoRA, mergedOpen · 4.0B43.5TR 35.6 L&M 39.1 BG 55.857.4EN 76.0 Logic 27.9 Math 45.0 AC 80.668.621.676.012.727.945.055.880.6
25Open-Jev-27B-v1.1 LoRA + headOpen · 27B42.8TR 44.6 L&M 38.1 BG 45.9Not applicable78.730.181.7Not run28.243.245.9Not run
26Perplexity Decider Flash v1.2Open · 4.0B41.4TR 34.7 L&M 38.9 BG 50.856.9EN 77.4 Logic 28.9 Math 44.1 AC 77.265.921.477.4Not run28.944.150.877.2
27StartLux-Decision-v1.1-2B-Instant fine-tuneOpen · 2.0B32.1TR 17.8 L&M 19.9 BG 58.741.2EN 62.1 Logic 13.4 Math 23.3 AC 66.041.17.962.14.313.423.358.766.0
28KnowLine-2B-Gen1 LoRA, mergedOpen · 2.0B31.2TR 21.0 L&M 17.6 BG 55.039.7EN 64.1 Logic 14.7 Math 19.1 AC 61.142.811.764.1Not run14.719.155.061.1
29d1-2BOpen · 1.9B30.5TR 11.7 L&M 18.6 BG 61.135.3EN 57.0 Logic 13.3 Math 21.3 AC 49.626.15.657.04.213.321.361.149.6
30GLiDE 2BOpen · 1.9B27.5TR 19.1 L&M 19.0 BG 44.441.1EN 62.5 Logic 15.3 Math 20.9 AC 65.639.010.762.5Not run15.320.944.465.6
31StartLux-Decision-v1.1-0.8B-Instant fine-tuneOpen · 0.8B22.4TR 10.7 L&M 10.0 BG 46.629.1EN 49.5 Logic 5.9 Math 12.1 AC 48.918.87.349.55.35.912.146.648.9
32KnowLine-0.8B-Gen1 LoRA, mergedOpen · 0.8B19.7TR 8.9 L&M 9.6 BG 40.626.4EN 48.6 Logic 8.3 Math 10.3 AC 38.522.83.048.64.88.310.340.638.5
33GLiDE 0.8BOpen · 0.8B19.7TR 11.6 L&M 13.9 BG 33.634.5EN 50.9 Logic 11.4 Math 15.1 AC 60.828.04.650.9Not run11.415.133.660.8
34OneJev1.6-0.8B full fine-tuneOpen · 0.8B17.7TR 11.4 L&M 9.8 BG 31.929.6EN 49.3 Logic 8.8 Math 10.3 AC 50.024.55.849.38.18.810.331.950.0
35GLiDE 0.4BOpen · 0.4B3.7TR 2.8 L&M 1.8 BG 6.524.6EN 49.3 Logic 4.0 Math 0.7 AC 44.45.71.649.3Not run4.00.76.544.4
36GLiClass v4 Edge fine-tuneOpen · 0.4B−0.8TR 4.2 L&M 3.7 BG −10.225.0EN 47.9 Logic 4.4 Math 3.3 AC 44.37.42.847.9Not run4.43.3−10.244.3
Adjudex 9B v5Open · 9.0BNot applicable62.1EN 82.2 Logic 33.0 Math 52.6 AC 80.476.026.782.2Not run33.052.6Not run80.4
Chinese Decision Pointer 4B LoRA, mergedOpen · 4.0BNot applicable54.3EN 75.5 Logic 25.4 Math 38.0 AC 78.163.620.575.5Not run25.438.0Not run78.1
Claude Fable 5.2 max · chatNot applicableNot applicable97.197.997.5Not runNot runNot runNot runNot run
Claude Haiku 5.5 max · chatNot applicableNot applicable83.630.490.9Not runNot runNot runNot runNot run
Clef-Flash post-trainedOpen · 9.0BNot applicable52.1EN 77.8 Logic 23.7 Math 37.2 AC 69.866.317.677.8Not run23.737.2Not run69.8
Copernicus-31BOpen · 31BNot applicable75.0EN 91.1 Logic 48.3 Math 71.7 AC 89.091.471.291.1Not run48.371.7Not run89.0
Decider 1Not applicableNot applicable18.34.944.7Not runNot runNot runNot run43.3
DeepSeek-V4-Flash chat285B MoE · 13B activeNot applicableNot applicableNot runNot runNot runNot runNot run99.1Not runNot run
DeepSeek-V4.1-Flash reasoning on · chat · first releaseOpen · 552B MoE · 16B activeNot applicableNot applicable91.889.3Not runNot runNot runNot runNot runNot run
DeepSeek-V4.1-Flash reasoning on · chat · 1017setups differOpen · 552B MoE · 16B activeNot applicableNot applicable92.688.590.6Not run95.299.5Not runNot run
DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseNot applicableNot applicableNot runNot runNot runNot runNot runNot run83.9Not run
Dexy-Decision-9B-V2 fine-tuneOpen · 9.0BNot applicable60.2EN 79.4 Logic 31.7 Math 48.5 AC 81.174.524.679.4Not run31.748.5Not run81.1
Erk-14B reasoning off · chatOpen · 14BNot applicableNot applicable65.316.765.0Not run42.888.4Not runNot run
GLM-5.3 low · chatNot applicableNot applicable84.661.284.7Not runNot runNot run73.8Not run
GLM-5.3-Flash reasoning on · chatsetups differOpen · 320B MoE · 18B activeNot applicableNot applicable86.767.987.0Not run91.498.5Not runNot run
GLiDE APINot applicableNot applicable84.849.3Not runNot run43.163.571.985.1
GLiDE 27BOpen · 27BNot applicable71.9EN 88.6 Logic 48.0 Math 65.0 AC 86.188.653.488.637.448.065.0Not run86.1
GLiDE 9BOpen · 7.9BNot applicable61.8EN 82.9 Logic 34.8 Math 47.4 AC 82.275.225.182.9Not run34.847.4Not run82.2
GLiNER2.5 MultiOpen · 0.3BNot applicableNot applicable−3.5−3.0−4.7Not runNot runNot runNot runNot run
GLiNER2.5-multi-Decide fine-tuneOpen · 0.3BNot applicableNot applicable1.3−0.21.5Not runNot runNot runNot runNot run
GPT-6 Luna low · chatNot applicableNot applicable84.250.282.9Not run47.492.1Not runNot run
GPT-6 Luna Decisions APINot applicableNot applicable81.942.980.115.8Not runNot run64.2Not run
GPT-6.1 Terra high · chatNot applicableNot applicable95.291.895.2Not run92.499.0Not runNot run
Gemini 3.8 Flash high · chatNot applicableNot applicable96.498.195.0Not runNot runNot run87.3Not run
Gemini 4 Argon low · chatNot applicableNot applicable96.295.195.0Not runNot runNot run89.3Not run
Gemini 4 Carbon high · chatNot applicableNot applicable97.098.592.9Not run95.999.5Not runNot run
Gemini 4 Hydrogen high · chatNot applicable93.5EN 95.9 Logic 95.1 Math 99.5 AC 83.796.698.195.9Not run95.199.5Not run83.7
Gemma 4 12B Unified default reasoning · chatOpen · 12BNot applicable85.9EN 83.5 Logic 84.7 Math 98.6 AC 77.0Not runNot run83.5Not run84.798.6Not run77.0
Gemma 4 12B Unified thinking on · chatOpen · 12BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run67.7
Gemma 4 E2B reasoning on · chatOpen · 5.1BNot applicable75.9EN 70.3 Logic 66.1 Math 96.0 AC 71.1Not runNot run70.3Not run66.196.0Not run71.1
Gemma 4 E2B default reasoning · chatOpen · 5.1BNot applicableNot applicableNot runNot run68.2Not runNot runNot runNot run55.2
Gemma 4 E2B thinking off · chatOpen · 5.1BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run54.8
Gemma 4 E4B default reasoning · chatOpen · 8.0BNot applicable83.0EN 78.1 Logic 79.0 Math 97.6 AC 77.272.036.378.128.679.097.6Not run77.2
Gemma 4 E4B thinking on · sampled (temperature 1.0) · chatOpen · 8.0BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run75.1
Granite-4.2-3B reasoning on · chatOpen · 3.0BNot applicable83.1EN 68.0 Logic 89.6 Math 99.0 AC 75.9Not runNot run68.0Not run89.699.0Not run75.9
Grok 4.7 high · chatNot applicableNot applicable93.789.791.8Not runNot run99.484.0Not run
Inkling 2 Small high · chat594B MoE · 17B activeNot applicableNot applicableNot runNot runNot runNot run94.699.4Not run86.2
Inkling-Small high · chat · Jul 2026Open · 276B MoE · 12B activeNot applicableNot applicable84.071.185.4Not run94.299.5Not runNot run
Jeffan-9B-1211Open · 9.0BNot applicable63.1EN 84.5 Logic 35.2 Math 49.8 AC 83.078.732.584.5Not run35.249.8Not run83.0
Karpo Decision 31BOpen · 31BNot applicable77.0EN 90.2 Logic 50.9 Math 76.9 AC 90.091.863.990.2Not run50.976.9Not run90.0
Kimi K3.1 Pro low · chatOpen · 2.8T MoE · 104B activeNot applicableNot applicable92.272.690.4Not runNot runNot run76.2Not run
KnowLine-12B-Gen1Open · 12BNot applicable65.1EN 83.3 Logic 38.2 Math 57.8 AC 80.982.540.683.3Not run38.257.8Not run80.9
KnowLine-31B-Gen2Open · 31BNot applicable75.5EN 90.4 Logic 48.7 Math 75.3 AC 87.791.870.090.4Not run48.775.3Not run87.7
KnowLine-9B-Gen1Open · 9.0BNot applicable59.7EN 82.4 Logic 30.3 Math 49.7 AC 76.174.829.782.4Not run30.349.7Not run76.1
KnowLine-9B-Gen2Open · 9.0BNot applicable59.2EN 81.5 Logic 29.9 Math 50.1 AC 75.374.527.681.5Not run29.950.1Not run75.3
LFM2.5-2.6B default reasoning · chatOpen · 2.7BNot applicableNot applicable31.811.267.3Not run69.292.5Not runNot run
LFM2.5-24B-A2B default reasoning · chatOpen · 24B MoE · 2.5B activeNot applicableNot applicableNot runNot run78.3Not run83.398.860.6Not run
LFM2.5-8B-A1B default reasoning · chatOpen · 8.3B MoE · 1.5B activeNot applicable66.6EN 63.0 Logic 50.2 Math 95.3 AC 57.77.0−1.263.0Not run50.295.3Not run57.7
LFM3-1.7B reasoning on · chatOpen · 1.7BNot applicable80.3EN 64.7 Logic 81.9 Math 98.7 AC 75.830.39.264.7Not run81.998.7Not run75.8
LFM3-1.7B reasoning off · chatOpen · 1.7BNot applicableNot applicable21.55.763.9Not runNot runNot runNot run74.9
LayaOpen · 0.4BNot applicableNot applicable1.1−2.23.6Not runNot runNot runNot runNot run
Laya MultilingualOpen · 0.3BNot applicableNot applicable−3.7−2.2−5.8Not runNot runNot runNot runNot run
Logits Sigma-1 Duo blendOpen · 57BNot applicable72.5EN 88.8 Logic 47.2 Math 69.0 AC 85.189.959.888.8Not run47.269.0Not run85.1
Metask-Jev-4B LoRA, mergedOpen · 4.5BNot applicable43.1EN 71.4 Logic 15.1 Math 23.9 AC 61.753.514.271.4Not run15.123.9Not run61.7
MiMo-V2.6-Flash high · chatOpen · 309B MoE · 15B activeNot applicableNot applicable83.257.486.3Not run93.999.3Not runNot run
Microsoft-Decision-1Not applicable61.5EN 82.2 Logic 30.6 Math 49.2 AC 83.966.818.782.2Not run30.649.2Not run83.9
MiniCPM5-2B reasoning off · chatOpen · 2.5BNot applicable84.3EN 72.7 Logic 86.1 Math 99.0 AC 79.2−16.8−18.572.7Not run86.199.0Not run79.2
MiniCPM5-2B reasoning off · chatOpen · 2.5BNot applicableNot applicableNot runNot run59.7Not runNot runNot runNot run67.0
MiniMax-M3.1-Flash high · chatOpen · 428B MoE · 23B activeNot applicableNot applicable78.749.380.6Not run92.497.8Not runNot run
OneJev-27B full fine-tuneOpen · 27BNot applicableNot applicable81.535.982.926.625.842.2Not runNot run
Perplexity Decider 27B v1Open · 27BNot applicableNot applicable82.536.985.6Not runNot runNot runNot runNot run
Perplexity Decider 27B v1.1setups differOpen · 27BNot applicableNot applicable86.346.8Not runNot runNot runNot run68.0Not run
Perplexity Decider 27B v1.2 API · with webOpen · 27BNot applicableNot applicable86.150.288.1Not run40.761.6Not runNot run
Perplexity Decider 27B v1.3Open · 27BNot applicable70.6EN 87.0 Logic 43.4 Math 65.6 AC 86.286.349.687.0Not run43.465.6Not run86.2
Perplexity Decider Flash v1.3Open · 4.0BNot applicable58.6EN 79.7 Logic 30.9 Math 44.9 AC 78.868.024.079.7Not run30.944.9Not run78.8
Qwen3-14B reasoning off · chatOpen · 15BNot applicableNot applicable65.118.174.6Not run56.794.7Not runNot run
Qwen3.5-2B reasoning off · chatOpen · 2.0BNot applicable72.4EN 69.8 Logic 56.0 Math 93.3 AC 70.317.92.069.8Not run56.093.3Not run70.3
Qwen3.5-2B reasoning off · chatOpen · 2.3BNot applicableNot applicableNot runNot run59.1Not runNot runNot runNot run50.2
Qwen3.5-397B-A17B high · chatOpen · 397B MoE · 17B activeNot applicableNot applicable88.874.089.3Not run93.9Not runNot runNot run
Qwen3.5-4B reasoning on · chatOpen · 4.0BNot applicable84.9EN 83.8 Logic 90.4 Math 99.3 AC 66.273.937.983.8Not run90.499.3Not run66.2
Qwen3.5-4B reasoning off · chatOpen · 4.0BNot applicableNot applicable60.620.975.5Not runNot runNot runNot run64.6
Qwen4-Max high · chatNot applicableNot applicable93.388.193.1Not run95.399.7Not runNot run
Raon-1 nano reasoning off · chatOpen · 3.1BNot applicable56.5EN 42.2 Logic 56.3 Math 86.9 AC 40.6Not runNot run42.2Not run56.386.9Not run40.6
StartLux-Decision-v1.1-9B-InstantOpen · 9.0BNot applicable60.9EN 83.5 Logic 33.2 Math 48.0 AC 78.873.524.483.5Not run33.248.0Not run78.8
Strands Decider 2B LoRA + headOpen · 2.0BNot applicable26.5EN 31.9 Logic 3.0 Math 10.8 AC 60.321.14.331.9Not run3.010.8Not run60.3
SubQ Snap PreviewNot applicableNot applicable84.644.682.4Not runNot runNot run63.8Not run
Valen-Decision-1.5-9BOpen · 9.0BNot applicable57.1EN 79.7 Logic 30.4 Math 44.2 AC 74.368.818.879.7Not run30.444.2Not run74.3
Verdin-4B-v1.1 LoRA, mergedOpen · 4.0BNot applicable54.3EN 77.8 Logic 24.7 Math 39.4 AC 75.464.020.577.8Not run24.739.4Not run75.4
celeris-1.1-decisionNot applicableNot applicableNot runNot run79.7Not run34.162.3Not runNot run
d1Not applicable58.3EN 83.5 Logic 26.0 Math 44.5 AC 79.278.532.683.5Not run26.044.5Not run79.2
d1-Omni-2B NVIDIA RTX PRO 6000 BlackwellOpen · 2.2BNot applicableNot applicableNot runNot runNot runNot runNot runNot run58.9Not run
d1.1Not applicable64.2EN 85.4 Logic 32.6 Math 56.5 AC 82.580.839.685.4Not run32.656.5Not run82.5
d1.2Not applicableNot applicable82.539.386.333.736.456.9Not runNot run
d1.2-ultraNot applicable69.0EN 87.9 Logic 39.4 Math 62.3 AC 86.487.248.687.931.739.462.3Not run86.4
d3 fine-tuneOpen · 26BNot applicableNot applicable86.548.487.235.643.163.0Not runNot run
d3-ultra fine-tuneOpen · 26BNot applicableNot applicable87.248.086.731.843.264.4Not runNot run
vera-medium-omniOpen · 4.2BNot applicable59.8EN 79.4 Logic 30.1 Math 46.0 AC 83.769.322.379.4Not run30.146.0Not run83.7

Perplexity Decider 27B v1.2 · DGX Spark, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

Cygnet, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

DeepSeek-V4.1-Flash · reasoning on · chat · 1017, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

GLM-5.3-Flash · reasoning on · chat, different setups of one model: Logic and Math: a hosted API; the other benches: our hardware.

Perplexity Decider 27B v1.1, different setups of one model: Backgammon: our hardware; the other benches: a hosted API.

How it's scored

Methodology: every rule, the sources and how models are matched across benches.

Head to head

Score 0–100, chance = 0 · higher is better ↑ · the difference in points; "tied" where the two are too close to tell apart (the rule)
BenchKnowLine-31B-Gen1.5REAX 1.3Difference
Combined ↑73.271.3KnowLine-31B-Gen1.5 ahead by 1.8
Decision Index ↑75.374.8tied (0.5)
Turkish Reading ↑91.090.5tied (0.6)
Turkish Grammar ↑71.464.7KnowLine-31B-Gen1.5 ahead by 6.6
English Reading ↑92.090.6tied (1.4)
Vision ↑37.3Not runonly KnowLine-31B-Gen1.5 ran it
Logic ↑48.849.1tied (−0.3)
Math ↑71.772.5tied (−0.8)
Backgammon ↑78.377.1tied (1.3)
Answer Check ↑88.887.0tied (1.9)

Full results

Decision models · highest first by Combined · Score 0–100, chance = 0 · ECE ↓ under each score
ModelSizeCombined ↑Decision Index ↑Turkish Reading ↑Turkish Grammar ↑English Reading ↑Vision ↑Logic ↑Math ↑Backgammon ↑Answer Check ↑Learning from experience ↑points gainedLatency ↓median, p95
KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B73.2TR 77.3 L&M 63.9 BG 78.375.3EN 92.0 Logic 48.8 Math 71.7 AC 88.891.0ECE 3.4 ↓71.4ECE 3.9 ↓92.0ECE 5.0 ↓37.3ECE 6.1 ↓48.8ECE 2.6 ↓71.7ECE 3.6 ↓78.388.8ECE 5.7 ↓–1,051 msp95 1,533 ms
REAX 1.3 blendREAX57.4B71.3TR 72.4 L&M 64.5 BG 77.174.8EN 90.6 Logic 49.1 Math 72.5 AC 87.090.5ECE 2.1 ↓64.7ECE 5.1 ↓90.6ECE 3.6 ↓Not run49.1ECE 3.7 ↓72.5ECE 7.1 ↓77.187.0ECE 4.8 ↓–not timed yet
Sieve 31B v3 full fine-tuneSthānika AI31.3B69.4TR 67.7 L&M 63.2 BG 77.373.8EN 88.3 Logic 47.3 Math 71.4 AC 88.189.7ECE 3.5 ↓58.3ECE 3.3 ↓88.3ECE 4.3 ↓Not run47.3ECE 2.1 ↓71.4ECE 4.3 ↓77.388.1ECE 7.0 ↓–not timed yet
d1-ultraLiquid AINot disclosed68.3TR 71.3 L&M 59.4 BG 74.371.1EN 88.3 Logic 43.7 Math 67.6 AC 84.790.1ECE 3.6 ↓63.4ECE 4.2 ↓88.3ECE 4.4 ↓37.6ECE 3.5 ↓43.7ECE 3.2 ↓67.6ECE 2.3 ↓74.384.7ECE 11.4 ↓–not timed yet
Turion Jev 31B LoRA, mergedTurion.AI31.3B68.2TR 75.1 L&M 54.1 BG 75.569.1EN 88.3 Logic 40.3 Math 61.3 AC 86.490.9ECE 9.5 ↓68.5ECE 12.0 ↓88.3ECE 11.6 ↓Not run40.3ECE 2.9 ↓61.3ECE 4.7 ↓75.586.4ECE 6.7 ↓–not timed yet
Gemini Decision 1 20261111GoogleNot disclosed68.0TR 71.9 L&M 55.1 BG 76.966.9EN 88.8 Logic 37.6 Math 64.2 AC 77.090.1ECE 2.8 ↓64.2ECE 2.9 ↓88.8ECE 4.5 ↓58.1ECE 5.2 ↓37.6ECE 3.1 ↓64.2ECE 2.5 ↓76.977.0ECE 3.9 ↓–not timed yet
Mammoth-1 post-trainedBase Labs27.4B64.8TR 58.9 L&M 59.1 BG 76.4Not applicable88.0ECE 2.0 ↓46.5ECE 7.9 ↓87.7ECE 2.4 ↓Not run43.1ECE 5.3 ↓67.5ECE 3.0 ↓76.4Not run–not timed yet
Drex Max fine-tuneNace.AI26.1B63.4TR 59.7 L&M 55.5 BG 74.9Not applicable87.0ECE 3.4 ↓48.0ECE 8.0 ↓87.0ECE 2.6 ↓33.5ECE 9.3 ↓41.3ECE 16.2 ↓62.9ECE 11.8 ↓74.9Not run–not timed yet
Drex Max v1.1Nace.AINot disclosed63.2TR 59.2 L&M 56.1 BG 74.5Not applicable85.7ECE 3.4 ↓48.0ECE 2.7 ↓85.6ECE 3.8 ↓31.4ECE 5.2 ↓43.4ECE 6.1 ↓62.6ECE 5.7 ↓74.5Not run–not timed yet
GLiDE v1.2Fastino26.1B62.6TR 60.6 L&M 56.1 BG 71.169.8EN 87.9 Logic 44.0 Math 62.3 AC 84.987.0ECE 3.5 ↓49.3ECE 3.0 ↓87.9ECE 2.7 ↓Not run44.0ECE 5.9 ↓62.3ECE 5.2 ↓71.184.9ECE 4.7 ↓–not timed yet
Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B61.2TR 58.8 L&M 55.2 BG 69.5Not applicable86.5ECE 3.1 ↓47.1ECE 6.2 ↓85.8ECE 3.7 ↓Not run41.7ECE 3.4 ↓62.2ECE 3.2 ↓69.5Not run–not timed yet
SubQ Snap 1.1SubquadraticNot disclosed58.5TR 54.8 L&M 53.8 BG 66.867.1EN 82.4 Logic 40.4 Math 60.7 AC 84.983.8ECE 3.6 ↓42.5ECE 14.5 ↓82.4ECE 3.2 ↓32.2ECE 11.7 ↓40.4ECE 12.9 ↓60.7ECE 9.5 ↓66.884.9ECE 2.3 ↓–not timed yet
celeris-1-decisionCelerisNot disclosed58.0TR 50.6 L&M 52.8 BG 70.6Not applicable81.5ECE 6.2 ↓37.4ECE 5.3 ↓80.1ECE 5.2 ↓Not run34.1ECE 10.7 ↓62.5ECE 4.3 ↓70.6Not run–not timed yet
Jev 1.14TypeSafeNot disclosed57.1TR 49.0 L&M 49.2 BG 73.164.5EN 82.6 Logic 35.2 Math 56.4 AC 83.681.3ECE 2.2 ↓35.2ECE 4.0 ↓82.6ECE 2.2 ↓29.1ECE 3.7 ↓35.2ECE 3.3 ↓56.4ECE 2.8 ↓73.183.6ECE 2.8 ↓–not timed yet
Jev 1.13TypeSafeNot disclosed55.3TR 46.9 L&M 47.5 BG 71.663.3EN 82.4 Logic 32.9 Math 55.1 AC 83.079.6ECE 3.8 ↓33.0ECE 6.5 ↓82.4ECE 3.1 ↓Not run32.9ECE 5.9 ↓55.1ECE 2.6 ↓71.683.0ECE 1.9 ↓–not timed yet
Drex 2.0Nace.AI35.1B3B active51.4TR 44.7 L&M 41.4 BG 68.3Not applicable76.8ECE 10.1 ↓31.0ECE 2.0 ↓82.4ECE 6.9 ↓Not run27.3ECE 4.3 ↓48.7ECE 1.9 ↓68.3Not run–not timed yet
Clef post-trainedCloudflare27.5B50.0TR 46.6 L&M 34.5 BG 69.0Not applicable81.3ECE 4.0 ↓31.8ECE 15.1 ↓83.3ECE 3.1 ↓Not run26.0ECE 15.7 ↓38.9ECE 11.0 ↓69.0Not run–not timed yet
Elephant-1 post-trainedBase Labs9.4B49.7TR 40.5 L&M 45.3 BG 63.260.8EN 81.7 Logic 33.0 Math 51.7 AC 76.975.6ECE 4.3 ↓25.5ECE 15.4 ↓81.7ECE 2.7 ↓Not run33.0ECE 8.6 ↓51.7ECE 4.3 ↓63.276.9ECE 3.7 ↓–not timed yet
Mercury DecideInceptionNot disclosed48.8TR 47.9 L&M 38.3 BG 60.4Not applicable80.6ECE 7.5 ↓33.9ECE 19.0 ↓82.2ECE 6.6 ↓Not run31.3ECE 15.4 ↓41.9ECE 15.8 ↓60.4Not run–not timed yet
Cerno 9B v1.2 full fine-tuneApertix AI8.9B47.5TR 43.1 L&M 39.6 BG 60.059.6EN 82.4 Logic 29.1 Math 45.0 AC 81.974.7ECE 3.0 ↓29.6ECE 14.2 ↓82.4ECE 4.2 ↓Not run29.1ECE 17.0 ↓45.0ECE 15.7 ↓60.081.9ECE 2.9 ↓–not timed yet
StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B47.4TR 34.8 L&M 40.5 BG 66.859.3EN 78.5 Logic 32.3 Math 44.8 AC 81.767.6ECE 2.6 ↓20.9ECE 10.3 ↓78.5ECE 3.2 ↓Not run32.3ECE 3.3 ↓44.8ECE 6.7 ↓66.881.7ECE 8.3 ↓–not timed yet
Cygnet unchanged baseBlockbrain Labs12B47.0TR 44.6 L&M 28.3 BG 68.251.8EN 79.9 Logic 24.1 Math 30.4 AC 72.978.5ECE 4.1 ↓30.2ECE 18.9 ↓79.9ECE 4.3 ↓Not run24.1ECE 11.2 ↓30.4ECE 5.9 ↓68.272.9ECE 6.5 ↓–not timed yet
Winnow-12B LoRA, mergedEldanRing11.9B44.6TR 44.3 L&M 30.5 BG 58.953.7EN 79.9 Logic 24.7 Math 33.6 AC 76.677.5ECE 9.3 ↓30.2ECE 24.1 ↓79.9ECE 8.7 ↓Not run24.7ECE 14.4 ↓33.6ECE 8.4 ↓58.976.6ECE 8.3 ↓–not timed yet
KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B43.5TR 35.6 L&M 39.1 BG 55.857.4EN 76.0 Logic 27.9 Math 45.0 AC 80.668.6ECE 6.2 ↓21.6ECE 16.6 ↓76.0ECE 5.8 ↓12.7ECE 11.0 ↓27.9ECE 13.7 ↓45.0ECE 8.6 ↓55.880.6ECE 4.5 ↓–99 msp95 149 ms
Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B42.8TR 44.6 L&M 38.1 BG 45.9Not applicable78.7ECE 43.6 ↓30.1ECE 16.4 ↓81.7ECE 45.3 ↓Not run28.2ECE 12.9 ↓43.2ECE 21.1 ↓45.9Not run–not timed yet
Perplexity Decider Flash v1.2Perplexity4.5B41.4TR 34.7 L&M 38.9 BG 50.856.9EN 77.4 Logic 28.9 Math 44.1 AC 77.265.9ECE 3.7 ↓21.4ECE 11.0 ↓77.4ECE 4.1 ↓Not run28.9ECE 6.0 ↓44.1ECE 10.4 ↓50.877.2ECE 2.8 ↓–not timed yet
StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B32.1TR 17.8 L&M 19.9 BG 58.741.2EN 62.1 Logic 13.4 Math 23.3 AC 66.041.1ECE 3.3 ↓7.9ECE 9.4 ↓62.1ECE 8.2 ↓4.3ECE 3.6 ↓13.4ECE 5.3 ↓23.3ECE 2.6 ↓58.766.0ECE 7.2 ↓–40 msp95 48 ms
KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B31.2TR 21.0 L&M 17.6 BG 55.039.7EN 64.1 Logic 14.7 Math 19.1 AC 61.142.8ECE 1.9 ↓11.7ECE 11.4 ↓64.1ECE 5.1 ↓Not run14.7ECE 8.3 ↓19.1ECE 9.4 ↓55.061.1ECE 2.7 ↓–62 msp95 87 ms
d1-2BLiquid AI1.9B30.5TR 11.7 L&M 18.6 BG 61.135.3EN 57.0 Logic 13.3 Math 21.3 AC 49.626.1ECE 4.4 ↓5.6ECE 6.7 ↓57.0ECE 7.9 ↓4.2ECE 1.1 ↓13.3ECE 6.9 ↓21.3ECE 5.6 ↓61.149.6ECE 7.2 ↓–not timed yet
GLiDE 2BFastino1.9B27.5TR 19.1 L&M 19.0 BG 44.441.1EN 62.5 Logic 15.3 Math 20.9 AC 65.639.0ECE 16.5 ↓10.7ECE 22.5 ↓62.5ECE 8.2 ↓Not run15.3ECE 16.2 ↓20.9ECE 17.2 ↓44.465.6ECE 4.0 ↓–34 msp95 47 ms
StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M22.4TR 10.7 L&M 10.0 BG 46.629.1EN 49.5 Logic 5.9 Math 12.1 AC 48.918.8ECE 5.0 ↓7.3ECE 4.6 ↓49.5ECE 8.7 ↓5.3ECE 1.7 ↓5.9ECE 7.8 ↓12.1ECE 4.8 ↓46.648.9ECE 5.6 ↓–28 msp95 33 ms
KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M19.7TR 8.9 L&M 9.6 BG 40.626.4EN 48.6 Logic 8.3 Math 10.3 AC 38.522.8ECE 5.5 ↓3.0ECE 10.8 ↓48.6ECE 8.1 ↓4.8ECE 1.5 ↓8.3ECE 8.4 ↓10.3ECE 10.5 ↓40.638.5ECE 14.6 ↓–33 msp95 50 ms
GLiDE 0.8BFastino750M19.7TR 11.6 L&M 13.9 BG 33.634.5EN 50.9 Logic 11.4 Math 15.1 AC 60.828.0ECE 21.8 ↓4.6ECE 26.1 ↓50.9ECE 12.5 ↓Not run11.4ECE 14.0 ↓15.1ECE 18.8 ↓33.660.8ECE 3.8 ↓–22 msp95 31 ms
OneJev1.6-0.8B full fine-tuneOmniJev850M17.7TR 11.4 L&M 9.8 BG 31.929.6EN 49.3 Logic 8.8 Math 10.3 AC 50.024.5ECE 7.3 ↓5.8ECE 10.3 ↓49.3ECE 3.7 ↓8.1ECE 4.2 ↓8.8ECE 11.2 ↓10.3ECE 10.0 ↓31.950.0ECE 4.1 ↓–17 msp95 23 ms
GLiDE 0.4BFastino400M3.7TR 2.8 L&M 1.8 BG 6.524.6EN 49.3 Logic 4.0 Math 0.7 AC 44.45.7ECE 7.2 ↓1.6ECE 8.3 ↓49.3ECE 4.8 ↓Not run4.0ECE 8.8 ↓0.7ECE 12.2 ↓6.544.4ECE 7.7 ↓–16 msp95 24 ms
GLiClass v4 Edge fine-tuneKnowledgator440M−0.8TR 4.2 L&M 3.7 BG −10.225.0EN 47.9 Logic 4.4 Math 3.3 AC 44.37.4ECE 11.9 ↓2.8ECE 10.0 ↓47.9ECE 3.8 ↓Not run4.4ECE 8.7 ↓3.3ECE 7.0 ↓−10.244.3ECE 9.8 ↓–25 msp95 53 ms
Adjudex 9B v5newsbang8.4BNot applicable62.1EN 82.2 Logic 33.0 Math 52.6 AC 80.476.0ECE 2.5 ↓26.7ECE 12.2 ↓82.2ECE 3.4 ↓Not run33.0ECE 7.2 ↓52.6ECE 4.8 ↓Not run80.4ECE 4.9 ↓–not timed yet
Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2BNot applicable54.3EN 75.5 Logic 25.4 Math 38.0 AC 78.163.6ECE 3.3 ↓20.5ECE 15.4 ↓75.5ECE 3.5 ↓Not run25.4ECE 15.5 ↓38.0ECE 13.7 ↓Not run78.1ECE 6.7 ↓–not timed yet
Clef-Flash post-trainedCloudflare9.5BNot applicable52.1EN 77.8 Logic 23.7 Math 37.2 AC 69.866.3ECE 5.6 ↓17.6ECE 20.3 ↓77.8ECE 4.0 ↓Not run23.7ECE 11.5 ↓37.2ECE 2.7 ↓Not run69.8ECE 5.9 ↓–not timed yet
Copernicus-31BPoindexter Labs31.3BNot applicable75.0EN 91.1 Logic 48.3 Math 71.7 AC 89.091.4ECE 6.3 ↓71.2ECE 10.5 ↓91.1ECE 7.8 ↓Not run48.3ECE 3.9 ↓71.7ECE 6.5 ↓Not run89.0ECE 11.0 ↓–not timed yet
Decider 1meraGPTNot disclosedNot applicableNot applicable18.3ECE 11.8 ↓4.9ECE 15.4 ↓44.7ECE 5.3 ↓Not runNot runNot runNot run43.3ECE 5.9 ↓–not timed yet
Dexy-Decision-9B-V2 fine-tunePatrick-188.4BNot applicable60.2EN 79.4 Logic 31.7 Math 48.5 AC 81.174.5ECE 3.2 ↓24.6ECE 12.7 ↓79.4ECE 4.2 ↓Not run31.7ECE 14.9 ↓48.5ECE 12.1 ↓Not run81.1ECE 2.5 ↓–not timed yet
GLiDE APIFastinoNot disclosedNot applicablenot one modelNot applicable84.8ECE 5.8 ↓49.3ECE 4.1 ↓Not runNot run43.1ECE 6.6 ↓63.5ECE 5.0 ↓71.985.1ECE 3.0 ↓–not timed yet
GLiDE 27BFastino26.1BNot applicable71.9EN 88.6 Logic 48.0 Math 65.0 AC 86.188.6ECE 2.4 ↓53.4ECE 2.7 ↓88.6ECE 1.7 ↓37.4ECE 4.3 ↓48.0ECE 4.4 ↓65.0ECE 4.1 ↓Not run86.1ECE 3.6 ↓–542 msp95 739 ms
GLiDE 9BFastino7.9BNot applicable61.8EN 82.9 Logic 34.8 Math 47.4 AC 82.275.2ECE 3.7 ↓25.1ECE 12.0 ↓82.9ECE 5.6 ↓Not run34.8ECE 6.7 ↓47.4ECE 7.1 ↓Not run82.2ECE 7.2 ↓–207 msp95 259 ms
GLiNER2.5-multi-Decide fine-tuneFastino290MNot applicableNot applicable1.3ECE 24.1 ↓−0.2ECE 22.7 ↓1.5ECE 27.2 ↓Not runNot runNot runNot runNot run–not timed yet
GPT-6 Luna Decisions APIOpenAINot disclosedNot applicableNot applicable81.9ECE 3.8 ↓42.9ECE 3.0 ↓80.1ECE 3.2 ↓15.8ECE 5.7 ↓Not runNot run64.2Not run–not timed yet
Jeffan-9B-1211tattrongvu8.9BNot applicable63.1EN 84.5 Logic 35.2 Math 49.8 AC 83.078.7ECE 11.4 ↓32.5ECE 3.0 ↓84.5ECE 8.9 ↓Not run35.2ECE 4.0 ↓49.8ECE 3.8 ↓Not run83.0ECE 3.3 ↓–not timed yet
Karpo Decision 31BMachinePulse31.3BNot applicable77.0EN 90.2 Logic 50.9 Math 76.9 AC 90.091.8ECE 1.2 ↓63.9ECE 3.4 ↓90.2ECE 2.4 ↓Not run50.9ECE 5.7 ↓76.9ECE 1.2 ↓Not run90.0ECE 2.8 ↓–not timed yet
KnowLine-12B-Gen1PelaAI12BNot applicable65.1EN 83.3 Logic 38.2 Math 57.8 AC 80.982.5ECE 11.4 ↓40.6ECE 4.3 ↓83.3ECE 11.1 ↓Not run38.2ECE 4.8 ↓57.8ECE 4.0 ↓Not run80.9ECE 6.8 ↓–not timed yet
KnowLine-31B-Gen2PelaAI31.3BNot applicable75.5EN 90.4 Logic 48.7 Math 75.3 AC 87.791.8ECE 6.3 ↓70.0ECE 5.7 ↓90.4ECE 7.7 ↓Not run48.7ECE 3.7 ↓75.3ECE 3.6 ↓Not run87.7ECE 7.2 ↓–not timed yet
KnowLine-9B-Gen1PelaAI9.7BNot applicable59.7EN 82.4 Logic 30.3 Math 49.7 AC 76.174.8ECE 1.7 ↓29.7ECE 6.8 ↓82.4ECE 3.5 ↓Not run30.3ECE 3.5 ↓49.7ECE 3.1 ↓Not run76.1ECE 3.6 ↓–not timed yet
KnowLine-9B-Gen2PelaAI9.7BNot applicable59.2EN 81.5 Logic 29.9 Math 50.1 AC 75.374.5ECE 8.2 ↓27.6ECE 5.1 ↓81.5ECE 8.7 ↓Not run29.9ECE 3.2 ↓50.1ECE 4.8 ↓Not run75.3ECE 4.5 ↓–not timed yet
LayaConvai Innovations420MNot applicableNot applicable1.1ECE 9.2 ↓−2.2ECE 14.9 ↓3.6ECE 12.1 ↓Not runNot runNot runNot runNot run–not timed yet
Laya MultilingualConvai Innovations320MNot applicableNot applicable−3.7ECE 33.1 ↓−2.2ECE 26.0 ↓−5.8ECE 44.4 ↓Not runNot runNot runNot runNot run–not timed yet
Logits Sigma-1 Duo blendLogits57.4BNot applicable72.5EN 88.8 Logic 47.2 Math 69.0 AC 85.189.9ECE 5.3 ↓59.8ECE 5.9 ↓88.8ECE 8.0 ↓Not run47.2ECE 5.0 ↓69.0ECE 7.5 ↓Not run85.1ECE 7.0 ↓–not timed yet
Metask-Jev-4B LoRA, mergedMetask Lab4.5BNot applicable43.1EN 71.4 Logic 15.1 Math 23.9 AC 61.753.5ECE 5.3 ↓14.2ECE 6.5 ↓71.4ECE 4.4 ↓Not run15.1ECE 2.5 ↓23.9ECE 3.3 ↓Not run61.7ECE 6.8 ↓–not timed yet
Microsoft-Decision-1MicrosoftNot disclosedNot applicable61.5EN 82.2 Logic 30.6 Math 49.2 AC 83.966.8ECE 8.9 ↓18.7ECE 24.4 ↓82.2ECE 2.8 ↓Not run30.6ECE 8.0 ↓49.2ECE 4.1 ↓Not run83.9ECE 2.4 ↓–not timed yet
OneJev-27B full fine-tuneOmniJev27.4BNot applicableNot applicable81.5ECE 9.9 ↓35.9ECE 3.7 ↓82.9ECE 9.7 ↓26.6ECE 4.9 ↓25.8ECE 4.6 ↓42.2ECE 3.6 ↓Not runNot run–not timed yet
Perplexity Decider 27B v1Perplexity26.1BNot applicableNot applicable82.5ECE 5.0 ↓36.9ECE 5.0 ↓85.6ECE 2.6 ↓Not runNot runNot runNot runNot run–not timed yet
Perplexity Decider 27B v1.1Perplexity26.1BNot applicableNot applicable86.3ECE 3.0 ↓46.8ECE 5.5 ↓Not runNot runNot runNot run68.0Not run–not timed yet
Perplexity Decider 27B v1.2 API · with webPerplexity26.1BNot applicableNot applicable86.1ECE 2.7 ↓50.2ECE 4.9 ↓88.1ECE 3.6 ↓Not run40.7ECE 4.2 ↓61.6ECE 3.7 ↓Not runNot run–not timed yet
Perplexity Decider 27B v1.3Perplexity26.1BNot applicable70.6EN 87.0 Logic 43.4 Math 65.6 AC 86.286.3ECE 2.4 ↓49.6ECE 6.1 ↓87.0ECE 3.4 ↓Not run43.4ECE 9.0 ↓65.6ECE 4.5 ↓Not run86.2ECE 2.8 ↓–not timed yet
Perplexity Decider Flash v1.3Perplexity4.5BNot applicable58.6EN 79.7 Logic 30.9 Math 44.9 AC 78.868.0ECE 6.2 ↓24.0ECE 15.3 ↓79.7ECE 2.7 ↓Not run30.9ECE 10.1 ↓44.9ECE 11.7 ↓Not run78.8ECE 4.2 ↓–not timed yet
StartLux-Decision-v1.1-9B-InstantStartLux9.7BNot applicable60.9EN 83.5 Logic 33.2 Math 48.0 AC 78.873.5ECE 3.6 ↓24.4ECE 11.3 ↓83.5ECE 6.6 ↓Not run33.2ECE 3.8 ↓48.0ECE 2.7 ↓Not run78.8ECE 8.2 ↓–not timed yet
Strands Decider 2B LoRA + headStrands Agents2.3BNot applicable26.5EN 31.9 Logic 3.0 Math 10.8 AC 60.321.1ECE 10.2 ↓4.3ECE 13.2 ↓31.9ECE 10.5 ↓Not run3.0ECE 12.0 ↓10.8ECE 8.8 ↓Not run60.3ECE 5.0 ↓–not timed yet
SubQ Snap PreviewSubquadraticNot disclosedNot applicableNot applicable84.6ECE 4.2 ↓44.6ECE 6.0 ↓82.4ECE 3.2 ↓Not runNot runNot run63.8Not run–not timed yet
Valen-Decision-1.5-9BValen-Team8.4BNot applicable57.1EN 79.7 Logic 30.4 Math 44.2 AC 74.368.8ECE 9.7 ↓18.8ECE 6.9 ↓79.7ECE 8.4 ↓Not run30.4ECE 5.8 ↓44.2ECE 4.1 ↓Not run74.3ECE 3.2 ↓–not timed yet
Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2BNot applicable54.3EN 77.8 Logic 24.7 Math 39.4 AC 75.464.0ECE 3.9 ↓20.5ECE 12.8 ↓77.8ECE 3.5 ↓Not run24.7ECE 10.8 ↓39.4ECE 5.7 ↓Not run75.4ECE 3.4 ↓–not timed yet
celeris-1.1-decisionCelerisNot disclosedNot applicableNot applicableNot runNot run79.7ECE 5.0 ↓Not run34.1ECE 9.8 ↓62.3ECE 4.5 ↓Not runNot run–not timed yet
d1Liquid AINot disclosedNot applicable58.3EN 83.5 Logic 26.0 Math 44.5 AC 79.278.5ECE 2.8 ↓32.6ECE 13.5 ↓83.5ECE 2.6 ↓Not run26.0ECE 10.9 ↓44.5ECE 3.3 ↓Not run79.2ECE 3.1 ↓–not timed yet
d1-Omni-2B NVIDIA RTX PRO 6000 BlackwellLiquid AI2.2BNot applicableNot applicableNot runNot runNot runNot runNot runNot run58.9Not run–not timed yet
d1.1Liquid AINot disclosedNot applicable64.2EN 85.4 Logic 32.6 Math 56.5 AC 82.580.8ECE 4.2 ↓39.6ECE 2.2 ↓85.4ECE 4.2 ↓Not run32.6ECE 4.4 ↓56.5ECE 2.2 ↓Not run82.5ECE 8.7 ↓–not timed yet
d1.2Liquid AINot disclosedNot applicableNot applicable82.5ECE 6.0 ↓39.3ECE 2.5 ↓86.3ECE 8.3 ↓33.7ECE 5.6 ↓36.4ECE 2.6 ↓56.9ECE 3.2 ↓Not runNot run–not timed yet
d1.2-ultraLiquid AINot disclosedNot applicable69.0EN 87.9 Logic 39.4 Math 62.3 AC 86.487.2ECE 5.6 ↓48.6ECE 3.1 ↓87.9ECE 8.5 ↓31.7ECE 2.8 ↓39.4ECE 3.0 ↓62.3ECE 4.6 ↓Not run86.4ECE 8.8 ↓–not timed yet
d3 fine-tunevLLM Semantic Router26.1BNot applicableNot applicable86.5ECE 3.5 ↓48.4ECE 2.4 ↓87.2ECE 3.6 ↓35.6ECE 3.7 ↓43.1ECE 4.6 ↓63.0ECE 4.5 ↓Not runNot run–not timed yet
d3-ultra fine-tunevLLM Semantic Router26.1BNot applicableNot applicable87.2ECE 3.8 ↓48.0ECE 16.3 ↓86.7ECE 2.8 ↓31.8ECE 18.8 ↓43.2ECE 15.1 ↓64.4ECE 8.6 ↓Not runNot run–not timed yet
vera-medium-omniscar-ai4.2BNot applicable59.8EN 79.4 Logic 30.1 Math 46.0 AC 83.769.3ECE 9.1 ↓22.3ECE 8.0 ↓79.4ECE 9.9 ↓Not run30.1ECE 4.7 ↓46.0ECE 3.0 ↓Not run83.7ECE 12.4 ↓–not timed yet
Chat models · highest first by Combined · Score 0–100, chance = 0 · ECE ↓ under each score
ModelSizeCombined ↑Decision Index ↑Turkish Reading ↑Turkish Grammar ↑English Reading ↑Vision ↑Logic ↑Math ↑Backgammon ↑Answer Check ↑Learning from experience ↑points gainedLatency ↓median, p95
GPT-6 Astra high · chatOpenAINot disclosed96.1TR 97.4 L&M 98.3 BG 92.5Not applicable96.4ECE 1.0 ↓97.9ECE 0.7 ↓95.2ECE 1.8 ↓Not run95.999.592.5Not run–not timed yet
GPT-6.1 Sol high · chatOpenAINot disclosed95.7TR 97.2 L&M 98.2 BG 91.6Not applicable95.6ECE 1.8 ↓97.9ECE 0.6 ↓94.5ECE 2.4 ↓Not run95.799.591.6Not run–not timed yet
Claude Fable 5.1 high · chatAnthropicNot disclosed93.7TR 95.2 L&M 97.7 BG 88.1Not applicable94.5ECE 10.9 ↓95.5ECE 12.5 ↓95.0ECE 11.8 ↓Not run94.099.788.1Not runBackgammon 9.4not timed yet
Claude Opus 5.5 high · chatAnthropicNot disclosed93.2TR 93.7 L&M 98.2 BG 87.7Not applicable94.5ECE 14.3 ↓93.4ECE 14.9 ↓93.1ECE 13.3 ↓Not run95.299.787.7Not runBackgammon 6.7not timed yet
Claude Sonnet 5.5 high · chatAnthropicNot disclosed88.3TR 81.3 L&M 97.8 BG 85.8Not applicable81.5ECE 13.8 ↓81.2ECE 16.9 ↓82.6ECE 13.9 ↓Not run94.399.685.8Not runBackgammon 7.0not timed yet
Muse Ember max · chatMetaNot disclosed87.7TR 82.0 L&M 97.7 BG 83.4Not applicable87.8ECE 7.5 ↓79.6ECE 6.3 ↓87.0ECE 7.0 ↓82.894.399.583.4Not run–not timed yet
Gemma 4 31B reasoning off · chatGoogle31.3B82.1TR 66.4 L&M 96.7 BG 83.4Not applicable88.6ECE 8.1 ↓57.0ECE 31.0 ↓88.1ECE 8.5 ↓Not run91.799.283.4Not runBackgammon 8.5not timed yet
Qwen3.8-27B low · chatAlibaba27.8B80.2TR 68.3 L&M 96.8 BG 75.4Not applicable86.7ECE 10.7 ↓60.5ECE 29.4 ↓86.3ECE 11.0 ↓Not run92.699.075.4Not runBackgammon 19.0not timed yet
Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed74.1TR 53.8 L&M 97.0 BG 71.5Not applicable84.0ECE 5.7 ↓41.0ECE 38.5 ↓83.8ECE 5.5 ↓Not run92.499.471.5Not runBackgammon 2.7not timed yet
Claude Fable 5.2 max · chatAnthropicNot disclosedNot applicableNot applicable97.1ECE 8.2 ↓97.9ECE 7.2 ↓97.5ECE 10.7 ↓Not runNot runNot runNot runNot run–not timed yet
Claude Haiku 5.5 max · chatAnthropicNot disclosedNot applicableNot applicable83.6ECE 18.9 ↓30.4ECE 16.4 ↓90.9ECE 18.4 ↓Not runNot runNot runNot runNot run–not timed yet
DeepSeek-V4-Flash chatDeepSeek285B13B activeNot applicableNot applicableNot runNot runNot runNot runNot run99.1Not runNot run–not timed yet
DeepSeek-V4.1-Flash reasoning on · chat · first releaseDeepSeek552B16B activeNot applicableNot applicable91.8ECE 6.6 ↓89.3ECE 8.6 ↓Not runNot runNot runNot runNot runNot run–not timed yet
DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B activeNot applicableNot applicable92.6ECE 6.0 ↓88.5ECE 7.3 ↓90.6ECE 7.5 ↓Not run95.299.5Not runNot run–not timed yet
DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseDeepSeekNot disclosedNot applicableNot applicableNot runNot runNot runNot runNot runNot run83.9Not runBackgammon 8.2not timed yet
Erk-14B reasoning off · chateCloud Technology14.8BNot applicableNot applicable65.3ECE 6.7 ↓16.7ECE 16.3 ↓65.0ECE 10.9 ↓Not run42.888.4Not runNot run–not timed yet
GLM-5.3 low · chatZ.ai753.3BNot applicableNot applicable84.6ECE 8.6 ↓61.2ECE 5.5 ↓84.7ECE 3.6 ↓Not runNot runNot run73.8Not runBackgammon 17.6not timed yet
GLM-5.3-Flash reasoning on · chatZ.ai320B18B activeNot applicableNot applicable86.7ECE 5.5 ↓67.9ECE 4.4 ↓87.0ECE 7.0 ↓Not run91.498.5Not runNot run–not timed yet
GLiNER2.5 MultiFastino290MNot applicableNot applicable−3.5ECE 35.1 ↓−3.0ECE 35.8 ↓−4.7ECE 36.4 ↓Not runNot runNot runNot runNot run–not timed yet
GPT-6 Luna low · chatOpenAINot disclosedNot applicableNot applicable84.2ECE 9.0 ↓50.2ECE 32.7 ↓82.9ECE 10.1 ↓Not run47.492.1Not runNot run–not timed yet
GPT-6.1 Terra high · chatOpenAINot disclosedNot applicableNot applicable95.2ECE 1.3 ↓91.8ECE 3.5 ↓95.2ECE 1.3 ↓Not run92.499.0Not runNot run–not timed yet
Gemini 3.8 Flash high · chatGoogleNot disclosedNot applicableNot applicable96.4ECE 1.3 ↓98.1ECE 1.0 ↓95.0ECE 1.7 ↓Not runNot runNot run87.3Not runBackgammon 8.2not timed yet
Gemini 4 Argon low · chatGoogleNot disclosedNot applicableNot applicable96.2ECE 1.1 ↓95.1ECE 2.7 ↓95.0ECE 1.5 ↓Not runNot runNot run89.3Not runBackgammon 8.5not timed yet
Gemini 4 Carbon high · chatGoogleNot disclosedNot applicableNot applicable97.0ECE 0.7 ↓98.5ECE 0.3 ↓92.9ECE 3.1 ↓Not run95.999.5Not runNot run–not timed yet
Gemini 4 Hydrogen high · chatGoogleNot disclosedNot applicable93.5EN 95.9 Logic 95.1 Math 99.5 AC 83.796.6ECE 2.3 ↓98.1ECE 0.9 ↓95.9ECE 3.1 ↓Not run95.199.5Not run83.7–not timed yet
Gemma 4 12B Unified default reasoning · chatGoogle12BNot applicable85.9EN 83.5 Logic 84.7 Math 98.6 AC 77.0Not runNot run83.5Not run84.798.6Not run77.0–not timed yet
Gemma 4 12B Unified thinking on · chatGoogle11.9BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run67.7–not timed yet
Gemma 4 E2B reasoning on · chatGoogle5.1BNot applicable75.9EN 70.3 Logic 66.1 Math 96.0 AC 71.1Not runNot run70.3Not run66.196.0Not run71.1–not timed yet
Gemma 4 E2B default reasoning · chatGoogle5.1BNot applicableNot applicableNot runNot run68.2Not runNot runNot runNot run55.2–not timed yet
Gemma 4 E2B thinking off · chatGoogle5.1BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run54.8–not timed yet
Gemma 4 E4B default reasoning · chatGoogle8BNot applicable83.0EN 78.1 Logic 79.0 Math 97.6 AC 77.272.036.378.128.679.097.6Not run77.2–not timed yet
Gemma 4 E4B thinking on · sampled (temperature 1.0) · chatGoogle8BNot applicableNot applicableNot runNot runNot runNot runNot runNot runNot run75.1–not timed yet
Granite-4.2-3B reasoning on · chatIBM3.7BNot applicable83.1EN 68.0 Logic 89.6 Math 99.0 AC 75.9Not runNot run68.0Not run89.699.0Not run75.9–not timed yet
Grok 4.7 high · chatxAINot disclosedNot applicableNot applicable93.7ECE 10.5 ↓89.7ECE 12.4 ↓91.8ECE 11.1 ↓Not runNot run99.484.0Not run–not timed yet
Inkling 2 Small high · chatThinking Machines594B17B activeNot applicableNot applicableNot runNot runNot runNot run94.699.4Not run86.2–not timed yet
Inkling-Small high · chat · Jul 2026Thinking Machines266B12B activeNot applicableNot applicable84.0ECE 3.2 ↓71.1ECE 9.9 ↓85.4ECE 2.6 ↓Not run94.299.5Not runNot run–not timed yet
Kimi K3.1 Pro low · chatMoonshot2.8T104B activeNot applicableNot applicable92.2ECE 4.3 ↓72.6ECE 16.2 ↓90.4ECE 5.7 ↓Not runNot runNot run76.2Not runBackgammon 4.8not timed yet
LFM2.5-2.6B default reasoning · chatLiquid AI2.7BNot applicableNot applicable31.811.267.3Not run69.292.5Not runNot run–not timed yet
LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B activeNot applicableNot applicableNot runNot run78.3Not run83.398.860.6Not run–not timed yet
LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B activeNot applicable66.6EN 63.0 Logic 50.2 Math 95.3 AC 57.77.0−1.263.0Not run50.295.3Not run57.7–not timed yet
LFM3-1.7B reasoning on · chatLiquid AI1.7BNot applicable80.3EN 64.7 Logic 81.9 Math 98.7 AC 75.830.39.264.7Not run81.998.7Not run75.8–not timed yet
LFM3-1.7B reasoning off · chatLiquid AI1.7BNot applicableNot applicable21.55.763.9Not runNot runNot runNot run74.9–not timed yet
MiMo-V2.6-Flash high · chatXiaomi309B15B activeNot applicableNot applicable83.2ECE 3.7 ↓57.4ECE 10.3 ↓86.3ECE 6.5 ↓Not run93.999.3Not runNot run–not timed yet
MiniCPM5-2B reasoning off · chatOpenBMB2.5BNot applicable84.3EN 72.7 Logic 86.1 Math 99.0 AC 79.2−16.8−18.572.7Not run86.199.0Not run79.2–not timed yet
MiniCPM5-2B reasoning off · chatOpenBMB2.5BNot applicableNot applicableNot runNot run59.7Not runNot runNot runNot run67.0–not timed yet
MiniMax-M3.1-Flash high · chatMiniMax428B23B activeNot applicableNot applicable78.7ECE 9.5 ↓49.3ECE 26.1 ↓80.6ECE 9.0 ↓Not run92.497.8Not runNot run–not timed yet
Qwen3-14B reasoning off · chatAlibaba14.8BNot applicableNot applicable65.1ECE 23.2 ↓18.1ECE 48.6 ↓74.6ECE 17.8 ↓Not run56.794.7Not runNot run–not timed yet
Qwen3.5-2B reasoning off · chatAlibaba2.3BNot applicable72.4EN 69.8 Logic 56.0 Math 93.3 AC 70.317.92.069.8Not run56.093.3Not run70.3–not timed yet
Qwen3.5-2B reasoning off · chatAlibaba2.3BNot applicableNot applicableNot runNot run59.1Not runNot runNot runNot run50.2–not timed yet
Qwen3.5-397B-A17B high · chatAlibaba397B17B activeNot applicableNot applicable88.8ECE 2.2 ↓74.0ECE 9.1 ↓89.3ECE 3.3 ↓Not run93.9Not runNot runNot run–not timed yet
Qwen3.5-4B reasoning on · chatAlibaba4.7BNot applicable84.9EN 83.8 Logic 90.4 Math 99.3 AC 66.273.937.983.8Not run90.499.3Not run66.2–not timed yet
Qwen3.5-4B reasoning off · chatAlibaba4.7BNot applicableNot applicable60.620.975.5Not runNot runNot runNot run64.6–not timed yet
Qwen4-Max high · chatAlibabaNot disclosedNot applicableNot applicable93.3ECE 7.0 ↓88.1ECE 8.3 ↓93.1ECE 8.5 ↓Not run95.399.7Not runNot run–not timed yet
Raon-1 nano reasoning off · chatKRAFTON3.5BNot applicable56.5EN 42.2 Logic 56.3 Math 86.9 AC 40.6Not runNot run42.2Not run56.386.9Not run40.6–not timed yet

Size, calibration and latency against score

Size against score

Combined: 28 decision models; 8 left out (models whose maker discloses no size are left out)

0204060801000.10.3131030100Parameters, billions (log scale)Score 0–100, chance = 0 ↑KnowLine-31B-Gen1.5: 73.2, Parameters, billions (log scale) 31.3REAX 1.3: 71.3, Parameters, billions (log scale) 57.4Sieve 31B v3: 69.4, Parameters, billions (log scale) 31.3Turion Jev 31B: 68.2, Parameters, billions (log scale) 31.3Mammoth-1: 64.8, Parameters, billions (log scale) 27.4Drex Max: 63.4, Parameters, billions (log scale) 26.1GLiDE · v1.2: 62.6, Parameters, billions (log scale) 26.1Perplexity Decider 27B v1.2 · DGX Spark: 61.2, Parameters, billions (log scale) 26.1Drex 2.0: 51.4, Parameters, billions (log scale) 35.1Clef: 50.0, Parameters, billions (log scale) 27.5Elephant-1: 49.7, Parameters, billions (log scale) 9.4Cerno 9B v1.2: 47.5, Parameters, billions (log scale) 8.9StartLux-Decision-v1.1-4B-Instant: 47.4, Parameters, billions (log scale) 4.7Cygnet: 47.0, Parameters, billions (log scale) 12Winnow-12B: 44.6, Parameters, billions (log scale) 11.9KnowLine 4B Gen5.5: 43.5, Parameters, billions (log scale) 4.7Open-Jev-27B-v1.1: 42.8, Parameters, billions (log scale) 27.8Perplexity Decider Flash v1.2: 41.4, Parameters, billions (log scale) 4.5StartLux-Decision-v1.1-2B-Instant: 32.1, Parameters, billions (log scale) 2.3KnowLine-2B-Gen1: 31.2, Parameters, billions (log scale) 2.3d1-2B: 30.5, Parameters, billions (log scale) 1.9GLiDE 2B: 27.5, Parameters, billions (log scale) 1.9StartLux-Decision-v1.1-0.8B-Instant: 22.4, Parameters, billions (log scale) 0.9KnowLine-0.8B-Gen1: 19.7, Parameters, billions (log scale) 0.9GLiDE 0.8B: 19.7, Parameters, billions (log scale) 0.8OneJev1.6-0.8B: 17.7, Parameters, billions (log scale) 0.8GLiDE 0.4B: 3.7, Parameters, billions (log scale) 0.4GLiClass v4 Edge: −0.8, Parameters, billions (log scale) 0.4Gemma 4 31B · reasoning off · chat: 82.1, Parameters, billions (log scale) 31.3Qwen3.8-27B · low · chat: 80.2, Parameters, billions (log scale) 27.8KnowLine-31B-Gen1.5REAX 1.3Mammoth-1Perplexity Decider 27B v1.2 · DGX SparkDrex 2.0Elephant-1Winnow-12BStartLux-Decision-v1.1-2B-InstantGLiDE 2BStartLux-Decision-v1.1-0.8B-InstantOneJev1.6-0.8BGLiDE 0.4BGLiClass v4 EdgeGemma 4 31B · reasoning off · chat

Calibration error ↓ against score

Combined: 36 decision models

0204060801000510152025Calibration error, ECE in points: mean of the benches' calibration errorsScore 0–100, chance = 0 ↑KnowLine-31B-Gen1.5: 73.2, Calibration error, ECE in points: mean of the benches' calibration errors 3REAX 1.3: 71.3, Calibration error, ECE in points: mean of the benches' calibration errors 4.4Sieve 31B v3: 69.4, Calibration error, ECE in points: mean of the benches' calibration errors 2.7d1-ultra: 68.3, Calibration error, ECE in points: mean of the benches' calibration errors 2.8Turion Jev 31B: 68.2, Calibration error, ECE in points: mean of the benches' calibration errors 7.3Gemini Decision 1 · 20261111: 68.0, Calibration error, ECE in points: mean of the benches' calibration errors 2Mammoth-1: 64.8, Calibration error, ECE in points: mean of the benches' calibration errors 4.4Drex Max: 63.4, Calibration error, ECE in points: mean of the benches' calibration errors 9.8Drex Max v1.1: 63.2, Calibration error, ECE in points: mean of the benches' calibration errors 4.3GLiDE · v1.2: 62.6, Calibration error, ECE in points: mean of the benches' calibration errors 4Perplexity Decider 27B v1.2 · DGX Spark: 61.2, Calibration error, ECE in points: mean of the benches' calibration errors 3.8SubQ Snap 1.1: 58.5, Calibration error, ECE in points: mean of the benches' calibration errors 10.8celeris-1-decision: 58.0, Calibration error, ECE in points: mean of the benches' calibration errors 5Jev 1.14: 57.1, Calibration error, ECE in points: mean of the benches' calibration errors 2.2Jev 1.13: 55.3, Calibration error, ECE in points: mean of the benches' calibration errors 4.2Drex 2.0: 51.4, Calibration error, ECE in points: mean of the benches' calibration errors 2.7Clef: 50.0, Calibration error, ECE in points: mean of the benches' calibration errors 11.6Elephant-1: 49.7, Calibration error, ECE in points: mean of the benches' calibration errors 8.7Mercury Decide: 48.8, Calibration error, ECE in points: mean of the benches' calibration errors 15.6Cerno 9B v1.2: 47.5, Calibration error, ECE in points: mean of the benches' calibration errors 13StartLux-Decision-v1.1-4B-Instant: 47.4, Calibration error, ECE in points: mean of the benches' calibration errors 6.2Cygnet: 47.0, Calibration error, ECE in points: mean of the benches' calibration errors 10.6Winnow-12B: 44.6, Calibration error, ECE in points: mean of the benches' calibration errors 15.1KnowLine 4B Gen5.5: 43.5, Calibration error, ECE in points: mean of the benches' calibration errors 11.9Open-Jev-27B-v1.1: 42.8, Calibration error, ECE in points: mean of the benches' calibration errors 21.4Perplexity Decider Flash v1.2: 41.4, Calibration error, ECE in points: mean of the benches' calibration errors 8.6StartLux-Decision-v1.1-2B-Instant: 32.1, Calibration error, ECE in points: mean of the benches' calibration errors 4.8KnowLine-2B-Gen1: 31.2, Calibration error, ECE in points: mean of the benches' calibration errors 8.7d1-2B: 30.5, Calibration error, ECE in points: mean of the benches' calibration errors 5.9GLiDE 2B: 27.5, Calibration error, ECE in points: mean of the benches' calibration errors 18.8StartLux-Decision-v1.1-0.8B-Instant: 22.4, Calibration error, ECE in points: mean of the benches' calibration errors 5.1KnowLine-0.8B-Gen1: 19.7, Calibration error, ECE in points: mean of the benches' calibration errors 9GLiDE 0.8B: 19.7, Calibration error, ECE in points: mean of the benches' calibration errors 21OneJev1.6-0.8B: 17.7, Calibration error, ECE in points: mean of the benches' calibration errors 9.9GLiDE 0.4B: 3.7, Calibration error, ECE in points: mean of the benches' calibration errors 9.5GLiClass v4 Edge: −0.8, Calibration error, ECE in points: mean of the benches' calibration errors 9.1KnowLine-31B-Gen1.5d1-ultraTurion Jev 31BDrex MaxDrex Max v1.1SubQ Snap 1.1celeris-1-decisionJev 1.14Drex 2.0ClefElephant-1Mercury DecideKnowLine 4B Gen5.5StartLux-Decision-v1.1-2B-InstantGLiDE 2BStartLux-Decision-v1.1-0.8B-InstantGLiDE 0.8BOneJev1.6-0.8BGLiDE 0.4BGLiClass v4 Edge

Latency ↓ against score

Combined: 11 decision models; 25 left out (latency: Reading · Turkish requests on spark2, a DGX Spark (GB10))

02040608010010301003001,0003,00010,000Median time per request, ms (log scale)Score 0–100, chance = 0 ↑KnowLine-31B-Gen1.5: 73.2, Median time per request, ms (log scale) 1,051KnowLine 4B Gen5.5: 43.5, Median time per request, ms (log scale) 99StartLux-Decision-v1.1-2B-Instant: 32.1, Median time per request, ms (log scale) 40KnowLine-2B-Gen1: 31.2, Median time per request, ms (log scale) 62GLiDE 2B: 27.5, Median time per request, ms (log scale) 34StartLux-Decision-v1.1-0.8B-Instant: 22.4, Median time per request, ms (log scale) 28KnowLine-0.8B-Gen1: 19.7, Median time per request, ms (log scale) 33GLiDE 0.8B: 19.7, Median time per request, ms (log scale) 22OneJev1.6-0.8B: 17.7, Median time per request, ms (log scale) 17GLiDE 0.4B: 3.7, Median time per request, ms (log scale) 16GLiClass v4 Edge: −0.8, Median time per request, ms (log scale) 25KnowLine-31B-Gen1.5KnowLine 4B Gen5.5StartLux-Decision-v1.1-2B-InstantGLiDE 2BStartLux-Decision-v1.1-0.8B-InstantOneJev1.6-0.8BGLiDE 0.4BGLiClass v4 Edge

Their score against ours

Jev Decision Index 0.4.1, Full score (6 November 2026, source): a sanity check, never our score; its 15 open decision models joined to ours by Hugging Face repo.

12 of the index's 15 models have a score here. Of their 66 pairs, 61 keep the index's order, 3 are reversed and 2 are tied on our score.

Index rankModelTheir score ↑Jev Decision Index 0.4.1, Full scoreOur score ↑Combined, score 0–100, chance = 0Our rankof these
1REAX 1.3decisionmodels/REAX-1.374.071.32
2GLiDE 27Bfastino/GLiDE-27B73.4not on this bench yet–
3KnowLine-31B-Gen1.5PelaAI/KnowLine-31B-Gen1.571.973.21
4Sieve 31B v3sthanika-ai/Sieve-31B-v371.169.43
5Turion Jev 31Bturion-ai/turion-jev-31b70.568.24
6d3-ultravllm-sr/d3-ultra70.2not on this bench yet–
7GLiDE 9Bfastino/GLiDE-9B62.5not on this bench yet–
8Elephant-1baselabs/elephant-160.349.75
9Cerno 9B v1.2apertixai/cerno-9b-v1.259.047.56
10KnowLine 4B Gen5.5PelaAI/KnowLine-4B-Gen5.556.043.58
11StartLux-Decision-v1.1-4B-Instantstartlux-models/StartLux-Decision-v1.1-4B-Instant54.847.47
12Perplexity Decider Flash v1.2perplexity-ai/pplx-decider-flash-v1.254.341.49
13GLiDE 0.8Bfastino/GLiDE-0.8B36.419.710
14GLiDE 0.4Bfastino/GLiDE-0.4B33.83.712
15OneJev1.6-0.8BOmniJev/OneJev1.6-0.8B29.617.711
ref.Jev 1.14reference, hostedno index score here57.1–

All benches

Turkish Reading

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B91.893.4%656 questions1.2d69cdd5
2KnowLine-31B-Gen2PelaAI31.3B91.8tied with Karpo Decision 31B93.4%656 questions6.3fd99a9e
3Copernicus-31BPoindexter Labs31.3B91.4tied with KnowLine-31B-Gen293.1%656 questions6.3be71525
4KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B91.0tied with Copernicus-31B92.8%656 questions3.48e56aa1
5Turion Jev 31B LoRA, mergedTurion.AI31.3B90.9tied with KnowLine-31B-Gen1.592.7%656 questions9.5866c0fc
6REAX 1.3 blendREAX57.4B90.5tied with Turion Jev 31B92.4%656 questions2.1fa4bf7e
7Gemini Decision 1 20261111GoogleNot disclosed90.1tied with REAX 1.392.1%656 questions2.8no repo
8d1-ultraLiquid AINot disclosed90.1tied with Gemini Decision 1 · 2026111192.1%656 questions3.6no repo
9Logits Sigma-1 Duo blendLogits57.4B89.9tied with d1-ultra91.9%656 questions5.3a0c1b6e
10Sieve 31B v3 full fine-tuneSthānika AI31.3B89.7tied with Logits Sigma-1 Duo91.8%656 questions3.58fcee85
11GLiDE 27BFastino26.1B88.6tied with Sieve 31B v390.9%656 questions2.44d32743
12Mammoth-1 post-trainedBase Labs27.4B88.0tied with GLiDE 27B90.4%656 questions2.061cec44
13d1.2-ultraLiquid AINot disclosed87.2tied with Mammoth-189.8%656 questions5.6no repo
14d3-ultra fine-tunevLLM Semantic Router26.1B87.2tied with d1.2-ultra89.8%656 questions3.852ba9e7
15Drex Max fine-tuneNace.AI26.1B87.0tied with d3-ultra89.6%656 questions3.4cac65e1
16GLiDE v1.2Fastino26.1B87.0tied with Drex Max89.6%656 questions3.5a7e4589
17Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B86.5tied with GLiDE · v1.289.2%656 questions3.1225c913
18d3 fine-tunevLLM Semantic Router26.1B86.5tied with Perplexity Decider 27B v1.2 · DGX Spark89.2%656 questions3.520f37b9
19Perplexity Decider 27B v1.1Perplexity26.1B86.3tied with d389.0%656 questions3.0no repo
20Perplexity Decider 27B v1.3Perplexity26.1B86.3tied with Perplexity Decider 27B v1.189.0%656 questions2.49c2ebb2
21Perplexity Decider 27B v1.2 API · with webPerplexity26.1B86.1tied with Perplexity Decider 27B v1.388.9%656 questions2.7no repo
22Drex Max v1.1Nace.AINot disclosed85.7tied with Perplexity Decider 27B v1.2 · API · with web88.6%656 questions3.4no repo
23GLiDE APIFastinoNot disclosed84.8tied with Drex Max v1.187.8%656 questions5.8no repo
24SubQ Snap PreviewSubquadraticNot disclosed84.6tied with GLiDE · API87.7%656 questions4.2no repo
25SubQ Snap 1.1SubquadraticNot disclosed83.8tied with SubQ Snap Preview87.0%656 questions3.6no repo
26KnowLine-12B-Gen1PelaAI12B82.5tied with SubQ Snap 1.186.0%656 questions11.4f26fbc2
27Perplexity Decider 27B v1Perplexity26.1B82.5tied with KnowLine-12B-Gen186.0%656 questions5.0no repo
28d1.2Liquid AINot disclosed82.5tied with Perplexity Decider 27B v186.0%656 questions6.0no repo
29GPT-6 Luna Decisions APIOpenAINot disclosed81.9tied with d1.285.5%656 questions3.8no repo
30OneJev-27B full fine-tuneOmniJev27.4B81.5tied with GPT-6 Luna · Decisions API85.2%656 questions9.9a5e5f55
31celeris-1-decisionCelerisNot disclosed81.5tied with OneJev-27B85.2%656 questions6.2no repo
32Clef post-trainedCloudflare27.5B81.3tied with celeris-1-decision85.1%656 questions4.02f3de3d
33Jev 1.14TypeSafeNot disclosed81.3tied with Clef85.1%656 questions2.2no repo
34d1.1Liquid AINot disclosed80.8tied with Jev 1.1484.6%656 questions4.2no repo
35Mercury DecideInceptionNot disclosed80.6tied with d1.184.5%656 questions7.5no repo
36Jev 1.13TypeSafeNot disclosed79.6tied with Mercury Decide83.7%656 questions3.8no repo
37Jeffan-9B-1211tattrongvu8.9B78.7tied with Jev 1.1382.9%656 questions11.4862d1a7
38Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B78.7tied with Jeffan-9B-121182.9%656 questions43.628cf730
39Cygnet unchanged baseBlockbrain Labs12B78.5tied with Open-Jev-27B-v1.182.8%656 questions4.1no repo
40d1Liquid AINot disclosed78.5tied with Cygnet82.8%656 questions2.8no repo
41Winnow-12B LoRA, mergedEldanRing11.9B77.5tied with d182.0%656 questions9.3b2b1421
42Drex 2.0Nace.AI35.1B3B active76.8tied with Winnow-12B81.4%656 questions10.1no repo
43Adjudex 9B v5newsbang8.4B76.0tied with Drex 2.080.8%656 questions2.5ffdcb82
44Elephant-1 post-trainedBase Labs9.4B75.6tied with Adjudex 9B v580.5%656 questions4.3b3f8a93
45GLiDE 9BFastino7.9B75.2tied with Elephant-180.2%656 questions3.7ab54df5
46KnowLine-9B-Gen1PelaAI9.7B74.8tied with GLiDE 9B79.9%656 questions1.7325378d
47Cerno 9B v1.2 full fine-tuneApertix AI8.9B74.7tied with KnowLine-9B-Gen179.7%656 questions3.0f02a632
48Dexy-Decision-9B-V2 fine-tunePatrick-188.4B74.5tied with Cerno 9B v1.279.6%656 questions3.24f434e2
49KnowLine-9B-Gen2PelaAI9.7B74.5tied with Dexy-Decision-9B-V279.6%656 questions8.28ec8eb6
50StartLux-Decision-v1.1-9B-InstantStartLux9.7B73.5tied with KnowLine-9B-Gen278.8%656 questions3.618a17f7
51vera-medium-omniscar-ai4.2B69.375.5%656 questions9.11f23a6c
52Valen-Decision-1.5-9BValen-Team8.4B68.8tied with vera-medium-omni75.0%656 questions9.75a51211
53KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B68.6tied with Valen-Decision-1.5-9B74.8%656 questions6.2779944d
54Perplexity Decider Flash v1.3Perplexity4.5B68.0tied with KnowLine 4B Gen5.574.4%656 questions6.2d06fb3f
55StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B67.6tied with Perplexity Decider Flash v1.374.1%656 questions2.6416088f
56Microsoft-Decision-1MicrosoftNot disclosed66.8tied with StartLux-Decision-v1.1-4B-Instant73.5%656 questions8.9no repo
57Clef-Flash post-trainedCloudflare9.5B66.3tied with Microsoft-Decision-173.0%656 questions5.6fde727a
58Perplexity Decider Flash v1.2Perplexity4.5B65.9tied with Clef-Flash72.7%656 questions3.76d771ad
59Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B64.0tied with Perplexity Decider Flash v1.271.2%656 questions3.945830d2
60Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B63.6tied with Verdin-4B-v1.170.9%656 questions3.3889f23e
61Metask-Jev-4B LoRA, mergedMetask Lab4.5B53.562.8%656 questions5.3ea20fe8
62KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B42.854.3%656 questions1.953245d4
63StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B41.1tied with KnowLine-2B-Gen152.9%656 questions3.3f1e6cd1
64GLiDE 2BFastino1.9B39.0tied with StartLux-Decision-v1.1-2B-Instant51.2%656 questions16.500f122c
65GLiDE 0.8BFastino750M28.042.4%656 questions21.89eacdee
66d1-2BLiquid AI1.9B26.1tied with GLiDE 0.8B40.9%656 questions4.410e7671
67OneJev1.6-0.8B full fine-tuneOmniJev850M24.5tied with d1-2B39.6%656 questions7.3bcd88a3
68KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M22.8tied with OneJev1.6-0.8B38.3%656 questions5.56b27831
69Strands Decider 2B LoRA + headStrands Agents2.3B21.1tied with KnowLine-0.8B-Gen136.9%656 questions10.22b52a62
70StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M18.8tied with Strands Decider 2B35.1%656 questions5.06496ae6
71Decider 1meraGPTNot disclosed18.3tied with StartLux-Decision-v1.1-0.8B-Instant34.6%656 questions11.8no repo
72GLiClass v4 Edge fine-tuneKnowledgator440M7.425.9%656 questions11.9885af1a
73GLiDE 0.4BFastino400M5.7tied with GLiClass v4 Edge24.5%656 questions7.26bda7ab
74GLiNER2.5-multi-Decide fine-tuneFastino290M1.321.0%656 questions24.1a35a0cd
75LayaConvai Innovations420M1.1tied with GLiNER2.5-multi-Decide20.9%656 questions9.2c5d7873
76Laya MultilingualConvai Innovations320M−3.717.1%656 questions33.1c5d7873

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Claude Fable 5.2 max · chatAnthropicNot disclosed97.197.7%656 questions8.2no repo
2Gemini 4 Carbon high · chatGoogleNot disclosed97.0tied with Claude Fable 5.2 · max · chat97.6%656 questions0.7no repo
3Gemini 4 Hydrogen high · chatGoogleNot disclosed96.6tied with Gemini 4 Carbon · high · chat97.3%656 questions2.3no repo
4GPT-6 Astra high · chatOpenAINot disclosed96.4tied with Gemini 4 Hydrogen · high · chat97.1%656 questions1.0no repo
5Gemini 3.8 Flash high · chatGoogleNot disclosed96.4tied with GPT-6 Astra · high · chat97.1%656 questions1.3no repo
6Gemini 4 Argon low · chatGoogleNot disclosed96.2tied with Gemini 3.8 Flash · high · chat97.0%656 questions1.1no repo
7GPT-6.1 Sol high · chatOpenAINot disclosed95.6tied with Gemini 4 Argon · low · chat96.5%656 questions1.8no repo
8GPT-6.1 Terra high · chatOpenAINot disclosed95.2tied with GPT-6.1 Sol · high · chat96.2%656 questions1.3no repo
9Claude Fable 5.1 high · chatAnthropicNot disclosed94.5tied with GPT-6.1 Terra · high · chat95.6%656 questions10.9no repo
10Claude Opus 5.5 high · chatAnthropicNot disclosed94.5tied with Claude Fable 5.1 · high · chat95.6%656 questions14.3no repo
11Grok 4.7 high · chatxAINot disclosed93.7tied with Claude Opus 5.5 · high · chat95.0%656 questions10.5no repo
12Qwen4-Max high · chatAlibabaNot disclosed93.3tied with Grok 4.7 · high · chat94.7%656 questions7.0no repo
13DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active92.6tied with Qwen4-Max · high · chat94.1%656 questions6.0no repo
14Kimi K3.1 Pro low · chatMoonshot2.8T104B active92.2tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101793.8%656 questions4.3no repo
15DeepSeek-V4.1-Flash reasoning on · chat · first releaseDeepSeek552B16B active91.8tied with Kimi K3.1 Pro · low · chat93.4%656 questions6.6no repo
16Qwen3.5-397B-A17B high · chatAlibaba397B17B active88.8tied with DeepSeek-V4.1-Flash · reasoning on · chat · first release91.0%656 questions2.2no repo
17Gemma 4 31B reasoning off · chatGoogle31.3B88.6tied with Qwen3.5-397B-A17B · high · chat90.9%656 questions8.1842da37
18Muse Ember max · chatMetaNot disclosed87.8tied with Gemma 4 31B · reasoning off · chat90.2%656 questions7.5no repo
19GLM-5.3-Flash reasoning on · chatZ.ai320B18B active86.7tied with Muse Ember · max · chat89.3%656 questions5.5no repo
20Qwen3.8-27B low · chatAlibaba27.8B86.7tied with GLM-5.3-Flash · reasoning on · chat89.3%656 questions10.71d4bf0f
21GLM-5.3 low · chatZ.ai753.3B84.6tied with Qwen3.8-27B · low · chat87.7%656 questions8.6no repo
22GPT-6 Luna low · chatOpenAINot disclosed84.2tied with GLM-5.3 · low · chat87.3%656 questions9.0no repo
23Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active84.0tied with GPT-6 Luna · low · chat87.2%656 questions3.2no repo
24Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed84.0tied with Inkling-Small · high · chat · Jul 202687.2%656 questions5.7no repo
25Claude Haiku 5.5 max · chatAnthropicNot disclosed83.6tied with Qwen3.8 Flash · no reasoning · chat86.9%656 questions18.9no repo
26MiMo-V2.6-Flash high · chatXiaomi309B15B active83.2tied with Claude Haiku 5.5 · max · chat86.6%656 questions3.7no repo
27Claude Sonnet 5.5 high · chatAnthropicNot disclosed81.5tied with MiMo-V2.6-Flash · high · chat85.2%656 questions13.8no repo
28MiniMax-M3.1-Flash high · chatMiniMax428B23B active78.7tied with Claude Sonnet 5.5 · high · chat82.9%656 questions9.5no repo
29Qwen3.5-4B reasoning on · chatAlibaba4.7B73.979.1%656 questions–851bf6e
30Gemma 4 E4B default reasoning · chatGoogle8B72.0tied with Qwen3.5-4B · reasoning on · chat77.6%656 questions–4ecc2ba
31Erk-14B reasoning off · chateCloud Technology14.8B65.372.3%656 questions6.78584b14
32Qwen3-14B reasoning off · chatAlibaba14.8B65.1tied with Erk-14B · reasoning off · chat72.1%656 questions23.240c0698
33Qwen3.5-4B reasoning off · chatAlibaba4.7B60.668.4%656 questions–851bf6e
34LFM2.5-2.6B default reasoning · chatLiquid AI2.7B31.845.4%656 questions–7141b56
35LFM3-1.7B reasoning on · chatLiquid AI1.7B30.3tied with LFM2.5-2.6B · default reasoning · chat44.2%656 questions–0dc4268
36LFM3-1.7B reasoning off · chatLiquid AI1.7B21.537.2%656 questions–0dc4268
37Qwen3.5-2B reasoning off · chatAlibaba2.3B17.9tied with LFM3-1.7B · reasoning off · chat34.3%656 questions–15852e8
38LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active7.025.6%656 questions–dd83f5c
39GLiNER2.5 MultiFastino290M−3.517.2%656 questions35.12ca71aa
40MiniCPM5-2B reasoning off · chatOpenBMB2.5B−16.86.6%656 questions–42433e9

656 questions per model · updated 16 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 656 Turkish reading questions (units 1 to 6) right (accuracy 100%) · feed: bench-scores.json

Turkish Grammar

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B71.477.1%1,542 questions3.98e56aa1
2Copernicus-31BPoindexter Labs31.3B71.2tied with KnowLine-31B-Gen1.577.0%1,542 questions10.5be71525
3KnowLine-31B-Gen2PelaAI31.3B70.0tied with Copernicus-31B76.0%1,542 questions5.7fd99a9e
4Turion Jev 31B LoRA, mergedTurion.AI31.3B68.5tied with KnowLine-31B-Gen274.8%1,542 questions12.0866c0fc
5REAX 1.3 blendREAX57.4B64.771.8%1,542 questions5.1fa4bf7e
6Gemini Decision 1 20261111GoogleNot disclosed64.2tied with REAX 1.371.3%1,542 questions2.9no repo
7Karpo Decision 31BMachinePulse31.3B63.9tied with Gemini Decision 1 · 2026111171.1%1,542 questions3.4d69cdd5
8d1-ultraLiquid AINot disclosed63.4tied with Karpo Decision 31B70.7%1,542 questions4.2no repo
9Logits Sigma-1 Duo blendLogits57.4B59.867.8%1,542 questions5.9a0c1b6e
10Sieve 31B v3 full fine-tuneSthānika AI31.3B58.3tied with Logits Sigma-1 Duo66.7%1,542 questions3.38fcee85
11GLiDE 27BFastino26.1B53.462.7%1,542 questions2.74d32743
12Perplexity Decider 27B v1.2 API · with webPerplexity26.1B50.260.2%1,542 questions4.9no repo
13Perplexity Decider 27B v1.3Perplexity26.1B49.6tied with Perplexity Decider 27B v1.2 · API · with web59.7%1,542 questions6.19c2ebb2
14GLiDE v1.2Fastino26.1B49.3tied with Perplexity Decider 27B v1.359.5%1,542 questions3.0a7e4589
15GLiDE APIFastinoNot disclosed49.3tied with GLiDE · v1.259.4%1,542 questions4.1no repo
16d1.2-ultraLiquid AINot disclosed48.6tied with GLiDE · API58.9%1,542 questions3.1no repo
17d3 fine-tunevLLM Semantic Router26.1B48.4tied with d1.2-ultra58.7%1,542 questions2.420f37b9
18Drex Max fine-tuneNace.AI26.1B48.0tied with d358.4%1,542 questions8.0cac65e1
19d3-ultra fine-tunevLLM Semantic Router26.1B48.0tied with Drex Max58.4%1,542 questions16.352ba9e7
20Drex Max v1.1Nace.AINot disclosed48.0tied with d3-ultra58.4%1,542 questions2.7no repo
21Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B47.1tied with Drex Max v1.157.7%1,542 questions6.2225c913
22Perplexity Decider 27B v1.1Perplexity26.1B46.8tied with Perplexity Decider 27B v1.2 · DGX Spark57.5%1,542 questions5.5no repo
23Mammoth-1 post-trainedBase Labs27.4B46.5tied with Perplexity Decider 27B v1.157.2%1,542 questions7.961cec44
24SubQ Snap PreviewSubquadraticNot disclosed44.6tied with Mammoth-155.7%1,542 questions6.0no repo
25GPT-6 Luna Decisions APIOpenAINot disclosed42.9tied with SubQ Snap Preview54.3%1,542 questions3.0no repo
26SubQ Snap 1.1SubquadraticNot disclosed42.5tied with GPT-6 Luna · Decisions API54.0%1,542 questions14.5no repo
27KnowLine-12B-Gen1PelaAI12B40.6tied with SubQ Snap 1.152.5%1,542 questions4.3f26fbc2
28d1.1Liquid AINot disclosed39.6tied with KnowLine-12B-Gen151.7%1,542 questions2.2no repo
29d1.2Liquid AINot disclosed39.3tied with d1.151.4%1,542 questions2.5no repo
30celeris-1-decisionCelerisNot disclosed37.4tied with d1.249.9%1,542 questions5.3no repo
31Perplexity Decider 27B v1Perplexity26.1B36.9tied with celeris-1-decision49.5%1,542 questions5.0no repo
32OneJev-27B full fine-tuneOmniJev27.4B35.9tied with Perplexity Decider 27B v148.7%1,542 questions3.7a5e5f55
33Jev 1.14TypeSafeNot disclosed35.2tied with OneJev-27B48.2%1,542 questions4.0no repo
34Mercury DecideInceptionNot disclosed33.9tied with Jev 1.1447.1%1,542 questions19.0no repo
35Jev 1.13TypeSafeNot disclosed33.0tied with Mercury Decide46.4%1,542 questions6.5no repo
36d1Liquid AINot disclosed32.6tied with Jev 1.1346.0%1,542 questions13.5no repo
37Jeffan-9B-1211tattrongvu8.9B32.5tied with d146.0%1,542 questions3.0862d1a7
38Clef post-trainedCloudflare27.5B31.8tied with Jeffan-9B-121145.5%1,542 questions15.12f3de3d
39Drex 2.0Nace.AI35.1B3B active31.0tied with Clef44.8%1,542 questions2.0no repo
40Cygnet unchanged baseBlockbrain Labs12B30.2tied with Drex 2.044.2%1,542 questions18.9no repo
41Winnow-12B LoRA, mergedEldanRing11.9B30.2tied with Cygnet44.2%1,542 questions24.1b2b1421
42Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B30.1tied with Winnow-12B44.1%1,542 questions16.428cf730
43KnowLine-9B-Gen1PelaAI9.7B29.7tied with Open-Jev-27B-v1.143.8%1,542 questions6.8325378d
44Cerno 9B v1.2 full fine-tuneApertix AI8.9B29.6tied with KnowLine-9B-Gen143.7%1,542 questions14.2f02a632
45KnowLine-9B-Gen2PelaAI9.7B27.6tied with Cerno 9B v1.242.1%1,542 questions5.18ec8eb6
46Adjudex 9B v5newsbang8.4B26.7tied with KnowLine-9B-Gen241.4%1,542 questions12.2ffdcb82
47Elephant-1 post-trainedBase Labs9.4B25.5tied with Adjudex 9B v540.4%1,542 questions15.4b3f8a93
48GLiDE 9BFastino7.9B25.1tied with Elephant-140.1%1,542 questions12.0ab54df5
49Dexy-Decision-9B-V2 fine-tunePatrick-188.4B24.6tied with GLiDE 9B39.7%1,542 questions12.74f434e2
50StartLux-Decision-v1.1-9B-InstantStartLux9.7B24.4tied with Dexy-Decision-9B-V239.5%1,542 questions11.318a17f7
51Perplexity Decider Flash v1.3Perplexity4.5B24.0tied with StartLux-Decision-v1.1-9B-Instant39.2%1,542 questions15.3d06fb3f
52vera-medium-omniscar-ai4.2B22.3tied with Perplexity Decider Flash v1.337.9%1,542 questions8.01f23a6c
53KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B21.6tied with vera-medium-omni37.3%1,542 questions16.6779944d
54Perplexity Decider Flash v1.2Perplexity4.5B21.4tied with KnowLine 4B Gen5.537.2%1,542 questions11.06d771ad
55StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B20.9tied with Perplexity Decider Flash v1.236.7%1,542 questions10.3416088f
56Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B20.5tied with StartLux-Decision-v1.1-4B-Instant36.4%1,542 questions15.4889f23e
57Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B20.5tied with Chinese Decision Pointer 4B36.4%1,542 questions12.845830d2
58Valen-Decision-1.5-9BValen-Team8.4B18.8tied with Verdin-4B-v1.135.0%1,542 questions6.95a51211
59Microsoft-Decision-1MicrosoftNot disclosed18.7tied with Valen-Decision-1.5-9B35.0%1,542 questions24.4no repo
60Clef-Flash post-trainedCloudflare9.5B17.6tied with Microsoft-Decision-134.1%1,542 questions20.3fde727a
61Metask-Jev-4B LoRA, mergedMetask Lab4.5B14.231.4%1,542 questions6.5ea20fe8
62KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B11.7tied with Metask-Jev-4B29.4%1,542 questions11.453245d4
63GLiDE 2BFastino1.9B10.7tied with KnowLine-2B-Gen128.5%1,542 questions22.500f122c
64StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B7.926.3%1,542 questions9.4f1e6cd1
65StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M7.3tied with StartLux-Decision-v1.1-2B-Instant25.8%1,542 questions4.66496ae6
66OneJev1.6-0.8B full fine-tuneOmniJev850M5.8tied with StartLux-Decision-v1.1-0.8B-Instant24.6%1,542 questions10.3bcd88a3
67d1-2BLiquid AI1.9B5.6tied with OneJev1.6-0.8B24.5%1,542 questions6.710e7671
68Decider 1meraGPTNot disclosed4.9tied with d1-2B23.9%1,542 questions15.4no repo
69GLiDE 0.8BFastino750M4.6tied with Decider 123.7%1,542 questions26.19eacdee
70Strands Decider 2B LoRA + headStrands Agents2.3B4.3tied with GLiDE 0.8B23.5%1,542 questions13.22b52a62
71KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M3.0tied with Strands Decider 2B22.4%1,542 questions10.86b27831
72GLiClass v4 Edge fine-tuneKnowledgator440M2.8tied with KnowLine-0.8B-Gen122.2%1,542 questions10.0885af1a
73GLiDE 0.4BFastino400M1.6tied with GLiClass v4 Edge21.3%1,542 questions8.36bda7ab
74GLiNER2.5-multi-Decide fine-tuneFastino290M−0.2tied with GLiDE 0.4B19.8%1,542 questions22.7a35a0cd
75LayaConvai Innovations420M−2.2tied with GLiNER2.5-multi-Decide18.2%1,542 questions14.9c5d7873
76Laya MultilingualConvai Innovations320M−2.2tied with Laya18.2%1,542 questions26.0c5d7873

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Gemini 4 Carbon high · chatGoogleNot disclosed98.598.8%1,542 questions0.3no repo
2Gemini 3.8 Flash high · chatGoogleNot disclosed98.1tied with Gemini 4 Carbon · high · chat98.5%1,542 questions1.0no repo
3Gemini 4 Hydrogen high · chatGoogleNot disclosed98.1tied with Gemini 3.8 Flash · high · chat98.4%1,542 questions0.9no repo
4Claude Fable 5.2 max · chatAnthropicNot disclosed97.9tied with Gemini 4 Hydrogen · high · chat98.3%1,542 questions7.2no repo
5GPT-6 Astra high · chatOpenAINot disclosed97.9tied with Claude Fable 5.2 · max · chat98.3%1,542 questions0.7no repo
6GPT-6.1 Sol high · chatOpenAINot disclosed97.9tied with GPT-6 Astra · high · chat98.3%1,542 questions0.6no repo
7Claude Fable 5.1 high · chatAnthropicNot disclosed95.5tied with GPT-6.1 Sol · high · chat96.4%1,542 questions12.5no repo
8Gemini 4 Argon low · chatGoogleNot disclosed95.1tied with Claude Fable 5.1 · high · chat96.1%1,542 questions2.7no repo
9Claude Opus 5.5 high · chatAnthropicNot disclosed93.4tied with Gemini 4 Argon · low · chat94.7%1,542 questions14.9no repo
10GPT-6.1 Terra high · chatOpenAINot disclosed91.8tied with Claude Opus 5.5 · high · chat93.5%1,542 questions3.5no repo
11Grok 4.7 high · chatxAINot disclosed89.7tied with GPT-6.1 Terra · high · chat91.8%1,542 questions12.4no repo
12DeepSeek-V4.1-Flash reasoning on · chat · first releaseDeepSeek552B16B active89.3tied with Grok 4.7 · high · chat91.4%1,542 questions8.6no repo
13DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active88.5tied with DeepSeek-V4.1-Flash · reasoning on · chat · first release90.8%1,542 questions7.3no repo
14Qwen4-Max high · chatAlibabaNot disclosed88.1tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101790.5%1,542 questions8.3no repo
15Claude Sonnet 5.5 high · chatAnthropicNot disclosed81.285.0%1,542 questions16.9no repo
16Muse Ember max · chatMetaNot disclosed79.6tied with Claude Sonnet 5.5 · high · chat83.7%1,542 questions6.3no repo
17Qwen3.5-397B-A17B high · chatAlibaba397B17B active74.079.2%1,542 questions9.1no repo
18Kimi K3.1 Pro low · chatMoonshot2.8T104B active72.6tied with Qwen3.5-397B-A17B · high · chat78.1%1,542 questions16.2no repo
19Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active71.1tied with Kimi K3.1 Pro · low · chat76.9%1,542 questions9.9no repo
20GLM-5.3-Flash reasoning on · chatZ.ai320B18B active67.974.3%1,542 questions4.4no repo
21GLM-5.3 low · chatZ.ai753.3B61.268.9%1,542 questions5.5no repo
22Qwen3.8-27B low · chatAlibaba27.8B60.5tied with GLM-5.3 · low · chat68.4%1,542 questions29.41d4bf0f
23MiMo-V2.6-Flash high · chatXiaomi309B15B active57.466.0%1,542 questions10.3no repo
24Gemma 4 31B reasoning off · chatGoogle31.3B57.0tied with MiMo-V2.6-Flash · high · chat65.6%1,542 questions31.0842da37
25GPT-6 Luna low · chatOpenAINot disclosed50.260.2%1,542 questions32.7no repo
26MiniMax-M3.1-Flash high · chatMiniMax428B23B active49.3tied with GPT-6 Luna · low · chat59.5%1,542 questions26.1no repo
27Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed41.052.8%1,542 questions38.5no repo
28Qwen3.5-4B reasoning on · chatAlibaba4.7B37.950.3%1,542 questions–851bf6e
29Gemma 4 E4B default reasoning · chatGoogle8B36.3tied with Qwen3.5-4B · reasoning on · chat49.0%1,542 questions–4ecc2ba
30Claude Haiku 5.5 max · chatAnthropicNot disclosed30.444.4%1,542 questions16.4no repo
31Qwen3.5-4B reasoning off · chatAlibaba4.7B20.936.7%1,542 questions–851bf6e
32Qwen3-14B reasoning off · chatAlibaba14.8B18.134.5%1,542 questions48.640c0698
33Erk-14B reasoning off · chateCloud Technology14.8B16.7tied with Qwen3-14B · reasoning off · chat33.4%1,542 questions16.38584b14
34LFM2.5-2.6B default reasoning · chatLiquid AI2.7B11.229.0%1,542 questions–7141b56
35LFM3-1.7B reasoning on · chatLiquid AI1.7B9.2tied with LFM2.5-2.6B · default reasoning · chat27.4%1,542 questions–0dc4268
36LFM3-1.7B reasoning off · chatLiquid AI1.7B5.724.6%1,542 questions–0dc4268
37Qwen3.5-2B reasoning off · chatAlibaba2.3B2.021.6%1,542 questions–15852e8
38LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active−1.219.0%1,542 questions–dd83f5c
39GLiNER2.5 MultiFastino290M−3.0tied with LFM2.5-8B-A1B · default reasoning · chat17.6%1,542 questions35.82ca71aa
40MiniCPM5-2B reasoning off · chatOpenBMB2.5B−18.55.2%1,542 questions–42433e9

1,542 questions per model · updated 16 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 1,542 Turkish grammar questions (units 7 to 20) right (accuracy 100%) · feed: bench-scores.json

Reading · English

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B92.093.6%547 questions5.08e56aa1
2Copernicus-31BPoindexter Labs31.3B91.1tied with KnowLine-31B-Gen1.592.9%547 questions7.8be71525
3REAX 1.3 blendREAX57.4B90.6tied with Copernicus-31B92.5%547 questions3.6fa4bf7e
4KnowLine-31B-Gen2PelaAI31.3B90.4tied with REAX 1.392.3%547 questions7.7fd99a9e
5Karpo Decision 31BMachinePulse31.3B90.2tied with KnowLine-31B-Gen292.1%547 questions2.4d69cdd5
6Gemini Decision 1 20261111GoogleNot disclosed88.8tied with Karpo Decision 31B91.0%547 questions4.5no repo
7Logits Sigma-1 Duo blendLogits57.4B88.8tied with Gemini Decision 1 · 2026111191.0%547 questions8.0a0c1b6e
8GLiDE 27BFastino26.1B88.6tied with Logits Sigma-1 Duo90.9%547 questions1.74d32743
9Sieve 31B v3 full fine-tuneSthānika AI31.3B88.3tied with GLiDE 27B90.7%547 questions4.38fcee85
10Turion Jev 31B LoRA, mergedTurion.AI31.3B88.3tied with Sieve 31B v390.7%547 questions11.6866c0fc
11d1-ultraLiquid AINot disclosed88.3tied with Turion Jev 31B90.7%547 questions4.4no repo
12Perplexity Decider 27B v1.2 API · with webPerplexity26.1B88.1tied with d1-ultra90.5%547 questions3.6no repo
13GLiDE v1.2Fastino26.1B87.9tied with Perplexity Decider 27B v1.2 · API · with web90.3%547 questions2.7a7e4589
14d1.2-ultraLiquid AINot disclosed87.9tied with GLiDE · v1.290.3%547 questions8.5no repo
15Mammoth-1 post-trainedBase Labs27.4B87.7tied with d1.2-ultra90.1%547 questions2.461cec44
16d3 fine-tunevLLM Semantic Router26.1B87.2tied with Mammoth-189.8%547 questions3.620f37b9
17Drex Max fine-tuneNace.AI26.1B87.0tied with d389.6%547 questions2.6cac65e1
18Perplexity Decider 27B v1.3Perplexity26.1B87.0tied with Drex Max89.6%547 questions3.49c2ebb2
19d3-ultra fine-tunevLLM Semantic Router26.1B86.7tied with Perplexity Decider 27B v1.389.4%547 questions2.852ba9e7
20d1.2Liquid AINot disclosed86.3tied with d3-ultra89.0%547 questions8.3no repo
21Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B85.8tied with d1.288.7%547 questions3.7225c913
22Drex Max v1.1Nace.AINot disclosed85.6tied with Perplexity Decider 27B v1.2 · DGX Spark88.5%547 questions3.8no repo
23Perplexity Decider 27B v1Perplexity26.1B85.6tied with Drex Max v1.188.5%547 questions2.6no repo
24d1.1Liquid AINot disclosed85.4tied with Perplexity Decider 27B v188.3%547 questions4.2no repo
25Jeffan-9B-1211tattrongvu8.9B84.5tied with d1.187.6%547 questions8.9862d1a7
26StartLux-Decision-v1.1-9B-InstantStartLux9.7B83.5tied with Jeffan-9B-121186.8%547 questions6.618a17f7
27d1Liquid AINot disclosed83.5tied with StartLux-Decision-v1.1-9B-Instant86.8%547 questions2.6no repo
28Clef post-trainedCloudflare27.5B83.3tied with d186.7%547 questions3.12f3de3d
29KnowLine-12B-Gen1PelaAI12B83.3tied with Clef86.7%547 questions11.1f26fbc2
30GLiDE 9BFastino7.9B82.9tied with KnowLine-12B-Gen186.3%547 questions5.6ab54df5
31OneJev-27B full fine-tuneOmniJev27.4B82.9tied with GLiDE 9B86.3%547 questions9.7a5e5f55
32Jev 1.14TypeSafeNot disclosed82.6tied with OneJev-27B86.1%547 questions2.2no repo
33Cerno 9B v1.2 full fine-tuneApertix AI8.9B82.4tied with Jev 1.1485.9%547 questions4.2f02a632
34Drex 2.0Nace.AI35.1B3B active82.4tied with Cerno 9B v1.285.9%547 questions6.9no repo
35Jev 1.13TypeSafeNot disclosed82.4tied with Drex 2.085.9%547 questions3.1no repo
36KnowLine-9B-Gen1PelaAI9.7B82.4tied with Jev 1.1385.9%547 questions3.5325378d
37SubQ Snap 1.1SubquadraticNot disclosed82.4tied with KnowLine-9B-Gen185.9%547 questions3.2no repo
38SubQ Snap PreviewSubquadraticNot disclosed82.4tied with SubQ Snap 1.185.9%547 questions3.2no repo
39Adjudex 9B v5newsbang8.4B82.2tied with SubQ Snap Preview85.7%547 questions3.4ffdcb82
40Mercury DecideInceptionNot disclosed82.2tied with Adjudex 9B v585.7%547 questions6.6no repo
41Microsoft-Decision-1MicrosoftNot disclosed82.2tied with Mercury Decide85.7%547 questions2.8no repo
42Elephant-1 post-trainedBase Labs9.4B81.7tied with Microsoft-Decision-185.4%547 questions2.7b3f8a93
43Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B81.7tied with Elephant-185.4%547 questions45.328cf730
44KnowLine-9B-Gen2PelaAI9.7B81.5tied with Open-Jev-27B-v1.185.2%547 questions8.78ec8eb6
45GPT-6 Luna Decisions APIOpenAINot disclosed80.1tied with KnowLine-9B-Gen284.1%547 questions3.2no repo
46celeris-1-decisionCelerisNot disclosed80.1tied with GPT-6 Luna · Decisions API84.1%547 questions5.2no repo
47Cygnet unchanged baseBlockbrain Labs12B79.9tied with celeris-1-decision83.9%547 questions4.3no repo
48Winnow-12B LoRA, mergedEldanRing11.9B79.9tied with Cygnet83.9%547 questions8.7b2b1421
49Perplexity Decider Flash v1.3Perplexity4.5B79.7tied with Winnow-12B83.7%547 questions2.7d06fb3f
50Valen-Decision-1.5-9BValen-Team8.4B79.7tied with Perplexity Decider Flash v1.383.7%547 questions8.45a51211
51celeris-1.1-decisionCelerisNot disclosed79.7tied with Valen-Decision-1.5-9B83.7%547 questions5.0no repo
52Dexy-Decision-9B-V2 fine-tunePatrick-188.4B79.4tied with celeris-1.1-decision83.5%547 questions4.24f434e2
53vera-medium-omniscar-ai4.2B79.4tied with Dexy-Decision-9B-V283.5%547 questions9.91f23a6c
54StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B78.5tied with vera-medium-omni82.8%547 questions3.2416088f
55Clef-Flash post-trainedCloudflare9.5B77.8tied with StartLux-Decision-v1.1-4B-Instant82.3%547 questions4.0fde727a
56Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B77.8tied with Clef-Flash82.3%547 questions3.545830d2
57Perplexity Decider Flash v1.2Perplexity4.5B77.4tied with Verdin-4B-v1.181.9%547 questions4.16d771ad
58KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B76.0tied with Perplexity Decider Flash v1.280.8%547 questions5.8779944d
59Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B75.5tied with KnowLine 4B Gen5.580.4%547 questions3.5889f23e
60Metask-Jev-4B LoRA, mergedMetask Lab4.5B71.4tied with Chinese Decision Pointer 4B77.1%547 questions4.4ea20fe8
61KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B64.171.3%547 questions5.153245d4
62GLiDE 2BFastino1.9B62.5tied with KnowLine-2B-Gen170.0%547 questions8.200f122c
63StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B62.1tied with GLiDE 2B69.7%547 questions8.2f1e6cd1
64d1-2BLiquid AI1.9B57.065.6%547 questions7.910e7671
65GLiDE 0.8BFastino750M50.960.7%547 questions12.59eacdee
66StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M49.5tied with GLiDE 0.8B59.6%547 questions8.76496ae6
67GLiDE 0.4BFastino400M49.3tied with StartLux-Decision-v1.1-0.8B-Instant59.4%547 questions4.86bda7ab
68OneJev1.6-0.8B full fine-tuneOmniJev850M49.3tied with GLiDE 0.4B59.4%547 questions3.7bcd88a3
69KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M48.6tied with OneJev1.6-0.8B58.9%547 questions8.16b27831
70GLiClass v4 Edge fine-tuneKnowledgator440M47.9tied with KnowLine-0.8B-Gen158.3%547 questions3.8885af1a
71Decider 1meraGPTNot disclosed44.7tied with GLiClass v4 Edge55.8%547 questions5.3no repo
72Strands Decider 2B LoRA + headStrands Agents2.3B31.945.5%547 questions10.52b52a62
73LayaConvai Innovations420M3.622.9%547 questions12.1c5d7873
74GLiNER2.5-multi-Decide fine-tuneFastino290M1.5tied with Laya21.2%547 questions27.2a35a0cd
75Laya MultilingualConvai Innovations320M−5.815.4%547 questions44.4c5d7873

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Claude Fable 5.2 max · chatAnthropicNot disclosed97.598.0%547 questions10.7no repo
2Gemini 4 Hydrogen high · chatGoogleNot disclosed95.9tied with Claude Fable 5.2 · max · chat96.7%547 questions3.1no repo
3GPT-6 Astra high · chatOpenAINot disclosed95.2tied with Gemini 4 Hydrogen · high · chat96.2%547 questions1.8no repo
4GPT-6.1 Terra high · chatOpenAINot disclosed95.2tied with GPT-6 Astra · high · chat96.2%547 questions1.3no repo
5Claude Fable 5.1 high · chatAnthropicNot disclosed95.0tied with GPT-6.1 Terra · high · chat96.0%547 questions11.8no repo
6Gemini 3.8 Flash high · chatGoogleNot disclosed95.0tied with Claude Fable 5.1 · high · chat96.0%547 questions1.7no repo
7Gemini 4 Argon low · chatGoogleNot disclosed95.0tied with Gemini 3.8 Flash · high · chat96.0%547 questions1.5no repo
8GPT-6.1 Sol high · chatOpenAINot disclosed94.5tied with Gemini 4 Argon · low · chat95.6%547 questions2.4no repo
9Claude Opus 5.5 high · chatAnthropicNot disclosed93.1tied with GPT-6.1 Sol · high · chat94.5%547 questions13.3no repo
10Qwen4-Max high · chatAlibabaNot disclosed93.1tied with Claude Opus 5.5 · high · chat94.5%547 questions8.5no repo
11Gemini 4 Carbon high · chatGoogleNot disclosed92.9tied with Qwen4-Max · high · chat94.3%547 questions3.1no repo
12Grok 4.7 high · chatxAINot disclosed91.8tied with Gemini 4 Carbon · high · chat93.4%547 questions11.1no repo
13Claude Haiku 5.5 max · chatAnthropicNot disclosed90.9tied with Grok 4.7 · high · chat92.7%547 questions18.4no repo
14DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active90.6tied with Claude Haiku 5.5 · max · chat92.5%547 questions7.5no repo
15Kimi K3.1 Pro low · chatMoonshot2.8T104B active90.4tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101792.3%547 questions5.7no repo
16Qwen3.5-397B-A17B high · chatAlibaba397B17B active89.3tied with Kimi K3.1 Pro · low · chat91.4%547 questions3.3no repo
17Gemma 4 31B reasoning off · chatGoogle31.3B88.1tied with Qwen3.5-397B-A17B · high · chat90.5%547 questions8.5842da37
18GLM-5.3-Flash reasoning on · chatZ.ai320B18B active87.0tied with Gemma 4 31B · reasoning off · chat89.6%547 questions7.0no repo
19Muse Ember max · chatMetaNot disclosed87.0tied with GLM-5.3-Flash · reasoning on · chat89.6%547 questions7.0no repo
20MiMo-V2.6-Flash high · chatXiaomi309B15B active86.3tied with Muse Ember · max · chat89.0%547 questions6.5no repo
21Qwen3.8-27B low · chatAlibaba27.8B86.3tied with MiMo-V2.6-Flash · high · chat89.0%547 questions11.01d4bf0f
22Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active85.4tied with Qwen3.8-27B · low · chat88.3%547 questions2.6no repo
23GLM-5.3 low · chatZ.ai753.3B84.7tied with Inkling-Small · high · chat · Jul 202687.8%547 questions3.6no repo
24Qwen3.5-4B reasoning on · chatAlibaba4.7B83.8tied with GLM-5.3 · low · chat87.0%547 questions–851bf6e
25Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed83.8tied with Qwen3.5-4B · reasoning on · chat87.0%547 questions5.5no repo
26Gemma 4 12B Unified default reasoning · chatGoogle12B83.5tied with Qwen3.8 Flash · no reasoning · chat86.8%547 questions–43c9d01
27GPT-6 Luna low · chatOpenAINot disclosed82.9tied with Gemma 4 12B Unified · default reasoning · chat86.3%547 questions10.1no repo
28Claude Sonnet 5.5 high · chatAnthropicNot disclosed82.6tied with GPT-6 Luna · low · chat86.1%547 questions13.9no repo
29MiniMax-M3.1-Flash high · chatMiniMax428B23B active80.6tied with Claude Sonnet 5.5 · high · chat84.5%547 questions9.0no repo
30LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active78.3tied with MiniMax-M3.1-Flash · high · chat82.6%547 questions–1769516
31Gemma 4 E4B default reasoning · chatGoogle8B78.1tied with LFM2.5-24B-A2B · default reasoning · chat82.4%547 questions–4ecc2ba
32Qwen3.5-4B reasoning off · chatAlibaba4.7B75.5tied with Gemma 4 E4B · default reasoning · chat80.4%547 questions–851bf6e
33Qwen3-14B reasoning off · chatAlibaba14.8B74.6tied with Qwen3.5-4B · reasoning off · chat79.7%547 questions17.840c0698
34MiniCPM5-2B reasoning off · chatOpenBMB2.5B72.7tied with Qwen3-14B · reasoning off · chat78.1%547 questions–42433e9
35Gemma 4 E2B reasoning on · chatGoogle5.1B70.3tied with MiniCPM5-2B · reasoning off · chat76.2%547 questions–4adf157
36Qwen3.5-2B reasoning off · chatAlibaba2.3B69.8tied with Gemma 4 E2B · reasoning on · chat75.9%547 questions–15852e8
37Gemma 4 E2B default reasoning · chatGoogle5.1B68.2tied with Qwen3.5-2B · reasoning off · chat74.6%547 questions–4adf157
38Granite-4.2-3B reasoning on · chatIBM3.7B68.0tied with Gemma 4 E2B · default reasoning · chat74.4%547 questions–4e9c275
39LFM2.5-2.6B default reasoning · chatLiquid AI2.7B67.3tied with Granite-4.2-3B · reasoning on · chat73.9%547 questions–7141b56
40Erk-14B reasoning off · chateCloud Technology14.8B65.0tied with LFM2.5-2.6B · default reasoning · chat72.0%547 questions10.98584b14
41LFM3-1.7B reasoning on · chatLiquid AI1.7B64.7tied with Erk-14B · reasoning off · chat71.8%547 questions–0dc4268
42LFM3-1.7B reasoning off · chatLiquid AI1.7B63.9tied with LFM3-1.7B · reasoning on · chat71.1%547 questions–0dc4268
43LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active63.0tied with LFM3-1.7B · reasoning off · chat70.4%547 questions–dd83f5c
44MiniCPM5-2B reasoning off · chatOpenBMB2.5B59.7tied with LFM2.5-8B-A1B · default reasoning · chat67.8%547 questions–42433e9
45Qwen3.5-2B reasoning off · chatAlibaba2.3B59.1tied with MiniCPM5-2B · reasoning off · chat67.3%547 questions–15852e8
46Raon-1 nano reasoning off · chatKRAFTON3.5B42.253.7%547 questions–fcf2d1d
47GLiNER2.5 MultiFastino290M−4.716.3%547 questions36.42ca71aa

547 questions per model · updated 16 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 547 kept reading questions, translated into English, right (accuracy 100%) · feed: bench-scores.json

Reading · Vision

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Gemini Decision 1 20261111GoogleNot disclosed58.166.4%763 questions5.2no repo
2d1-ultraLiquid AINot disclosed37.650.1%763 questions3.5no repo
3GLiDE 27BFastino26.1B37.4tied with d1-ultra49.9%763 questions4.34d32743
4KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B37.3tied with GLiDE 27B49.8%763 questions6.18e56aa1
5d3 fine-tunevLLM Semantic Router26.1B35.6tied with KnowLine-31B-Gen1.548.5%763 questions3.720f37b9
6d1.2Liquid AINot disclosed33.7tied with d346.9%763 questions5.6no repo
7Drex Max fine-tuneNace.AI26.1B33.5tied with d1.246.8%763 questions9.3cac65e1
8SubQ Snap 1.1SubquadraticNot disclosed32.2tied with Drex Max45.7%763 questions11.7no repo
9d3-ultra fine-tunevLLM Semantic Router26.1B31.8tied with SubQ Snap 1.145.5%763 questions18.852ba9e7
10d1.2-ultraLiquid AINot disclosed31.7tied with d3-ultra45.3%763 questions2.8no repo
11Drex Max v1.1Nace.AINot disclosed31.4tied with d1.2-ultra45.1%763 questions5.2no repo
12Jev 1.14TypeSafeNot disclosed29.1tied with Drex Max v1.143.3%763 questions3.7no repo
13OneJev-27B full fine-tuneOmniJev27.4B26.6tied with Jev 1.1441.3%763 questions4.9a5e5f55
14GPT-6 Luna Decisions APIOpenAINot disclosed15.832.6%763 questions5.7no repo
15KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B12.7tied with GPT-6 Luna · Decisions API30.1%763 questions11.0779944d
16OneJev1.6-0.8B full fine-tuneOmniJev850M8.126.5%763 questions4.2bcd88a3
17StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M5.3tied with OneJev1.6-0.8B24.2%763 questions1.76496ae6
18KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M4.8tied with StartLux-Decision-v1.1-0.8B-Instant23.9%763 questions1.56b27831
19StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B4.3tied with KnowLine-0.8B-Gen123.5%763 questions3.6f1e6cd1
20d1-2BLiquid AI1.9B4.2tied with StartLux-Decision-v1.1-2B-Instant23.3%763 questions1.110e7671

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Muse Ember max · chatMetaNot disclosed82.886.2%763 questions–no repo
2Gemma 4 E4B default reasoning · chatGoogle8B28.642.9%763 questions–4ecc2ba

763 questions per model · updated 16 November 2026 · 0 = chance: 1 of each question's options right on every question (accuracy 20% with 5 options); 100 = every one of the 763 questions that depend on the printed page right, read from the page image alone (accuracy 100%) · feed: bench-scores.json

Logic

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B50.960.7%1,125 questions5.7d69cdd5
2REAX 1.3 blendREAX57.4B49.1tied with Karpo Decision 31B59.3%1,125 questions3.7fa4bf7e
3KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B48.8tied with REAX 1.359.0%1,125 questions2.68e56aa1
4KnowLine-31B-Gen2PelaAI31.3B48.7tied with KnowLine-31B-Gen1.558.9%1,125 questions3.7fd99a9e
5Copernicus-31BPoindexter Labs31.3B48.3tied with KnowLine-31B-Gen258.7%1,125 questions3.9be71525
6GLiDE 27BFastino26.1B48.0tied with Copernicus-31B58.4%1,125 questions4.44d32743
7Sieve 31B v3 full fine-tuneSthānika AI31.3B47.3tied with GLiDE 27B57.9%1,125 questions2.18fcee85
8Logits Sigma-1 Duo blendLogits57.4B47.2tied with Sieve 31B v357.8%1,125 questions5.0a0c1b6e
9GLiDE v1.2Fastino26.1B44.055.2%1,125 questions5.9a7e4589
10d1-ultraLiquid AINot disclosed43.7tied with GLiDE · v1.254.9%1,125 questions3.2no repo
11Drex Max v1.1Nace.AINot disclosed43.4tied with d1-ultra54.8%1,125 questions6.1no repo
12Perplexity Decider 27B v1.3Perplexity26.1B43.4tied with Drex Max v1.154.8%1,125 questions9.09c2ebb2
13d3-ultra fine-tunevLLM Semantic Router26.1B43.2tied with Perplexity Decider 27B v1.354.6%1,125 questions15.152ba9e7
14GLiDE APIFastinoNot disclosed43.1tied with d3-ultra54.5%1,125 questions6.6no repo
15Mammoth-1 post-trainedBase Labs27.4B43.1tied with GLiDE · API54.5%1,125 questions5.361cec44
16d3 fine-tunevLLM Semantic Router26.1B43.1tied with Mammoth-154.5%1,125 questions4.620f37b9
17Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B41.7tied with d353.3%1,125 questions3.4no repo
18Drex Max fine-tuneNace.AI26.1B41.3tied with Perplexity Decider 27B v1.2 · DGX Spark53.1%1,125 questions16.2cac65e1
19Perplexity Decider 27B v1.2 API · with webPerplexity26.1B40.7tied with Drex Max52.5%1,125 questions4.2no repo
20SubQ Snap 1.1SubquadraticNot disclosed40.4tied with Perplexity Decider 27B v1.2 · API · with web52.4%1,125 questions12.9no repo
21Turion Jev 31B LoRA, mergedTurion.AI31.3B40.3tied with SubQ Snap 1.152.3%1,125 questions2.9866c0fc
22d1.2-ultraLiquid AINot disclosed39.4tied with Turion Jev 31B51.6%1,125 questions3.0no repo
23KnowLine-12B-Gen1PelaAI12B38.2tied with d1.2-ultra50.6%1,125 questions4.8f26fbc2
24Gemini Decision 1 20261111GoogleNot disclosed37.6tied with KnowLine-12B-Gen150.0%1,125 questions3.1no repo
25d1.2Liquid AINot disclosed36.4tied with Gemini Decision 1 · 2026111149.2%1,125 questions2.6no repo
26Jeffan-9B-1211tattrongvu8.9B35.2tied with d1.248.2%1,125 questions4.0862d1a7
27Jev 1.14TypeSafeNot disclosed35.2tied with Jeffan-9B-121148.2%1,125 questions3.3no repo
28GLiDE 9BFastino7.9B34.8tied with Jev 1.1447.8%1,125 questions6.7ab54df5
29celeris-1-decisionCelerisNot disclosed34.1tied with GLiDE 9B47.3%1,125 questions10.7no repo
30celeris-1.1-decisionCelerisNot disclosed34.1tied with celeris-1-decision47.3%1,125 questions9.8no repo
31StartLux-Decision-v1.1-9B-InstantStartLux9.7B33.2tied with celeris-1.1-decision46.6%1,125 questions3.818a17f7
32Adjudex 9B v5newsbang8.4B33.0tied with StartLux-Decision-v1.1-9B-Instant46.4%1,125 questions7.2ffdcb82
33Elephant-1 post-trainedBase Labs9.4B33.0tied with Adjudex 9B v546.4%1,125 questions8.6b3f8a93
34Jev 1.13TypeSafeNot disclosed32.9tied with Elephant-146.3%1,125 questions5.9no repo
35d1.1Liquid AINot disclosed32.6tied with Jev 1.1346.0%1,125 questions4.4no repo
36StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B32.3tied with d1.145.9%1,125 questions3.3416088f
37Dexy-Decision-9B-V2 fine-tunePatrick-188.4B31.7tied with StartLux-Decision-v1.1-4B-Instant45.3%1,125 questions14.94f434e2
38Mercury DecideInceptionNot disclosed31.3tied with Dexy-Decision-9B-V245.1%1,125 questions15.4no repo
39Perplexity Decider Flash v1.3Perplexity4.5B30.9tied with Mercury Decide44.7%1,125 questions10.1d06fb3f
40Microsoft-Decision-1MicrosoftNot disclosed30.6tied with Perplexity Decider Flash v1.344.4%1,125 questions8.0no repo
41Valen-Decision-1.5-9BValen-Team8.4B30.4tied with Microsoft-Decision-144.4%1,125 questions5.85a51211
42KnowLine-9B-Gen1PelaAI9.7B30.3tied with Valen-Decision-1.5-9B44.3%1,125 questions3.5325378d
43vera-medium-omniscar-ai4.2B30.1tied with KnowLine-9B-Gen144.1%1,125 questions4.71f23a6c
44KnowLine-9B-Gen2PelaAI9.7B29.9tied with vera-medium-omni43.9%1,125 questions3.28ec8eb6
45Cerno 9B v1.2 full fine-tuneApertix AI8.9B29.1tied with KnowLine-9B-Gen243.3%1,125 questions17.0f02a632
46Perplexity Decider Flash v1.2Perplexity4.5B28.9tied with Cerno 9B v1.243.1%1,125 questions6.06d771ad
47Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B28.2tied with Perplexity Decider Flash v1.242.6%1,125 questions12.928cf730
48KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B27.9tied with Open-Jev-27B-v1.142.3%1,125 questions13.7779944d
49Drex 2.0Nace.AI35.1B3B active27.3tied with KnowLine 4B Gen5.541.9%1,125 questions4.3no repo
50Clef post-trainedCloudflare27.5B26.0tied with Drex 2.040.8%1,125 questions15.72f3de3d
51d1Liquid AINot disclosed26.0tied with Clef40.8%1,125 questions10.9no repo
52OneJev-27B full fine-tuneOmniJev27.4B25.8tied with d140.6%1,125 questions4.6a5e5f55
53Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B25.4tied with OneJev-27B40.4%1,125 questions15.5889f23e
54Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B24.7tied with Chinese Decision Pointer 4B39.7%1,125 questions10.845830d2
55Winnow-12B LoRA, mergedEldanRing11.9B24.7tied with Verdin-4B-v1.139.7%1,125 questions14.4b2b1421
56Cygnet unchanged baseBlockbrain Labs12B24.1tied with Winnow-12B39.3%1,125 questions11.2no repo
57Clef-Flash post-trainedCloudflare9.5B23.7tied with Cygnet38.9%1,125 questions11.5fde727a
58GLiDE 2BFastino1.9B15.332.3%1,125 questions16.200f122c
59Metask-Jev-4B LoRA, mergedMetask Lab4.5B15.1tied with GLiDE 2B32.1%1,125 questions2.5ea20fe8
60KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B14.7tied with Metask-Jev-4B31.7%1,125 questions8.353245d4
61StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B13.4tied with KnowLine-2B-Gen130.8%1,125 questions5.3f1e6cd1
62d1-2BLiquid AI1.9B13.3tied with StartLux-Decision-v1.1-2B-Instant30.7%1,125 questions6.910e7671
63GLiDE 0.8BFastino750M11.4tied with d1-2B29.2%1,125 questions14.09eacdee
64OneJev1.6-0.8B full fine-tuneOmniJev850M8.8tied with GLiDE 0.8B27.0%1,125 questions11.2bcd88a3
65KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M8.3tied with OneJev1.6-0.8B26.7%1,125 questions8.46b27831
66StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M5.9tied with KnowLine-0.8B-Gen124.7%1,125 questions7.86496ae6
67GLiClass v4 Edge fine-tuneKnowledgator440M4.4tied with StartLux-Decision-v1.1-0.8B-Instant23.6%1,125 questions8.7885af1a
68GLiDE 0.4BFastino400M4.0tied with GLiClass v4 Edge23.2%1,125 questions8.86bda7ab
69Strands Decider 2B LoRA + headStrands Agents2.3B3.0tied with GLiDE 0.4B22.4%1,125 questions12.02b52a62

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1GPT-6 Astra high · chatOpenAINot disclosed95.996.7%1,125 questions–no repo
2Gemini 4 Carbon high · chatGoogleNot disclosed95.9tied with GPT-6 Astra · high · chat96.7%1,125 questions–no repo
3GPT-6.1 Sol high · chatOpenAINot disclosed95.7tied with Gemini 4 Carbon · high · chat96.5%1,125 questions–no repo
4Qwen4-Max high · chatAlibabaNot disclosed95.3tied with GPT-6.1 Sol · high · chat96.3%1,125 questions–no repo
5Claude Opus 5.5 high · chatAnthropicNot disclosed95.2tied with Qwen4-Max · high · chat96.2%1,125 questions–no repo
6DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active95.2tied with Claude Opus 5.5 · high · chat96.2%1,125 questions–no repo
7Gemini 4 Hydrogen high · chatGoogleNot disclosed95.1tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101796.1%1,125 questions–no repo
8Inkling 2 Small high · chatThinking Machines594B17B active94.6tied with Gemini 4 Hydrogen · high · chat95.6%1,125 questions–no repo
9Claude Sonnet 5.5 high · chatAnthropicNot disclosed94.3tied with Inkling 2 Small · high · chat95.5%1,125 questions–no repo
10Muse Ember max · chatMetaNot disclosed94.3tied with Claude Sonnet 5.5 · high · chat95.5%1,125 questions–no repo
11Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active94.2tied with Muse Ember · max · chat95.4%1,125 questions–no repo
12Claude Fable 5.1 high · chatAnthropicNot disclosed94.0tied with Inkling-Small · high · chat · Jul 202695.2%1,125 questions–no repo
13MiMo-V2.6-Flash high · chatXiaomi309B15B active93.9tied with Claude Fable 5.1 · high · chat95.1%1,125 questions–no repo
14Qwen3.5-397B-A17B high · chatAlibaba397B17B active93.9tied with MiMo-V2.6-Flash · high · chat95.1%1,125 questions–no repo
15Qwen3.8-27B low · chatAlibaba27.8B92.6tied with Qwen3.5-397B-A17B · high · chat94.0%1,125 questions–1d4bf0f
16GPT-6.1 Terra high · chatOpenAINot disclosed92.4tied with Qwen3.8-27B · low · chat94.0%1,125 questions–no repo
17MiniMax-M3.1-Flash high · chatMiniMax428B23B active92.4tied with GPT-6.1 Terra · high · chat94.0%1,125 questions–no repo
18Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed92.4tied with MiniMax-M3.1-Flash · high · chat94.0%1,125 questions–no repo
19Gemma 4 31B reasoning off · chatGoogle31.3B91.7tied with Qwen3.8 Flash · no reasoning · chat93.3%1,125 questions–842da37
20GLM-5.3-Flash reasoning on · chatZ.ai320B18B active91.4tied with Gemma 4 31B · reasoning off · chat93.2%1,125 questions–no repo
21Qwen3.5-4B reasoning on · chatAlibaba4.7B90.4tied with GLM-5.3-Flash · reasoning on · chat92.4%1,125 questions–851bf6e
22Granite-4.2-3B reasoning on · chatIBM3.7B89.6tied with Qwen3.5-4B · reasoning on · chat91.6%1,125 questions–4e9c275
23MiniCPM5-2B reasoning off · chatOpenBMB2.5B86.188.9%1,125 questions–42433e9
24Gemma 4 12B Unified default reasoning · chatGoogle12B84.7tied with MiniCPM5-2B · reasoning off · chat87.7%1,125 questions–43c9d01
25LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active83.3tied with Gemma 4 12B Unified · default reasoning · chat86.7%1,125 questions–1769516
26LFM3-1.7B reasoning on · chatLiquid AI1.7B81.9tied with LFM2.5-24B-A2B · default reasoning · chat85.5%1,125 questions–0dc4268
27Gemma 4 E4B default reasoning · chatGoogle8B79.0tied with LFM3-1.7B · reasoning on · chat83.2%1,125 questions–4ecc2ba
28LFM2.5-2.6B default reasoning · chatLiquid AI2.7B69.275.4%1,125 questions–7141b56
29Gemma 4 E2B reasoning on · chatGoogle5.1B66.172.9%1,125 questions–4adf157
30Qwen3-14B reasoning off · chatAlibaba14.8B56.765.3%1,125 questions–40c0698
31Raon-1 nano reasoning off · chatKRAFTON3.5B56.3tied with Qwen3-14B · reasoning off · chat65.1%1,125 questions–fcf2d1d
32Qwen3.5-2B reasoning off · chatAlibaba2.3B56.0tied with Raon-1 nano · reasoning off · chat64.8%1,125 questions–15852e8
33LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active50.260.2%1,125 questions–dd83f5c
34GPT-6 Luna low · chatOpenAINot disclosed47.4tied with LFM2.5-8B-A1B · default reasoning · chat58.0%1,125 questions–no repo
35Erk-14B reasoning off · chateCloud Technology14.8B42.854.2%1,125 questions–8584b14

1,125 questions per model · updated 16 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 1,125 Logic questions right (accuracy 100%) · feed: bench-scores.json

Math

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B76.981.6%2,163 questions1.2d69cdd5
2KnowLine-31B-Gen2PelaAI31.3B75.3tied with Karpo Decision 31B80.3%2,163 questions3.6fd99a9e
3REAX 1.3 blendREAX57.4B72.578.0%2,163 questions7.1fa4bf7e
4KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B71.7tied with REAX 1.377.4%2,163 questions3.68e56aa1
5Copernicus-31BPoindexter Labs31.3B71.7tied with KnowLine-31B-Gen1.577.3%2,163 questions6.5be71525
6Sieve 31B v3 full fine-tuneSthānika AI31.3B71.4tied with Copernicus-31B77.1%2,163 questions4.38fcee85
7Logits Sigma-1 Duo blendLogits57.4B69.075.2%2,163 questions7.5a0c1b6e
8d1-ultraLiquid AINot disclosed67.6tied with Logits Sigma-1 Duo74.1%2,163 questions2.3no repo
9Mammoth-1 post-trainedBase Labs27.4B67.5tied with d1-ultra74.0%2,163 questions3.061cec44
10Perplexity Decider 27B v1.3Perplexity26.1B65.6tied with Mammoth-172.5%2,163 questions4.59c2ebb2
11GLiDE 27BFastino26.1B65.0tied with Perplexity Decider 27B v1.372.0%2,163 questions4.14d32743
12d3-ultra fine-tunevLLM Semantic Router26.1B64.4tied with GLiDE 27B71.5%2,163 questions8.652ba9e7
13Gemini Decision 1 20261111GoogleNot disclosed64.2tied with d3-ultra71.3%2,163 questions2.5no repo
14GLiDE APIFastinoNot disclosed63.5tied with Gemini Decision 1 · 2026111170.8%2,163 questions5.0no repo
15d3 fine-tunevLLM Semantic Router26.1B63.0tied with GLiDE · API70.4%2,163 questions4.520f37b9
16Drex Max fine-tuneNace.AI26.1B62.9tied with d370.3%2,163 questions11.8cac65e1
17Drex Max v1.1Nace.AINot disclosed62.6tied with Drex Max70.1%2,163 questions5.7no repo
18celeris-1-decisionCelerisNot disclosed62.5tied with Drex Max v1.170.0%2,163 questions4.3no repo
19GLiDE v1.2Fastino26.1B62.3tied with celeris-1-decision69.9%2,163 questions5.2a7e4589
20celeris-1.1-decisionCelerisNot disclosed62.3tied with GLiDE · v1.269.9%2,163 questions4.5no repo
21d1.2-ultraLiquid AINot disclosed62.3tied with celeris-1.1-decision69.9%2,163 questions4.6no repo
22Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B62.2tied with d1.2-ultra69.8%2,163 questions3.2no repo
23Perplexity Decider 27B v1.2 API · with webPerplexity26.1B61.6tied with Perplexity Decider 27B v1.2 · DGX Spark69.3%2,163 questions3.7no repo
24Turion Jev 31B LoRA, mergedTurion.AI31.3B61.3tied with Perplexity Decider 27B v1.2 · API · with web69.0%2,163 questions4.7866c0fc
25SubQ Snap 1.1SubquadraticNot disclosed60.7tied with Turion Jev 31B68.6%2,163 questions9.5no repo
26KnowLine-12B-Gen1PelaAI12B57.866.3%2,163 questions4.0f26fbc2
27d1.2Liquid AINot disclosed56.9tied with KnowLine-12B-Gen165.5%2,163 questions3.2no repo
28d1.1Liquid AINot disclosed56.5tied with d1.265.2%2,163 questions2.2no repo
29Jev 1.14TypeSafeNot disclosed56.4tied with d1.165.1%2,163 questions2.8no repo
30Jev 1.13TypeSafeNot disclosed55.1tied with Jev 1.1464.1%2,163 questions2.6no repo
31Adjudex 9B v5newsbang8.4B52.662.1%2,163 questions4.8ffdcb82
32Elephant-1 post-trainedBase Labs9.4B51.7tied with Adjudex 9B v561.3%2,163 questions4.3b3f8a93
33KnowLine-9B-Gen2PelaAI9.7B50.1tied with Elephant-160.1%2,163 questions4.88ec8eb6
34Jeffan-9B-1211tattrongvu8.9B49.8tied with KnowLine-9B-Gen259.8%2,163 questions3.8862d1a7
35KnowLine-9B-Gen1PelaAI9.7B49.7tied with Jeffan-9B-121159.8%2,163 questions3.1325378d
36Microsoft-Decision-1MicrosoftNot disclosed49.2tied with KnowLine-9B-Gen159.4%2,163 questions4.1no repo
37Drex 2.0Nace.AI35.1B3B active48.7tied with Microsoft-Decision-159.0%2,163 questions1.9no repo
38Dexy-Decision-9B-V2 fine-tunePatrick-188.4B48.5tied with Drex 2.058.8%2,163 questions12.14f434e2
39StartLux-Decision-v1.1-9B-InstantStartLux9.7B48.0tied with Dexy-Decision-9B-V258.4%2,163 questions2.718a17f7
40GLiDE 9BFastino7.9B47.4tied with StartLux-Decision-v1.1-9B-Instant57.9%2,163 questions7.1ab54df5
41vera-medium-omniscar-ai4.2B46.0tied with GLiDE 9B56.8%2,163 questions3.01f23a6c
42Cerno 9B v1.2 full fine-tuneApertix AI8.9B45.0tied with vera-medium-omni56.0%2,163 questions15.7f02a632
43KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B45.0tied with Cerno 9B v1.256.0%2,163 questions8.6779944d
44Perplexity Decider Flash v1.3Perplexity4.5B44.9tied with KnowLine 4B Gen5.555.9%2,163 questions11.7d06fb3f
45StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B44.8tied with Perplexity Decider Flash v1.355.8%2,163 questions6.7416088f
46d1Liquid AINot disclosed44.5tied with StartLux-Decision-v1.1-4B-Instant55.6%2,163 questions3.3no repo
47Valen-Decision-1.5-9BValen-Team8.4B44.2tied with d155.3%2,163 questions4.15a51211
48Perplexity Decider Flash v1.2Perplexity4.5B44.1tied with Valen-Decision-1.5-9B55.2%2,163 questions10.46d771ad
49Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B43.2tied with Perplexity Decider Flash v1.254.6%2,163 questions21.128cf730
50OneJev-27B full fine-tuneOmniJev27.4B42.2tied with Open-Jev-27B-v1.153.8%2,163 questions3.6a5e5f55
51Mercury DecideInceptionNot disclosed41.9tied with OneJev-27B53.5%2,163 questions15.8no repo
52Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B39.451.5%2,163 questions5.745830d2
53Clef post-trainedCloudflare27.5B38.9tied with Verdin-4B-v1.151.1%2,163 questions11.02f3de3d
54Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B38.0tied with Clef50.4%2,163 questions13.7889f23e
55Clef-Flash post-trainedCloudflare9.5B37.2tied with Chinese Decision Pointer 4B49.8%2,163 questions2.7fde727a
56Winnow-12B LoRA, mergedEldanRing11.9B33.646.9%2,163 questions8.4b2b1421
57Cygnet unchanged baseBlockbrain Labs12B30.444.3%2,163 questions5.9no repo
58Metask-Jev-4B LoRA, mergedMetask Lab4.5B23.939.2%2,163 questions3.3ea20fe8
59StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B23.3tied with Metask-Jev-4B38.7%2,163 questions2.6f1e6cd1
60d1-2BLiquid AI1.9B21.3tied with StartLux-Decision-v1.1-2B-Instant37.1%2,163 questions5.610e7671
61GLiDE 2BFastino1.9B20.9tied with d1-2B36.8%2,163 questions17.200f122c
62KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B19.1tied with GLiDE 2B35.3%2,163 questions9.453245d4
63GLiDE 0.8BFastino750M15.132.1%2,163 questions18.89eacdee
64StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M12.129.7%2,163 questions4.86496ae6
65Strands Decider 2B LoRA + headStrands Agents2.3B10.8tied with StartLux-Decision-v1.1-0.8B-Instant28.7%2,163 questions8.82b52a62
66OneJev1.6-0.8B full fine-tuneOmniJev850M10.3tied with Strands Decider 2B28.2%2,163 questions10.0bcd88a3
67KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M10.3tied with OneJev1.6-0.8B28.2%2,163 questions10.56b27831
68GLiClass v4 Edge fine-tuneKnowledgator440M3.322.7%2,163 questions7.0885af1a
69GLiDE 0.4BFastino400M0.720.6%2,163 questions12.26bda7ab

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Claude Opus 5.5 high · chatAnthropicNot disclosed99.799.8%2,163 questions–no repo
2Qwen4-Max high · chatAlibabaNot disclosed99.7tied with Claude Opus 5.5 · high · chat99.8%2,163 questions–no repo
3Claude Fable 5.1 high · chatAnthropicNot disclosed99.7tied with Qwen4-Max · high · chat99.7%2,163 questions–no repo
4Claude Sonnet 5.5 high · chatAnthropicNot disclosed99.6tied with Claude Fable 5.1 · high · chat99.7%2,163 questions–no repo
5DeepSeek-V4.1-Flash reasoning on · chat · 1017DeepSeek552B16B active99.5tied with Claude Sonnet 5.5 · high · chat99.6%2,163 questions–no repo
6GPT-6 Astra high · chatOpenAINot disclosed99.5tied with DeepSeek-V4.1-Flash · reasoning on · chat · 101799.6%2,163 questions–no repo
7GPT-6.1 Sol high · chatOpenAINot disclosed99.5tied with GPT-6 Astra · high · chat99.6%2,163 questions–no repo
8Inkling-Small high · chat · Jul 2026Thinking Machines266B12B active99.5tied with GPT-6.1 Sol · high · chat99.6%2,163 questions–no repo
9Gemini 4 Carbon high · chatGoogleNot disclosed99.5tied with Inkling-Small · high · chat · Jul 202699.6%2,163 questions–no repo
10Gemini 4 Hydrogen high · chatGoogleNot disclosed99.5tied with Gemini 4 Carbon · high · chat99.6%2,163 questions–no repo
11Muse Ember max · chatMetaNot disclosed99.5tied with Gemini 4 Hydrogen · high · chat99.6%2,163 questions–no repo
12Grok 4.7 high · chatxAINot disclosed99.4tied with Muse Ember · max · chat99.5%2,163 questions–no repo
13Inkling 2 Small high · chatThinking Machines594B17B active99.4tied with Grok 4.7 · high · chat99.5%2,163 questions–no repo
14Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed99.4tied with Inkling 2 Small · high · chat99.5%2,163 questions–no repo
15MiMo-V2.6-Flash high · chatXiaomi309B15B active99.3tied with Qwen3.8 Flash · no reasoning · chat99.4%2,163 questions–no repo
16Qwen3.5-4B reasoning on · chatAlibaba4.7B99.3tied with MiMo-V2.6-Flash · high · chat99.4%2,163 questions–851bf6e
17Gemma 4 31B reasoning off · chatGoogle31.3B99.2tied with Qwen3.5-4B · reasoning on · chat99.4%2,163 questions–842da37
18DeepSeek-V4-Flash chatDeepSeek285B13B active99.1tied with Gemma 4 31B · reasoning off · chat99.3%2,163 questions–no repo
19MiniCPM5-2B reasoning off · chatOpenBMB2.5B99.0tied with DeepSeek-V4-Flash · chat99.2%2,163 questions–42433e9
20GPT-6.1 Terra high · chatOpenAINot disclosed99.0tied with MiniCPM5-2B · reasoning off · chat99.2%2,163 questions–no repo
21Granite-4.2-3B reasoning on · chatIBM3.7B99.0tied with GPT-6.1 Terra · high · chat99.2%2,163 questions–4e9c275
22Qwen3.8-27B low · chatAlibaba27.8B99.0tied with Granite-4.2-3B · reasoning on · chat99.2%2,163 questions–1d4bf0f
23LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active98.8tied with Qwen3.8-27B · low · chat99.0%2,163 questions–1769516
24LFM3-1.7B reasoning on · chatLiquid AI1.7B98.7tied with LFM2.5-24B-A2B · default reasoning · chat98.9%2,163 questions–0dc4268
25Gemma 4 12B Unified default reasoning · chatGoogle12B98.6tied with LFM3-1.7B · reasoning on · chat98.8%2,163 questions–43c9d01
26GLM-5.3-Flash reasoning on · chatZ.ai320B18B active98.5tied with Gemma 4 12B Unified · default reasoning · chat98.8%2,163 questions–no repo
27MiniMax-M3.1-Flash high · chatMiniMax428B23B active97.8tied with GLM-5.3-Flash · reasoning on · chat98.2%2,163 questions–no repo
28Gemma 4 E4B default reasoning · chatGoogle8B97.6tied with MiniMax-M3.1-Flash · high · chat98.1%2,163 questions–4ecc2ba
29Gemma 4 E2B reasoning on · chatGoogle5.1B96.0tied with Gemma 4 E4B · default reasoning · chat96.8%2,163 questions–4adf157
30LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active95.3tied with Gemma 4 E2B · reasoning on · chat96.3%2,163 questions–dd83f5c
31Qwen3-14B reasoning off · chatAlibaba14.8B94.7tied with LFM2.5-8B-A1B · default reasoning · chat95.8%2,163 questions–40c0698
32Qwen3.5-2B reasoning off · chatAlibaba2.3B93.3tied with Qwen3-14B · reasoning off · chat94.6%2,163 questions–15852e8
33LFM2.5-2.6B default reasoning · chatLiquid AI2.7B92.5tied with Qwen3.5-2B · reasoning off · chat94.0%2,163 questions–7141b56
34GPT-6 Luna low · chatOpenAINot disclosed92.1tied with LFM2.5-2.6B · default reasoning · chat93.7%2,163 questions–no repo
35Erk-14B reasoning off · chateCloud Technology14.8B88.490.8%2,163 questions–8584b14
36Raon-1 nano reasoning off · chatKRAFTON3.5B86.9tied with Erk-14B · reasoning off · chat89.6%2,163 questions–fcf2d1d

2,163 questions per model · updated 16 November 2026 · 0 = chance: 1 of 5 options right on every question (accuracy 20%); 100 = every one of the 2,163 Math questions right (accuracy 100%) · feed: bench-scores.json

Backgammon

Decision models

RankModelSizeScore ↑0–100, chance = 0Error rate (pr, lower is better)Calibration error ↓ECE, pointsCommit
1KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B78.326.21521,928 decisions–8e56aa1
2Sieve 31B v3 full fine-tuneSthānika AI31.3B77.3tied with KnowLine-31B-Gen1.527.49141,740 decisions–8fcee85
3REAX 1.3 blendREAX57.4B77.1tied with Sieve 31B v327.77231,998 decisions–fa4bf7e
4Gemini Decision 1 20261111GoogleNot disclosed76.9tied with REAX 1.327.96861,704 decisions–no repo
5Mammoth-1 post-trainedBase Labs27.4B76.4tied with Gemini Decision 1 · 2026111128.54322,021 decisions–61cec44
6Turion Jev 31B LoRA, mergedTurion.AI31.3B75.5tied with Mammoth-129.69571,855 decisions–866c0fc
7Drex Max fine-tuneNace.AI26.1B74.9tied with Turion Jev 31B30.39942,148 decisions–cac65e1
8Drex Max v1.1Nace.AINot disclosed74.5tied with Drex Max30.93832,154 decisions–no repo
9d1-ultraLiquid AINot disclosed74.3tied with Drex Max v1.130.33742,071 decisions–no repo
10Jev 1.14TypeSafeNot disclosed73.1tied with d1-ultra31.73451,912 decisions–no repo
11GLiDE APIFastinoNot disclosed71.9tied with Jev 1.1433.18262,159 decisions–no repo
12Jev 1.13TypeSafeNot disclosed71.6tied with GLiDE · API34.40941,911 decisions–no repo
13GLiDE v1.2Fastino26.1B71.1tied with Jev 1.1335.00462,003 decisions–a7e4589
14celeris-1-decisionCelerisNot disclosed70.6tied with GLiDE · v1.235.64281,922 decisions–no repo
15Perplexity Decider 27B v1.2 DGX SparkPerplexity26.1B69.5tied with celeris-1-decision36.93342,028 decisions–225c913
16Clef post-trainedCloudflare27.5B69.0tied with Perplexity Decider 27B v1.2 · DGX Spark36.53641,978 decisions–no repo
17Drex 2.0Nace.AI35.1B3B active68.3tied with Clef38.441,937 decisions–no repo
18Cygnet unchanged baseBlockbrain Labs12B68.2tied with Drex 2.038.55032,016 decisions–no repo
19Perplexity Decider 27B v1.1Perplexity26.1B68.0tied with Cygnet38.78292,052 decisions–no repo
20StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B66.8tied with Perplexity Decider 27B v1.140.14491,989 decisions–416088f
21SubQ Snap 1.1SubquadraticNot disclosed66.8tied with StartLux-Decision-v1.1-4B-Instant40.19972,326 decisions–no repo
22GPT-6 Luna Decisions APIOpenAINot disclosed64.243.31071,962 decisions–no repo
23SubQ Snap PreviewSubquadraticNot disclosed63.8tied with GPT-6 Luna · Decisions API43.87062,200 decisions–no repo
24Elephant-1 post-trainedBase Labs9.4B63.2tied with SubQ Snap Preview44.56162,044 decisions–b3f8a93
25d1-2BLiquid AI1.9B61.1tied with Elephant-147.10251,980 decisions–10e7671
26Mercury DecideInceptionNot disclosed60.4tied with d1-2B47.9481,853 decisions–no repo
27Cerno 9B v1.2 full fine-tuneApertix AI8.9B60.0tied with Mercury Decide48.46732,108 decisions–f02a632
28Winnow-12B LoRA, mergedEldanRing11.9B58.9tied with Cerno 9B v1.249.72772,015 decisions–no repo
29d1-Omni-2B NVIDIA RTX PRO 6000 BlackwellLiquid AI2.2B58.9tied with Winnow-12B49.79611,710 decisions–81222da
30StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B58.7tied with d1-Omni-2B · NVIDIA RTX PRO 6000 Blackwell50.04171,771 decisions–f1e6cd1
31KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B55.853.58962,153 decisions–779944d
32KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B55.0tied with KnowLine 4B Gen5.554.54791,955 decisions–53245d4
33Perplexity Decider Flash v1.2Perplexity4.5B50.859.64222,006 decisions–no repo
34StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M46.664.71331,882 decisions–6496ae6
35Open-Jev-27B-v1.1 LoRA + headZefan Cai27.8B45.9tied with StartLux-Decision-v1.1-0.8B-Instant65.55712,002 decisions–no repo
36GLiDE 2BFastino1.9B44.4tied with Open-Jev-27B-v1.167.35151,676 decisions–00f122c
37KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M40.671.94121,728 decisions–6b27831
38GLiDE 0.8BFastino750M33.680.36731,832 decisions–9eacdee
39OneJev1.6-0.8B full fine-tuneOmniJev850M31.9tied with GLiDE 0.8B82.51931,763 decisions–bcd88a3
40GLiDE 0.4BFastino400M6.5113.2191,355 decisions–6bda7ab
41GLiClass v4 Edge fine-tuneKnowledgator440M−10.2133.5221,405 decisions–885af1a

Chat models

RankModelSizeScore ↑0–100, chance = 0Error rate (pr, lower is better)Calibration error ↓ECE, pointsCommit
1GPT-6 Astra high · chatOpenAINot disclosed92.58.828561,489 decisions–no repo
2GPT-6.1 Sol high · chatOpenAINot disclosed91.6tied with GPT-6 Astra · high · chat9.899021,389 decisions–no repo
3Gemini 4 Argon low · chatGoogleNot disclosed89.3tied with GPT-6.1 Sol · high · chat12.57231,332 decisions–no repo
4Claude Fable 5.1 high · chatAnthropicNot disclosed88.1tied with Gemini 4 Argon · low · chat13.9551,286 decisions–no repo
5Claude Opus 5.5 high · chatAnthropicNot disclosed87.7tied with Claude Fable 5.1 · high · chat14.44361,477 decisions–no repo
6Gemini 3.8 Flash high · chatGoogleNot disclosed87.3tied with Claude Opus 5.5 · high · chat14.94642,168 decisions–no repo
7Claude Sonnet 5.5 high · chatAnthropicNot disclosed85.8tied with Gemini 3.8 Flash · high · chat16.71942,164 decisions–no repo
8Grok 4.7 high · chatxAINot disclosed84.0tied with Claude Sonnet 5.5 · high · chat18.78421,337 decisions–no repo
9DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseDeepSeekNot disclosed83.9tied with Grok 4.7 · high · chat18.99182,188 decisions–no repo
10Gemma 4 31B reasoning off · chatGoogle31.3B83.4tied with DeepSeek-V4.1-Flash · low · API + NVIDIA RTX PRO 6000 Blackwell · first release19.57512,122 decisions–no repo
11Muse Ember max · chatMetaNot disclosed83.4tied with Gemma 4 31B · reasoning off · chat20.10641,667 decisions–no repo
12Kimi K3.1 Pro low · chatMoonshot2.8T104B active76.228.04551,167 decisions–no repo
13Qwen3.8-27B low · chatAlibaba27.8B75.4tied with Kimi K3.1 Pro · low · chat29.03262,108 decisions–no repo
14GLM-5.3 low · chatZ.ai753.3B73.8tied with Qwen3.8-27B · low · chat30.86621,989 decisions–no repo
15Qwen3.8 Flash no reasoning · chatAlibabaNot disclosed71.533.57772,027 decisions–no repo
16LFM2.5-24B-A2B default reasoning · chatLiquid AI24B2.5B active60.647.68632,193 decisions–no repo

1,167 to 2,326 decisions per model · updated 16 November 2026 · 0 = 0 = a random legal play's error rate in the same league (League 1: 121.1, League 2: 117.9, League 3: 117.6; late entries: the league whose field they played); below 0 = worse than a random legal play. score = 100 × (1 − error rate / Random's error rate in that league); error rate (PR) = 500 × mean GNUbg equity loss per decision; 100 = 100 = error rate 0: the engine's best play on every decision · feed: bench-scores.json

Answer Check · does the text give this answer?

Decision models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Karpo Decision 31BMachinePulse31.3B90.093.3%2,069 pairs2.8d69cdd5
2Copernicus-31BPoindexter Labs31.3B89.0tied with Karpo Decision 31B92.7%2,069 pairs11.0be71525
3KnowLine-31B-Gen1.5 LoRA, mergedPelaAI31.3B88.8tied with Copernicus-31B92.6%2,069 pairs5.78e56aa1
4Sieve 31B v3 full fine-tuneSthānika AI31.3B88.1tied with KnowLine-31B-Gen1.592.1%2,069 pairs7.08fcee85
5KnowLine-31B-Gen2PelaAI31.3B87.7tied with Sieve 31B v391.8%2,069 pairs7.2fd99a9e
6REAX 1.3 blendREAX57.4B87.0tied with KnowLine-31B-Gen291.3%2,069 pairs4.8fa4bf7e
7Turion Jev 31B LoRA, mergedTurion.AI31.3B86.4tied with REAX 1.391.0%2,069 pairs6.7866c0fc
8d1.2-ultraLiquid AINot disclosed86.4tied with Turion Jev 31B91.0%2,069 pairs8.8no repo
9Perplexity Decider 27B v1.3Perplexity26.1B86.2tied with d1.2-ultra90.8%2,069 pairs2.89c2ebb2
10GLiDE 27BFastino26.1B86.1tied with Perplexity Decider 27B v1.390.7%2,069 pairs3.64d32743
11Logits Sigma-1 Duo blendLogits57.4B85.1tied with GLiDE 27B90.1%2,069 pairs7.0a0c1b6e
12GLiDE APIFastinoNot disclosed85.1tied with Logits Sigma-1 Duo90.0%2,069 pairs3.0no repo
13GLiDE v1.2Fastino26.1B84.9tied with GLiDE · API89.9%2,069 pairs4.7a7e4589
14SubQ Snap 1.1SubquadraticNot disclosed84.9tied with GLiDE · v1.289.9%2,069 pairs2.3no repo
15d1-ultraLiquid AINot disclosed84.7tied with SubQ Snap 1.189.8%2,069 pairs11.4no repo
16Microsoft-Decision-1MicrosoftNot disclosed83.9tied with d1-ultra89.3%2,069 pairs2.4no repo
17vera-medium-omniscar-ai4.2B83.7tied with Microsoft-Decision-189.1%2,069 pairs12.41f23a6c
18Jev 1.14TypeSafeNot disclosed83.6tied with vera-medium-omni89.1%2,069 pairs2.8no repo
19Jeffan-9B-1211tattrongvu8.9B83.0tied with Jev 1.1488.6%2,069 pairs3.3862d1a7
20Jev 1.13TypeSafeNot disclosed83.0tied with Jeffan-9B-121188.6%2,069 pairs1.9no repo
21d1.1Liquid AINot disclosed82.5tied with Jev 1.1388.4%2,069 pairs8.7no repo
22GLiDE 9BFastino7.9B82.2tied with d1.188.2%2,069 pairs7.2ab54df5
23Cerno 9B v1.2 full fine-tuneApertix AI8.9B81.9tied with GLiDE 9B87.9%2,069 pairs2.9f02a632
24StartLux-Decision-v1.1-4B-Instant fine-tuneStartLux4.7B81.7tied with Cerno 9B v1.287.8%2,069 pairs8.3416088f
25Dexy-Decision-9B-V2 fine-tunePatrick-188.4B81.1tied with StartLux-Decision-v1.1-4B-Instant87.4%2,069 pairs2.54f434e2
26KnowLine-12B-Gen1PelaAI12B80.9tied with Dexy-Decision-9B-V287.2%2,069 pairs6.8f26fbc2
27KnowLine 4B Gen5.5 LoRA, mergedPelaAI4.7B80.6tied with KnowLine-12B-Gen187.1%2,069 pairs4.5779944d
28Adjudex 9B v5newsbang8.4B80.4tied with KnowLine 4B Gen5.587.0%2,069 pairs4.9ffdcb82
29d1Liquid AINot disclosed79.2tied with Adjudex 9B v586.1%2,069 pairs3.1no repo
30Perplexity Decider Flash v1.3Perplexity4.5B78.8tied with d185.9%2,069 pairs4.2d06fb3f
31StartLux-Decision-v1.1-9B-InstantStartLux9.7B78.8tied with Perplexity Decider Flash v1.385.9%2,069 pairs8.218a17f7
32Chinese Decision Pointer 4B LoRA, mergedquincyqiang4.2B78.1tied with StartLux-Decision-v1.1-9B-Instant85.4%2,069 pairs6.7889f23e
33Perplexity Decider Flash v1.2Perplexity4.5B77.2tied with Chinese Decision Pointer 4B84.8%2,069 pairs2.86d771ad
34Gemini Decision 1 20261111GoogleNot disclosed77.0tied with Perplexity Decider Flash v1.284.7%2,069 pairs3.9no repo
35Elephant-1 post-trainedBase Labs9.4B76.9tied with Gemini Decision 1 · 2026111184.6%2,069 pairs3.7b3f8a93
36Winnow-12B LoRA, mergedEldanRing11.9B76.6tied with Elephant-184.4%2,069 pairs8.3b2b1421
37KnowLine-9B-Gen1PelaAI9.7B76.1tied with Winnow-12B84.1%2,069 pairs3.6325378d
38Verdin-4B-v1.1 LoRA, mergedvictorbennett4.2B75.4tied with KnowLine-9B-Gen183.6%2,069 pairs3.445830d2
39KnowLine-9B-Gen2PelaAI9.7B75.3tied with Verdin-4B-v1.183.5%2,069 pairs4.58ec8eb6
40Valen-Decision-1.5-9BValen-Team8.4B74.3tied with KnowLine-9B-Gen282.8%2,069 pairs3.25a51211
41Cygnet unchanged baseBlockbrain Labs12B72.9tied with Valen-Decision-1.5-9B81.9%2,069 pairs6.5no repo
42Clef-Flash post-trainedCloudflare9.5B69.879.8%2,069 pairs5.9fde727a
43StartLux-Decision-v1.1-2B-Instant fine-tuneStartLux2.3B66.077.3%2,069 pairs7.2f1e6cd1
44GLiDE 2BFastino1.9B65.6tied with StartLux-Decision-v1.1-2B-Instant77.1%2,069 pairs4.000f122c
45Metask-Jev-4B LoRA, mergedMetask Lab4.5B61.774.5%2,069 pairs6.8ea20fe8
46KnowLine-2B-Gen1 LoRA, mergedPelaAI2.3B61.1tied with Metask-Jev-4B74.0%2,069 pairs2.753245d4
47GLiDE 0.8BFastino750M60.8tied with KnowLine-2B-Gen173.9%2,069 pairs3.89eacdee
48Strands Decider 2B LoRA + headStrands Agents2.3B60.3tied with GLiDE 0.8B73.6%2,069 pairs5.02b52a62
49OneJev1.6-0.8B full fine-tuneOmniJev850M50.066.7%2,069 pairs4.1bcd88a3
50d1-2BLiquid AI1.9B49.6tied with OneJev1.6-0.8B66.4%2,069 pairs7.210e7671
51StartLux-Decision-v1.1-0.8B-Instant fine-tuneStartLux870M48.9tied with d1-2B65.9%2,069 pairs5.66496ae6
52GLiDE 0.4BFastino400M44.462.9%2,069 pairs7.76bda7ab
53GLiClass v4 Edge fine-tuneKnowledgator440M44.3tied with GLiDE 0.4B62.9%2,069 pairs9.8885af1a
54Decider 1meraGPTNot disclosed43.3tied with GLiClass v4 Edge62.2%2,069 pairs5.9no repo
55KnowLine-0.8B-Gen1 LoRA, mergedPelaAI870M38.559.0%2,069 pairs14.66b27831

Chat models

RankModelSizeScore ↑0–100, chance = 0AccuracyCalibration error ↓ECE, pointsCommit
1Inkling 2 Small high · chatThinking Machines594B17B active86.290.8%2,069 pairs–no repo
2Gemini 4 Hydrogen high · chatGoogleNot disclosed83.789.1%2,069 pairs–no repo
3MiniCPM5-2B reasoning off · chatOpenBMB2.5B79.286.2%2,069 pairs–42433e9
4Gemma 4 E4B default reasoning · chatGoogle8B77.2tied with MiniCPM5-2B · reasoning off · chat84.8%2,069 pairs–4ecc2ba
5Gemma 4 12B Unified default reasoning · chatGoogle12B77.0tied with Gemma 4 E4B · default reasoning · chat84.7%2,069 pairs–43c9d01
6Granite-4.2-3B reasoning on · chatIBM3.7B75.9tied with Gemma 4 12B Unified · default reasoning · chat83.9%2,069 pairs–4e9c275
7LFM3-1.7B reasoning on · chatLiquid AI1.7B75.8tied with Granite-4.2-3B · reasoning on · chat83.9%2,069 pairs–0dc4268
8Gemma 4 E4B thinking on · sampled (temperature 1.0) · chatGoogle8B75.1tied with LFM3-1.7B · reasoning on · chat83.4%2,069 pairs–4ecc2ba
9LFM3-1.7B reasoning off · chatLiquid AI1.7B74.9tied with Gemma 4 E4B · thinking on · sampled (temperature 1.0) · chat83.3%2,069 pairs–0dc4268
10Gemma 4 E2B reasoning on · chatGoogle5.1B71.180.8%2,069 pairs–4adf157
11Qwen3.5-2B reasoning off · chatAlibaba2.3B70.3tied with Gemma 4 E2B · reasoning on · chat80.2%2,069 pairs–15852e8
12Gemma 4 12B Unified thinking on · chatGoogle11.9B67.778.4%2,069 pairs–43c9d01
13MiniCPM5-2B reasoning off · chatOpenBMB2.5B67.0tied with Gemma 4 12B Unified · thinking on · chat78.0%2,069 pairs–42433e9
14Qwen3.5-4B reasoning on · chatAlibaba4.7B66.2tied with MiniCPM5-2B · reasoning off · chat77.5%2,069 pairs–851bf6e
15Qwen3.5-4B reasoning off · chatAlibaba4.7B64.6tied with Qwen3.5-4B · reasoning on · chat76.4%2,069 pairs–851bf6e
16LFM2.5-8B-A1B default reasoning · chatLiquid AI8.5B1.7B active57.771.8%2,069 pairs–dd83f5c
17Gemma 4 E2B default reasoning · chatGoogle5.1B55.270.1%2,069 pairs–4adf157
18Gemma 4 E2B thinking off · chatGoogle5.1B54.8tied with Gemma 4 E2B · default reasoning · chat69.8%2,069 pairs–4adf157
19Qwen3.5-2B reasoning off · chatAlibaba2.3B50.266.8%2,069 pairs–15852e8
20Raon-1 nano reasoning off · chatKRAFTON3.5B40.660.4%2,069 pairs–fcf2d1d

2,069 pairs per model · updated 16 November 2026 · 0 = uniform chance: one of the three labels at random on every pair (accuracy 1/3); 100 = every one of the 2,069 pairs labelled right (accuracy 100%) · feed: bench-scores.json

Learning from experience

ModelPoints gained ↑
Qwen3.8-27B low · chatAlibabaBackgammon 19.0
GLM-5.3 low · chatZ.aiBackgammon 17.6
Claude Fable 5.1 high · chatAnthropicBackgammon 9.4
Gemini 4 Argon low · chatGoogleBackgammon 8.5
Gemma 4 31B reasoning off · chatGoogleBackgammon 8.5
DeepSeek-V4.1-Flash low · API + NVIDIA RTX PRO 6000 Blackwell · first releaseDeepSeekBackgammon 8.2
Gemini 3.8 Flash high · chatGoogleBackgammon 8.2
Claude Sonnet 5.5 high · chatAnthropicBackgammon 7.0
Claude Opus 5.5 high · chatAnthropicBackgammon 6.7
Kimi K3.1 Pro low · chatMoonshotBackgammon 4.8
Qwen3.8 Flash no reasoning · chatAlibabaBackgammon 2.7