Logic and Math · Accuracy

Accuracy on Logic and Math

Logic and Math pooled: 3,288 questions; only setups that finished both. How to read this page · Method

Accuracy

Share of the questions answered right, with the Wilson 95% interval and the tie-group letters · higher is better; the line is chance (20%)

  1. Karpo Decision 31B

    Karpo Decision 31B · MachinePulse · decision model

    Parameters: 31B dense

    Accuracy 74.4% (95%: 72.9%–75.9%)

    Logic 60.7% · Math 81.6%

    Its run record (Method) →

    74.4% (73%–76%) a
  2. KnowLine-31B-Gen2

    KnowLine-31B-Gen2 · PelaAI · decision model

    Parameters: 31B dense

    Base model: Gemma 4 31B

    Accuracy 73.0% (95%: 71.4%–74.5%)

    Logic 58.9% · Math 80.3%

    Its run record (Method) →

    73.0% (71%–74%) ab
  3. REAX 1.3

    REAX 1.3 · REAX · decision model

    Parameters: 57.36B dense

    Accuracy 71.6% (95%: 70.0%–73.1%)

    Logic 59.3% · Math 78.0%

    Its run record (Method) →

    71.6% (70%–73%) b
  4. KnowLine-31B-Gen1.5

    KnowLine-31B-Gen1.5 · PelaAI · decision model

    Parameters: 31B dense

    Accuracy 71.1% (95%: 69.5%–72.6%)

    Logic 59.0% · Math 77.4%

    Its run record (Method) →

    71.1% (70%–73%) bc
  5. Copernicus-31B

    Copernicus-31B · Poindexter Labs · decision model

    Parameters: 31B dense

    Accuracy 71.0% (95%: 69.4%–72.5%)

    Logic 58.7% · Math 77.3%

    Its run record (Method) →

    71.0% (69%–72%) bc
  6. Sieve 31B v3

    Sieve 31B v3 · Sthānika AI · decision model

    Parameters: 31B dense

    Accuracy 70.5% (95%: 68.9%–72.1%)

    Logic 57.9% · Math 77.1%

    Its run record (Method) →

    70.5% (69%–72%) bcd
  7. Logits Sigma-1 Duo

    Logits Sigma-1 Duo · Logits · decision model

    Parameters: 57.36B dense

    Accuracy 69.3% (95%: 67.7%–70.8%)

    Logic 57.8% · Math 75.2%

    Its run record (Method) →

    69.3% (68%–71%) cde
  8. d1-ultra

    d1-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

    Accuracy 67.5% (95%: 65.9%–69.1%)

    Logic 54.9% · Math 74.1%

    Its run record (Method) →

    67.5% (66%–69%) defg
  9. GLiDE 27B

    GLiDE 27B · Fastino · decision model

    Parameters: 27B dense

    Accuracy 67.3% (95%: 65.7%–68.9%)

    Logic 58.4% · Math 72.0%

    Its run record (Method) →

    67.3% (66%–69%) efh
  10. Mammoth-1

    Mammoth-1 · Base Labs · decision model

    Parameters: 27B dense

    Accuracy 67.3% (95%: 65.7%–68.9%)

    Logic 54.5% · Math 74.0%

    Its run record (Method) →

    67.3% (66%–69%) efghi
  11. Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell

    Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 66.4% (95%: 64.8%–68.0%)

    Logic 54.8% · Math 72.5%

    Its run record (Method) →

    66.4% (65%–68%) fghij
  12. d3-ultra

    d3-ultra · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Accuracy 65.7% (95%: 64.1%–67.3%)

    Logic 54.6% · Math 71.5%

    Its run record (Method) →

    65.7% (64%–67%) fghijk
  13. GLiDE API

    GLiDE API · Fastino · decision model

    Parameters: not disclosed by maker

    Accuracy 65.2% (95%: 63.6%–66.8%)

    Logic 54.5% · Math 70.8%

    Its run record (Method) →

    65.2% (64%–67%) fghijkl
  14. d3

    d3 · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Accuracy 65.0% (95%: 63.3%–66.6%)

    Logic 54.5% · Math 70.4%

    Its run record (Method) →

    65.0% (63%–67%) fghijkl
  15. Drex Max v1.1

    Drex Max v1.1 · Nace.AI · decision model

    Parameters: not disclosed by maker

    Accuracy 64.8% (95%: 63.2%–66.5%)

    Logic 54.8% · Math 70.1%

    Its run record (Method) →

    64.8% (63%–66%) fghijkl
  16. GLiDE v1.2

    GLiDE v1.2 · Fastino · decision model

    Parameters: 27B dense

    Accuracy 64.8% (95%: 63.2%–66.5%)

    Logic 55.2% · Math 69.9%

    Its run record (Method) →

    64.8% (63%–66%) gijkl
  17. Drex Max

    Drex Max · Nace.AI · decision model

    Parameters: 27B dense

    Accuracy 64.4% (95%: 62.8%–66.0%)

    Logic 53.1% · Math 70.3%

    Its run record (Method) →

    64.4% (63%–66%) gijkl
  18. Perplexity Decider 27B v1.2 API

    Perplexity Decider 27B v1.2 API · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 64.1% (95%: 62.5%–65.8%)

    Logic 53.3% · Math 69.8%

    Its run record (Method) →

    64.1% (62%–66%) jklm
  19. Gemini Decision 1 20261111

    Gemini Decision 1 20261111 · Google · decision model

    Parameters: not disclosed by maker (closed weights)

    Accuracy 64.1% (95%: 62.4%–65.7%)

    Logic 50.0% · Math 71.3%

    Its run record (Method) →

    64.1% (62%–66%) hijklm
  20. d1.2-ultra

    d1.2-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 63.6% (95%: 61.9%–65.2%)

    Logic 51.6% · Math 69.9%

    Its run record (Method) →

    63.6% (62%–65%) jklm
  21. Perplexity Decider 27B v1.2 API · with web

    Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 63.6% (95%: 61.9%–65.2%)

    Logic 52.5% · Math 69.3%

    Its run record (Method) →

    63.6% (62%–65%) klm
  22. Turion Jev 31B

    Turion Jev 31B · Turion.AI · decision model

    Parameters: 31B dense

    Accuracy 63.3% (95%: 61.6%–64.9%)

    Logic 52.3% · Math 69.0%

    Its run record (Method) →

    63.3% (62%–65%) jklmn
  23. SubQ Snap 1.1

    SubQ Snap 1.1 · Subquadratic · decision model

    Parameters: not disclosed by maker

    Shown once with SubQ Snap Preview: the same answer to every question

    Accuracy 63.0% (95%: 61.4%–64.7%)

    Logic 52.4% · Math 68.6%

    Its run record (Method) →

    63.0% (61%–65%) klmn
  24. celeris-1-decision

    celeris-1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Accuracy 62.2% (95%: 60.6%–63.9%)

    Logic 47.3% · Math 70.0%

    Its run record (Method) →

    62.2% (61%–64%) lmno
  25. celeris-1.1-decision

    celeris-1.1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Accuracy 62.1% (95%: 60.5%–63.8%)

    Logic 47.3% · Math 69.9%

    Its run record (Method) →

    62.1% (60%–64%) lmnop
  26. KnowLine-12B-Gen1

    KnowLine-12B-Gen1 · PelaAI · decision model

    Parameters: 12B dense

    Base model: Gemma 4 12B

    Accuracy 60.9% (95%: 59.2%–62.5%)

    Logic 50.6% · Math 66.3%

    Its run record (Method) →

    60.9% (59%–63%) mnopq
  27. d1.2

    d1.2 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 59.9% (95%: 58.2%–61.6%)

    Logic 49.2% · Math 65.5%

    Its run record (Method) →

    59.9% (58%–62%) nopqr
  28. Jev 1.14

    Jev 1.14 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Accuracy 59.3% (95%: 57.6%–61.0%)

    Logic 48.2% · Math 65.1%

    Its run record (Method) →

    59.3% (58%–61%) opqrs
  29. d1.1

    d1.1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 58.6% (95%: 56.9%–60.3%)

    Logic 46.0% · Math 65.2%

    Its run record (Method) →

    58.6% (57%–60%) pqrs
  30. Jev 1.13

    Jev 1.13 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Accuracy 58.0% (95%: 56.3%–59.7%)

    Logic 46.3% · Math 64.1%

    Its run record (Method) →

    58.0% (56%–60%) qrst
  31. Adjudex 9B v5

    Adjudex 9B v5 · newsbang · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 56.7% (95%: 55.0%–58.4%)

    Logic 46.4% · Math 62.1%

    Its run record (Method) →

    56.7% (55%–58%) rstu
  32. Elephant-1

    Elephant-1 · Base Labs · decision model

    Parameters: 9B dense

    Accuracy 56.2% (95%: 54.5%–57.9%)

    Logic 46.4% · Math 61.3%

    Its run record (Method) →

    56.2% (55%–58%) stu
  33. Jeffan-9B-1211

    Jeffan-9B-1211 · tattrongvu · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 55.8% (95%: 54.1%–57.5%)

    Logic 48.2% · Math 59.8%

    Its run record (Method) →

    55.8% (54%–58%) stu
  34. KnowLine-9B-Gen2

    KnowLine-9B-Gen2 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 54.6% (95%: 52.9%–56.3%)

    Logic 43.9% · Math 60.1%

    Its run record (Method) →

    54.6% (53%–56%) tuv
  35. GLiDE 9B

    GLiDE 9B · Fastino · decision model

    Parameters: 7.94B dense

    Accuracy 54.5% (95%: 52.8%–56.2%)

    Logic 47.8% · Math 57.9%

    Its run record (Method) →

    54.5% (53%–56%) uv
  36. KnowLine-9B-Gen1

    KnowLine-9B-Gen1 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 54.5% (95%: 52.8%–56.2%)

    Logic 44.3% · Math 59.8%

    Its run record (Method) →

    54.5% (53%–56%) uv
  37. StartLux-Decision-v1.1-9B-Instant

    StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

    Parameters: 9B dense

    Accuracy 54.4% (95%: 52.7%–56.1%)

    Logic 46.6% · Math 58.4%

    Its run record (Method) →

    54.4% (53%–56%) uv
  38. Microsoft-Decision-1

    Microsoft-Decision-1 · Microsoft · decision model

    Parameters: not disclosed by maker

    Base model: Qwen3.5-9B

    Accuracy 54.3% (95%: 52.6%–56.0%)

    Logic 44.4% · Math 59.4%

    Its run record (Method) →

    54.3% (53%–56%) uvw
  39. Dexy-Decision-9B-V2

    Dexy-Decision-9B-V2 · Patrick-18 · decision model

    Parameters: 9B dense

    Base model: vllm-sr/d3-flash

    Accuracy 54.2% (95%: 52.5%–55.9%)

    Logic 45.3% · Math 58.8%

    Its run record (Method) →

    54.2% (52%–56%) uvw
  40. Drex 2.0

    Drex 2.0 · Nace.AI · decision model

    Parameters: 35B MoE · 3B active

    Accuracy 53.1% (95%: 51.4%–54.8%)

    Logic 41.9% · Math 59.0%

    Its run record (Method) →

    53.1% (51%–55%) uvwx
  41. vera-medium-omni

    vera-medium-omni · scar-ai · decision model

    Parameters: 4.2B dense

    Base model: Qwen3.5-4B

    Accuracy 52.5% (95%: 50.8%–54.2%)

    Logic 44.1% · Math 56.8%

    Its run record (Method) →

    52.5% (51%–54%) vwxy
  42. StartLux-Decision-v1.1-4B-Instant

    StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

    Parameters: 4B dense

    Accuracy 52.4% (95%: 50.7%–54.1%)

    Logic 45.9% · Math 55.8%

    Its run record (Method) →

    52.4% (51%–54%) vwxy
  43. Perplexity Decider Flash v1.3

    Perplexity Decider Flash v1.3 · Perplexity · decision model

    Parameters: 4B dense

    Accuracy 52.1% (95%: 50.4%–53.8%)

    Logic 44.7% · Math 55.9%

    Its run record (Method) →

    52.1% (50%–54%) vwxy
  44. Cerno 9B v1.2

    Cerno 9B v1.2 · Apertix AI · decision model

    Parameters: 9B dense

    Accuracy 51.7% (95%: 50.0%–53.4%)

    Logic 43.3% · Math 56.0%

    Its run record (Method) →

    51.7% (50%–53%) vwxy
  45. Valen-Decision-1.5-9B

    Valen-Decision-1.5-9B · Valen-Team · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 51.6% (95%: 49.9%–53.3%)

    Logic 44.4% · Math 55.3%

    Its run record (Method) →

    51.6% (50%–53%) vwxy
  46. KnowLine 4B Gen5.5

    KnowLine 4B Gen5.5 · PelaAI · decision model

    Parameters: 4B dense

    Accuracy 51.3% (95%: 49.6%–53.0%)

    Logic 42.3% · Math 56.0%

    Its run record (Method) →

    51.3% (50%–53%) vwxyz
  47. Perplexity Decider Flash v1.2

    Perplexity Decider Flash v1.2 · Perplexity · decision model

    Parameters: 4B dense

    Accuracy 51.1% (95%: 49.4%–52.8%)

    Logic 43.1% · Math 55.2%

    Its run record (Method) →

    51.1% (49%–53%) vwxyz
  48. Mercury Decide

    Mercury Decide · Inception · decision model

    Parameters: not disclosed by maker

    Accuracy 50.6% (95%: 48.9%–52.3%)

    Logic 45.1% · Math 53.5%

    Its run record (Method) →

    50.6% (49%–52%) wxyzA
  49. d1

    d1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 50.5% (95%: 48.8%–52.3%)

    Logic 40.8% · Math 55.6%

    Its run record (Method) →

    50.5% (49%–52%) wxyzA
  50. Open-Jev-27B-v1.1

    Open-Jev-27B-v1.1 · Zefan Cai · decision model

    Parameters: 27B dense

    Base model: Qwen3.8-27B + adapter

    Accuracy 50.5% (95%: 48.7%–52.2%)

    Logic 42.6% · Math 54.6%

    Its run record (Method) →

    50.5% (49%–52%) xyzA
  51. OneJev-27B

    OneJev-27B · OmniJev · decision model

    Parameters: 27B dense

    Accuracy 49.3% (95%: 47.6%–51.0%)

    Logic 40.6% · Math 53.8%

    Its run record (Method) →

    49.3% (48%–51%) yzAB
  52. Clef

    Clef · Cloudflare · decision model

    Parameters: 27B dense

    Accuracy 47.6% (95%: 45.9%–49.3%)

    Logic 40.8% · Math 51.1%

    Its run record (Method) →

    47.6% (46%–49%) zABC
  53. Verdin-4B-v1.1

    Verdin-4B-v1.1 · victorbennett · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Accuracy 47.5% (95%: 45.8%–49.2%)

    Logic 39.7% · Math 51.5%

    Its run record (Method) →

    47.5% (46%–49%) ABC
  54. Chinese Decision Pointer 4B

    Chinese Decision Pointer 4B · quincyqiang · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Accuracy 47.0% (95%: 45.3%–48.7%)

    Logic 40.4% · Math 50.4%

    Its run record (Method) →

    47.0% (45%–49%) ABC
  55. Clef-Flash

    Clef-Flash · Cloudflare · decision model

    Parameters: 9B dense

    Accuracy 46.1% (95%: 44.4%–47.8%)

    Logic 38.9% · Math 49.8%

    Its run record (Method) →

    46.1% (44%–48%) BCD
  56. Winnow-12B

    Winnow-12B · EldanRing · decision model

    Parameters: 12B dense

    Accuracy 44.4% (95%: 42.7%–46.1%)

    Logic 39.7% · Math 46.9%

    Its run record (Method) →

    44.4% (43%–46%) CD
  57. Cygnet

    Cygnet · Blockbrain Labs · decision model

    Parameters: 11.95B dense

    Base model: Gemma 4 12B IT

    Accuracy 42.6% (95%: 40.9%–44.3%)

    Logic 39.3% · Math 44.3%

    Its run record (Method) →

    42.6% (41%–44%) D
  58. Metask-Jev-4B

    Metask-Jev-4B · Metask Lab · decision model

    Parameters: 4.54B dense

    Accuracy 36.7% (95%: 35.1%–38.4%)

    Logic 32.1% · Math 39.2%

    Its run record (Method) →

    36.7% (35%–38%) E
  59. StartLux-Decision-v1.1-2B-Instant

    StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

    Parameters: 2B dense

    Accuracy 35.9% (95%: 34.3%–37.6%)

    Logic 30.8% · Math 38.7%

    Its run record (Method) →

    35.9% (34%–38%) E
  60. GLiDE 2B

    GLiDE 2B · Fastino · decision model

    Parameters: 1.88B dense

    Accuracy 35.2% (95%: 33.6%–36.9%)

    Logic 32.3% · Math 36.8%

    Its run record (Method) →

    35.2% (34%–37%) E
  61. d1-2B

    d1-2B · Liquid AI · decision model

    Parameters: 1.92B dense

    Base model: LFM3-VL-2B

    Accuracy 34.9% (95%: 33.3%–36.5%)

    Logic 30.7% · Math 37.1%

    Its run record (Method) →

    34.9% (33%–37%) EF
  62. KnowLine-2B-Gen1

    KnowLine-2B-Gen1 · PelaAI · decision model

    Parameters: 2B dense

    Base model: Qwen3.5-2B

    Accuracy 34.1% (95%: 32.5%–35.7%)

    Logic 31.7% · Math 35.3%

    Its run record (Method) →

    34.1% (32%–36%) EF
  63. GLiDE 0.8B

    GLiDE 0.8B · Fastino · decision model

    Parameters: 0.75B dense

    Accuracy 31.1% (95%: 29.5%–32.7%)

    Logic 29.2% · Math 32.1%

    Its run record (Method) →

    31.1% (30%–33%) FG
  64. StartLux-Decision-v1.1-0.8B-Instant

    StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Accuracy 28.0% (95%: 26.5%–29.5%)

    Logic 24.7% · Math 29.7%

    Its run record (Method) →

    28.0% (26%–30%) GH
  65. OneJev1.6-0.8B

    OneJev1.6-0.8B · OmniJev · decision model

    Parameters: 0.8B dense

    Accuracy 27.8% (95%: 26.3%–29.4%)

    Logic 27.0% · Math 28.2%

    Its run record (Method) →

    27.8% (26%–29%) GH
  66. KnowLine-0.8B-Gen1

    KnowLine-0.8B-Gen1 · PelaAI · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Accuracy 27.7% (95%: 26.2%–29.2%)

    Logic 26.7% · Math 28.2%

    Its run record (Method) →

    27.7% (26%–29%) GH
  67. Strands Decider 2B

    Strands Decider 2B · Strands Agents · decision model

    Parameters: 2B dense

    Accuracy 26.5% (95%: 25.0%–28.1%)

    Logic 22.4% · Math 28.7%

    Its run record (Method) →

    26.5% (25%–28%) HI
  68. GLiClass v4 Edge

    GLiClass v4 Edge · decision model

    Parameters: 0.4B dense

    Base model: DeBERTa-v3-large

    Accuracy 23.0% (95%: 21.6%–24.4%)

    Logic 23.6% · Math 22.7%

    Its run record (Method) →

    23.0% (22%–24%) IJ
  69. GLiDE 0.4B

    GLiDE 0.4B · Fastino · decision model

    Parameters: 0.40B dense

    Accuracy 21.5% (95%: 20.1%–22.9%)

    Logic 23.2% · Math 20.6%

    Its run record (Method) →

    21.5% (20%–23%) J

MachinePulse PelaAI REAX Poindexter Labs Sthānika AI Logits Liquid AI Fastino Base Labs Perplexity vLLM Semantic Router Nace.AI Google Turion.AI Subquadratic Celeris TypeSafe newsbang tattrongvu StartLux Microsoft Patrick-18 scar-ai Apertix AI Valen-Team Inception Zefan Cai OmniJev Cloudflare victorbennett quincyqiang EldanRing Blockbrain Labs Metask Lab Strands Agents Other Chance, 20% (five options)Bars run to each accuracy; the whisker is its 95% interval. Ranks show observed scores; small gaps need not be meaningful.

Accuracy by topic

SetupWebLogicall 1,125 questionsMathall 2,163 questionsLogic · Numerical reasoning524 questionsLogic · Verbal reasoning469 questionsLogic · Formal logic132 questionsMath · Basic concepts238 questionsMath · Digits and place value204 questionsMath · Number bases17 questionsMath · Division and divisibility157 questionsMath · Prime factorisation and GCD/LCM109 questionsMath · Rational numbers134 questionsMath · Decimals110 questionsMath · Ordering101 questionsMath · Simple inequalities161 questionsMath · Absolute value105 questionsMath · Exponents226 questionsMath · Radicals257 questionsMath · Factorisation344 questions
Karpo Decision 31B Open · 31B—60.7%683/112581.6%1764/216350.8%266/52464.0%300/46988.6%117/13277.7%185/23865.2%133/20464.7%11/1768.8%108/15774.3%81/10985.1%114/13492.7%102/11092.1%93/10188.2%142/16181.9%86/10585.8%194/22683.7%215/25787.2%300/344
KnowLine-31B-Gen2 Open · 31B—58.9%663/112580.3%1736/216347.1%247/52463.8%299/46988.6%117/13275.6%180/23860.3%123/20464.7%11/1770.1%110/15775.2%82/10979.1%106/13490.9%100/11093.1%94/10185.1%137/16181.9%86/10588.5%200/22682.9%213/25785.5%294/344
REAX 1.3 Open · 57B—59.3%667/112578.0%1687/216347.5%249/52464.4%302/46987.9%116/13276.9%183/23857.8%118/20464.7%11/1761.8%97/15776.1%83/10976.1%102/13492.7%102/11084.2%85/10183.9%135/16179.0%83/10585.4%193/22677.8%200/25785.8%295/344
KnowLine-31B-Gen1.5 Open · 31B—59.0%664/112577.4%1674/216347.1%247/52464.2%301/46987.9%116/13276.9%183/23858.8%120/20452.9%9/1763.7%100/15770.6%77/10976.1%102/13488.2%97/11089.1%90/10182.0%132/16179.0%83/10585.0%192/22680.2%206/25782.3%283/344
Copernicus-31B Open · 31B—58.7%660/112577.3%1673/216347.1%247/52463.8%299/46986.4%114/13276.1%181/23854.9%112/20458.8%10/1763.1%99/15771.6%78/10979.1%106/13489.1%98/11092.1%93/10181.4%131/16176.2%80/10585.4%193/22679.4%204/25783.7%288/344
Sieve 31B v3 Open · 31B—57.9%651/112577.1%1668/216345.8%240/52462.9%295/46987.9%116/13275.6%180/23855.9%114/20452.9%9/1763.7%100/15769.7%76/10981.3%109/13490.9%100/11086.1%87/10180.1%129/16180.0%84/10582.7%187/22679.4%204/25784.0%289/344
Logits Sigma-1 Duo Open · 57B—57.8%650/112575.2%1627/216344.7%234/52463.8%299/46988.6%117/13271.4%170/23851.5%105/20464.7%11/1759.9%94/15775.2%82/10975.4%101/13490.0%99/11085.1%86/10181.4%131/16175.2%79/10581.4%184/22678.6%202/25782.3%283/344
d1-ultra Closed · Efficient—54.9%618/112574.1%1603/216344.7%234/52457.4%269/46987.1%115/13271.8%171/23853.4%109/20470.6%12/1765.6%103/15759.6%65/10978.4%105/13484.5%93/11077.2%78/10172.7%117/16178.1%82/10579.6%180/22679.4%204/25782.6%284/344
GLiDE 27B Open · 27B—58.4%657/112572.0%1557/216342.0%220/52468.0%319/46989.4%118/13269.3%165/23852.0%106/20452.9%9/1761.8%97/15760.6%66/10973.1%98/13490.9%100/11081.2%82/10177.6%125/16170.5%74/10577.4%175/22673.9%190/25778.5%270/344
Mammoth-1 Open · 27B—54.5%613/112574.0%1600/216342.7%224/52458.6%275/46986.4%114/13268.1%162/23850.0%102/20435.3%6/1757.3%90/15762.4%68/10977.6%104/13488.2%97/11081.2%82/10179.5%128/16179.0%83/10584.5%191/22678.6%202/25782.8%285/344
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—54.8%616/112572.5%1568/216341.4%217/52459.9%281/46989.4%118/13269.7%166/23852.5%107/20476.5%13/1755.4%87/15760.6%66/10977.6%104/13486.4%95/11083.2%84/10177.6%125/16173.3%77/10576.5%173/22676.7%197/25779.7%274/344
d3-ultra Open · 26B—54.6%614/112571.5%1547/216342.2%221/52459.3%278/46987.1%115/13267.2%160/23850.0%102/20452.9%9/1756.1%88/15773.4%80/10969.4%93/13488.2%97/11085.1%86/10177.6%125/16175.2%79/10575.2%170/22672.0%185/25779.4%273/344
GLiDE API Closed · Efficient—54.5%613/112570.8%1531/216338.7%203/52463.1%296/46986.4%114/13267.2%160/23854.4%111/20447.1%8/1756.7%89/15759.6%65/10971.6%96/13488.2%97/11080.2%81/10173.9%119/16175.2%79/10574.3%168/22675.1%193/25777.0%265/344
d3 Open · 26B—54.5%613/112570.4%1523/216340.6%213/52460.6%284/46987.9%116/13265.1%155/23852.5%107/20447.1%8/1758.6%92/15756.0%61/10967.9%91/13486.4%95/11080.2%81/10177.6%125/16170.5%74/10576.5%173/22673.2%188/25779.4%273/344
Drex Max v1.1 Closed · Efficient (provisional)—54.8%616/112570.1%1516/216341.4%217/52460.8%285/46986.4%114/13269.7%166/23852.9%108/20458.8%10/1754.8%86/15756.0%61/10967.9%91/13488.2%97/11080.2%81/10175.8%122/16172.4%76/10576.1%172/22670.4%181/25777.0%265/344
GLiDE v1.2 Open · 27B—55.2%621/112569.9%1511/216340.5%212/52463.1%296/46985.6%113/13267.2%160/23852.9%108/20452.9%9/1756.7%89/15760.6%66/10973.9%99/13488.2%97/11078.2%79/10172.0%116/16171.4%75/10575.2%170/22670.4%181/25776.2%262/344
Drex Max Open · 27B—53.1%597/112570.3%1521/216342.4%222/52455.7%261/46986.4%114/13268.1%162/23852.5%107/20452.9%9/1756.1%88/15758.7%64/10967.9%91/13489.1%98/11079.2%80/10177.0%124/16169.5%73/10575.7%171/22671.6%184/25778.5%270/344
Perplexity Decider 27B v1.2 API Open · 27B—53.3%600/112569.8%1509/216339.9%209/52458.4%274/46988.6%117/13266.0%157/23850.5%103/20452.9%9/1755.4%87/15760.6%66/10972.4%97/13483.6%92/11076.2%77/10174.5%120/16170.5%74/10577.4%175/22670.8%182/25778.5%270/344
Gemini Decision 1 20261111 Closed · Efficient—50.0%563/112571.3%1543/216339.3%206/52453.1%249/46981.8%108/13265.1%155/23848.0%98/20458.8%10/1761.8%97/15765.1%71/10975.4%101/13481.8%90/11079.2%80/10170.8%114/16176.2%80/10577.0%174/22676.7%197/25780.2%276/344
d1.2-ultra Closed · Undisclosed—51.6%580/112569.9%1511/216338.2%200/52457.4%269/46984.1%111/13267.2%160/23846.1%94/20447.1%8/1754.8%86/15759.6%65/10975.4%101/13484.5%93/11082.2%83/10175.8%122/16169.5%73/10575.2%170/22673.2%188/25777.9%268/344
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web52.5%591/112569.3%1499/216339.1%205/52458.0%272/46986.4%114/13268.1%162/23848.5%99/20458.8%10/1756.7%89/15761.5%67/10967.9%91/13482.7%91/11080.2%81/10175.2%121/16166.7%70/10577.9%176/22670.4%181/25775.9%261/344
Turion Jev 31B Open · 31B—52.3%588/112569.0%1493/216336.6%192/52460.1%282/46986.4%114/13270.2%167/23841.2%84/20441.2%7/1756.1%88/15756.9%62/10976.1%102/13481.8%90/11084.2%85/10176.4%123/16171.4%75/10573.0%165/22673.2%188/25774.7%257/344
SubQ Snap 1.1 Closed · Efficient—52.4%589/112568.6%1483/216337.2%195/52460.3%283/46984.1%111/13265.1%155/23851.5%105/20452.9%9/1752.9%83/15767.0%73/10970.1%94/13482.7%91/11076.2%77/10174.5%120/16172.4%76/10574.8%169/22668.1%175/25774.4%256/344
celeris-1-decision Closed · Efficient—47.3%532/112570.0%1514/216337.2%195/52450.5%237/46975.8%100/13272.7%173/23853.4%109/20458.8%10/1757.3%90/15759.6%65/10970.1%94/13492.7%102/11075.2%76/10168.9%111/16169.5%73/10572.6%164/22673.5%189/25775.0%258/344
celeris-1.1-decision Closed · Efficient—47.3%532/112569.9%1511/216337.8%198/52449.5%232/46977.3%102/13271.4%170/23853.4%109/20452.9%9/1758.6%92/15759.6%65/10970.1%94/13492.7%102/11074.3%75/10168.9%111/16170.5%74/10572.1%163/22673.5%189/25775.0%258/344
KnowLine-12B-Gen1 Open · 12B—50.6%569/112566.3%1433/216336.6%192/52456.7%266/46984.1%111/13266.8%159/23855.9%114/20441.2%7/1756.1%88/15754.1%59/10965.7%88/13477.3%85/11077.2%78/10167.7%109/16169.5%73/10567.3%152/22666.1%170/25773.0%251/344
d1.2 Closed · Undisclosed—49.2%553/112565.5%1417/216339.7%208/52451.6%242/46978.0%103/13262.6%149/23840.2%82/20441.2%7/1752.9%83/15760.6%66/10965.7%88/13475.5%83/11078.2%79/10166.5%107/16167.6%71/10572.1%163/22673.2%188/25773.0%251/344
Jev 1.14 Closed · Efficient—48.2%542/112565.1%1409/216335.7%187/52453.1%249/46980.3%106/13259.7%142/23842.6%87/20458.8%10/1752.2%82/15758.7%64/10964.9%87/13478.2%86/11076.2%77/10166.5%107/16174.3%78/10570.4%159/22671.2%183/25771.8%247/344
d1.1 Closed · Efficient—46.0%518/112565.2%1410/216334.7%182/52450.7%238/46974.2%98/13263.0%150/23844.1%90/20441.2%7/1750.3%79/15756.0%61/10963.4%85/13472.7%80/11073.3%74/10164.6%104/16166.7%70/10572.1%163/22675.9%195/25773.3%252/344
Jev 1.13 Closed · Efficient—46.3%521/112564.1%1386/216333.2%174/52452.0%244/46978.0%103/13256.7%135/23847.5%97/20441.2%7/1751.0%80/15755.0%60/10963.4%85/13478.2%86/11076.2%77/10164.0%103/16174.3%78/10569.9%158/22670.0%180/25769.8%240/344
Adjudex 9B v5 Open · 9.0B—46.4%522/112562.1%1343/216334.0%178/52449.9%234/46983.3%110/13261.3%146/23843.1%88/20441.2%7/1751.0%80/15752.3%57/10963.4%85/13476.4%84/11064.4%65/10164.0%103/16164.8%68/10564.2%145/22668.9%177/25769.2%238/344
Elephant-1 Open · 9.0B—46.4%522/112561.3%1327/216333.2%174/52451.0%239/46982.6%109/13259.2%141/23840.2%82/20441.2%7/1752.2%82/15756.0%61/10963.4%85/13472.7%80/11072.3%73/10160.9%98/16165.7%69/10563.7%144/22664.2%165/25769.8%240/344
Jeffan-9B-1211 Open · 9.0B—48.2%542/112559.8%1294/216334.2%179/52453.7%252/46984.1%111/13258.4%139/23838.2%78/20423.5%4/1738.9%61/15753.2%58/10956.7%76/13473.6%81/11071.3%72/10164.0%103/16163.8%67/10564.6%146/22663.4%163/25771.5%246/344
KnowLine-9B-Gen2 Open · 9.0B—43.9%494/112560.1%1300/216333.4%175/52445.2%212/46981.1%107/13257.6%137/23841.2%84/20435.3%6/1749.0%77/15750.5%55/10960.4%81/13477.3%85/11064.4%65/10162.7%101/16166.7%70/10566.4%150/22667.3%173/25762.8%216/344
GLiDE 9B Open · 7.9B—47.8%538/112557.9%1253/216332.1%168/52455.7%261/46982.6%109/13251.3%122/23838.2%78/20423.5%4/1743.3%68/15748.6%53/10955.2%74/13473.6%81/11063.4%64/10162.7%101/16167.6%71/10562.8%142/22664.2%165/25766.9%230/344
KnowLine-9B-Gen1 Open · 9.0B—44.3%498/112559.8%1293/216332.4%170/52447.5%223/46979.5%105/13259.7%142/23842.2%86/20417.6%3/1751.0%80/15753.2%58/10961.2%82/13472.7%80/11063.4%64/10166.5%107/16161.0%64/10561.5%139/22666.1%170/25763.4%218/344
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—46.6%524/112558.4%1264/216332.3%169/52452.5%246/46982.6%109/13259.2%141/23838.2%78/20435.3%6/1746.5%73/15749.5%54/10954.5%73/13470.9%78/11065.3%66/10164.6%104/16162.9%66/10560.6%137/22662.3%160/25766.3%228/344
Microsoft-Decision-1 Closed · Efficient—44.4%500/112559.4%1284/216334.0%178/52446.9%220/46977.3%102/13261.3%146/23840.2%82/20423.5%4/1748.4%76/15752.3%57/10953.7%72/13470.0%77/11063.4%64/10160.9%98/16161.9%65/10566.4%150/22663.8%164/25766.6%229/344
Dexy-Decision-9B-V2 Open · 9.0B—45.3%510/112558.8%1272/216334.9%183/52447.3%222/46979.5%105/13256.3%134/23839.7%81/20435.3%6/1742.7%67/15756.0%61/10953.7%72/13472.7%80/11070.3%71/10160.2%97/16161.0%64/10564.2%145/22660.7%156/25769.2%238/344
Drex 2.0 Open · 35B MoE · 3.0B active—41.9%471/112559.0%1276/216334.2%179/52441.4%194/46974.2%98/13255.5%132/23840.7%83/20429.4%5/1749.7%78/15751.4%56/10957.5%77/13470.0%77/11065.3%66/10164.0%103/16161.9%65/10562.4%141/22664.6%166/25766.0%227/344
vera-medium-omni Open · 4.2B—44.1%496/112556.8%1229/216331.9%167/52448.6%228/46976.5%101/13255.9%133/23836.3%74/20435.3%6/1742.7%67/15748.6%53/10950.7%68/13470.9%78/11067.3%68/10162.7%101/16163.8%67/10558.4%132/22661.1%157/25765.4%225/344
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—45.9%516/112555.8%1208/216332.4%170/52450.7%238/46981.8%108/13254.2%129/23840.7%83/20417.6%3/1741.4%65/15749.5%54/10952.2%70/13474.5%82/11064.4%65/10159.6%96/16158.1%61/10555.8%126/22661.5%158/25762.8%216/344
Perplexity Decider Flash v1.3 Open · 4.0B—44.7%503/112555.9%1210/216330.7%161/52450.1%235/46981.1%107/13255.5%132/23836.3%74/20429.4%5/1744.6%70/15750.5%55/10953.0%71/13469.1%76/11060.4%61/10160.2%97/16159.0%62/10558.8%133/22661.9%159/25762.5%215/344
Cerno 9B v1.2 Open · 9.0B—43.3%487/112556.0%1212/216331.1%163/52448.0%225/46975.0%99/13255.0%131/23841.7%85/20435.3%6/1743.9%69/15748.6%53/10953.0%71/13467.3%74/11066.3%67/10160.2%97/16156.2%59/10558.8%133/22661.1%157/25761.0%210/344
Valen-Decision-1.5-9B Open · 9.0B—44.4%499/112555.3%1197/216334.2%179/52445.8%215/46979.5%105/13252.9%126/23834.8%71/20447.1%8/1746.5%73/15745.9%50/10953.0%71/13461.8%68/11065.3%66/10154.7%88/16157.1%60/10558.4%132/22665.0%167/25763.1%217/344
KnowLine 4B Gen5.5 Open · 4.0B—42.3%476/112556.0%1211/216330.2%158/52447.5%223/46972.0%95/13252.1%124/23837.7%77/20423.5%4/1740.8%64/15753.2%58/10960.4%81/13471.8%79/11060.4%61/10157.8%93/16161.9%65/10559.3%134/22659.9%154/25763.1%217/344
Perplexity Decider Flash v1.2 Open · 4.0B—43.1%485/112555.2%1195/216331.3%164/52446.1%216/46979.5%105/13251.7%123/23839.2%80/20429.4%5/1740.8%64/15752.3%57/10957.5%77/13470.9%78/11063.4%64/10156.5%91/16157.1%60/10559.7%135/22661.9%159/25758.7%202/344
Mercury Decide Closed · Efficient—45.1%507/112553.5%1157/216333.4%175/52451.6%242/46968.2%90/13256.7%135/23836.3%74/20441.2%7/1738.9%61/15744.0%48/10944.0%59/13467.3%74/11075.2%76/10157.1%92/16156.2%59/10550.9%115/22658.8%151/25759.9%206/344
d1 Closed · Efficient—40.8%459/112555.6%1203/216329.0%152/52444.3%208/46975.0%99/13252.1%124/23839.7%81/20435.3%6/1740.8%64/15742.2%46/10951.5%69/13459.1%65/11063.4%64/10160.9%98/16161.9%65/10558.0%131/22661.9%159/25767.2%231/344
Open-Jev-27B-v1.1 Open · 27B—42.6%479/112554.6%1180/216331.1%163/52447.1%221/46972.0%95/13248.7%116/23836.3%74/20447.1%8/1748.4%76/15738.5%42/10947.0%63/13473.6%81/11068.3%69/10162.1%100/16153.3%56/10556.2%127/22661.5%158/25761.0%210/344
OneJev-27B Open · 27B—40.6%457/112553.8%1163/216330.0%157/52445.0%211/46967.4%89/13252.9%126/23832.8%67/20447.1%8/1743.9%69/15738.5%42/10947.0%63/13467.3%74/11065.3%66/10160.2%97/16155.2%58/10554.0%122/22659.9%154/25763.1%217/344
Clef Open · 27B—40.8%459/112551.1%1105/216330.2%158/52446.1%216/46964.4%85/13254.6%130/23836.3%74/20452.9%9/1738.9%61/15740.4%44/10936.6%49/13459.1%65/11057.4%58/10157.8%93/16149.5%52/10553.1%120/22654.1%139/25761.3%211/344
Verdin-4B-v1.1 Open · 4.0B—39.7%447/112551.5%1114/216328.4%149/52442.4%199/46975.0%99/13255.9%133/23834.3%70/20417.6%3/1733.8%53/15754.1%59/10944.0%59/13464.5%71/11060.4%61/10150.3%81/16145.7%48/10555.3%125/22659.9%154/25757.3%197/344
Chinese Decision Pointer 4B Open · 4.0B—40.4%454/112550.4%1091/216329.0%152/52443.9%206/46972.7%96/13247.1%112/23834.3%70/20435.3%6/1743.3%68/15739.4%43/10949.3%66/13462.7%69/11056.4%57/10156.5%91/16145.7%48/10549.1%111/22655.6%143/25760.2%207/344
Clef-Flash Open · 9.0B—38.9%438/112549.8%1077/216330.0%157/52440.1%188/46970.5%93/13249.6%118/23833.3%68/20452.9%9/1742.0%66/15743.1%47/10942.5%57/13457.3%63/11059.4%60/10157.8%93/16153.3%56/10549.1%111/22655.3%142/25754.4%187/344
Winnow-12B Open · 12B—39.7%447/112546.9%1014/216326.5%139/52445.4%213/46972.0%95/13251.3%122/23831.9%65/20441.2%7/1735.7%56/15738.5%42/10947.0%63/13461.8%68/11067.3%68/10152.2%84/16145.7%48/10545.6%103/22646.3%119/25749.1%169/344
Cygnet Open · 12B—39.3%442/112544.3%959/216326.9%141/52447.3%222/46959.8%79/13243.7%104/23830.9%63/20441.2%7/1733.1%52/15739.4%43/10936.6%49/13450.9%56/11059.4%60/10148.4%78/16145.7%48/10549.6%112/22647.9%123/25747.7%164/344
Metask-Jev-4B Open · 4.5B—32.1%361/112539.2%847/216324.6%129/52434.5%162/46953.0%70/13240.3%96/23831.4%64/20411.8%2/1730.6%48/15732.1%35/10932.1%43/13439.1%43/11042.6%43/10143.5%70/16134.3%36/10538.1%86/22647.9%123/25745.9%158/344
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—30.8%346/112538.7%836/216325.8%135/52429.9%140/46953.8%71/13233.6%80/23825.0%51/20435.3%6/1732.5%51/15734.9%38/10932.8%44/13450.0%55/11054.5%55/10146.0%74/16146.7%49/10540.3%91/22643.6%112/25737.8%130/344
GLiDE 2B Open · 1.9B—32.3%363/112536.8%795/216326.5%139/52433.0%155/46952.3%69/13232.8%78/23827.9%57/20423.5%4/1730.6%48/15736.7%40/10932.8%44/13440.9%45/11040.6%41/10138.5%62/16142.9%45/10535.8%81/22642.4%109/25741.0%141/344
d1-2B Open · 1.9B—30.7%345/112537.1%802/216324.8%130/52431.8%149/46950.0%66/13229.4%70/23833.3%68/20435.3%6/1737.6%59/15735.8%39/10934.3%46/13446.4%51/11039.6%40/10132.9%53/16139.0%41/10541.6%94/22640.5%104/25738.1%131/344
KnowLine-2B-Gen1 Open · 2.0B—31.7%357/112535.3%763/216325.0%131/52432.2%151/46956.8%75/13231.1%74/23825.5%52/20411.8%2/1729.9%47/15726.6%29/10935.1%47/13440.0%44/11051.5%52/10137.3%60/16135.2%37/10537.2%84/22638.1%98/25739.8%137/344
GLiDE 0.8B Open · 0.8B—29.2%328/112532.1%694/216325.6%134/52429.0%136/46943.9%58/13230.3%72/23827.0%55/20429.4%5/1729.9%47/15733.9%37/10926.1%35/13426.4%29/11034.7%35/10136.0%58/16123.8%25/10529.6%67/22634.6%89/25740.7%140/344
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—24.7%278/112529.7%642/216319.5%102/52425.6%120/46942.4%56/13227.7%66/23820.6%42/20429.4%5/1723.6%37/15735.8%39/10929.1%39/13438.2%42/11042.6%43/10136.0%58/16127.6%29/10530.5%69/22628.4%73/25729.1%100/344
OneJev1.6-0.8B Open · 0.8B—27.0%304/112528.2%611/216324.0%126/52427.5%129/46937.1%49/13226.5%63/23822.5%46/20411.8%2/1725.5%40/15733.9%37/10925.4%34/13428.2%31/11028.7%29/10132.9%53/16128.6%30/10529.2%66/22624.1%62/25734.3%118/344
KnowLine-0.8B-Gen1 Open · 0.8B—26.7%300/112528.2%610/216325.2%132/52424.3%114/46940.9%54/13227.3%65/23821.6%44/20435.3%6/1721.0%33/15731.2%34/10923.9%32/13430.9%34/11029.7%30/10133.5%54/16127.6%29/10530.1%68/22633.5%86/25727.6%95/344
Strands Decider 2B Open · 2.0B—22.4%252/112528.7%620/216319.3%101/52424.1%113/46928.8%38/13229.8%71/23824.0%49/20423.5%4/1725.5%40/15730.3%33/10927.6%37/13430.9%34/11037.6%38/10131.7%51/16125.7%27/10529.6%67/22625.3%65/25730.2%104/344
GLiClass v4 Edge Open · 0.4B—23.6%265/112522.7%490/216323.9%125/52422.0%103/46928.0%37/13222.7%54/23823.0%47/20429.4%5/1721.7%34/15730.3%33/10919.4%26/13424.5%27/11023.8%24/10118.0%29/16124.8%26/10522.1%50/22619.1%49/25725.0%86/344
GLiDE 0.4B Open · 0.4B—23.2%261/112520.6%445/216320.8%109/52424.1%113/46929.5%39/13219.3%46/23817.6%36/2045.9%1/1723.6%37/15725.7%28/10918.7%25/13410.9%12/11022.8%23/10123.6%38/16119.0%20/10522.6%51/22618.3%47/25723.5%81/344

Share right of the questions each setup answered in each topic; under it right / answered.

SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 3,276 of 3,288); shown once.

Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.

Every setup, every measure

SetupWebAccuracy% of N (N ≤ 3,288) · Wilson 95%Tie groupHolm · McNemarRight / NquestionsKept at ≥99%right / wrongAUROCright vs wrongDecision Score (AA formula)0–100 · chosen-answer BrierNon-hallucinationof answers not right, % “I don't know” · higher is betterMedian latencymsp90ms, nearest rankCost$ per 1,000 qper 1M q
Karpo Decision 31B Open · 31B

Karpo Decision 31B · MachinePulse · decision model

Parameters: 31B dense

Accuracy 74.4% (95%: 72.9%–75.9%)

Logic 60.7% · Math 81.6%

Its run record (Method) →

—74.4%72.9%–75.9%a2,447 / 3,2881,006 right of 1,009 kept3 wrong · 2,279 skipped0.88281.13chosen-answer Brier 0.12155.7%answered 98%162 ms198 ms$0.118$118estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen2 Open · 31B

KnowLine-31B-Gen2 · PelaAI · decision model

Parameters: 31B dense

Base model: Gemma 4 31B

Accuracy 73.0% (95%: 71.4%–74.5%)

Logic 58.9% · Math 80.3%

Its run record (Method) →

—73.0%71.4%–74.5%ab2,399 / 3,288523 right of 524 kept1 wrong · 2,764 skipped0.88280.27chosen-answer Brier 0.12420.1%answered 100%116 ms177 ms$0.093$92.98estimate · RTX PRO 6000 Blackwell
REAX 1.3 Open · 57B

REAX 1.3 · REAX · decision model

Parameters: 57.36B dense

Accuracy 71.6% (95%: 70.0%–73.1%)

Logic 59.3% · Math 78.0%

Its run record (Method) →

—71.6%70.0%–73.1%b2,354 / 3,288229 right of 229 kept0 mistakes · 3,059 skipped0.87879.30chosen-answer Brier 0.13003.8%answered 99%200 ms260 ms$0.148$148estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen1.5 Open · 31B

KnowLine-31B-Gen1.5 · PelaAI · decision model

Parameters: 31B dense

Accuracy 71.1% (95%: 69.5%–72.6%)

Logic 59.0% · Math 77.4%

Its run record (Method) →

—71.1%69.5%–72.6%bc2,338 / 3,288553 right of 554 kept1 wrong · 2,734 skipped0.87078.92chosen-answer Brier 0.13270.7%answered 100%811 ms1,111 ms$0.052$51.85estimate · DGX Spark
Copernicus-31B Open · 31B

Copernicus-31B · Poindexter Labs · decision model

Parameters: 31B dense

Accuracy 71.0% (95%: 69.4%–72.5%)

Logic 58.7% · Math 77.3%

Its run record (Method) →

—71.0%69.4%–72.5%bc2,333 / 3,288384 right of 384 kept0 mistakes · 2,904 skipped0.87378.76chosen-answer Brier 0.13441.8%answered 99%128 ms218 ms$0.102$102estimate · RTX PRO 6000 Blackwell
Sieve 31B v3 Open · 31B

Sieve 31B v3 · Sthānika AI · decision model

Parameters: 31B dense

Accuracy 70.5% (95%: 68.9%–72.1%)

Logic 57.9% · Math 77.1%

Its run record (Method) →

—70.5%68.9%–72.1%bcd2,319 / 3,288294 right of 296 kept2 wrong · 2,992 skipped0.86578.49chosen-answer Brier 0.13543.8%answered 99%676 ms1,173 ms$0.047$47.29estimate · DGX Spark
Logits Sigma-1 Duo Open · 57B

Logits Sigma-1 Duo · Logits · decision model

Parameters: 57.36B dense

Accuracy 69.3% (95%: 67.7%–70.8%)

Logic 57.8% · Math 75.2%

Its run record (Method) →

—69.3%67.7%–70.8%cde2,277 / 3,28823 right of 23 kept0 mistakes · 3,265 skipped0.86877.60chosen-answer Brier 0.14054.4%answered 99%207 ms324 ms$0.165$165estimate · RTX PRO 6000 Blackwell
d1-ultra Closed · Efficient

d1-ultra · Liquid AI · decision model

Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

Accuracy 67.5% (95%: 65.9%–69.1%)

Logic 54.9% · Math 74.1%

Its run record (Method) →

—67.5%65.9%–69.1%defg2,221 / 3,288233 right of 233 kept0 mistakes · 3,055 skipped0.85276.50chosen-answer Brier 0.145589.1%answered 52%180 ms426 ms$0.008$8.37
GLiDE 27B Open · 27B

GLiDE 27B · Fastino · decision model

Parameters: 27B dense

Accuracy 67.3% (95%: 65.7%–68.9%)

Logic 58.4% · Math 72.0%

Its run record (Method) →

—67.3%65.7%–68.9%efh2,214 / 3,288272 right of 274 kept2 wrong · 3,014 skipped0.85276.29chosen-answer Brier 0.147531.9%answered 89%450 ms587 ms$0.029$28.55estimate · DGX Spark
Mammoth-1 Open · 27B

Mammoth-1 · Base Labs · decision model

Parameters: 27B dense

Accuracy 67.3% (95%: 65.7%–68.9%)

Logic 54.5% · Math 74.0%

Its run record (Method) →

—67.3%65.7%–68.9%efghi2,213 / 3,288518 right of 520 kept2 wrong · 2,768 skipped0.86876.68chosen-answer Brier 0.139471.4%answered 69%150 ms232 ms$0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B

Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

Parameters: 27B dense

Accuracy 66.4% (95%: 64.8%–68.0%)

Logic 54.8% · Math 72.5%

Its run record (Method) →

—66.4%64.8%–68.0%fghij2,184 / 3,288385 right of 385 kept0 mistakes · 2,903 skipped0.84475.44chosen-answer Brier 0.155413.5%answered 95%97 ms125 ms$0.071$71.10estimate · RTX PRO 6000 Blackwell
d3-ultra Open · 26B

d3-ultra · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Accuracy 65.7% (95%: 64.1%–67.3%)

Logic 54.6% · Math 71.5%

Its run record (Method) →

—65.7%64.1%–67.3%fghijk2,161 / 3,28839 right of 39 kept0 mistakes · 3,249 skipped0.83574.31chosen-answer Brier 0.17117.5%answered 97%129 ms168 ms$0.096$95.71estimate · RTX PRO 6000 Blackwell
GLiDE API Closed · Efficient

GLiDE API · Fastino · decision model

Parameters: not disclosed by maker

Accuracy 65.2% (95%: 63.6%–66.8%)

Logic 54.5% · Math 70.8%

Its run record (Method) →

—65.2%63.6%–66.8%fghijkl2,144 / 3,288213 right of 213 kept0 mistakes · 3,075 skipped0.85275.04chosen-answer Brier 0.151349.2%answered 80%266 ms284 ms$0.036$36.42
d3 Open · 26B

d3 · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Accuracy 65.0% (95%: 63.3%–66.6%)

Logic 54.5% · Math 70.4%

Its run record (Method) →

—65.0%63.3%–66.6%fghijkl2,136 / 3,288216 right of 217 kept1 wrong · 3,071 skipped0.85875.16chosen-answer Brier 0.14649.3%answered 97%57 ms86 ms$0.046$46.28estimate · RTX PRO 6000 Blackwell
Drex Max v1.1 Closed · Efficient (provisional)

Drex Max v1.1 · Nace.AI · decision model

Parameters: not disclosed by maker

Accuracy 64.8% (95%: 63.2%–66.5%)

Logic 54.8% · Math 70.1%

Its run record (Method) →

—64.8%63.2%–66.5%fghijkl2,132 / 3,288404 right of 405 kept1 wrong · 2,883 skipped0.84674.61chosen-answer Brier 0.156248.3%answered 79%327 ms518 ms$0.016$16.13
GLiDE v1.2 Open · 27B

GLiDE v1.2 · Fastino · decision model

Parameters: 27B dense

Accuracy 64.8% (95%: 63.2%–66.5%)

Logic 55.2% · Math 69.9%

Its run record (Method) →

—64.8%63.2%–66.5%gijkl2,132 / 3,288176 right of 176 kept0 mistakes · 3,112 skipped0.84374.68chosen-answer Brier 0.154842.3%answered 83%81 ms136 ms$0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell
Drex Max Open · 27B

Drex Max · Nace.AI · decision model

Parameters: 27B dense

Accuracy 64.4% (95%: 62.8%–66.0%)

Logic 53.1% · Math 70.3%

Its run record (Method) →

—64.4%62.8%–66.0%gijkl2,118 / 3,288866 right of 884 kept18 wrong · 2,404 skipped0.84173.20chosen-answer Brier 0.18025.7%answered 98%81 ms130 ms$0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.2 API Open · 27B

Perplexity Decider 27B v1.2 API · Perplexity · decision model

Parameters: 27B dense

Accuracy 64.1% (95%: 62.5%–65.8%)

Logic 53.3% · Math 69.8%

Its run record (Method) →

—64.1%62.5%–65.8%jklm2,109 / 3,288409 right of 411 kept2 wrong · 2,877 skipped0.84974.56chosen-answer Brier 0.150210.3%answered 96%230 ms439 ms$0.005$5.38
Gemini Decision 1 20261111 Closed · Efficient

Gemini Decision 1 20261111 · Google · decision model

Parameters: not disclosed by maker (closed weights)

Accuracy 64.1% (95%: 62.4%–65.7%)

Logic 50.0% · Math 71.3%

Its run record (Method) →

—64.1%62.4%–65.7%hijklm2,106 / 3,288257 right of 257 kept0 mistakes · 3,031 skipped0.83874.23chosen-answer Brier 0.15593.0%answered 99%227 ms347 ms$0.027$27.07
d1.2-ultra Closed · Undisclosed

d1.2-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 63.6% (95%: 61.9%–65.2%)

Logic 51.6% · Math 69.9%

Its run record (Method) →

—63.6%61.9%–65.2%jklm2,091 / 3,288128 right of 128 kept0 mistakes · 3,160 skipped0.85674.31chosen-answer Brier 0.149890.2%answered 48%217 ms347 ms$0.008$8.37
Perplexity Decider 27B v1.2 API · with web Open · 27B

Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

Parameters: 27B dense

Accuracy 63.6% (95%: 61.9%–65.2%)

Logic 52.5% · Math 69.3%

Its run record (Method) →

with web63.6%61.9%–65.2%klm2,090 / 3,288404 right of 405 kept1 wrong · 2,883 skipped0.85474.28chosen-answer Brier 0.150125.8%answered 90%430 ms707 ms$1.025$1,025
Turion Jev 31B Open · 31B

Turion Jev 31B · Turion.AI · decision model

Parameters: 31B dense

Accuracy 63.3% (95%: 61.6%–64.9%)

Logic 52.3% · Math 69.0%

Its run record (Method) →

—63.3%61.6%–64.9%jklmn2,081 / 3,28850 right of 50 kept0 mistakes · 3,238 skipped0.84473.88chosen-answer Brier 0.15531.9%answered 99%301 ms448 ms$0.020$20.01estimate · DGX Spark
SubQ Snap 1.1 Closed · Efficient

SubQ Snap 1.1 · Subquadratic · decision model

Parameters: not disclosed by maker

Shown once with SubQ Snap Preview: the same answer to every question

Accuracy 63.0% (95%: 61.4%–64.7%)

Logic 52.4% · Math 68.6%

Its run record (Method) →

—63.0%61.4%–64.7%klmn2,072 / 3,288430 right of 433 kept3 wrong · 2,855 skipped0.82772.66chosen-answer Brier 0.177058.3%answered 73%382 ms391 ms$0.011$10.75
celeris-1-decision Closed · Efficient

celeris-1-decision · Celeris · decision model

Parameters: not disclosed by maker

Accuracy 62.2% (95%: 60.6%–63.9%)

Logic 47.3% · Math 70.0%

Its run record (Method) →

—62.2%60.6%–63.9%lmno2,046 / 3,288No answers at this threshold0.82972.82chosen-answer Brier 0.165987.2%answered 50%388 ms392 ms$0.011$11.25
celeris-1.1-decision Closed · Efficient

celeris-1.1-decision · Celeris · decision model

Parameters: not disclosed by maker

Accuracy 62.1% (95%: 60.5%–63.8%)

Logic 47.3% · Math 69.9%

Its run record (Method) →

—62.1%60.5%–63.8%lmnop2,043 / 3,288No answers at this threshold0.83072.78chosen-answer Brier 0.165786.7%answered 51%381 ms388 ms$0.001$1.13
KnowLine-12B-Gen1 Open · 12B

KnowLine-12B-Gen1 · PelaAI · decision model

Parameters: 12B dense

Base model: Gemma 4 12B

Accuracy 60.9% (95%: 59.2%–62.5%)

Logic 50.6% · Math 66.3%

Its run record (Method) →

—60.9%59.2%–62.5%mnopq2,002 / 3,28813 right of 13 kept0 mistakes · 3,275 skipped0.81071.86chosen-answer Brier 0.17164.5%answered 98%51 ms76 ms$0.041$41.44estimate · RTX PRO 6000 Blackwell
d1.2 Closed · Undisclosed

d1.2 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 59.9% (95%: 58.2%–61.6%)

Logic 49.2% · Math 65.5%

Its run record (Method) →

—59.9%58.2%–61.6%nopqr1,970 / 3,288169 right of 169 kept0 mistakes · 3,119 skipped0.81271.40chosen-answer Brier 0.171266.9%answered 65%208 ms390 ms$0.004$3.98
Jev 1.14 Closed · Efficient

Jev 1.14 · TypeSafe · decision model

Parameters: not disclosed by maker

Accuracy 59.3% (95%: 57.6%–61.0%)

Logic 48.2% · Math 65.1%

Its run record (Method) →

—59.3%57.6%–61.0%opqrs1,951 / 3,288263 right of 263 kept0 mistakes · 3,025 skipped0.81071.00chosen-answer Brier 0.173415.4%answered 93%164 ms188 ms$0.021$21.27
d1.1 Closed · Efficient

d1.1 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 58.6% (95%: 56.9%–60.3%)

Logic 46.0% · Math 65.2%

Its run record (Method) →

—58.6%56.9%–60.3%pqrs1,928 / 3,288231 right of 231 kept0 mistakes · 3,057 skipped0.81170.69chosen-answer Brier 0.172686.0%answered 46%174 ms285 ms$0.004$3.98
Jev 1.13 Closed · Efficient

Jev 1.13 · TypeSafe · decision model

Parameters: not disclosed by maker

Accuracy 58.0% (95%: 56.3%–59.7%)

Logic 46.3% · Math 64.1%

Its run record (Method) →

—58.0%56.3%–59.7%qrst1,907 / 3,288313 right of 313 kept0 mistakes · 2,975 skipped0.81770.41chosen-answer Brier 0.171810.2%answered 96%166 ms205 ms$0.021$21.27
Adjudex 9B v5 Open · 9.0B

Adjudex 9B v5 · newsbang · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 56.7% (95%: 55.0%–58.4%)

Logic 46.4% · Math 62.1%

Its run record (Method) →

—56.7%55.0%–58.4%rstu1,865 / 3,288251 right of 253 kept2 wrong · 3,035 skipped0.80269.27chosen-answer Brier 0.18187.0%answered 97%38 ms45 ms$0.028$28.19estimate · RTX PRO 6000 Blackwell
Elephant-1 Open · 9.0B

Elephant-1 · Base Labs · decision model

Parameters: 9B dense

Accuracy 56.2% (95%: 54.5%–57.9%)

Logic 46.4% · Math 61.3%

Its run record (Method) →

—56.2%54.5%–57.9%stu1,849 / 3,288235 right of 236 kept1 wrong · 3,052 skipped0.79768.96chosen-answer Brier 0.183169.4%answered 60%80 ms94 ms$0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell
Jeffan-9B-1211 Open · 9.0B

Jeffan-9B-1211 · tattrongvu · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 55.8% (95%: 54.1%–57.5%)

Logic 48.2% · Math 59.8%

Its run record (Method) →

—55.8%54.1%–57.5%stu1,836 / 3,28874 right of 74 kept0 mistakes · 3,214 skipped0.79868.92chosen-answer Brier 0.179913.5%answered 94%39 ms53 ms$0.032$32.40estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen2 Open · 9.0B

KnowLine-9B-Gen2 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 54.6% (95%: 52.9%–56.3%)

Logic 43.9% · Math 60.1%

Its run record (Method) →

—54.6%52.9%–56.3%tuv1,794 / 3,288No answers at this threshold0.78667.99chosen-answer Brier 0.185810.4%answered 95%42 ms57 ms$0.034$34.08estimate · RTX PRO 6000 Blackwell
GLiDE 9B Open · 7.9B

GLiDE 9B · Fastino · decision model

Parameters: 7.94B dense

Accuracy 54.5% (95%: 52.8%–56.2%)

Logic 47.8% · Math 57.9%

Its run record (Method) →

—54.5%52.8%–56.2%uv1,791 / 3,288105 right of 105 kept0 mistakes · 3,183 skipped0.78667.72chosen-answer Brier 0.190226.9%answered 86%177 ms222 ms$0.011$11.45estimate · DGX Spark
KnowLine-9B-Gen1 Open · 9.0B

KnowLine-9B-Gen1 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 54.5% (95%: 52.8%–56.2%)

Logic 44.3% · Math 59.8%

Its run record (Method) →

—54.5%52.8%–56.2%uv1,791 / 3,288153 right of 153 kept0 mistakes · 3,135 skipped0.78267.96chosen-answer Brier 0.18543.4%answered 98%38 ms55 ms$0.030$29.87estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-9B-Instant Open · 9.0B

StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

Parameters: 9B dense

Accuracy 54.4% (95%: 52.7%–56.1%)

Logic 46.6% · Math 58.4%

Its run record (Method) →

—54.4%52.7%–56.1%uv1,788 / 3,28863 right of 63 kept0 mistakes · 3,225 skipped0.77967.73chosen-answer Brier 0.18939.6%answered 95%142 ms165 ms$0.009$8.92estimate · DGX Spark
Microsoft-Decision-1 Closed · Efficient

Microsoft-Decision-1 · Microsoft · decision model

Parameters: not disclosed by maker

Base model: Qwen3.5-9B

Accuracy 54.3% (95%: 52.6%–56.0%)

Logic 44.4% · Math 59.4%

Its run record (Method) →

—54.3%52.6%–56.0%uvw1,784 / 3,28847 right of 47 kept0 mistakes · 3,241 skipped0.76967.42chosen-answer Brier 0.194111.4%answered 94%312 ms360 ms$0.009$9.07
Dexy-Decision-9B-V2 Open · 9.0B

Dexy-Decision-9B-V2 · Patrick-18 · decision model

Parameters: 9B dense

Base model: vllm-sr/d3-flash

Accuracy 54.2% (95%: 52.5%–55.9%)

Logic 45.3% · Math 58.8%

Its run record (Method) →

—54.2%52.5%–55.9%uvw1,782 / 3,288263 right of 266 kept3 wrong · 3,022 skipped0.79566.98chosen-answer Brier 0.20236.1%answered 97%46 ms51 ms$0.033$32.61estimate · RTX PRO 6000 Blackwell
Drex 2.0 Open · 35B MoE · 3.0B active

Drex 2.0 · Nace.AI · decision model

Parameters: 35B MoE · 3B active

Accuracy 53.1% (95%: 51.4%–54.8%)

Logic 41.9% · Math 59.0%

Its run record (Method) →

—53.1%51.4%–54.8%uvwx1,747 / 3,288207 right of 207 kept0 mistakes · 3,081 skipped0.78967.48chosen-answer Brier 0.181828.1%answered 85%339 ms472 ms$0.012$12.17
vera-medium-omni Open · 4.2B

vera-medium-omni · scar-ai · decision model

Parameters: 4.2B dense

Base model: Qwen3.5-4B

Accuracy 52.5% (95%: 50.8%–54.2%)

Logic 44.1% · Math 56.8%

Its run record (Method) →

—52.5%50.8%–54.2%vwxy1,725 / 3,28839 right of 39 kept0 mistakes · 3,249 skipped0.77766.69chosen-answer Brier 0.19097.9%answered 96%31 ms36 ms$0.024$24.19estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-4B-Instant Open · 4.0B

StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

Parameters: 4B dense

Accuracy 52.4% (95%: 50.7%–54.1%)

Logic 45.9% · Math 55.8%

Its run record (Method) →

—52.4%50.7%–54.1%vwxy1,724 / 3,288117 right of 117 kept0 mistakes · 3,171 skipped0.76366.33chosen-answer Brier 0.197814.4%answered 93%68 ms80 ms$0.004$4.28estimate · DGX Spark
Perplexity Decider Flash v1.3 Open · 4.0B

Perplexity Decider Flash v1.3 · Perplexity · decision model

Parameters: 4B dense

Accuracy 52.1% (95%: 50.4%–53.8%)

Logic 44.7% · Math 55.9%

Its run record (Method) →

—52.1%50.4%–53.8%vwxy1,713 / 3,288130 right of 131 kept1 wrong · 3,157 skipped0.78265.93chosen-answer Brier 0.202415.0%answered 92%29 ms35 ms$0.024$23.77estimate · RTX PRO 6000 Blackwell
Cerno 9B v1.2 Open · 9.0B

Cerno 9B v1.2 · Apertix AI · decision model

Parameters: 9B dense

Accuracy 51.7% (95%: 50.0%–53.4%)

Logic 43.3% · Math 56.0%

Its run record (Method) →

—51.7%50.0%–53.4%vwxy1,699 / 3,288270 right of 281 kept11 wrong · 3,007 skipped0.77664.84chosen-answer Brier 0.21993.1%answered 99%163 ms240 ms$0.011$10.82estimate · DGX Spark
Valen-Decision-1.5-9B Open · 9.0B

Valen-Decision-1.5-9B · Valen-Team · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 51.6% (95%: 49.9%–53.3%)

Logic 44.4% · Math 55.3%

Its run record (Method) →

—51.6%49.9%–53.3%vwxy1,696 / 3,288165 right of 167 kept2 wrong · 3,121 skipped0.76565.99chosen-answer Brier 0.19607.1%answered 96%59 ms65 ms$0.041$41.44estimate · RTX PRO 6000 Blackwell
KnowLine 4B Gen5.5 Open · 4.0B

KnowLine 4B Gen5.5 · PelaAI · decision model

Parameters: 4B dense

Accuracy 51.3% (95%: 49.6%–53.0%)

Logic 42.3% · Math 56.0%

Its run record (Method) →

—51.3%49.6%–53.0%vwxyz1,687 / 3,28871 right of 72 kept1 wrong · 3,216 skipped0.75265.07chosen-answer Brier 0.21164.4%answered 98%76 ms104 ms$0.005$5.02estimate · DGX Spark
Perplexity Decider Flash v1.2 Open · 4.0B

Perplexity Decider Flash v1.2 · Perplexity · decision model

Parameters: 4B dense

Accuracy 51.1% (95%: 49.4%–52.8%)

Logic 43.1% · Math 55.2%

Its run record (Method) →

—51.1%49.4%–52.8%vwxyz1,680 / 3,288185 right of 190 kept5 wrong · 3,098 skipped0.76965.47chosen-answer Brier 0.201618.9%answered 90%41 ms51 ms$0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell
Mercury Decide Closed · Efficient

Mercury Decide · Inception · decision model

Parameters: not disclosed by maker

Accuracy 50.6% (95%: 48.9%–52.3%)

Logic 45.1% · Math 53.5%

Its run record (Method) →

—50.6%48.9%–52.3%wxyzA1,664 / 3,288356 right of 372 kept16 wrong · 2,916 skipped0.77764.36chosen-answer Brier 0.218910.1%answered 95%293 ms349 ms$0.005$5.46
d1 Closed · Efficient

d1 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 50.5% (95%: 48.8%–52.3%)

Logic 40.8% · Math 55.6%

Its run record (Method) →

—50.5%48.8%–52.3%wxyzA1,662 / 3,288207 right of 208 kept1 wrong · 3,080 skipped0.75765.29chosen-answer Brier 0.19986.4%answered 97%185 ms292 ms$0.004$3.98
Open-Jev-27B-v1.1 Open · 27B

Open-Jev-27B-v1.1 · Zefan Cai · decision model

Parameters: 27B dense

Base model: Qwen3.8-27B + adapter

Accuracy 50.5% (95%: 48.7%–52.2%)

Logic 42.6% · Math 54.6%

Its run record (Method) →

—50.5%48.7%–52.2%xyzA1,659 / 3,288No answers at this threshold0.75962.91chosen-answer Brier 0.246514.5%answered 92%250 ms482 ms$0.212$212estimate · RTX PRO 6000 Blackwell
OneJev-27B Open · 27B

OneJev-27B · OmniJev · decision model

Parameters: 27B dense

Accuracy 49.3% (95%: 47.6%–51.0%)

Logic 40.6% · Math 53.8%

Its run record (Method) →

—49.3%47.6%–51.0%yzAB1,620 / 3,28830 right of 30 kept0 mistakes · 3,258 skipped0.77065.05chosen-answer Brier 0.19171.6%answered 99%80 ms134 ms$0.069$69.42estimate · RTX PRO 6000 Blackwell
Clef Open · 27B

Clef · Cloudflare · decision model

Parameters: 27B dense

Accuracy 47.6% (95%: 45.9%–49.3%)

Logic 40.8% · Math 51.1%

Its run record (Method) →

—47.6%45.9%–49.3%zABC1,564 / 3,28818 right of 18 kept0 mistakes · 3,270 skipped0.74762.92chosen-answer Brier 0.21722.4%answered 99%185 ms216 ms$0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell
Verdin-4B-v1.1 Open · 4.0B

Verdin-4B-v1.1 · victorbennett · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Accuracy 47.5% (95%: 45.8%–49.2%)

Logic 39.7% · Math 51.5%

Its run record (Method) →

—47.5%45.8%–49.2%ABC1,561 / 3,288161 right of 165 kept4 wrong · 3,123 skipped0.74763.44chosen-answer Brier 0.20598.9%answered 95%46 ms52 ms$0.033$33.45estimate · RTX PRO 6000 Blackwell
Chinese Decision Pointer 4B Open · 4.0B

Chinese Decision Pointer 4B · quincyqiang · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Accuracy 47.0% (95%: 45.3%–48.7%)

Logic 40.4% · Math 50.4%

Its run record (Method) →

—47.0%45.3%–48.7%ABC1,545 / 3,288188 right of 194 kept6 wrong · 3,094 skipped0.74362.33chosen-answer Brier 0.22337.8%answered 96%30 ms42 ms$0.025$25.03estimate · RTX PRO 6000 Blackwell
Clef-Flash Open · 9.0B

Clef-Flash · Cloudflare · decision model

Parameters: 9B dense

Accuracy 46.1% (95%: 44.4%–47.8%)

Logic 38.9% · Math 49.8%

Its run record (Method) →

—46.1%44.4%–47.8%BCD1,515 / 3,288No answers at this threshold0.70762.13chosen-answer Brier 0.21816.9%answered 96%103 ms131 ms$0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell
Winnow-12B Open · 12B

Winnow-12B · EldanRing · decision model

Parameters: 12B dense

Accuracy 44.4% (95%: 42.7%–46.1%)

Logic 39.7% · Math 46.9%

Its run record (Method) →

—44.4%42.7%–46.1%CD1,461 / 3,28852 right of 53 kept1 wrong · 3,235 skipped0.73761.44chosen-answer Brier 0.215517.1%answered 90%55 ms72 ms$0.043$42.91estimate · RTX PRO 6000 Blackwell
Cygnet Open · 12B

Cygnet · Blockbrain Labs · decision model

Parameters: 11.95B dense

Base model: Gemma 4 12B IT

Accuracy 42.6% (95%: 40.9%–44.3%)

Logic 39.3% · Math 44.3%

Its run record (Method) →

—42.6%40.9%–44.3%D1,401 / 3,288No answers at this threshold0.72960.88chosen-answer Brier 0.20867.3%answered 96%38 ms44 ms$0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell
Metask-Jev-4B Open · 4.5B

Metask-Jev-4B · Metask Lab · decision model

Parameters: 4.54B dense

Accuracy 36.7% (95%: 35.1%–38.4%)

Logic 32.1% · Math 39.2%

Its run record (Method) →

—36.7%35.1%–38.4%E1,208 / 3,288No answers at this threshold0.65957.83chosen-answer Brier 0.21082.0%answered 99%117 ms146 ms$0.086$85.83estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-2B-Instant Open · 2.0B

StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

Parameters: 2B dense

Accuracy 35.9% (95%: 34.3%–37.6%)

Logic 30.8% · Math 38.7%

Its run record (Method) →

—35.9%34.3%–37.6%E1,182 / 3,2884 right of 4 kept0 mistakes · 3,284 skipped0.67557.77chosen-answer Brier 0.204116.4%answered 89%36 ms42 ms$0.002$2.36estimate · DGX Spark
GLiDE 2B Open · 1.9B

GLiDE 2B · Fastino · decision model

Parameters: 1.88B dense

Accuracy 35.2% (95%: 33.6%–36.9%)

Logic 32.3% · Math 36.8%

Its run record (Method) →

—35.2%33.6%–36.9%E1,158 / 3,28829 right of 30 kept1 wrong · 3,258 skipped0.63755.25chosen-answer Brier 0.247223.1%answered 84%28 ms37 ms$0.002$1.94estimate · DGX Spark
d1-2B Open · 1.9B

d1-2B · Liquid AI · decision model

Parameters: 1.92B dense

Base model: LFM3-VL-2B

Accuracy 34.9% (95%: 33.3%–36.5%)

Logic 30.7% · Math 37.1%

Its run record (Method) →

—34.9%33.3%–36.5%EF1,147 / 3,2887 right of 7 kept0 mistakes · 3,281 skipped0.64856.81chosen-answer Brier 0.212782.5%answered 30%17 ms21 ms$0.014$14.30estimate · RTX PRO 6000 Blackwell
KnowLine-2B-Gen1 Open · 2.0B

KnowLine-2B-Gen1 · PelaAI · decision model

Parameters: 2B dense

Base model: Qwen3.5-2B

Accuracy 34.1% (95%: 32.5%–35.7%)

Logic 31.7% · Math 35.3%

Its run record (Method) →

—34.1%32.5%–35.7%EF1,120 / 3,28816 right of 16 kept0 mistakes · 3,272 skipped0.62656.04chosen-answer Brier 0.21996.8%answered 95%51 ms63 ms$0.003$3.40estimate · DGX Spark
GLiDE 0.8B Open · 0.8B

GLiDE 0.8B · Fastino · decision model

Parameters: 0.75B dense

Accuracy 31.1% (95%: 29.5%–32.7%)

Logic 29.2% · Math 32.1%

Its run record (Method) →

—31.1%29.5%–32.7%FG1,022 / 3,28813 right of 13 kept0 mistakes · 3,275 skipped0.60253.32chosen-answer Brier 0.244422.7%answered 83%20 ms25 ms$0.001$1.38estimate · DGX Spark
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B

StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Accuracy 28.0% (95%: 26.5%–29.5%)

Logic 24.7% · Math 29.7%

Its run record (Method) →

—28.0%26.5%–29.5%GH920 / 3,288No answers at this threshold0.60554.00chosen-answer Brier 0.199914.9%answered 89%24 ms29 ms$0.002$1.62estimate · DGX Spark
OneJev1.6-0.8B Open · 0.8B

OneJev1.6-0.8B · OmniJev · decision model

Parameters: 0.8B dense

Accuracy 27.8% (95%: 26.3%–29.4%)

Logic 27.0% · Math 28.2%

Its run record (Method) →

—27.8%26.3%–29.4%GH915 / 3,2882 right of 2 kept0 mistakes · 3,286 skipped0.57653.31chosen-answer Brier 0.21202.0%answered 99%15 ms17 ms$0.001$1.11estimate · DGX Spark
KnowLine-0.8B-Gen1 Open · 0.8B

KnowLine-0.8B-Gen1 · PelaAI · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Accuracy 27.7% (95%: 26.2%–29.2%)

Logic 26.7% · Math 28.2%

Its run record (Method) →

—27.7%26.2%–29.2%GH910 / 3,2882 right of 2 kept0 mistakes · 3,286 skipped0.59053.55chosen-answer Brier 0.20570.6%answered 100%26 ms35 ms$0.002$1.80estimate · DGX Spark
Strands Decider 2B Open · 2.0B

Strands Decider 2B · Strands Agents · decision model

Parameters: 2B dense

Accuracy 26.5% (95%: 25.0%–28.1%)

Logic 22.4% · Math 28.7%

Its run record (Method) →

—26.5%25.0%–28.1%HI872 / 3,288No answers at this threshold0.57353.20chosen-answer Brier 0.20125.0%answered 96%35 ms42 ms$0.027$26.72estimate · RTX PRO 6000 Blackwell
GLiClass v4 Edge Open · 0.4B

GLiClass v4 Edge · decision model

Parameters: 0.4B dense

Base model: DeBERTa-v3-large

Accuracy 23.0% (95%: 21.6%–24.4%)

Logic 23.6% · Math 22.7%

Its run record (Method) →

—23.0%21.6%–24.4%IJ755 / 3,288No answers at this threshold0.51152.00chosen-answer Brier 0.189615.6%answered 87%12 ms23 ms$0.001$1.01estimate · DGX Spark
GLiDE 0.4B Open · 0.4B

GLiDE 0.4B · Fastino · decision model

Parameters: 0.40B dense

Accuracy 21.5% (95%: 20.1%–22.9%)

Logic 23.2% · Math 20.6%

Its run record (Method) →

—21.5%20.1%–22.9%J706 / 3,288No answers at this threshold0.52051.39chosen-answer Brier 0.186973.9%answered 32%11 ms14 ms$0.001$0.83estimate · DGX Spark

Tie groups and “not measured”

estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.

Decision Score (AA formula): Artificial Analysis's formula applied to our questions. Not their Index. 50 × (1 + accuracy − chosen-answer Brier), the chosen-answer Brier over the probability of the chosen answer, on decision models' standard runs only. Chat models and the “I don't know” runs are excluded.