Math · Accuracy

Accuracy on Math

2,163 questions, asked in English. How to read this page · Method

Accuracy

Share of the questions answered right, with the Wilson 95% interval and the tie-group letters · higher is better; the line is chance (20%)

  1. Karpo Decision 31B

    Karpo Decision 31B · MachinePulse · decision model

    Parameters: 31B dense

    Accuracy 81.6% (95%: 79.9%–83.1%)

    Logic 60.7%

    Its run record (Method) →

    81.6% (80%–83%) a
  2. KnowLine-31B-Gen2

    KnowLine-31B-Gen2 · PelaAI · decision model

    Parameters: 31B dense

    Base model: Gemma 4 31B

    Accuracy 80.3% (95%: 78.5%–81.9%)

    Logic 58.9%

    Its run record (Method) →

    80.3% (79%–82%) ab
  3. REAX 1.3

    REAX 1.3 · REAX · decision model

    Parameters: 57.36B dense

    Accuracy 78.0% (95%: 76.2%–79.7%)

    Logic 59.3%

    Its run record (Method) →

    78.0% (76%–80%) bc
  4. KnowLine-31B-Gen1.5

    KnowLine-31B-Gen1.5 · PelaAI · decision model

    Parameters: 31B dense

    Accuracy 77.4% (95%: 75.6%–79.1%)

    Logic 59.0%

    Its run record (Method) →

    77.4% (76%–79%) cd
  5. Copernicus-31B

    Copernicus-31B · Poindexter Labs · decision model

    Parameters: 31B dense

    Accuracy 77.3% (95%: 75.5%–79.1%)

    Logic 58.7%

    Its run record (Method) →

    77.3% (76%–79%) cd
  6. Sieve 31B v3

    Sieve 31B v3 · Sthānika AI · decision model

    Parameters: 31B dense

    Accuracy 77.1% (95%: 75.3%–78.8%)

    Logic 57.9%

    Its run record (Method) →

    77.1% (75%–79%) cd
  7. Logits Sigma-1 Duo

    Logits Sigma-1 Duo · Logits · decision model

    Parameters: 57.36B dense

    Accuracy 75.2% (95%: 73.4%–77.0%)

    Logic 57.8%

    Its run record (Method) →

    75.2% (73%–77%) de
  8. d1-ultra

    d1-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

    Accuracy 74.1% (95%: 72.2%–75.9%)

    Logic 54.9%

    Its run record (Method) →

    74.1% (72%–76%) defg
  9. Mammoth-1

    Mammoth-1 · Base Labs · decision model

    Parameters: 27B dense

    Accuracy 74.0% (95%: 72.1%–75.8%)

    Logic 54.5%

    Its run record (Method) →

    74.0% (72%–76%) defh
  10. Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell

    Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 72.5% (95%: 70.6%–74.3%)

    Logic 54.8%

    Its run record (Method) →

    72.5% (71%–74%) efghi
  11. GLiDE 27B

    GLiDE 27B · Fastino · decision model

    Parameters: 27B dense

    Accuracy 72.0% (95%: 70.1%–73.8%)

    Logic 58.4%

    Its run record (Method) →

    72.0% (70%–74%) efghij
  12. d3-ultra

    d3-ultra · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Accuracy 71.5% (95%: 69.6%–73.4%)

    Logic 54.6%

    Its run record (Method) →

    71.5% (70%–73%) fghij
  13. Gemini Decision 1 20261111

    Gemini Decision 1 20261111 · Google · decision model

    Parameters: not disclosed by maker (closed weights)

    Accuracy 71.3% (95%: 69.4%–73.2%)

    Logic 50.0%

    Its run record (Method) →

    71.3% (69%–73%) efghij
  14. GLiDE API

    GLiDE API · Fastino · decision model

    Parameters: not disclosed by maker

    Accuracy 70.8% (95%: 68.8%–72.7%)

    Logic 54.5%

    Its run record (Method) →

    70.8% (69%–73%) fghij
  15. d3

    d3 · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Accuracy 70.4% (95%: 68.5%–72.3%)

    Logic 54.5%

    Its run record (Method) →

    70.4% (68%–72%) gij
  16. Drex Max

    Drex Max · Nace.AI · decision model

    Parameters: 27B dense

    Accuracy 70.3% (95%: 68.4%–72.2%)

    Logic 53.1%

    Its run record (Method) →

    70.3% (68%–72%) gijk
  17. Drex Max v1.1

    Drex Max v1.1 · Nace.AI · decision model

    Parameters: not disclosed by maker

    Accuracy 70.1% (95%: 68.1%–72.0%)

    Logic 54.8%

    Its run record (Method) →

    70.1% (68%–72%) ijk
  18. celeris-1-decision

    celeris-1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Accuracy 70.0% (95%: 68.0%–71.9%)

    Logic 47.3%

    Its run record (Method) →

    70.0% (68%–72%) hijk
  19. GLiDE v1.2

    GLiDE v1.2 · Fastino · decision model

    Parameters: 27B dense

    Accuracy 69.9% (95%: 67.9%–71.8%)

    Logic 55.2%

    Its run record (Method) →

    69.9% (68%–72%) ijk
  20. celeris-1.1-decision

    celeris-1.1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Accuracy 69.9% (95%: 67.9%–71.8%)

    Logic 47.3%

    Its run record (Method) →

    69.9% (68%–72%) ijkl
  21. d1.2-ultra

    d1.2-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 69.9% (95%: 67.9%–71.8%)

    Logic 51.6%

    Its run record (Method) →

    69.9% (68%–72%) ijk
  22. Perplexity Decider 27B v1.2 API

    Perplexity Decider 27B v1.2 API · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 69.8% (95%: 67.8%–71.7%)

    Logic 53.3%

    Its run record (Method) →

    69.8% (68%–72%) ijkl
  23. Perplexity Decider 27B v1.2 API · with web

    Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 69.3% (95%: 67.3%–71.2%)

    Logic 52.5%

    Its run record (Method) →

    69.3% (67%–71%) jklm
  24. Turion Jev 31B

    Turion Jev 31B · Turion.AI · decision model

    Parameters: 31B dense

    Accuracy 69.0% (95%: 67.0%–70.9%)

    Logic 52.3%

    Its run record (Method) →

    69.0% (67%–71%) ijklmn
  25. SubQ Snap 1.1

    SubQ Snap 1.1 · Subquadratic · decision model

    Parameters: not disclosed by maker

    Shown once with SubQ Snap Preview: the same answer to every question

    Accuracy 68.6% (95%: 66.6%–70.5%)

    Logic 52.4%

    Its run record (Method) →

    68.6% (67%–70%) jklmn
  26. KnowLine-12B-Gen1

    KnowLine-12B-Gen1 · PelaAI · decision model

    Parameters: 12B dense

    Base model: Gemma 4 12B

    Accuracy 66.3% (95%: 64.2%–68.2%)

    Logic 50.6%

    Its run record (Method) →

    66.3% (64%–68%) klmno
  27. d1.2

    d1.2 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 65.5% (95%: 63.5%–67.5%)

    Logic 49.2%

    Its run record (Method) →

    65.5% (63%–67%) lmnop
  28. d1.1

    d1.1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 65.2% (95%: 63.2%–67.2%)

    Logic 46.0%

    Its run record (Method) →

    65.2% (63%–67%) mnop
  29. Jev 1.14

    Jev 1.14 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Accuracy 65.1% (95%: 63.1%–67.1%)

    Logic 48.2%

    Its run record (Method) →

    65.1% (63%–67%) nop
  30. Jev 1.13

    Jev 1.13 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Accuracy 64.1% (95%: 62.0%–66.1%)

    Logic 46.3%

    Its run record (Method) →

    64.1% (62%–66%) opq
  31. Adjudex 9B v5

    Adjudex 9B v5 · newsbang · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 62.1% (95%: 60.0%–64.1%)

    Logic 46.4%

    Its run record (Method) →

    62.1% (60%–64%) opqr
  32. Elephant-1

    Elephant-1 · Base Labs · decision model

    Parameters: 9B dense

    Accuracy 61.3% (95%: 59.3%–63.4%)

    Logic 46.4%

    Its run record (Method) →

    61.3% (59%–63%) pqr
  33. KnowLine-9B-Gen2

    KnowLine-9B-Gen2 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 60.1% (95%: 58.0%–62.1%)

    Logic 43.9%

    Its run record (Method) →

    60.1% (58%–62%) qrs
  34. Jeffan-9B-1211

    Jeffan-9B-1211 · tattrongvu · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 59.8% (95%: 57.7%–61.9%)

    Logic 48.2%

    Its run record (Method) →

    59.8% (58%–62%) qrs
  35. KnowLine-9B-Gen1

    KnowLine-9B-Gen1 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 59.8% (95%: 57.7%–61.8%)

    Logic 44.3%

    Its run record (Method) →

    59.8% (58%–62%) qrs
  36. Microsoft-Decision-1

    Microsoft-Decision-1 · Microsoft · decision model

    Parameters: not disclosed by maker

    Base model: Qwen3.5-9B

    Accuracy 59.4% (95%: 57.3%–61.4%)

    Logic 44.4%

    Its run record (Method) →

    59.4% (57%–61%) rst
  37. Drex 2.0

    Drex 2.0 · Nace.AI · decision model

    Parameters: 35B MoE · 3B active

    Accuracy 59.0% (95%: 56.9%–61.0%)

    Logic 41.9%

    Its run record (Method) →

    59.0% (57%–61%) rstu
  38. Dexy-Decision-9B-V2

    Dexy-Decision-9B-V2 · Patrick-18 · decision model

    Parameters: 9B dense

    Base model: vllm-sr/d3-flash

    Accuracy 58.8% (95%: 56.7%–60.9%)

    Logic 45.3%

    Its run record (Method) →

    58.8% (57%–61%) rstu
  39. StartLux-Decision-v1.1-9B-Instant

    StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

    Parameters: 9B dense

    Accuracy 58.4% (95%: 56.3%–60.5%)

    Logic 46.6%

    Its run record (Method) →

    58.4% (56%–60%) rstuv
  40. GLiDE 9B

    GLiDE 9B · Fastino · decision model

    Parameters: 7.94B dense

    Accuracy 57.9% (95%: 55.8%–60.0%)

    Logic 47.8%

    Its run record (Method) →

    57.9% (56%–60%) rstuvw
  41. vera-medium-omni

    vera-medium-omni · scar-ai · decision model

    Parameters: 4.2B dense

    Base model: Qwen3.5-4B

    Accuracy 56.8% (95%: 54.7%–58.9%)

    Logic 44.1%

    Its run record (Method) →

    56.8% (55%–59%) stuvw
  42. Cerno 9B v1.2

    Cerno 9B v1.2 · Apertix AI · decision model

    Parameters: 9B dense

    Accuracy 56.0% (95%: 53.9%–58.1%)

    Logic 43.3%

    Its run record (Method) →

    56.0% (54%–58%) stuvw
  43. KnowLine 4B Gen5.5

    KnowLine 4B Gen5.5 · PelaAI · decision model

    Parameters: 4B dense

    Accuracy 56.0% (95%: 53.9%–58.1%)

    Logic 42.3%

    Its run record (Method) →

    56.0% (54%–58%) stuvw
  44. Perplexity Decider Flash v1.3

    Perplexity Decider Flash v1.3 · Perplexity · decision model

    Parameters: 4B dense

    Accuracy 55.9% (95%: 53.8%–58.0%)

    Logic 44.7%

    Its run record (Method) →

    55.9% (54%–58%) stuvw
  45. StartLux-Decision-v1.1-4B-Instant

    StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

    Parameters: 4B dense

    Accuracy 55.8% (95%: 53.7%–57.9%)

    Logic 45.9%

    Its run record (Method) →

    55.8% (54%–58%) stuvwx
  46. d1

    d1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 55.6% (95%: 53.5%–57.7%)

    Logic 40.8%

    Its run record (Method) →

    55.6% (54%–58%) stuvwxy
  47. Valen-Decision-1.5-9B

    Valen-Decision-1.5-9B · Valen-Team · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 55.3% (95%: 53.2%–57.4%)

    Logic 44.4%

    Its run record (Method) →

    55.3% (53%–57%) tuvwxy
  48. Perplexity Decider Flash v1.2

    Perplexity Decider Flash v1.2 · Perplexity · decision model

    Parameters: 4B dense

    Accuracy 55.2% (95%: 53.1%–57.3%)

    Logic 43.1%

    Its run record (Method) →

    55.2% (53%–57%) tuvwxy
  49. Open-Jev-27B-v1.1

    Open-Jev-27B-v1.1 · Zefan Cai · decision model

    Parameters: 27B dense

    Base model: Qwen3.8-27B + adapter

    Accuracy 54.6% (95%: 52.4%–56.6%)

    Logic 42.6%

    Its run record (Method) →

    54.6% (52%–57%) uvwxyz
  50. OneJev-27B

    OneJev-27B · OmniJev · decision model

    Parameters: 27B dense

    Accuracy 53.8% (95%: 51.7%–55.9%)

    Logic 40.6%

    Its run record (Method) →

    53.8% (52%–56%) vwxyzA
  51. Mercury Decide

    Mercury Decide · Inception · decision model

    Parameters: not disclosed by maker

    Accuracy 53.5% (95%: 51.4%–55.6%)

    Logic 45.1%

    Its run record (Method) →

    53.5% (51%–56%) wxyzA
  52. Verdin-4B-v1.1

    Verdin-4B-v1.1 · victorbennett · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Accuracy 51.5% (95%: 49.4%–53.6%)

    Logic 39.7%

    Its run record (Method) →

    51.5% (49%–54%) yzAB
  53. Clef

    Clef · Cloudflare · decision model

    Parameters: 27B dense

    Accuracy 51.1% (95%: 49.0%–53.2%)

    Logic 40.8%

    Its run record (Method) →

    51.1% (49%–53%) xyzAB
  54. Chinese Decision Pointer 4B

    Chinese Decision Pointer 4B · quincyqiang · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Accuracy 50.4% (95%: 48.3%–52.5%)

    Logic 40.4%

    Its run record (Method) →

    50.4% (48%–53%) zAB
  55. Clef-Flash

    Clef-Flash · Cloudflare · decision model

    Parameters: 9B dense

    Accuracy 49.8% (95%: 47.7%–51.9%)

    Logic 38.9%

    Its run record (Method) →

    49.8% (48%–52%) AB
  56. Winnow-12B

    Winnow-12B · EldanRing · decision model

    Parameters: 12B dense

    Accuracy 46.9% (95%: 44.8%–49.0%)

    Logic 39.7%

    Its run record (Method) →

    46.9% (45%–49%) BC
  57. Cygnet

    Cygnet · Blockbrain Labs · decision model

    Parameters: 11.95B dense

    Base model: Gemma 4 12B IT

    Accuracy 44.3% (95%: 42.3%–46.4%)

    Logic 39.3%

    Its run record (Method) →

    44.3% (42%–46%) C
  58. Metask-Jev-4B

    Metask-Jev-4B · Metask Lab · decision model

    Parameters: 4.54B dense

    Accuracy 39.2% (95%: 37.1%–41.2%)

    Logic 32.1%

    Its run record (Method) →

    39.2% (37%–41%) D
  59. StartLux-Decision-v1.1-2B-Instant

    StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

    Parameters: 2B dense

    Accuracy 38.7% (95%: 36.6%–40.7%)

    Logic 30.8%

    Its run record (Method) →

    38.7% (37%–41%) D
  60. d1-2B

    d1-2B · Liquid AI · decision model

    Parameters: 1.92B dense

    Base model: LFM3-VL-2B

    Accuracy 37.1% (95%: 35.1%–39.1%)

    Logic 30.7%

    Its run record (Method) →

    37.1% (35%–39%) D
  61. GLiDE 2B

    GLiDE 2B · Fastino · decision model

    Parameters: 1.88B dense

    Accuracy 36.8% (95%: 34.7%–38.8%)

    Logic 32.3%

    Its run record (Method) →

    36.8% (35%–39%) D
  62. KnowLine-2B-Gen1

    KnowLine-2B-Gen1 · PelaAI · decision model

    Parameters: 2B dense

    Base model: Qwen3.5-2B

    Accuracy 35.3% (95%: 33.3%–37.3%)

    Logic 31.7%

    Its run record (Method) →

    35.3% (33%–37%) DE
  63. GLiDE 0.8B

    GLiDE 0.8B · Fastino · decision model

    Parameters: 0.75B dense

    Accuracy 32.1% (95%: 30.2%–34.1%)

    Logic 29.2%

    Its run record (Method) →

    32.1% (30%–34%) EF
  64. StartLux-Decision-v1.1-0.8B-Instant

    StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Accuracy 29.7% (95%: 27.8%–31.6%)

    Logic 24.7%

    Its run record (Method) →

    29.7% (28%–32%) F
  65. Strands Decider 2B

    Strands Decider 2B · Strands Agents · decision model

    Parameters: 2B dense

    Accuracy 28.7% (95%: 26.8%–30.6%)

    Logic 22.4%

    Its run record (Method) →

    28.7% (27%–31%) F
  66. OneJev1.6-0.8B

    OneJev1.6-0.8B · OmniJev · decision model

    Parameters: 0.8B dense

    Accuracy 28.2% (95%: 26.4%–30.2%)

    Logic 27.0%

    Its run record (Method) →

    28.2% (26%–30%) F
  67. KnowLine-0.8B-Gen1

    KnowLine-0.8B-Gen1 · PelaAI · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Accuracy 28.2% (95%: 26.3%–30.1%)

    Logic 26.7%

    Its run record (Method) →

    28.2% (26%–30%) F
  68. GLiClass v4 Edge

    GLiClass v4 Edge · decision model

    Parameters: 0.4B dense

    Base model: DeBERTa-v3-large

    Accuracy 22.7% (95%: 20.9%–24.5%)

    Logic 23.6%

    Its run record (Method) →

    22.7% (21%–24%) G
  69. GLiDE 0.4B

    GLiDE 0.4B · Fastino · decision model

    Parameters: 0.40B dense

    Accuracy 20.6% (95%: 18.9%–22.3%)

    Logic 23.2%

    Its run record (Method) →

    20.6% (19%–22%) G

MachinePulse PelaAI REAX Poindexter Labs Sthānika AI Logits Liquid AI Base Labs Perplexity Fastino vLLM Semantic Router Google Nace.AI Celeris Turion.AI Subquadratic TypeSafe newsbang tattrongvu Microsoft Patrick-18 StartLux scar-ai Apertix AI Valen-Team Zefan Cai OmniJev Inception victorbennett Cloudflare quincyqiang EldanRing Blockbrain Labs Metask Lab Strands Agents Other Chance, 20% (five options)Bars run to each accuracy; the whisker is its 95% interval. Ranks show observed scores; small gaps need not be meaningful.

Accuracy by topic

SetupWebBasic concepts238 questionsDigits and place value204 questionsNumber bases17 questionsDivision and divisibility157 questionsPrime factorisation and GCD/LCM109 questionsRational numbers134 questionsDecimals110 questionsOrdering101 questionsSimple inequalities161 questionsAbsolute value105 questionsExponents226 questionsRadicals257 questionsFactorisation344 questions
Karpo Decision 31B Open · 31B—77.7%185/23865.2%133/20464.7%11/1768.8%108/15774.3%81/10985.1%114/13492.7%102/11092.1%93/10188.2%142/16181.9%86/10585.8%194/22683.7%215/25787.2%300/344
KnowLine-31B-Gen2 Open · 31B—75.6%180/23860.3%123/20464.7%11/1770.1%110/15775.2%82/10979.1%106/13490.9%100/11093.1%94/10185.1%137/16181.9%86/10588.5%200/22682.9%213/25785.5%294/344
REAX 1.3 Open · 57B—76.9%183/23857.8%118/20464.7%11/1761.8%97/15776.1%83/10976.1%102/13492.7%102/11084.2%85/10183.9%135/16179.0%83/10585.4%193/22677.8%200/25785.8%295/344
KnowLine-31B-Gen1.5 Open · 31B—76.9%183/23858.8%120/20452.9%9/1763.7%100/15770.6%77/10976.1%102/13488.2%97/11089.1%90/10182.0%132/16179.0%83/10585.0%192/22680.2%206/25782.3%283/344
Copernicus-31B Open · 31B—76.1%181/23854.9%112/20458.8%10/1763.1%99/15771.6%78/10979.1%106/13489.1%98/11092.1%93/10181.4%131/16176.2%80/10585.4%193/22679.4%204/25783.7%288/344
Sieve 31B v3 Open · 31B—75.6%180/23855.9%114/20452.9%9/1763.7%100/15769.7%76/10981.3%109/13490.9%100/11086.1%87/10180.1%129/16180.0%84/10582.7%187/22679.4%204/25784.0%289/344
Logits Sigma-1 Duo Open · 57B—71.4%170/23851.5%105/20464.7%11/1759.9%94/15775.2%82/10975.4%101/13490.0%99/11085.1%86/10181.4%131/16175.2%79/10581.4%184/22678.6%202/25782.3%283/344
d1-ultra Closed · Efficient—71.8%171/23853.4%109/20470.6%12/1765.6%103/15759.6%65/10978.4%105/13484.5%93/11077.2%78/10172.7%117/16178.1%82/10579.6%180/22679.4%204/25782.6%284/344
Mammoth-1 Open · 27B—68.1%162/23850.0%102/20435.3%6/1757.3%90/15762.4%68/10977.6%104/13488.2%97/11081.2%82/10179.5%128/16179.0%83/10584.5%191/22678.6%202/25782.8%285/344
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—69.7%166/23852.5%107/20476.5%13/1755.4%87/15760.6%66/10977.6%104/13486.4%95/11083.2%84/10177.6%125/16173.3%77/10576.5%173/22676.7%197/25779.7%274/344
GLiDE 27B Open · 27B—69.3%165/23852.0%106/20452.9%9/1761.8%97/15760.6%66/10973.1%98/13490.9%100/11081.2%82/10177.6%125/16170.5%74/10577.4%175/22673.9%190/25778.5%270/344
d3-ultra Open · 26B—67.2%160/23850.0%102/20452.9%9/1756.1%88/15773.4%80/10969.4%93/13488.2%97/11085.1%86/10177.6%125/16175.2%79/10575.2%170/22672.0%185/25779.4%273/344
Gemini Decision 1 20261111 Closed · Efficient—65.1%155/23848.0%98/20458.8%10/1761.8%97/15765.1%71/10975.4%101/13481.8%90/11079.2%80/10170.8%114/16176.2%80/10577.0%174/22676.7%197/25780.2%276/344
GLiDE API Closed · Efficient—67.2%160/23854.4%111/20447.1%8/1756.7%89/15759.6%65/10971.6%96/13488.2%97/11080.2%81/10173.9%119/16175.2%79/10574.3%168/22675.1%193/25777.0%265/344
d3 Open · 26B—65.1%155/23852.5%107/20447.1%8/1758.6%92/15756.0%61/10967.9%91/13486.4%95/11080.2%81/10177.6%125/16170.5%74/10576.5%173/22673.2%188/25779.4%273/344
Drex Max Open · 27B—68.1%162/23852.5%107/20452.9%9/1756.1%88/15758.7%64/10967.9%91/13489.1%98/11079.2%80/10177.0%124/16169.5%73/10575.7%171/22671.6%184/25778.5%270/344
Drex Max v1.1 Closed · Efficient (provisional)—69.7%166/23852.9%108/20458.8%10/1754.8%86/15756.0%61/10967.9%91/13488.2%97/11080.2%81/10175.8%122/16172.4%76/10576.1%172/22670.4%181/25777.0%265/344
celeris-1-decision Closed · Efficient—72.7%173/23853.4%109/20458.8%10/1757.3%90/15759.6%65/10970.1%94/13492.7%102/11075.2%76/10168.9%111/16169.5%73/10572.6%164/22673.5%189/25775.0%258/344
GLiDE v1.2 Open · 27B—67.2%160/23852.9%108/20452.9%9/1756.7%89/15760.6%66/10973.9%99/13488.2%97/11078.2%79/10172.0%116/16171.4%75/10575.2%170/22670.4%181/25776.2%262/344
celeris-1.1-decision Closed · Efficient—71.4%170/23853.4%109/20452.9%9/1758.6%92/15759.6%65/10970.1%94/13492.7%102/11074.3%75/10168.9%111/16170.5%74/10572.1%163/22673.5%189/25775.0%258/344
d1.2-ultra Closed · Undisclosed—67.2%160/23846.1%94/20447.1%8/1754.8%86/15759.6%65/10975.4%101/13484.5%93/11082.2%83/10175.8%122/16169.5%73/10575.2%170/22673.2%188/25777.9%268/344
Perplexity Decider 27B v1.2 API Open · 27B—66.0%157/23850.5%103/20452.9%9/1755.4%87/15760.6%66/10972.4%97/13483.6%92/11076.2%77/10174.5%120/16170.5%74/10577.4%175/22670.8%182/25778.5%270/344
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web68.1%162/23848.5%99/20458.8%10/1756.7%89/15761.5%67/10967.9%91/13482.7%91/11080.2%81/10175.2%121/16166.7%70/10577.9%176/22670.4%181/25775.9%261/344
Turion Jev 31B Open · 31B—70.2%167/23841.2%84/20441.2%7/1756.1%88/15756.9%62/10976.1%102/13481.8%90/11084.2%85/10176.4%123/16171.4%75/10573.0%165/22673.2%188/25774.7%257/344
SubQ Snap 1.1 Closed · Efficient—65.1%155/23851.5%105/20452.9%9/1752.9%83/15767.0%73/10970.1%94/13482.7%91/11076.2%77/10174.5%120/16172.4%76/10574.8%169/22668.1%175/25774.4%256/344
KnowLine-12B-Gen1 Open · 12B—66.8%159/23855.9%114/20441.2%7/1756.1%88/15754.1%59/10965.7%88/13477.3%85/11077.2%78/10167.7%109/16169.5%73/10567.3%152/22666.1%170/25773.0%251/344
d1.2 Closed · Undisclosed—62.6%149/23840.2%82/20441.2%7/1752.9%83/15760.6%66/10965.7%88/13475.5%83/11078.2%79/10166.5%107/16167.6%71/10572.1%163/22673.2%188/25773.0%251/344
d1.1 Closed · Efficient—63.0%150/23844.1%90/20441.2%7/1750.3%79/15756.0%61/10963.4%85/13472.7%80/11073.3%74/10164.6%104/16166.7%70/10572.1%163/22675.9%195/25773.3%252/344
Jev 1.14 Closed · Efficient—59.7%142/23842.6%87/20458.8%10/1752.2%82/15758.7%64/10964.9%87/13478.2%86/11076.2%77/10166.5%107/16174.3%78/10570.4%159/22671.2%183/25771.8%247/344
Jev 1.13 Closed · Efficient—56.7%135/23847.5%97/20441.2%7/1751.0%80/15755.0%60/10963.4%85/13478.2%86/11076.2%77/10164.0%103/16174.3%78/10569.9%158/22670.0%180/25769.8%240/344
Adjudex 9B v5 Open · 9.0B—61.3%146/23843.1%88/20441.2%7/1751.0%80/15752.3%57/10963.4%85/13476.4%84/11064.4%65/10164.0%103/16164.8%68/10564.2%145/22668.9%177/25769.2%238/344
Elephant-1 Open · 9.0B—59.2%141/23840.2%82/20441.2%7/1752.2%82/15756.0%61/10963.4%85/13472.7%80/11072.3%73/10160.9%98/16165.7%69/10563.7%144/22664.2%165/25769.8%240/344
KnowLine-9B-Gen2 Open · 9.0B—57.6%137/23841.2%84/20435.3%6/1749.0%77/15750.5%55/10960.4%81/13477.3%85/11064.4%65/10162.7%101/16166.7%70/10566.4%150/22667.3%173/25762.8%216/344
Jeffan-9B-1211 Open · 9.0B—58.4%139/23838.2%78/20423.5%4/1738.9%61/15753.2%58/10956.7%76/13473.6%81/11071.3%72/10164.0%103/16163.8%67/10564.6%146/22663.4%163/25771.5%246/344
KnowLine-9B-Gen1 Open · 9.0B—59.7%142/23842.2%86/20417.6%3/1751.0%80/15753.2%58/10961.2%82/13472.7%80/11063.4%64/10166.5%107/16161.0%64/10561.5%139/22666.1%170/25763.4%218/344
Microsoft-Decision-1 Closed · Efficient—61.3%146/23840.2%82/20423.5%4/1748.4%76/15752.3%57/10953.7%72/13470.0%77/11063.4%64/10160.9%98/16161.9%65/10566.4%150/22663.8%164/25766.6%229/344
Drex 2.0 Open · 35B MoE · 3.0B active—55.5%132/23840.7%83/20429.4%5/1749.7%78/15751.4%56/10957.5%77/13470.0%77/11065.3%66/10164.0%103/16161.9%65/10562.4%141/22664.6%166/25766.0%227/344
Dexy-Decision-9B-V2 Open · 9.0B—56.3%134/23839.7%81/20435.3%6/1742.7%67/15756.0%61/10953.7%72/13472.7%80/11070.3%71/10160.2%97/16161.0%64/10564.2%145/22660.7%156/25769.2%238/344
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—59.2%141/23838.2%78/20435.3%6/1746.5%73/15749.5%54/10954.5%73/13470.9%78/11065.3%66/10164.6%104/16162.9%66/10560.6%137/22662.3%160/25766.3%228/344
GLiDE 9B Open · 7.9B—51.3%122/23838.2%78/20423.5%4/1743.3%68/15748.6%53/10955.2%74/13473.6%81/11063.4%64/10162.7%101/16167.6%71/10562.8%142/22664.2%165/25766.9%230/344
vera-medium-omni Open · 4.2B—55.9%133/23836.3%74/20435.3%6/1742.7%67/15748.6%53/10950.7%68/13470.9%78/11067.3%68/10162.7%101/16163.8%67/10558.4%132/22661.1%157/25765.4%225/344
Cerno 9B v1.2 Open · 9.0B—55.0%131/23841.7%85/20435.3%6/1743.9%69/15748.6%53/10953.0%71/13467.3%74/11066.3%67/10160.2%97/16156.2%59/10558.8%133/22661.1%157/25761.0%210/344
KnowLine 4B Gen5.5 Open · 4.0B—52.1%124/23837.7%77/20423.5%4/1740.8%64/15753.2%58/10960.4%81/13471.8%79/11060.4%61/10157.8%93/16161.9%65/10559.3%134/22659.9%154/25763.1%217/344
Perplexity Decider Flash v1.3 Open · 4.0B—55.5%132/23836.3%74/20429.4%5/1744.6%70/15750.5%55/10953.0%71/13469.1%76/11060.4%61/10160.2%97/16159.0%62/10558.8%133/22661.9%159/25762.5%215/344
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—54.2%129/23840.7%83/20417.6%3/1741.4%65/15749.5%54/10952.2%70/13474.5%82/11064.4%65/10159.6%96/16158.1%61/10555.8%126/22661.5%158/25762.8%216/344
d1 Closed · Efficient—52.1%124/23839.7%81/20435.3%6/1740.8%64/15742.2%46/10951.5%69/13459.1%65/11063.4%64/10160.9%98/16161.9%65/10558.0%131/22661.9%159/25767.2%231/344
Valen-Decision-1.5-9B Open · 9.0B—52.9%126/23834.8%71/20447.1%8/1746.5%73/15745.9%50/10953.0%71/13461.8%68/11065.3%66/10154.7%88/16157.1%60/10558.4%132/22665.0%167/25763.1%217/344
Perplexity Decider Flash v1.2 Open · 4.0B—51.7%123/23839.2%80/20429.4%5/1740.8%64/15752.3%57/10957.5%77/13470.9%78/11063.4%64/10156.5%91/16157.1%60/10559.7%135/22661.9%159/25758.7%202/344
Open-Jev-27B-v1.1 Open · 27B—48.7%116/23836.3%74/20447.1%8/1748.4%76/15738.5%42/10947.0%63/13473.6%81/11068.3%69/10162.1%100/16153.3%56/10556.2%127/22661.5%158/25761.0%210/344
OneJev-27B Open · 27B—52.9%126/23832.8%67/20447.1%8/1743.9%69/15738.5%42/10947.0%63/13467.3%74/11065.3%66/10160.2%97/16155.2%58/10554.0%122/22659.9%154/25763.1%217/344
Mercury Decide Closed · Efficient—56.7%135/23836.3%74/20441.2%7/1738.9%61/15744.0%48/10944.0%59/13467.3%74/11075.2%76/10157.1%92/16156.2%59/10550.9%115/22658.8%151/25759.9%206/344
Verdin-4B-v1.1 Open · 4.0B—55.9%133/23834.3%70/20417.6%3/1733.8%53/15754.1%59/10944.0%59/13464.5%71/11060.4%61/10150.3%81/16145.7%48/10555.3%125/22659.9%154/25757.3%197/344
Clef Open · 27B—54.6%130/23836.3%74/20452.9%9/1738.9%61/15740.4%44/10936.6%49/13459.1%65/11057.4%58/10157.8%93/16149.5%52/10553.1%120/22654.1%139/25761.3%211/344
Chinese Decision Pointer 4B Open · 4.0B—47.1%112/23834.3%70/20435.3%6/1743.3%68/15739.4%43/10949.3%66/13462.7%69/11056.4%57/10156.5%91/16145.7%48/10549.1%111/22655.6%143/25760.2%207/344
Clef-Flash Open · 9.0B—49.6%118/23833.3%68/20452.9%9/1742.0%66/15743.1%47/10942.5%57/13457.3%63/11059.4%60/10157.8%93/16153.3%56/10549.1%111/22655.3%142/25754.4%187/344
Winnow-12B Open · 12B—51.3%122/23831.9%65/20441.2%7/1735.7%56/15738.5%42/10947.0%63/13461.8%68/11067.3%68/10152.2%84/16145.7%48/10545.6%103/22646.3%119/25749.1%169/344
Cygnet Open · 12B—43.7%104/23830.9%63/20441.2%7/1733.1%52/15739.4%43/10936.6%49/13450.9%56/11059.4%60/10148.4%78/16145.7%48/10549.6%112/22647.9%123/25747.7%164/344
Metask-Jev-4B Open · 4.5B—40.3%96/23831.4%64/20411.8%2/1730.6%48/15732.1%35/10932.1%43/13439.1%43/11042.6%43/10143.5%70/16134.3%36/10538.1%86/22647.9%123/25745.9%158/344
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—33.6%80/23825.0%51/20435.3%6/1732.5%51/15734.9%38/10932.8%44/13450.0%55/11054.5%55/10146.0%74/16146.7%49/10540.3%91/22643.6%112/25737.8%130/344
d1-2B Open · 1.9B—29.4%70/23833.3%68/20435.3%6/1737.6%59/15735.8%39/10934.3%46/13446.4%51/11039.6%40/10132.9%53/16139.0%41/10541.6%94/22640.5%104/25738.1%131/344
GLiDE 2B Open · 1.9B—32.8%78/23827.9%57/20423.5%4/1730.6%48/15736.7%40/10932.8%44/13440.9%45/11040.6%41/10138.5%62/16142.9%45/10535.8%81/22642.4%109/25741.0%141/344
KnowLine-2B-Gen1 Open · 2.0B—31.1%74/23825.5%52/20411.8%2/1729.9%47/15726.6%29/10935.1%47/13440.0%44/11051.5%52/10137.3%60/16135.2%37/10537.2%84/22638.1%98/25739.8%137/344
GLiDE 0.8B Open · 0.8B—30.3%72/23827.0%55/20429.4%5/1729.9%47/15733.9%37/10926.1%35/13426.4%29/11034.7%35/10136.0%58/16123.8%25/10529.6%67/22634.6%89/25740.7%140/344
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—27.7%66/23820.6%42/20429.4%5/1723.6%37/15735.8%39/10929.1%39/13438.2%42/11042.6%43/10136.0%58/16127.6%29/10530.5%69/22628.4%73/25729.1%100/344
Strands Decider 2B Open · 2.0B—29.8%71/23824.0%49/20423.5%4/1725.5%40/15730.3%33/10927.6%37/13430.9%34/11037.6%38/10131.7%51/16125.7%27/10529.6%67/22625.3%65/25730.2%104/344
OneJev1.6-0.8B Open · 0.8B—26.5%63/23822.5%46/20411.8%2/1725.5%40/15733.9%37/10925.4%34/13428.2%31/11028.7%29/10132.9%53/16128.6%30/10529.2%66/22624.1%62/25734.3%118/344
KnowLine-0.8B-Gen1 Open · 0.8B—27.3%65/23821.6%44/20435.3%6/1721.0%33/15731.2%34/10923.9%32/13430.9%34/11029.7%30/10133.5%54/16127.6%29/10530.1%68/22633.5%86/25727.6%95/344
GLiClass v4 Edge Open · 0.4B—22.7%54/23823.0%47/20429.4%5/1721.7%34/15730.3%33/10919.4%26/13424.5%27/11023.8%24/10118.0%29/16124.8%26/10522.1%50/22619.1%49/25725.0%86/344
GLiDE 0.4B Open · 0.4B—19.3%46/23817.6%36/2045.9%1/1723.6%37/15725.7%28/10918.7%25/13410.9%12/11022.8%23/10123.6%38/16119.0%20/10522.6%51/22618.3%47/25723.5%81/344

Share right of the questions each setup answered in each topic; under it right / answered.

SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 2,151 of 2,163); shown once.

Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.

Every setup, every measure

SetupWebAccuracy% of N (N ≤ 2,163) · Wilson 95%Tie groupHolm · McNemarRight / NquestionsKept at ≥99%right / wrongAUROCright vs wrongDecision Score (AA formula)0–100 · chosen-answer BrierNon-hallucinationof answers not right, % “I don't know” · higher is betterMedian latencymsp90ms, nearest rankCost$ per 1,000 qper 1M q
Karpo Decision 31B Open · 31B

Karpo Decision 31B · MachinePulse · decision model

Parameters: 31B dense

Accuracy 81.6% (95%: 79.9%–83.1%)

Logic 60.7%

Its run record (Method) →

—81.6%79.9%–83.1%a1,764 / 2,163837 right of 837 kept0 mistakes · 1,326 skipped0.90385.99chosen-answer Brier 0.09584.8%answered 99%155 ms186 ms$0.113$113estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen2 Open · 31B

KnowLine-31B-Gen2 · PelaAI · decision model

Parameters: 31B dense

Base model: Gemma 4 31B

Accuracy 80.3% (95%: 78.5%–81.9%)

Logic 58.9%

Its run record (Method) →

—80.3%78.5%–81.9%ab1,736 / 2,163469 right of 470 kept1 wrong · 1,693 skipped0.89885.05chosen-answer Brier 0.10160.2%answered 100%114 ms120 ms$0.083$83.14estimate · RTX PRO 6000 Blackwell
REAX 1.3 Open · 57B

REAX 1.3 · REAX · decision model

Parameters: 57.36B dense

Accuracy 78.0% (95%: 76.2%–79.7%)

Logic 59.3%

Its run record (Method) →

—78.0%76.2%–79.7%bc1,687 / 2,163178 right of 178 kept0 mistakes · 1,985 skipped0.90383.57chosen-answer Brier 0.10842.3%answered 99%189 ms232 ms$0.142$142estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen1.5 Open · 31B

KnowLine-31B-Gen1.5 · PelaAI · decision model

Parameters: 31B dense

Accuracy 77.4% (95%: 75.6%–79.1%)

Logic 59.0%

Its run record (Method) →

—77.4%75.6%–79.1%cd1,674 / 2,163464 right of 465 kept1 wrong · 1,698 skipped0.89583.18chosen-answer Brier 0.11031.1%answered 100%764 ms867 ms$0.047$46.87estimate · DGX Spark
Copernicus-31B Open · 31B

Copernicus-31B · Poindexter Labs · decision model

Parameters: 31B dense

Accuracy 77.3% (95%: 75.5%–79.1%)

Logic 58.7%

Its run record (Method) →

—77.3%75.5%–79.1%cd1,673 / 2,163318 right of 318 kept0 mistakes · 1,845 skipped0.89983.06chosen-answer Brier 0.11221.7%answered 100%118 ms142 ms$0.084$84.42estimate · RTX PRO 6000 Blackwell
Sieve 31B v3 Open · 31B

Sieve 31B v3 · Sthānika AI · decision model

Parameters: 31B dense

Accuracy 77.1% (95%: 75.3%–78.8%)

Logic 57.9%

Its run record (Method) →

—77.1%75.3%–78.8%cd1,668 / 2,163215 right of 215 kept0 mistakes · 1,948 skipped0.89182.89chosen-answer Brier 0.11332.1%answered 100%637 ms735 ms$0.039$39.14estimate · DGX Spark
Logits Sigma-1 Duo Open · 57B

Logits Sigma-1 Duo · Logits · decision model

Parameters: 57.36B dense

Accuracy 75.2% (95%: 73.4%–77.0%)

Logic 57.8%

Its run record (Method) →

—75.2%73.4%–77.0%de1,627 / 2,16315 right of 15 kept0 mistakes · 2,148 skipped0.89981.60chosen-answer Brier 0.12012.1%answered 99%196 ms221 ms$0.141$141estimate · RTX PRO 6000 Blackwell
d1-ultra Closed · Efficient

d1-ultra · Liquid AI · decision model

Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

Accuracy 74.1% (95%: 72.2%–75.9%)

Logic 54.9%

Its run record (Method) →

—74.1%72.2%–75.9%defg1,603 / 2,163213 right of 213 kept0 mistakes · 1,950 skipped0.87680.88chosen-answer Brier 0.123587.4%answered 61%179 ms426 ms$0.006$6.14
Mammoth-1 Open · 27B

Mammoth-1 · Base Labs · decision model

Parameters: 27B dense

Accuracy 74.0% (95%: 72.1%–75.8%)

Logic 54.5%

Its run record (Method) →

—74.0%72.1%–75.8%defh1,600 / 2,163412 right of 412 kept0 mistakes · 1,751 skipped0.89581.16chosen-answer Brier 0.116470.6%answered 75%140 ms173 ms$0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B

Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

Parameters: 27B dense

Accuracy 72.5% (95%: 70.6%–74.3%)

Logic 54.8%

Its run record (Method) →

—72.5%70.6%–74.3%efghi1,568 / 2,163323 right of 323 kept0 mistakes · 1,840 skipped0.85679.32chosen-answer Brier 0.13859.4%answered 97%91 ms103 ms$0.066$65.87estimate · RTX PRO 6000 Blackwell
GLiDE 27B Open · 27B

GLiDE 27B · Fastino · decision model

Parameters: 27B dense

Accuracy 72.0% (95%: 70.1%–73.8%)

Logic 58.4%

Its run record (Method) →

—72.0%70.1%–73.8%efghij1,557 / 2,163198 right of 199 kept1 wrong · 1,964 skipped0.86479.22chosen-answer Brier 0.135421.6%answered 94%425 ms461 ms$0.026$26.16estimate · DGX Spark
d3-ultra Open · 26B

d3-ultra · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Accuracy 71.5% (95%: 69.6%–73.4%)

Logic 54.6%

Its run record (Method) →

—71.5%69.6%–73.4%fghij1,547 / 2,16311 right of 11 kept0 mistakes · 2,152 skipped0.85778.39chosen-answer Brier 0.14745.3%answered 98%118 ms148 ms$0.089$88.58estimate · RTX PRO 6000 Blackwell
Gemini Decision 1 20261111 Closed · Efficient

Gemini Decision 1 20261111 · Google · decision model

Parameters: not disclosed by maker (closed weights)

Accuracy 71.3% (95%: 69.4%–73.2%)

Logic 50.0%

Its run record (Method) →

—71.3%69.4%–73.2%efghij1,543 / 2,163214 right of 214 kept0 mistakes · 1,949 skipped0.85978.84chosen-answer Brier 0.13663.9%answered 99%226 ms351 ms$0.022$22.02
GLiDE API Closed · Efficient

GLiDE API · Fastino · decision model

Parameters: not disclosed by maker

Accuracy 70.8% (95%: 68.8%–72.7%)

Logic 54.5%

Its run record (Method) →

—70.8%68.8%–72.7%fghij1,531 / 2,163161 right of 161 kept0 mistakes · 2,002 skipped0.86578.56chosen-answer Brier 0.136740.6%answered 86%264 ms279 ms$0.028$28.30
d3 Open · 26B

d3 · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Accuracy 70.4% (95%: 68.5%–72.3%)

Logic 54.5%

Its run record (Method) →

—70.4%68.5%–72.3%gij1,523 / 2,163162 right of 163 kept1 wrong · 2,000 skipped0.87878.71chosen-answer Brier 0.12994.7%answered 99%55 ms62 ms$0.041$40.61estimate · RTX PRO 6000 Blackwell
Drex Max Open · 27B

Drex Max · Nace.AI · decision model

Parameters: 27B dense

Accuracy 70.3% (95%: 68.4%–72.2%)

Logic 53.1%

Its run record (Method) →

—70.3%68.4%–72.2%gijk1,521 / 2,163719 right of 729 kept10 wrong · 1,434 skipped0.86177.10chosen-answer Brier 0.16114.5%answered 99%76 ms89 ms$0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell
Drex Max v1.1 Closed · Efficient (provisional)

Drex Max v1.1 · Nace.AI · decision model

Parameters: not disclosed by maker

Accuracy 70.1% (95%: 68.1%–72.0%)

Logic 54.8%

Its run record (Method) →

—70.1%68.1%–72.0%ijk1,516 / 2,163315 right of 315 kept0 mistakes · 1,848 skipped0.86778.01chosen-answer Brier 0.140743.5%answered 84%326 ms473 ms$0.013$12.88
celeris-1-decision Closed · Efficient

celeris-1-decision · Celeris · decision model

Parameters: not disclosed by maker

Accuracy 70.0% (95%: 68.0%–71.9%)

Logic 47.3%

Its run record (Method) →

—70.0%68.0%–71.9%hijk1,514 / 2,163No answers at this threshold0.85877.96chosen-answer Brier 0.140782.7%answered 62%387 ms391 ms$0.009$9.10
GLiDE v1.2 Open · 27B

GLiDE v1.2 · Fastino · decision model

Parameters: 27B dense

Accuracy 69.9% (95%: 67.9%–71.8%)

Logic 55.2%

Its run record (Method) →

—69.9%67.9%–71.8%ijk1,511 / 2,163131 right of 131 kept0 mistakes · 2,032 skipped0.86277.95chosen-answer Brier 0.139635.8%answered 88%79 ms84 ms$0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell
celeris-1.1-decision Closed · Efficient

celeris-1.1-decision · Celeris · decision model

Parameters: not disclosed by maker

Accuracy 69.9% (95%: 67.9%–71.8%)

Logic 47.3%

Its run record (Method) →

—69.9%67.9%–71.8%ijkl1,511 / 2,163No answers at this threshold0.85977.91chosen-answer Brier 0.140382.9%answered 62%381 ms386 ms$0.001$0.91
d1.2-ultra Closed · Undisclosed

d1.2-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 69.9% (95%: 67.9%–71.8%)

Logic 51.6%

Its run record (Method) →

—69.9%67.9%–71.8%ijk1,511 / 2,163115 right of 115 kept0 mistakes · 2,048 skipped0.87678.23chosen-answer Brier 0.134086.7%answered 58%215 ms347 ms$0.006$6.14
Perplexity Decider 27B v1.2 API Open · 27B

Perplexity Decider 27B v1.2 API · Perplexity · decision model

Parameters: 27B dense

Accuracy 69.8% (95%: 67.8%–71.7%)

Logic 53.3%

Its run record (Method) →

—69.8%67.8%–71.7%ijkl1,509 / 2,163331 right of 331 kept0 mistakes · 1,832 skipped0.86978.16chosen-answer Brier 0.13457.0%answered 98%226 ms368 ms$0.004$4.29
Perplexity Decider 27B v1.2 API · with web Open · 27B

Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

Parameters: 27B dense

Accuracy 69.3% (95%: 67.3%–71.2%)

Logic 52.5%

Its run record (Method) →

with web69.3%67.3%–71.2%jklm1,499 / 2,163326 right of 326 kept0 mistakes · 1,837 skipped0.87377.94chosen-answer Brier 0.134215.4%answered 95%422 ms641 ms$1.023$1,023
Turion Jev 31B Open · 31B

Turion Jev 31B · Turion.AI · decision model

Parameters: 31B dense

Accuracy 69.0% (95%: 67.0%–70.9%)

Logic 52.3%

Its run record (Method) →

—69.0%67.0%–70.9%ijklmn1,493 / 2,16343 right of 43 kept0 mistakes · 2,120 skipped0.86477.52chosen-answer Brier 0.13992.2%answered 99%278 ms329 ms$0.017$17.46estimate · DGX Spark
SubQ Snap 1.1 Closed · Efficient

SubQ Snap 1.1 · Subquadratic · decision model

Parameters: not disclosed by maker

Shown once with SubQ Snap Preview: the same answer to every question

Accuracy 68.6% (95%: 66.6%–70.5%)

Logic 52.4%

Its run record (Method) →

—68.6%66.6%–70.5%jklmn1,483 / 2,163360 right of 361 kept1 wrong · 1,802 skipped0.83976.15chosen-answer Brier 0.162652.5%answered 80%381 ms390 ms$0.009$8.59
KnowLine-12B-Gen1 Open · 12B

KnowLine-12B-Gen1 · PelaAI · decision model

Parameters: 12B dense

Base model: Gemma 4 12B

Accuracy 66.3% (95%: 64.2%–68.2%)

Logic 50.6%

Its run record (Method) →

—66.3%64.2%–68.2%klmno1,433 / 2,1637 right of 7 kept0 mistakes · 2,156 skipped0.81974.96chosen-answer Brier 0.16321.1%answered 100%49 ms55 ms$0.036$36.13estimate · RTX PRO 6000 Blackwell
d1.2 Closed · Undisclosed

d1.2 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 65.5% (95%: 63.5%–67.5%)

Logic 49.2%

Its run record (Method) →

—65.5%63.5%–67.5%lmnop1,417 / 2,163141 right of 141 kept0 mistakes · 2,022 skipped0.83874.98chosen-answer Brier 0.155656.3%answered 76%204 ms386 ms$0.003$2.92
d1.1 Closed · Efficient

d1.1 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 65.2% (95%: 63.2%–67.2%)

Logic 46.0%

Its run record (Method) →

—65.2%63.2%–67.2%mnop1,410 / 2,163183 right of 183 kept0 mistakes · 1,980 skipped0.82974.62chosen-answer Brier 0.159582.5%answered 56%172 ms290 ms$0.003$2.92
Jev 1.14 Closed · Efficient

Jev 1.14 · TypeSafe · decision model

Parameters: not disclosed by maker

Accuracy 65.1% (95%: 63.1%–67.1%)

Logic 48.2%

Its run record (Method) →

—65.1%63.1%–67.1%nop1,409 / 2,163202 right of 202 kept0 mistakes · 1,961 skipped0.82974.56chosen-answer Brier 0.16027.7%answered 97%163 ms185 ms$0.019$18.99
Jev 1.13 Closed · Efficient

Jev 1.13 · TypeSafe · decision model

Parameters: not disclosed by maker

Accuracy 64.1% (95%: 62.0%–66.1%)

Logic 46.3%

Its run record (Method) →

—64.1%62.0%–66.1%opq1,386 / 2,163248 right of 248 kept0 mistakes · 1,915 skipped0.83474.08chosen-answer Brier 0.15925.7%answered 98%166 ms206 ms$0.019$18.99
Adjudex 9B v5 Open · 9.0B

Adjudex 9B v5 · newsbang · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 62.1% (95%: 60.0%–64.1%)

Logic 46.4%

Its run record (Method) →

—62.1%60.0%–64.1%opqr1,343 / 2,163197 right of 198 kept1 wrong · 1,965 skipped0.80672.28chosen-answer Brier 0.17523.1%answered 99%36 ms44 ms$0.027$26.54estimate · RTX PRO 6000 Blackwell
Elephant-1 Open · 9.0B

Elephant-1 · Base Labs · decision model

Parameters: 9B dense

Accuracy 61.3% (95%: 59.3%–63.4%)

Logic 46.4%

Its run record (Method) →

—61.3%59.3%–63.4%pqr1,327 / 2,163182 right of 183 kept1 wrong · 1,980 skipped0.80371.86chosen-answer Brier 0.176365.0%answered 67%80 ms92 ms$0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen2 Open · 9.0B

KnowLine-9B-Gen2 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 60.1% (95%: 58.0%–62.1%)

Logic 43.9%

Its run record (Method) →

—60.1%58.0%–62.1%qrs1,300 / 2,163No answers at this threshold0.80371.15chosen-answer Brier 0.17793.2%answered 99%41 ms46 ms$0.031$31.02estimate · RTX PRO 6000 Blackwell
Jeffan-9B-1211 Open · 9.0B

Jeffan-9B-1211 · tattrongvu · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 59.8% (95%: 57.7%–61.9%)

Logic 48.2%

Its run record (Method) →

—59.8%57.7%–61.9%qrs1,294 / 2,16364 right of 64 kept0 mistakes · 2,099 skipped0.81471.32chosen-answer Brier 0.17196.6%answered 97%37 ms51 ms$0.029$28.78estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen1 Open · 9.0B

KnowLine-9B-Gen1 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 59.8% (95%: 57.7%–61.8%)

Logic 44.3%

Its run record (Method) →

—59.8%57.7%–61.8%qrs1,293 / 2,163123 right of 123 kept0 mistakes · 2,040 skipped0.79971.08chosen-answer Brier 0.17620.9%answered 100%37 ms43 ms$0.027$26.54estimate · RTX PRO 6000 Blackwell
Microsoft-Decision-1 Closed · Efficient

Microsoft-Decision-1 · Microsoft · decision model

Parameters: not disclosed by maker

Base model: Qwen3.5-9B

Accuracy 59.4% (95%: 57.3%–61.4%)

Logic 44.4%

Its run record (Method) →

—59.4%57.3%–61.4%rst1,284 / 2,16327 right of 27 kept0 mistakes · 2,136 skipped0.78770.49chosen-answer Brier 0.18395.1%answered 98%310 ms359 ms$0.007$6.81
Drex 2.0 Open · 35B MoE · 3.0B active

Drex 2.0 · Nace.AI · decision model

Parameters: 35B MoE · 3B active

Accuracy 59.0% (95%: 56.9%–61.0%)

Logic 41.9%

Its run record (Method) →

—59.0%56.9%–61.0%rstu1,276 / 2,163174 right of 174 kept0 mistakes · 1,989 skipped0.81370.98chosen-answer Brier 0.170320.7%answered 91%334 ms432 ms$0.009$8.92
Dexy-Decision-9B-V2 Open · 9.0B

Dexy-Decision-9B-V2 · Patrick-18 · decision model

Parameters: 9B dense

Base model: vllm-sr/d3-flash

Accuracy 58.8% (95%: 56.7%–60.9%)

Logic 45.3%

Its run record (Method) →

—58.8%56.7%–60.9%rstu1,272 / 2,163207 right of 210 kept3 wrong · 1,953 skipped0.80469.67chosen-answer Brier 0.19461.7%answered 99%45 ms50 ms$0.032$31.98estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-9B-Instant Open · 9.0B

StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

Parameters: 9B dense

Accuracy 58.4% (95%: 56.3%–60.5%)

Logic 46.6%

Its run record (Method) →

—58.4%56.3%–60.5%rstuv1,264 / 2,16349 right of 49 kept0 mistakes · 2,114 skipped0.78169.90chosen-answer Brier 0.18644.2%answered 98%140 ms146 ms$0.008$8.34estimate · DGX Spark
GLiDE 9B Open · 7.9B

GLiDE 9B · Fastino · decision model

Parameters: 7.94B dense

Accuracy 57.9% (95%: 55.8%–60.0%)

Logic 47.8%

Its run record (Method) →

—57.9%55.8%–60.0%rstuvw1,253 / 2,16373 right of 73 kept0 mistakes · 2,090 skipped0.79869.69chosen-answer Brier 0.185524.6%answered 89%174 ms182 ms$0.010$10.49estimate · DGX Spark
vera-medium-omni Open · 4.2B

vera-medium-omni · scar-ai · decision model

Parameters: 4.2B dense

Base model: Qwen3.5-4B

Accuracy 56.8% (95%: 54.7%–58.9%)

Logic 44.1%

Its run record (Method) →

—56.8%54.7%–58.9%stuvw1,229 / 2,16333 right of 33 kept0 mistakes · 2,130 skipped0.78969.18chosen-answer Brier 0.18463.8%answered 98%31 ms35 ms$0.023$23.34estimate · RTX PRO 6000 Blackwell
Cerno 9B v1.2 Open · 9.0B

Cerno 9B v1.2 · Apertix AI · decision model

Parameters: 9B dense

Accuracy 56.0% (95%: 53.9%–58.1%)

Logic 43.3%

Its run record (Method) →

—56.0%53.9%–58.1%stuvw1,212 / 2,163244 right of 254 kept10 wrong · 1,909 skipped0.78067.15chosen-answer Brier 0.21741.2%answered 99%153 ms175 ms$0.010$9.51estimate · DGX Spark
KnowLine 4B Gen5.5 Open · 4.0B

KnowLine 4B Gen5.5 · PelaAI · decision model

Parameters: 4B dense

Accuracy 56.0% (95%: 53.9%–58.1%)

Logic 42.3%

Its run record (Method) →

—56.0%53.9%–58.1%stuvw1,211 / 2,16354 right of 54 kept0 mistakes · 2,109 skipped0.76567.89chosen-answer Brier 0.20211.6%answered 99%74 ms84 ms$0.005$4.56estimate · DGX Spark
Perplexity Decider Flash v1.3 Open · 4.0B

Perplexity Decider Flash v1.3 · Perplexity · decision model

Parameters: 4B dense

Accuracy 55.9% (95%: 53.8%–58.0%)

Logic 44.7%

Its run record (Method) →

—55.9%53.8%–58.0%stuvw1,210 / 2,163113 right of 114 kept1 wrong · 2,049 skipped0.78767.89chosen-answer Brier 0.20158.4%answered 96%28 ms31 ms$0.022$22.38estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-4B-Instant Open · 4.0B

StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

Parameters: 4B dense

Accuracy 55.8% (95%: 53.7%–57.9%)

Logic 45.9%

Its run record (Method) →

—55.8%53.7%–57.9%stuvwx1,208 / 2,16396 right of 96 kept0 mistakes · 2,067 skipped0.77968.25chosen-answer Brier 0.193413.4%answered 94%67 ms69 ms$0.004$4.10estimate · DGX Spark
d1 Closed · Efficient

d1 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 55.6% (95%: 53.5%–57.7%)

Logic 40.8%

Its run record (Method) →

—55.6%53.5%–57.7%stuvwxy1,203 / 2,163152 right of 152 kept0 mistakes · 2,011 skipped0.76768.16chosen-answer Brier 0.19302.9%answered 99%184 ms297 ms$0.003$2.92
Valen-Decision-1.5-9B Open · 9.0B

Valen-Decision-1.5-9B · Valen-Team · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 55.3% (95%: 53.2%–57.4%)

Logic 44.4%

Its run record (Method) →

—55.3%53.2%–57.4%tuvwxy1,197 / 2,163130 right of 130 kept0 mistakes · 2,033 skipped0.79068.37chosen-answer Brier 0.18602.6%answered 99%58 ms64 ms$0.041$40.93estimate · RTX PRO 6000 Blackwell
Perplexity Decider Flash v1.2 Open · 4.0B

Perplexity Decider Flash v1.2 · Perplexity · decision model

Parameters: 4B dense

Accuracy 55.2% (95%: 53.1%–57.3%)

Logic 43.1%

Its run record (Method) →

—55.2%53.1%–57.3%tuvwxy1,195 / 2,163150 right of 155 kept5 wrong · 2,008 skipped0.78167.61chosen-answer Brier 0.200212.0%answered 94%40 ms47 ms$0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell
Open-Jev-27B-v1.1 Open · 27B

Open-Jev-27B-v1.1 · Zefan Cai · decision model

Parameters: 27B dense

Base model: Qwen3.8-27B + adapter

Accuracy 54.6% (95%: 52.4%–56.6%)

Logic 42.6%

Its run record (Method) →

—54.6%52.4%–56.6%uvwxyz1,180 / 2,163No answers at this threshold0.76464.53chosen-answer Brier 0.25495.0%answered 98%233 ms272 ms$0.169$169estimate · RTX PRO 6000 Blackwell
OneJev-27B Open · 27B

OneJev-27B · OmniJev · decision model

Parameters: 27B dense

Accuracy 53.8% (95%: 51.7%–55.9%)

Logic 40.6%

Its run record (Method) →

—53.8%51.7%–55.9%vwxyzA1,163 / 2,16326 right of 26 kept0 mistakes · 2,137 skipped0.78367.54chosen-answer Brier 0.18691.8%answered 99%79 ms106 ms$0.061$60.76estimate · RTX PRO 6000 Blackwell
Mercury Decide Closed · Efficient

Mercury Decide · Inception · decision model

Parameters: not disclosed by maker

Accuracy 53.5% (95%: 51.4%–55.6%)

Logic 45.1%

Its run record (Method) →

—53.5%51.4%–55.6%wxyzA1,157 / 2,163272 right of 281 kept9 wrong · 1,882 skipped0.78966.02chosen-answer Brier 0.21463.2%answered 99%288 ms342 ms$0.004$4.38
Verdin-4B-v1.1 Open · 4.0B

Verdin-4B-v1.1 · victorbennett · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Accuracy 51.5% (95%: 49.4%–53.6%)

Logic 39.7%

Its run record (Method) →

—51.5%49.4%–53.6%yzAB1,114 / 2,163126 right of 130 kept4 wrong · 2,033 skipped0.75265.62chosen-answer Brier 0.20266.4%answered 97%47 ms52 ms$0.034$33.58estimate · RTX PRO 6000 Blackwell
Clef Open · 27B

Clef · Cloudflare · decision model

Parameters: 27B dense

Accuracy 51.1% (95%: 49.0%–53.2%)

Logic 40.8%

Its run record (Method) →

—51.1%49.0%–53.2%xyzAB1,105 / 2,16312 right of 12 kept0 mistakes · 2,151 skipped0.76765.20chosen-answer Brier 0.20701.6%answered 99%181 ms210 ms$0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell
Chinese Decision Pointer 4B Open · 4.0B

Chinese Decision Pointer 4B · quincyqiang · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Accuracy 50.4% (95%: 48.3%–52.5%)

Logic 40.4%

Its run record (Method) →

—50.4%48.3%–52.5%zAB1,091 / 2,163149 right of 155 kept6 wrong · 2,008 skipped0.74664.11chosen-answer Brier 0.22213.4%answered 98%28 ms32 ms$0.023$22.70estimate · RTX PRO 6000 Blackwell
Clef-Flash Open · 9.0B

Clef-Flash · Cloudflare · decision model

Parameters: 9B dense

Accuracy 49.8% (95%: 47.7%–51.9%)

Logic 38.9%

Its run record (Method) →

—49.8%47.7%–51.9%AB1,077 / 2,163No answers at this threshold0.73064.44chosen-answer Brier 0.20922.0%answered 99%99 ms127 ms$0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell
Winnow-12B Open · 12B

Winnow-12B · EldanRing · decision model

Parameters: 12B dense

Accuracy 46.9% (95%: 44.8%–49.0%)

Logic 39.7%

Its run record (Method) →

—46.9%44.8%–49.0%BC1,014 / 2,16320 right of 21 kept1 wrong · 2,142 skipped0.74363.00chosen-answer Brier 0.20879.2%answered 95%52 ms61 ms$0.038$38.37estimate · RTX PRO 6000 Blackwell
Cygnet Open · 12B

Cygnet · Blockbrain Labs · decision model

Parameters: 11.95B dense

Base model: Gemma 4 12B IT

Accuracy 44.3% (95%: 42.3%–46.4%)

Logic 39.3%

Its run record (Method) →

—44.3%42.3%–46.4%C959 / 2,163No answers at this threshold0.73561.87chosen-answer Brier 0.20604.9%answered 97%37 ms41 ms$0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell
Metask-Jev-4B Open · 4.5B

Metask-Jev-4B · Metask Lab · decision model

Parameters: 4.54B dense

Accuracy 39.2% (95%: 37.1%–41.2%)

Logic 32.1%

Its run record (Method) →

—39.2%37.1%–41.2%D847 / 2,163No answers at this threshold0.65158.63chosen-answer Brier 0.21902.2%answered 99%113 ms144 ms$0.083$83.14estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-2B-Instant Open · 2.0B

StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

Parameters: 2B dense

Accuracy 38.7% (95%: 36.6%–40.7%)

Logic 30.8%

Its run record (Method) →

—38.7%36.6%–40.7%D836 / 2,1632 right of 2 kept0 mistakes · 2,161 skipped0.68558.95chosen-answer Brier 0.20757.2%answered 95%36 ms37 ms$0.002$2.27estimate · DGX Spark
d1-2B Open · 1.9B

d1-2B · Liquid AI · decision model

Parameters: 1.92B dense

Base model: LFM3-VL-2B

Accuracy 37.1% (95%: 35.1%–39.1%)

Logic 30.7%

Its run record (Method) →

—37.1%35.1%–39.1%D802 / 2,1634 right of 4 kept0 mistakes · 2,159 skipped0.65657.72chosen-answer Brier 0.216380.2%answered 34%16 ms21 ms$0.014$14.07estimate · RTX PRO 6000 Blackwell
GLiDE 2B Open · 1.9B

GLiDE 2B · Fastino · decision model

Parameters: 1.88B dense

Accuracy 36.8% (95%: 34.7%–38.8%)

Logic 32.3%

Its run record (Method) →

—36.8%34.7%–38.8%D795 / 2,16315 right of 16 kept1 wrong · 2,147 skipped0.63555.70chosen-answer Brier 0.253617.3%answered 89%28 ms30 ms$0.002$1.78estimate · DGX Spark
KnowLine-2B-Gen1 Open · 2.0B

KnowLine-2B-Gen1 · PelaAI · decision model

Parameters: 2B dense

Base model: Qwen3.5-2B

Accuracy 35.3% (95%: 33.3%–37.3%)

Logic 31.7%

Its run record (Method) →

—35.3%33.3%–37.3%DE763 / 2,1638 right of 8 kept0 mistakes · 2,155 skipped0.63656.49chosen-answer Brier 0.22291.7%answered 99%50 ms54 ms$0.003$3.20estimate · DGX Spark
GLiDE 0.8B Open · 0.8B

GLiDE 0.8B · Fastino · decision model

Parameters: 0.75B dense

Accuracy 32.1% (95%: 30.2%–34.1%)

Logic 29.2%

Its run record (Method) →

—32.1%30.2%–34.1%EF694 / 2,1634 right of 4 kept0 mistakes · 2,159 skipped0.58953.21chosen-answer Brier 0.256620.8%answered 85%20 ms21 ms$0.001$1.28estimate · DGX Spark
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B

StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Accuracy 29.7% (95%: 27.8%–31.6%)

Logic 24.7%

Its run record (Method) →

—29.7%27.8%–31.6%F642 / 2,163No answers at this threshold0.58354.40chosen-answer Brier 0.208811.3%answered 92%24 ms25 ms$0.002$1.53estimate · DGX Spark
Strands Decider 2B Open · 2.0B

Strands Decider 2B · Strands Agents · decision model

Parameters: 2B dense

Accuracy 28.7% (95%: 26.8%–30.6%)

Logic 22.4%

Its run record (Method) →

—28.7%26.8%–30.6%F620 / 2,163No answers at this threshold0.56953.90chosen-answer Brier 0.20863.2%answered 98%35 ms41 ms$0.026$26.22estimate · RTX PRO 6000 Blackwell
OneJev1.6-0.8B Open · 0.8B

OneJev1.6-0.8B · OmniJev · decision model

Parameters: 0.8B dense

Accuracy 28.2% (95%: 26.4%–30.2%)

Logic 27.0%

Its run record (Method) →

—28.2%26.4%–30.2%F611 / 2,163No answers at this threshold0.59253.62chosen-answer Brier 0.21000.6%answered 100%15 ms17 ms$0.001$1.09estimate · DGX Spark
KnowLine-0.8B-Gen1 Open · 0.8B

KnowLine-0.8B-Gen1 · PelaAI · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Accuracy 28.2% (95%: 26.3%–30.1%)

Logic 26.7%

Its run record (Method) →

—28.2%26.3%–30.1%F610 / 2,163No answers at this threshold0.59653.60chosen-answer Brier 0.21000.3%answered 100%25 ms28 ms$0.002$1.69estimate · DGX Spark
GLiClass v4 Edge Open · 0.4B

GLiClass v4 Edge · decision model

Parameters: 0.4B dense

Base model: DeBERTa-v3-large

Accuracy 22.7% (95%: 20.9%–24.5%)

Logic 23.6%

Its run record (Method) →

—22.7%20.9%–24.5%G490 / 2,163No answers at this threshold0.50152.01chosen-answer Brier 0.186317.2%answered 86%12 ms13 ms$0.001$0.85estimate · DGX Spark
GLiDE 0.4B Open · 0.4B

GLiDE 0.4B · Fastino · decision model

Parameters: 0.40B dense

Accuracy 20.6% (95%: 18.9%–22.3%)

Logic 23.2%

Its run record (Method) →

—20.6%18.9%–22.3%G445 / 2,163No answers at this threshold0.50050.98chosen-answer Brier 0.186272.8%answered 33%11 ms12 ms$0.001$0.77estimate · DGX Spark

Tie groups and “not measured”

estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.

Decision Score (AA formula): Artificial Analysis's formula applied to our questions. Not their Index. 50 × (1 + accuracy − chosen-answer Brier), the chosen-answer Brier over the probability of the chosen answer, on decision models' standard runs only. Chat models and the “I don't know” runs are excluded.