Logic · Accuracy

Accuracy on Logic

1,125 questions, asked in English. How to read this page · Method

Accuracy

Share of the questions answered right, with the Wilson 95% interval and the tie-group letters · higher is better; the line is chance (20%)

  1. Karpo Decision 31B

    Karpo Decision 31B · MachinePulse · decision model

    Parameters: 31B dense

    Accuracy 60.7% (95%: 57.8%–63.5%)

    Math 81.6%

    Its run record (Method) →

    60.7% (58%–64%) ab
  2. REAX 1.3

    REAX 1.3 · REAX · decision model

    Parameters: 57.36B dense

    Accuracy 59.3% (95%: 56.4%–62.1%)

    Math 78.0%

    Its run record (Method) →

    59.3% (56%–62%) ac
  3. KnowLine-31B-Gen1.5

    KnowLine-31B-Gen1.5 · PelaAI · decision model

    Parameters: 31B dense

    Accuracy 59.0% (95%: 56.1%–61.9%)

    Math 77.4%

    Its run record (Method) →

    59.0% (56%–62%) abcd
  4. KnowLine-31B-Gen2

    KnowLine-31B-Gen2 · PelaAI · decision model

    Parameters: 31B dense

    Base model: Gemma 4 31B

    Accuracy 58.9% (95%: 56.0%–61.8%)

    Math 80.3%

    Its run record (Method) →

    58.9% (56%–62%) abcd
  5. Copernicus-31B

    Copernicus-31B · Poindexter Labs · decision model

    Parameters: 31B dense

    Accuracy 58.7% (95%: 55.8%–61.5%)

    Math 77.3%

    Its run record (Method) →

    58.7% (56%–62%) abcd
  6. GLiDE 27B

    GLiDE 27B · Fastino · decision model

    Parameters: 27B dense

    Accuracy 58.4% (95%: 55.5%–61.2%)

    Math 72.0%

    Its run record (Method) →

    58.4% (55%–61%) abcde
  7. Sieve 31B v3

    Sieve 31B v3 · Sthānika AI · decision model

    Parameters: 31B dense

    Accuracy 57.9% (95%: 55.0%–60.7%)

    Math 77.1%

    Its run record (Method) →

    57.9% (55%–61%) abcdf
  8. Logits Sigma-1 Duo

    Logits Sigma-1 Duo · Logits · decision model

    Parameters: 57.36B dense

    Accuracy 57.8% (95%: 54.9%–60.6%)

    Math 75.2%

    Its run record (Method) →

    57.8% (55%–61%) abcdefg
  9. GLiDE v1.2

    GLiDE v1.2 · Fastino · decision model

    Parameters: 27B dense

    Accuracy 55.2% (95%: 52.3%–58.1%)

    Math 69.9%

    Its run record (Method) →

    55.2% (52%–58%) abcdefgh
  10. d1-ultra

    d1-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

    Accuracy 54.9% (95%: 52.0%–57.8%)

    Math 74.1%

    Its run record (Method) →

    54.9% (52%–58%) abcdefgh
  11. Drex Max v1.1

    Drex Max v1.1 · Nace.AI · decision model

    Parameters: not disclosed by maker

    Accuracy 54.8% (95%: 51.8%–57.6%)

    Math 70.1%

    Its run record (Method) →

    54.8% (52%–58%) abcdefghi
  12. Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell

    Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 54.8% (95%: 51.8%–57.6%)

    Math 72.5%

    Its run record (Method) →

    54.8% (52%–58%) abcdefgh
  13. d3-ultra

    d3-ultra · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Accuracy 54.6% (95%: 51.7%–57.5%)

    Math 71.5%

    Its run record (Method) →

    54.6% (52%–57%) bdefghij
  14. GLiDE API

    GLiDE API · Fastino · decision model

    Parameters: not disclosed by maker

    Accuracy 54.5% (95%: 51.6%–57.4%)

    Math 70.8%

    Its run record (Method) →

    54.5% (52%–57%) cdefghi
  15. Mammoth-1

    Mammoth-1 · Base Labs · decision model

    Parameters: 27B dense

    Accuracy 54.5% (95%: 51.6%–57.4%)

    Math 74.0%

    Its run record (Method) →

    54.5% (52%–57%) cdefghij
  16. d3

    d3 · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Accuracy 54.5% (95%: 51.6%–57.4%)

    Math 70.4%

    Its run record (Method) →

    54.5% (52%–57%) cdefgh
  17. Perplexity Decider 27B v1.2 API

    Perplexity Decider 27B v1.2 API · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 53.3% (95%: 50.4%–56.2%)

    Math 69.8%

    Its run record (Method) →

    53.3% (50%–56%) defghijk
  18. Drex Max

    Drex Max · Nace.AI · decision model

    Parameters: 27B dense

    Accuracy 53.1% (95%: 50.1%–56.0%)

    Math 70.3%

    Its run record (Method) →

    53.1% (50%–56%) defghijk
  19. Perplexity Decider 27B v1.2 API · with web

    Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

    Parameters: 27B dense

    Accuracy 52.5% (95%: 49.6%–55.4%)

    Math 69.3%

    Its run record (Method) →

    52.5% (50%–55%) fghijkl
  20. SubQ Snap 1.1

    SubQ Snap 1.1 · Subquadratic · decision model

    Parameters: not disclosed by maker

    Shown once with SubQ Snap Preview: the same answer to every question

    Accuracy 52.4% (95%: 49.4%–55.3%)

    Math 68.6%

    Its run record (Method) →

    52.4% (49%–55%) fghijklm
  21. Turion Jev 31B

    Turion Jev 31B · Turion.AI · decision model

    Parameters: 31B dense

    Accuracy 52.3% (95%: 49.3%–55.2%)

    Math 69.0%

    Its run record (Method) →

    52.3% (49%–55%) eghijklmn
  22. d1.2-ultra

    d1.2-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 51.6% (95%: 48.6%–54.5%)

    Math 69.9%

    Its run record (Method) →

    51.6% (49%–54%) hijklmno
  23. KnowLine-12B-Gen1

    KnowLine-12B-Gen1 · PelaAI · decision model

    Parameters: 12B dense

    Base model: Gemma 4 12B

    Accuracy 50.6% (95%: 47.7%–53.5%)

    Math 66.3%

    Its run record (Method) →

    50.6% (48%–53%) hijklmnop
  24. Gemini Decision 1 20261111

    Gemini Decision 1 20261111 · Google · decision model

    Parameters: not disclosed by maker (closed weights)

    Accuracy 50.0% (95%: 47.1%–53.0%)

    Math 71.3%

    Its run record (Method) →

    50.0% (47%–53%) hijklmnopq
  25. d1.2

    d1.2 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 49.2% (95%: 46.2%–52.1%)

    Math 65.5%

    Its run record (Method) →

    49.2% (46%–52%) hijklmnopqr
  26. Jeffan-9B-1211

    Jeffan-9B-1211 · tattrongvu · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 48.2% (95%: 45.3%–51.1%)

    Math 59.8%

    Its run record (Method) →

    48.2% (45%–51%) jklmnopqrs
  27. Jev 1.14

    Jev 1.14 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Accuracy 48.2% (95%: 45.3%–51.1%)

    Math 65.1%

    Its run record (Method) →

    48.2% (45%–51%) ijklmnopqrs
  28. GLiDE 9B

    GLiDE 9B · Fastino · decision model

    Parameters: 7.94B dense

    Accuracy 47.8% (95%: 44.9%–50.7%)

    Math 57.9%

    Its run record (Method) →

    47.8% (45%–51%) klmnopqrs
  29. celeris-1-decision

    celeris-1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Accuracy 47.3% (95%: 44.4%–50.2%)

    Math 70.0%

    Its run record (Method) →

    47.3% (44%–50%) klmnopqrstu
  30. celeris-1.1-decision

    celeris-1.1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Accuracy 47.3% (95%: 44.4%–50.2%)

    Math 69.9%

    Its run record (Method) →

    47.3% (44%–50%) klmnopqrst
  31. StartLux-Decision-v1.1-9B-Instant

    StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

    Parameters: 9B dense

    Accuracy 46.6% (95%: 43.7%–49.5%)

    Math 58.4%

    Its run record (Method) →

    46.6% (44%–49%) lmnopqrstuv
  32. Adjudex 9B v5

    Adjudex 9B v5 · newsbang · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 46.4% (95%: 43.5%–49.3%)

    Math 62.1%

    Its run record (Method) →

    46.4% (44%–49%) lmnopqrstuv
  33. Elephant-1

    Elephant-1 · Base Labs · decision model

    Parameters: 9B dense

    Accuracy 46.4% (95%: 43.5%–49.3%)

    Math 61.3%

    Its run record (Method) →

    46.4% (44%–49%) lmnopqrstuv
  34. Jev 1.13

    Jev 1.13 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Accuracy 46.3% (95%: 43.4%–49.2%)

    Math 64.1%

    Its run record (Method) →

    46.3% (43%–49%) lmnopqrstuv
  35. d1.1

    d1.1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 46.0% (95%: 43.2%–49.0%)

    Math 65.2%

    Its run record (Method) →

    46.0% (43%–49%) mnopqrstuvw
  36. StartLux-Decision-v1.1-4B-Instant

    StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

    Parameters: 4B dense

    Accuracy 45.9% (95%: 43.0%–48.8%)

    Math 55.8%

    Its run record (Method) →

    45.9% (43%–49%) nopqrstuvx
  37. Dexy-Decision-9B-V2

    Dexy-Decision-9B-V2 · Patrick-18 · decision model

    Parameters: 9B dense

    Base model: vllm-sr/d3-flash

    Accuracy 45.3% (95%: 42.4%–48.3%)

    Math 58.8%

    Its run record (Method) →

    45.3% (42%–48%) opqrstuvwxy
  38. Mercury Decide

    Mercury Decide · Inception · decision model

    Parameters: not disclosed by maker

    Accuracy 45.1% (95%: 42.2%–48.0%)

    Math 53.5%

    Its run record (Method) →

    45.1% (42%–48%) pqrstuvwxy
  39. Perplexity Decider Flash v1.3

    Perplexity Decider Flash v1.3 · Perplexity · decision model

    Parameters: 4B dense

    Accuracy 44.7% (95%: 41.8%–47.6%)

    Math 55.9%

    Its run record (Method) →

    44.7% (42%–48%) pqrstuvwxy
  40. Microsoft-Decision-1

    Microsoft-Decision-1 · Microsoft · decision model

    Parameters: not disclosed by maker

    Base model: Qwen3.5-9B

    Accuracy 44.4% (95%: 41.6%–47.4%)

    Math 59.4%

    Its run record (Method) →

    44.4% (42%–47%) pqrstuvwxy
  41. Valen-Decision-1.5-9B

    Valen-Decision-1.5-9B · Valen-Team · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 44.4% (95%: 41.5%–47.3%)

    Math 55.3%

    Its run record (Method) →

    44.4% (41%–47%) pqrstuvwxy
  42. KnowLine-9B-Gen1

    KnowLine-9B-Gen1 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 44.3% (95%: 41.4%–47.2%)

    Math 59.8%

    Its run record (Method) →

    44.3% (41%–47%) pqrstuvwxy
  43. vera-medium-omni

    vera-medium-omni · scar-ai · decision model

    Parameters: 4.2B dense

    Base model: Qwen3.5-4B

    Accuracy 44.1% (95%: 41.2%–47.0%)

    Math 56.8%

    Its run record (Method) →

    44.1% (41%–47%) qrstuvwxy
  44. KnowLine-9B-Gen2

    KnowLine-9B-Gen2 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Accuracy 43.9% (95%: 41.0%–46.8%)

    Math 60.1%

    Its run record (Method) →

    43.9% (41%–47%) qrstuvwxy
  45. Cerno 9B v1.2

    Cerno 9B v1.2 · Apertix AI · decision model

    Parameters: 9B dense

    Accuracy 43.3% (95%: 40.4%–46.2%)

    Math 56.0%

    Its run record (Method) →

    43.3% (40%–46%) rstuvwxy
  46. Perplexity Decider Flash v1.2

    Perplexity Decider Flash v1.2 · Perplexity · decision model

    Parameters: 4B dense

    Accuracy 43.1% (95%: 40.2%–46.0%)

    Math 55.2%

    Its run record (Method) →

    43.1% (40%–46%) qrstuvwxy
  47. Open-Jev-27B-v1.1

    Open-Jev-27B-v1.1 · Zefan Cai · decision model

    Parameters: 27B dense

    Base model: Qwen3.8-27B + adapter

    Accuracy 42.6% (95%: 39.7%–45.5%)

    Math 54.6%

    Its run record (Method) →

    42.6% (40%–45%) rstuvwxy
  48. KnowLine 4B Gen5.5

    KnowLine 4B Gen5.5 · PelaAI · decision model

    Parameters: 4B dense

    Accuracy 42.3% (95%: 39.5%–45.2%)

    Math 56.0%

    Its run record (Method) →

    42.3% (39%–45%) stuvwxy
  49. Drex 2.0

    Drex 2.0 · Nace.AI · decision model

    Parameters: 35B MoE · 3B active

    Accuracy 41.9% (95%: 39.0%–44.8%)

    Math 59.0%

    Its run record (Method) →

    41.9% (39%–45%) stuvwxy
  50. Clef

    Clef · Cloudflare · decision model

    Parameters: 27B dense

    Accuracy 40.8% (95%: 38.0%–43.7%)

    Math 51.1%

    Its run record (Method) →

    40.8% (38%–44%) uvwxy
  51. d1

    d1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Accuracy 40.8% (95%: 38.0%–43.7%)

    Math 55.6%

    Its run record (Method) →

    40.8% (38%–44%) tuvwxy
  52. OneJev-27B

    OneJev-27B · OmniJev · decision model

    Parameters: 27B dense

    Accuracy 40.6% (95%: 37.8%–43.5%)

    Math 53.8%

    Its run record (Method) →

    40.6% (38%–44%) tuvwxy
  53. Chinese Decision Pointer 4B

    Chinese Decision Pointer 4B · quincyqiang · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Accuracy 40.4% (95%: 37.5%–43.3%)

    Math 50.4%

    Its run record (Method) →

    40.4% (38%–43%) vwxy
  54. Verdin-4B-v1.1

    Verdin-4B-v1.1 · victorbennett · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Accuracy 39.7% (95%: 36.9%–42.6%)

    Math 51.5%

    Its run record (Method) →

    39.7% (37%–43%) wy
  55. Winnow-12B

    Winnow-12B · EldanRing · decision model

    Parameters: 12B dense

    Accuracy 39.7% (95%: 36.9%–42.6%)

    Math 46.9%

    Its run record (Method) →

    39.7% (37%–43%) xy
  56. Cygnet

    Cygnet · Blockbrain Labs · decision model

    Parameters: 11.95B dense

    Base model: Gemma 4 12B IT

    Accuracy 39.3% (95%: 36.5%–42.2%)

    Math 44.3%

    Its run record (Method) →

    39.3% (36%–42%) xy
  57. Clef-Flash

    Clef-Flash · Cloudflare · decision model

    Parameters: 9B dense

    Accuracy 38.9% (95%: 36.1%–41.8%)

    Math 49.8%

    Its run record (Method) →

    38.9% (36%–42%) yz
  58. GLiDE 2B

    GLiDE 2B · Fastino · decision model

    Parameters: 1.88B dense

    Accuracy 32.3% (95%: 29.6%–35.1%)

    Math 36.8%

    Its run record (Method) →

    32.3% (30%–35%) zA
  59. Metask-Jev-4B

    Metask-Jev-4B · Metask Lab · decision model

    Parameters: 4.54B dense

    Accuracy 32.1% (95%: 29.4%–34.9%)

    Math 39.2%

    Its run record (Method) →

    32.1% (29%–35%) zA
  60. KnowLine-2B-Gen1

    KnowLine-2B-Gen1 · PelaAI · decision model

    Parameters: 2B dense

    Base model: Qwen3.5-2B

    Accuracy 31.7% (95%: 29.1%–34.5%)

    Math 35.3%

    Its run record (Method) →

    31.7% (29%–35%) A
  61. StartLux-Decision-v1.1-2B-Instant

    StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

    Parameters: 2B dense

    Accuracy 30.8% (95%: 28.1%–33.5%)

    Math 38.7%

    Its run record (Method) →

    30.8% (28%–34%) AB
  62. d1-2B

    d1-2B · Liquid AI · decision model

    Parameters: 1.92B dense

    Base model: LFM3-VL-2B

    Accuracy 30.7% (95%: 28.0%–33.4%)

    Math 37.1%

    Its run record (Method) →

    30.7% (28%–33%) AB
  63. GLiDE 0.8B

    GLiDE 0.8B · Fastino · decision model

    Parameters: 0.75B dense

    Accuracy 29.2% (95%: 26.6%–31.9%)

    Math 32.1%

    Its run record (Method) →

    29.2% (27%–32%) ABC
  64. OneJev1.6-0.8B

    OneJev1.6-0.8B · OmniJev · decision model

    Parameters: 0.8B dense

    Accuracy 27.0% (95%: 24.5%–29.7%)

    Math 28.2%

    Its run record (Method) →

    27.0% (25%–30%) ABC
  65. KnowLine-0.8B-Gen1

    KnowLine-0.8B-Gen1 · PelaAI · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Accuracy 26.7% (95%: 24.2%–29.3%)

    Math 28.2%

    Its run record (Method) →

    26.7% (24%–29%) ABC
  66. StartLux-Decision-v1.1-0.8B-Instant

    StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Accuracy 24.7% (95%: 22.3%–27.3%)

    Math 29.7%

    Its run record (Method) →

    24.7% (22%–27%) BC
  67. GLiClass v4 Edge

    GLiClass v4 Edge · decision model

    Parameters: 0.4B dense

    Base model: DeBERTa-v3-large

    Accuracy 23.6% (95%: 21.2%–26.1%)

    Math 22.7%

    Its run record (Method) →

    23.6% (21%–26%) BC
  68. GLiDE 0.4B

    GLiDE 0.4B · Fastino · decision model

    Parameters: 0.40B dense

    Accuracy 23.2% (95%: 20.8%–25.8%)

    Math 20.6%

    Its run record (Method) →

    23.2% (21%–26%) C
  69. Strands Decider 2B

    Strands Decider 2B · Strands Agents · decision model

    Parameters: 2B dense

    Accuracy 22.4% (95%: 20.1%–24.9%)

    Math 28.7%

    Its run record (Method) →

    22.4% (20%–25%) C

MachinePulse REAX PelaAI Poindexter Labs Fastino Sthānika AI Logits Liquid AI Nace.AI Perplexity vLLM Semantic Router Base Labs Subquadratic Turion.AI Google tattrongvu TypeSafe Celeris StartLux newsbang Patrick-18 Inception Microsoft Valen-Team scar-ai Apertix AI Zefan Cai Cloudflare OmniJev quincyqiang victorbennett EldanRing Blockbrain Labs Metask Lab Other Strands Agents Chance, 20% (five options)Bars run to each accuracy; the whisker is its 95% interval. Ranks show observed scores; small gaps need not be meaningful.

Accuracy by topic

SetupWebNumerical reasoning524 questionsVerbal reasoning469 questionsFormal logic132 questions
Karpo Decision 31B Open · 31B—50.8%266/52464.0%300/46988.6%117/132
REAX 1.3 Open · 57B—47.5%249/52464.4%302/46987.9%116/132
KnowLine-31B-Gen1.5 Open · 31B—47.1%247/52464.2%301/46987.9%116/132
KnowLine-31B-Gen2 Open · 31B—47.1%247/52463.8%299/46988.6%117/132
Copernicus-31B Open · 31B—47.1%247/52463.8%299/46986.4%114/132
GLiDE 27B Open · 27B—42.0%220/52468.0%319/46989.4%118/132
Sieve 31B v3 Open · 31B—45.8%240/52462.9%295/46987.9%116/132
Logits Sigma-1 Duo Open · 57B—44.7%234/52463.8%299/46988.6%117/132
GLiDE v1.2 Open · 27B—40.5%212/52463.1%296/46985.6%113/132
d1-ultra Closed · Efficient—44.7%234/52457.4%269/46987.1%115/132
Drex Max v1.1 Closed · Efficient (provisional)—41.4%217/52460.8%285/46986.4%114/132
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—41.4%217/52459.9%281/46989.4%118/132
d3-ultra Open · 26B—42.2%221/52459.3%278/46987.1%115/132
GLiDE API Closed · Efficient—38.7%203/52463.1%296/46986.4%114/132
Mammoth-1 Open · 27B—42.7%224/52458.6%275/46986.4%114/132
d3 Open · 26B—40.6%213/52460.6%284/46987.9%116/132
Perplexity Decider 27B v1.2 API Open · 27B—39.9%209/52458.4%274/46988.6%117/132
Drex Max Open · 27B—42.4%222/52455.7%261/46986.4%114/132
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web39.1%205/52458.0%272/46986.4%114/132
SubQ Snap 1.1 Closed · Efficient—37.2%195/52460.3%283/46984.1%111/132
Turion Jev 31B Open · 31B—36.6%192/52460.1%282/46986.4%114/132
d1.2-ultra Closed · Undisclosed—38.2%200/52457.4%269/46984.1%111/132
KnowLine-12B-Gen1 Open · 12B—36.6%192/52456.7%266/46984.1%111/132
Gemini Decision 1 20261111 Closed · Efficient—39.3%206/52453.1%249/46981.8%108/132
d1.2 Closed · Undisclosed—39.7%208/52451.6%242/46978.0%103/132
Jeffan-9B-1211 Open · 9.0B—34.2%179/52453.7%252/46984.1%111/132
Jev 1.14 Closed · Efficient—35.7%187/52453.1%249/46980.3%106/132
GLiDE 9B Open · 7.9B—32.1%168/52455.7%261/46982.6%109/132
celeris-1-decision Closed · Efficient—37.2%195/52450.5%237/46975.8%100/132
celeris-1.1-decision Closed · Efficient—37.8%198/52449.5%232/46977.3%102/132
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—32.3%169/52452.5%246/46982.6%109/132
Adjudex 9B v5 Open · 9.0B—34.0%178/52449.9%234/46983.3%110/132
Elephant-1 Open · 9.0B—33.2%174/52451.0%239/46982.6%109/132
Jev 1.13 Closed · Efficient—33.2%174/52452.0%244/46978.0%103/132
d1.1 Closed · Efficient—34.7%182/52450.7%238/46974.2%98/132
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—32.4%170/52450.7%238/46981.8%108/132
Dexy-Decision-9B-V2 Open · 9.0B—34.9%183/52447.3%222/46979.5%105/132
Mercury Decide Closed · Efficient—33.4%175/52451.6%242/46968.2%90/132
Perplexity Decider Flash v1.3 Open · 4.0B—30.7%161/52450.1%235/46981.1%107/132
Microsoft-Decision-1 Closed · Efficient—34.0%178/52446.9%220/46977.3%102/132
Valen-Decision-1.5-9B Open · 9.0B—34.2%179/52445.8%215/46979.5%105/132
KnowLine-9B-Gen1 Open · 9.0B—32.4%170/52447.5%223/46979.5%105/132
vera-medium-omni Open · 4.2B—31.9%167/52448.6%228/46976.5%101/132
KnowLine-9B-Gen2 Open · 9.0B—33.4%175/52445.2%212/46981.1%107/132
Cerno 9B v1.2 Open · 9.0B—31.1%163/52448.0%225/46975.0%99/132
Perplexity Decider Flash v1.2 Open · 4.0B—31.3%164/52446.1%216/46979.5%105/132
Open-Jev-27B-v1.1 Open · 27B—31.1%163/52447.1%221/46972.0%95/132
KnowLine 4B Gen5.5 Open · 4.0B—30.2%158/52447.5%223/46972.0%95/132
Drex 2.0 Open · 35B MoE · 3.0B active—34.2%179/52441.4%194/46974.2%98/132
Clef Open · 27B—30.2%158/52446.1%216/46964.4%85/132
d1 Closed · Efficient—29.0%152/52444.3%208/46975.0%99/132
OneJev-27B Open · 27B—30.0%157/52445.0%211/46967.4%89/132
Chinese Decision Pointer 4B Open · 4.0B—29.0%152/52443.9%206/46972.7%96/132
Verdin-4B-v1.1 Open · 4.0B—28.4%149/52442.4%199/46975.0%99/132
Winnow-12B Open · 12B—26.5%139/52445.4%213/46972.0%95/132
Cygnet Open · 12B—26.9%141/52447.3%222/46959.8%79/132
Clef-Flash Open · 9.0B—30.0%157/52440.1%188/46970.5%93/132
GLiDE 2B Open · 1.9B—26.5%139/52433.0%155/46952.3%69/132
Metask-Jev-4B Open · 4.5B—24.6%129/52434.5%162/46953.0%70/132
KnowLine-2B-Gen1 Open · 2.0B—25.0%131/52432.2%151/46956.8%75/132
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—25.8%135/52429.9%140/46953.8%71/132
d1-2B Open · 1.9B—24.8%130/52431.8%149/46950.0%66/132
GLiDE 0.8B Open · 0.8B—25.6%134/52429.0%136/46943.9%58/132
OneJev1.6-0.8B Open · 0.8B—24.0%126/52427.5%129/46937.1%49/132
KnowLine-0.8B-Gen1 Open · 0.8B—25.2%132/52424.3%114/46940.9%54/132
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—19.5%102/52425.6%120/46942.4%56/132
GLiClass v4 Edge Open · 0.4B—23.9%125/52422.0%103/46928.0%37/132
GLiDE 0.4B Open · 0.4B—20.8%109/52424.1%113/46929.5%39/132
Strands Decider 2B Open · 2.0B—19.3%101/52424.1%113/46928.8%38/132

Share right of the questions each setup answered in each topic; under it right / answered.

Decision style

k = 5 by our rule (methods agree, ARI ≥ 0.80); it is also the most stable.

Skill and “I don't know”, 5 styles

68 decision models, one dot each in its maker's colour: Logic accuracy across, the “I don't know” rate up. Each style is a shaded area and a dot shape; its name is computed from the data.

10%20%30%40%50%60%70%80%0%10%20%30%40%50%60%70%80%Logic accuracy (standard run)“I don't know” rate (Logic + Math)StrongStrongStrong, cautiousStrong, cautiousMiddleMiddleLow-scoringLow-scoringLow-scoring, cautiousLow-scoring, cautious

Karpo Decision 31B: Strong; Logic 60.7%, Math 81.6%, “I don't know” 1.52%

REAX 1.3: Strong; Logic 59.3%, Math 78.0%, “I don't know” 1.09%

KnowLine-31B-Gen1.5: Strong; Logic 59.0%, Math 77.4%, “I don't know” 0.21%

KnowLine-31B-Gen2: Strong; Logic 58.9%, Math 80.3%, “I don't know” 0.03%

Copernicus-31B: Strong; Logic 58.7%, Math 77.3%, “I don't know” 0.52%

GLiDE 27B: Strong; Logic 58.4%, Math 72.0%, “I don't know” 11.47%

Sieve 31B v3: Strong; Logic 57.9%, Math 77.1%, “I don't know” 1.13%

Logits Sigma-1 Duo: Strong; Logic 57.8%, Math 75.2%, “I don't know” 1.37%

GLiDE (v1.2): Strong; Logic 55.2%, Math 69.9%, “I don't know” 17.18%

d1-ultra: Strong, cautious; Logic 54.9%, Math 74.1%, “I don't know” 47.75%

Drex Max v1.1: Strong; Logic 54.8%, Math 70.1%, “I don't know” 20.77%

Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): Strong; Logic 54.8%, Math 72.5%, “I don't know” 4.93%

d3-ultra: Strong; Logic 54.6%, Math 71.5%, “I don't know” 2.65%

GLiDE (API): Strong; Logic 54.5%, Math 70.8%, “I don't know” 20.47%

Mammoth-1: Strong, cautious; Logic 54.5%, Math 74.0%, “I don't know” 30.63%; Ward puts it in Strong

d3: Strong; Logic 54.5%, Math 70.4%, “I don't know” 3.41%

Perplexity Decider 27B v1.2 (API): Strong; Logic 53.3%, Math 69.8%, “I don't know” 3.86%

Drex Max: Strong; Logic 53.1%, Math 70.3%, “I don't know” 2.07%

SubQ Snap 1.1: Strong, cautious; Logic 52.4%, Math 68.6%, “I don't know” 26.82%; Ward puts it in Strong

Turion Jev 31B: Strong; Logic 52.3%, Math 69.0%, “I don't know” 0.67%

d1.2-ultra: Strong, cautious; Logic 51.6%, Math 69.9%, “I don't know” 51.92%

KnowLine-12B-Gen1: Strong; Logic 50.6%, Math 66.3%, “I don't know” 1.79%

Gemini Decision 1 (20261111): Strong; Logic 50.0%, Math 71.3%, “I don't know” 1.13%

d1.2: Strong, cautious; Logic 49.2%, Math 65.5%, “I don't know” 34.91%

Jeffan-9B-1211: Middle; Logic 48.2%, Math 59.8%, “I don't know” 6.27%

Jev 1.14: Middle; Logic 48.2%, Math 65.1%, “I don't know” 6.66%

GLiDE 9B: Middle; Logic 47.8%, Math 57.9%, “I don't know” 13.53%

celeris-1-decision: Strong, cautious; Logic 47.3%, Math 70.0%, “I don't know” 49.82%

celeris-1.1-decision: Strong, cautious; Logic 47.3%, Math 69.9%, “I don't know” 49.24%

StartLux-Decision-v1.1-9B-Instant: Middle; Logic 46.6%, Math 58.4%, “I don't know” 4.59%

Adjudex 9B v5: Middle; Logic 46.4%, Math 62.1%, “I don't know” 3.13%

Elephant-1: Strong, cautious; Logic 46.4%, Math 61.3%, “I don't know” 39.72%

Jev 1.13: Middle; Logic 46.3%, Math 64.1%, “I don't know” 4.47%

d1.1: Strong, cautious; Logic 46.0%, Math 65.2%, “I don't know” 53.53%

StartLux-Decision-v1.1-4B-Instant: Middle; Logic 45.9%, Math 55.8%, “I don't know” 7.36%

Dexy-Decision-9B-V2: Middle; Logic 45.3%, Math 58.8%, “I don't know” 2.89%

Mercury Decide: Middle; Logic 45.1%, Math 53.5%, “I don't know” 5.14%

Perplexity Decider Flash v1.3: Middle; Logic 44.7%, Math 55.9%, “I don't know” 7.60%

Microsoft-Decision-1: Middle; Logic 44.4%, Math 59.4%, “I don't know” 5.50%

Valen-Decision-1.5-9B: Middle; Logic 44.4%, Math 55.3%, “I don't know” 3.59%

KnowLine-9B-Gen1: Middle; Logic 44.3%, Math 59.8%, “I don't know” 1.58%

vera-medium-omni: Middle; Logic 44.1%, Math 56.8%, “I don't know” 3.86%

KnowLine-9B-Gen2: Middle; Logic 43.9%, Math 60.1%, “I don't know” 4.99%

Cerno 9B v1.2: Middle; Logic 43.3%, Math 56.0%, “I don't know” 1.49%

Perplexity Decider Flash v1.2: Middle; Logic 43.1%, Math 55.2%, “I don't know” 10.01%

Open-Jev-27B-v1.1: Middle; Logic 42.6%, Math 54.6%, “I don't know” 7.57%

KnowLine 4B Gen5.5: Middle; Logic 42.3%, Math 56.0%, “I don't know” 2.19%

Drex 2.0: Middle; Logic 41.9%, Math 59.0%, “I don't know” 14.66%

Clef: Middle; Logic 40.8%, Math 51.1%, “I don't know” 1.25%

d1: Middle; Logic 40.8%, Math 55.6%, “I don't know” 3.35%

OneJev-27B: Middle; Logic 40.6%, Math 53.8%, “I don't know” 0.79%

Chinese Decision Pointer 4B: Middle; Logic 40.4%, Math 50.4%, “I don't know” 4.20%

Verdin-4B-v1.1: Middle; Logic 39.7%, Math 51.5%, “I don't know” 4.84%

Winnow-12B: Middle; Logic 39.7%, Math 46.9%, “I don't know” 10.19%

Cygnet: Middle; Logic 39.3%, Math 44.3%, “I don't know” 4.26%

Clef-Flash: Middle; Logic 38.9%, Math 49.8%, “I don't know” 3.80%

GLiDE 2B: Low-scoring; Logic 32.3%, Math 36.8%, “I don't know” 15.91%

Metask-Jev-4B: Low-scoring; Logic 32.1%, Math 39.2%, “I don't know” 1.28%

KnowLine-2B-Gen1: Low-scoring; Logic 31.7%, Math 35.3%, “I don't know” 4.65%

StartLux-Decision-v1.1-2B-Instant: Low-scoring; Logic 30.8%, Math 38.7%, “I don't know” 11.07%

d1-2B: Low-scoring, cautious; Logic 30.7%, Math 37.1%, “I don't know” 70.41%

GLiDE 0.8B: Low-scoring; Logic 29.2%, Math 32.1%, “I don't know” 16.76%

OneJev1.6-0.8B: Low-scoring; Logic 27.0%, Math 28.2%, “I don't know” 1.46%

KnowLine-0.8B-Gen1: Low-scoring; Logic 26.7%, Math 28.2%, “I don't know” 0.46%

StartLux-Decision-v1.1-0.8B-Instant: Low-scoring; Logic 24.7%, Math 29.7%, “I don't know” 11.13%

GLiClass v4 Edge: Low-scoring; Logic 23.6%, Math 22.7%, “I don't know” 12.65%

GLiDE 0.4B: Low-scoring, cautious; Logic 23.2%, Math 20.6%, “I don't know” 68.28%

Strands Decider 2B: Low-scoring; Logic 22.4%, Math 28.7%, “I don't know” 3.74%

Strong Strong, cautious Middle Low-scoring Low-scoring, cautious MachinePulse REAX PelaAI Poindexter Labs Fastino Sthānika AI Logits Liquid AI Nace.AI Perplexity vLLM Semantic Router Base Labs Subquadratic Turion.AI Google tattrongvu TypeSafe Celeris StartLux newsbang Patrick-18 Inception Microsoft Valen-Team scar-ai Apertix AI Zefan Cai Cloudflare OmniJev quincyqiang victorbennett EldanRing Blockbrain Labs Metask Lab Other Strands Agents

The groups come from the data, not from our rules; borderline models can switch groups.
Shaded areas only outline each style's dots; they are not confidence regions.

Styles (k = 5)

Styleits members under itModelsLogicmean accuracyMathmean accuracy“I don't know”mean rateMean zLogic / Math / IDK
Strong1955.7%73.2%5.07%+1.08 / +1.00 / -0.44
Karpo Decision 31B, REAX 1.3, KnowLine-31B-Gen1.5, KnowLine-31B-Gen2, Copernicus-31B, GLiDE 27B, Sieve 31B v3, Logits Sigma-1 Duo, GLiDE v1.2, Drex Max v1.1, Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell, d3-ultra, GLiDE API, d3, Perplexity Decider 27B v1.2 API, Drex Max, Turion Jev 31B, KnowLine-12B-Gen1, Gemini Decision 1 20261111
Strong, cautious949.9%68.7%42.70%+0.50 / +0.70 / +1.78
d1-ultra, Mammoth-1, SubQ Snap 1.1, d1.2-ultra, d1.2, celeris-1-decision, celeris-1.1-decision, Elephant-1, d1.1
Middle2843.5%56.0%5.35%-0.15 / -0.14 / -0.42
Jeffan-9B-1211, Jev 1.14, GLiDE 9B, StartLux-Decision-v1.1-9B-Instant, Adjudex 9B v5, Jev 1.13, StartLux-Decision-v1.1-4B-Instant, Dexy-Decision-9B-V2, Mercury Decide, Perplexity Decider Flash v1.3, Microsoft-Decision-1, Valen-Decision-1.5-9B, KnowLine-9B-Gen1, vera-medium-omni, KnowLine-9B-Gen2, Cerno 9B v1.2, Perplexity Decider Flash v1.2, Open-Jev-27B-v1.1, KnowLine 4B Gen5.5, Drex 2.0, Clef, d1, OneJev-27B, Chinese Decision Pointer 4B, Verdin-4B-v1.1, Winnow-12B, Cygnet, Clef-Flash
Low-scoring1028.0%31.9%7.91%-1.72 / -1.74 / -0.27
GLiDE 2B, Metask-Jev-4B, KnowLine-2B-Gen1, StartLux-Decision-v1.1-2B-Instant, GLiDE 0.8B, OneJev1.6-0.8B, KnowLine-0.8B-Gen1, StartLux-Decision-v1.1-0.8B-Instant, GLiClass v4 Edge, Strands Decider 2B
Low-scoring, cautious226.9%28.8%69.34%-1.83 / -1.94 / +3.34
d1-2B, GLiDE 0.4B

Models (k = 5)

ModelStylek-meansLogicMath“I don't know”Wardits styleSame partnersquestions resampledSame partnersmodels resampled
Karpo Decision 31BStrong60.7%81.6%1.52%same99%96%
REAX 1.3Strong59.3%78.0%1.09%same99%96%
KnowLine-31B-Gen1.5Strong59.0%77.4%0.21%same99%95%
KnowLine-31B-Gen2Strong58.9%80.3%0.03%same99%95%
Copernicus-31BStrong58.7%77.3%0.52%same99%96%
GLiDE 27BStrong58.4%72.0%11.47%same99%95%
Sieve 31B v3Strong57.9%77.1%1.13%same99%95%
Logits Sigma-1 DuoStrong57.8%75.2%1.37%same99%96%
GLiDE v1.2Strong55.2%69.9%17.18%same99%87%
Drex Max v1.1Strong54.8%70.1%20.77%same99%79%
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 BlackwellStrong54.8%72.5%4.93%same99%96%
d3-ultraStrong54.6%71.5%2.65%same99%96%
GLiDE APIStrong54.5%70.8%20.47%same99%83%
d3Strong54.5%70.4%3.41%same99%96%
Perplexity Decider 27B v1.2 APIStrong53.3%69.8%3.86%same99%96%
Drex MaxStrong53.1%70.3%2.07%same99%96%
Turion Jev 31BStrong52.3%69.0%0.67%same99%95%
KnowLine-12B-Gen1Strong50.6%66.3%1.79%same94%92%
Gemini Decision 1 20261111Strong50.0%71.3%1.13%same99%95%
d1-ultraStrong, cautious54.9%74.1%47.75%same100%96%
Mammoth-1Strong, cautious54.5%74.0%30.63%Strong100%88%
SubQ Snap 1.1Strong, cautious52.4%68.6%26.82%Strong100%83%
d1.2-ultraStrong, cautious51.6%69.9%51.92%same100%95%
d1.2Strong, cautious49.2%65.5%34.91%same100%94%
celeris-1-decisionStrong, cautious47.3%70.0%49.82%same100%95%
celeris-1.1-decisionStrong, cautious47.3%69.9%49.24%same100%95%
Elephant-1Strong, cautious46.4%61.3%39.72%same100%95%
d1.1Strong, cautious46.0%65.2%53.53%same100%95%
Jeffan-9B-1211Middle48.2%59.8%6.27%same99%98%
Jev 1.14Middle48.2%65.1%6.66%same77%83%
GLiDE 9BMiddle47.8%57.9%13.53%same99%98%
StartLux-Decision-v1.1-9B-InstantMiddle46.6%58.4%4.59%same99%98%
Adjudex 9B v5Middle46.4%62.1%3.13%same99%98%
Jev 1.13Middle46.3%64.1%4.47%same98%98%
StartLux-Decision-v1.1-4B-InstantMiddle45.9%55.8%7.36%same99%98%
Dexy-Decision-9B-V2Middle45.3%58.8%2.89%same99%99%
Mercury DecideMiddle45.1%53.5%5.14%same99%99%
Perplexity Decider Flash v1.3Middle44.7%55.9%7.60%same99%98%
Microsoft-Decision-1Middle44.4%59.4%5.50%same99%98%
Valen-Decision-1.5-9BMiddle44.4%55.3%3.59%same99%99%
KnowLine-9B-Gen1Middle44.3%59.8%1.58%same99%98%
vera-medium-omniMiddle44.1%56.8%3.86%same99%98%
KnowLine-9B-Gen2Middle43.9%60.1%4.99%same99%99%
Cerno 9B v1.2Middle43.3%56.0%1.49%same99%98%
Perplexity Decider Flash v1.2Middle43.1%55.2%10.01%same99%98%
Open-Jev-27B-v1.1Middle42.6%54.6%7.57%same99%98%
KnowLine 4B Gen5.5Middle42.3%56.0%2.19%same99%98%
Drex 2.0Middle41.9%59.0%14.66%same99%98%
ClefMiddle40.8%51.1%1.25%same99%97%
d1Middle40.8%55.6%3.35%same99%98%
OneJev-27BMiddle40.6%53.8%0.79%same99%98%
Chinese Decision Pointer 4BMiddle40.4%50.4%4.20%same99%97%
Verdin-4B-v1.1Middle39.7%51.5%4.84%same99%98%
Winnow-12BMiddle39.7%46.9%10.19%same99%97%
CygnetMiddle39.3%44.3%4.26%same98%97%
Clef-FlashMiddle38.9%49.8%3.80%same99%97%
GLiDE 2BLow-scoring32.3%36.8%15.91%same100%100%
Metask-Jev-4BLow-scoring32.1%39.2%1.28%same100%100%
KnowLine-2B-Gen1Low-scoring31.7%35.3%4.65%same100%100%
StartLux-Decision-v1.1-2B-InstantLow-scoring30.8%38.7%11.07%same100%100%
GLiDE 0.8BLow-scoring29.2%32.1%16.76%same100%100%
OneJev1.6-0.8BLow-scoring27.0%28.2%1.46%same100%100%
KnowLine-0.8B-Gen1Low-scoring26.7%28.2%0.46%same100%100%
StartLux-Decision-v1.1-0.8B-InstantLow-scoring24.7%29.7%11.13%same100%100%
GLiClass v4 EdgeLow-scoring23.6%22.7%12.65%same100%100%
Strands Decider 2BLow-scoring22.4%28.7%3.74%same100%100%
d1-2BLow-scoring, cautious30.7%37.1%70.41%same100%99%
GLiDE 0.4BLow-scoring, cautious23.2%20.6%68.28%same100%100%

How many styles

kk-means vs WardARIStability k-meansquestions resampledStability Wardquestions resampledStability k-meansmodels resampledStability Wardmodels resampledMean stabilitySilhouettek-meansSilhouetteWard
20.730.910.820.700.620.760.500.46
30.790.830.790.610.600.710.490.51
40.920.960.930.890.860.910.550.54
5 ✓0.940.980.940.970.910.950.590.58
60.980.980.940.870.890.920.580.57
How the styles are found

Models. The Logic leaderboard's decision rows (one row per model by the site's rule) whose setup has complete English standard runs on Logic (1,125 questions) and Math (2,163) and a complete “I don't know” run on both: 68 models. Rows with web or on a sample are left out, as is any row missing a run (never filled in). Left out: Perplexity Decider 27B v1.2 (API · with web) (with web).

Numbers. Logic and Math accuracy: the standard runs, as on this site. “I don't know” rate: the answers that chose “I don't know” (option F) over all answers of the model's two “I don't know” runs, Logic and Math pooled (3,288 answers); not the non-hallucination rate F / (F + wrong). Each number is turned into a z-score over these models (mean 0, SD 1).

Clustering. k-means (best of 100 seeded restarts, seed 4317) and Ward clustering on the three z-scores for k = 2–6. Agreement: the adjusted Rand index (ARI) of the two (1 = the same groups; label-free). Stability: 500 bootstrap replicates (seed 4317) of each resampling unit, the questions (one resampled question set per benchmark, the same for every model and for both its runs) and the models (drawn with replacement; each model drawn is compared once). In every replicate the three numbers, the z-scores and both clusterings are refitted (k-means: 20 restarts) and compared with the full-data result by ARI on the models present. Same partners: per model, the mean share of its style partners it keeps in a replicate (Jaccard of the two partner sets, itself included), k-means at that k.

Choosing k. The rule for k is chosen by us (analyst-selected): among the k at which k-means and Ward agree (ARI ≥ 0.80), the one with the highest mean stability (the mean of the four bootstrap stabilities: k-means and Ward, questions and models resampled); a tie to the smaller k. If no k agrees, the k with the highest min(agreement, mean stability). Chosen: k = 5.

Names. A style's name from its mean z-scores (Logic, Math, IDK): skill = the mean of the two accuracy z-scores, Strong at or above +0.5, Low-scoring at or below −0.5, else Middle; IDK at or above +0.5 'cautious', at or below −0.5 'decisive'. Middle with an IDK word is that word alone ('Cautious', 'Decisive'). Styles are listed by mean skill, highest first.

SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 1,125 of 1,125); shown once.

Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.

Every setup, every measure

SetupWebAccuracy% of N (N ≤ 1,125) · Wilson 95%Tie groupHolm · McNemarRight / NquestionsKept at ≥99%right / wrongAUROCright vs wrongDecision Score (AA formula)0–100 · chosen-answer BrierNon-hallucinationof answers not right, % “I don't know” · higher is betterMedian latencymsp90ms, nearest rankCost$ per 1,000 qper 1M q
Karpo Decision 31B Open · 31B

Karpo Decision 31B · MachinePulse · decision model

Parameters: 31B dense

Accuracy 60.7% (95%: 57.8%–63.5%)

Math 81.6%

Its run record (Method) →

—60.7%57.8%–63.5%ab683 / 1,125169 right of 172 kept3 wrong · 953 skipped0.81671.80chosen-answer Brier 0.17116.6%answered 97%173 ms211 ms$0.128$128estimate · RTX PRO 6000 Blackwell
REAX 1.3 Open · 57B

REAX 1.3 · REAX · decision model

Parameters: 57.36B dense

Accuracy 59.3% (95%: 56.4%–62.1%)

Math 78.0%

Its run record (Method) →

—59.3%56.4%–62.1%ac667 / 1,12551 right of 51 kept0 mistakes · 1,074 skipped0.81671.07chosen-answer Brier 0.17145.3%answered 98%215 ms278 ms$0.160$160estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen1.5 Open · 31B

KnowLine-31B-Gen1.5 · PelaAI · decision model

Parameters: 31B dense

Accuracy 59.0% (95%: 56.1%–61.9%)

Math 77.4%

Its run record (Method) →

—59.0%56.1%–61.9%abcd664 / 1,12589 right of 89 kept0 mistakes · 1,036 skipped0.80470.72chosen-answer Brier 0.17570.4%answered 100%983 ms1,279 ms$0.061$61.43estimate · DGX Spark
KnowLine-31B-Gen2 Open · 31B

KnowLine-31B-Gen2 · PelaAI · decision model

Parameters: 31B dense

Base model: Gemma 4 31B

Accuracy 58.9% (95%: 56.0%–61.8%)

Math 80.3%

Its run record (Method) →

—58.9%56.0%–61.8%abcd663 / 1,12554 right of 54 kept0 mistakes · 1,071 skipped0.82571.09chosen-answer Brier 0.16750.0%answered 100%151 ms204 ms$0.112$112estimate · RTX PRO 6000 Blackwell
Copernicus-31B Open · 31B

Copernicus-31B · Poindexter Labs · decision model

Parameters: 31B dense

Accuracy 58.7% (95%: 55.8%–61.5%)

Math 77.3%

Its run record (Method) →

—58.7%55.8%–61.5%abcd660 / 1,12566 right of 66 kept0 mistakes · 1,059 skipped0.80170.48chosen-answer Brier 0.17711.9%answered 99%185 ms255 ms$0.136$136estimate · RTX PRO 6000 Blackwell
GLiDE 27B Open · 27B

GLiDE 27B · Fastino · decision model

Parameters: 27B dense

Accuracy 58.4% (95%: 55.5%–61.2%)

Math 72.0%

Its run record (Method) →

—58.4%55.5%–61.2%abcde657 / 1,12574 right of 75 kept1 wrong · 1,050 skipped0.82270.67chosen-answer Brier 0.170744.3%answered 79%544 ms655 ms$0.033$33.16estimate · DGX Spark
Sieve 31B v3 Open · 31B

Sieve 31B v3 · Sthānika AI · decision model

Parameters: 31B dense

Accuracy 57.9% (95%: 55.0%–60.7%)

Math 77.1%

Its run record (Method) →

—57.9%55.0%–60.7%abcdf651 / 1,12579 right of 81 kept2 wrong · 1,044 skipped0.79770.03chosen-answer Brier 0.17815.5%answered 98%1,050 ms1,335 ms$0.063$62.95estimate · DGX Spark
Logits Sigma-1 Duo Open · 57B

Logits Sigma-1 Duo · Logits · decision model

Parameters: 57.36B dense

Accuracy 57.8% (95%: 54.9%–60.6%)

Math 75.2%

Its run record (Method) →

—57.8%54.9%–60.6%abcdefg650 / 1,1258 right of 8 kept0 mistakes · 1,117 skipped0.79669.90chosen-answer Brier 0.17987.1%answered 97%285 ms399 ms$0.211$211estimate · RTX PRO 6000 Blackwell
GLiDE v1.2 Open · 27B

GLiDE v1.2 · Fastino · decision model

Parameters: 27B dense

Accuracy 55.2% (95%: 52.3%–58.1%)

Math 69.9%

Its run record (Method) →

—55.2%52.3%–58.1%abcdefgh621 / 1,12545 right of 45 kept0 mistakes · 1,080 skipped0.79368.39chosen-answer Brier 0.184250.2%answered 73%111 ms140 ms$0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell
d1-ultra Closed · Efficient

d1-ultra · Liquid AI · decision model

Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

Accuracy 54.9% (95%: 52.0%–57.8%)

Math 74.1%

Its run record (Method) →

—54.9%52.0%–57.8%abcdefgh618 / 1,12520 right of 20 kept0 mistakes · 1,105 skipped0.78368.09chosen-answer Brier 0.187691.0%answered 35%182 ms425 ms$0.013$12.64
Drex Max v1.1 Closed · Efficient (provisional)

Drex Max v1.1 · Nace.AI · decision model

Parameters: not disclosed by maker

Accuracy 54.8% (95%: 51.8%–57.6%)

Math 70.1%

Its run record (Method) →

—54.8%51.8%–57.6%abcdefghi616 / 1,12589 right of 90 kept1 wrong · 1,035 skipped0.79368.07chosen-answer Brier 0.186154.1%answered 69%329 ms1,540 ms$0.022$22.37
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B

Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

Parameters: 27B dense

Accuracy 54.8% (95%: 51.8%–57.6%)

Math 72.5%

Its run record (Method) →

—54.8%51.8%–57.6%abcdefgh616 / 1,12562 right of 62 kept0 mistakes · 1,063 skipped0.80367.98chosen-answer Brier 0.188118.1%answered 91%108 ms131 ms$0.081$81.16estimate · RTX PRO 6000 Blackwell
d3-ultra Open · 26B

d3-ultra · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Accuracy 54.6% (95%: 51.7%–57.5%)

Math 71.5%

Its run record (Method) →

—54.6%51.7%–57.5%bdefghij614 / 1,12528 right of 28 kept0 mistakes · 1,097 skipped0.77866.46chosen-answer Brier 0.216610.1%answered 95%156 ms185 ms$0.109$109estimate · RTX PRO 6000 Blackwell
GLiDE API Closed · Efficient

GLiDE API · Fastino · decision model

Parameters: not disclosed by maker

Accuracy 54.5% (95%: 51.6%–57.4%)

Math 70.8%

Its run record (Method) →

—54.5%51.6%–57.4%cdefghi613 / 1,12552 right of 52 kept0 mistakes · 1,073 skipped0.81468.27chosen-answer Brier 0.179460.0%answered 67%271 ms291 ms$0.052$52.03
Mammoth-1 Open · 27B

Mammoth-1 · Base Labs · decision model

Parameters: 27B dense

Accuracy 54.5% (95%: 51.6%–57.4%)

Math 74.0%

Its run record (Method) →

—54.5%51.6%–57.4%cdefghij613 / 1,125106 right of 108 kept2 wrong · 1,017 skipped0.79568.07chosen-answer Brier 0.183572.3%answered 58%221 ms244 ms$0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell
d3 Open · 26B

d3 · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Accuracy 54.5% (95%: 51.6%–57.4%)

Math 70.4%

Its run record (Method) →

—54.5%51.6%–57.4%cdefgh613 / 1,12554 right of 54 kept0 mistakes · 1,071 skipped0.80668.34chosen-answer Brier 0.178114.8%answered 93%76 ms98 ms$0.057$57.18estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.2 API Open · 27B

Perplexity Decider 27B v1.2 API · Perplexity · decision model

Parameters: 27B dense

Accuracy 53.3% (95%: 50.4%–56.2%)

Math 69.8%

Its run record (Method) →

—53.3%50.4%–56.2%defghijk600 / 1,12578 right of 80 kept2 wrong · 1,045 skipped0.79867.64chosen-answer Brier 0.180514.4%answered 93%238 ms552 ms$0.007$7.46
Drex Max Open · 27B

Drex Max · Nace.AI · decision model

Parameters: 27B dense

Accuracy 53.1% (95%: 50.1%–56.0%)

Math 70.3%

Its run record (Method) →

—53.1%50.1%–56.0%defghijk597 / 1,125147 right of 155 kept8 wrong · 970 skipped0.78665.68chosen-answer Brier 0.21707.3%answered 97%106 ms143 ms$0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.2 API · with web Open · 27B

Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

Parameters: 27B dense

Accuracy 52.5% (95%: 49.6%–55.4%)

Math 69.3%

Its run record (Method) →

with web52.5%49.6%–55.4%fghijkl591 / 1,12578 right of 79 kept1 wrong · 1,046 skipped0.80067.24chosen-answer Brier 0.180637.7%answered 80%448 ms827 ms$1.030$1,030
SubQ Snap 1.1 Closed · Efficient

SubQ Snap 1.1 · Subquadratic · decision model

Parameters: not disclosed by maker

Shown once with SubQ Snap Preview: the same answer to every question

Accuracy 52.4% (95%: 49.4%–55.3%)

Math 68.6%

Its run record (Method) →

—52.4%49.4%–55.3%fghijklm589 / 1,12570 right of 72 kept2 wrong · 1,053 skipped0.78765.94chosen-answer Brier 0.204765.7%answered 61%384 ms393 ms$0.015$14.92
Turion Jev 31B Open · 31B

Turion Jev 31B · Turion.AI · decision model

Parameters: 31B dense

Accuracy 52.3% (95%: 49.3%–55.2%)

Math 69.0%

Its run record (Method) →

—52.3%49.3%–55.2%eghijklmn588 / 1,1257 right of 7 kept0 mistakes · 1,118 skipped0.78266.89chosen-answer Brier 0.18491.5%answered 99%385 ms533 ms$0.025$24.91estimate · DGX Spark
d1.2-ultra Closed · Undisclosed

d1.2-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 51.6% (95%: 48.6%–54.5%)

Math 69.9%

Its run record (Method) →

—51.6%48.6%–54.5%hijklmno580 / 1,12513 right of 13 kept0 mistakes · 1,112 skipped0.80066.77chosen-answer Brier 0.180294.5%answered 28%220 ms341 ms$0.013$12.64
KnowLine-12B-Gen1 Open · 12B

KnowLine-12B-Gen1 · PelaAI · decision model

Parameters: 12B dense

Base model: Gemma 4 12B

Accuracy 50.6% (95%: 47.7%–53.5%)

Math 66.3%

Its run record (Method) →

—50.6%47.7%–53.5%hijklmnop569 / 1,1256 right of 6 kept0 mistakes · 1,119 skipped0.77465.90chosen-answer Brier 0.18788.8%answered 95%68 ms88 ms$0.052$51.64estimate · RTX PRO 6000 Blackwell
Gemini Decision 1 20261111 Closed · Efficient

Gemini Decision 1 20261111 · Google · decision model

Parameters: not disclosed by maker (closed weights)

Accuracy 50.0% (95%: 47.1%–53.0%)

Math 71.3%

Its run record (Method) →

—50.0%47.1%–53.0%hijklmnopq563 / 1,12543 right of 43 kept0 mistakes · 1,082 skipped0.76865.37chosen-answer Brier 0.19302.0%answered 99%228 ms344 ms$0.037$36.77
d1.2 Closed · Undisclosed

d1.2 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 49.2% (95%: 46.2%–52.1%)

Math 65.5%

Its run record (Method) →

—49.2%46.2%–52.1%hijklmnopqr553 / 1,12528 right of 28 kept0 mistakes · 1,097 skipped0.74264.51chosen-answer Brier 0.201378.8%answered 43%222 ms390 ms$0.006$6.02
Jeffan-9B-1211 Open · 9.0B

Jeffan-9B-1211 · tattrongvu · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 48.2% (95%: 45.3%–51.1%)

Math 59.8%

Its run record (Method) →

—48.2%45.3%–51.1%jklmnopqrs542 / 1,12510 right of 10 kept0 mistakes · 1,115 skipped0.75364.33chosen-answer Brier 0.195223.4%answered 87%52 ms66 ms$0.039$39.35estimate · RTX PRO 6000 Blackwell
Jev 1.14 Closed · Efficient

Jev 1.14 · TypeSafe · decision model

Parameters: not disclosed by maker

Accuracy 48.2% (95%: 45.3%–51.1%)

Math 65.1%

Its run record (Method) →

—48.2%45.3%–51.1%ijklmnopqrs542 / 1,12561 right of 61 kept0 mistakes · 1,064 skipped0.75064.14chosen-answer Brier 0.199024.5%answered 86%166 ms193 ms$0.026$25.64
GLiDE 9B Open · 7.9B

GLiDE 9B · Fastino · decision model

Parameters: 7.94B dense

Accuracy 47.8% (95%: 44.9%–50.7%)

Math 57.9%

Its run record (Method) →

—47.8%44.9%–50.7%klmnopqrs538 / 1,12532 right of 32 kept0 mistakes · 1,093 skipped0.74963.95chosen-answer Brier 0.199330.4%answered 82%208 ms251 ms$0.013$13.29estimate · DGX Spark
celeris-1-decision Closed · Efficient

celeris-1-decision · Celeris · decision model

Parameters: not disclosed by maker

Accuracy 47.3% (95%: 44.4%–50.2%)

Math 70.0%

Its run record (Method) →

—47.3%44.4%–50.2%klmnopqrstu532 / 1,125No answers at this threshold0.73562.94chosen-answer Brier 0.214192.3%answered 28%390 ms394 ms$0.015$15.40
celeris-1.1-decision Closed · Efficient

celeris-1.1-decision · Celeris · decision model

Parameters: not disclosed by maker

Accuracy 47.3% (95%: 44.4%–50.2%)

Math 69.9%

Its run record (Method) →

—47.3%44.4%–50.2%klmnopqrst532 / 1,125No answers at this threshold0.73362.92chosen-answer Brier 0.214491.0%answered 29%383 ms390 ms$0.002$1.54
StartLux-Decision-v1.1-9B-Instant Open · 9.0B

StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

Parameters: 9B dense

Accuracy 46.6% (95%: 43.7%–49.5%)

Math 58.4%

Its run record (Method) →

—46.6%43.7%–49.5%lmnopqrstuv524 / 1,12514 right of 14 kept0 mistakes · 1,111 skipped0.76263.55chosen-answer Brier 0.194717.4%answered 90%160 ms175 ms$0.010$10.04estimate · DGX Spark
Adjudex 9B v5 Open · 9.0B

Adjudex 9B v5 · newsbang · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 46.4% (95%: 43.5%–49.3%)

Math 62.1%

Its run record (Method) →

—46.4%43.5%–49.3%lmnopqrstuv522 / 1,12554 right of 55 kept1 wrong · 1,070 skipped0.77163.48chosen-answer Brier 0.194512.2%answered 93%41 ms46 ms$0.031$31.36estimate · RTX PRO 6000 Blackwell
Elephant-1 Open · 9.0B

Elephant-1 · Base Labs · decision model

Parameters: 9B dense

Accuracy 46.4% (95%: 43.5%–49.3%)

Math 61.3%

Its run record (Method) →

—46.4%43.5%–49.3%lmnopqrstuv522 / 1,12553 right of 53 kept0 mistakes · 1,072 skipped0.76863.39chosen-answer Brier 0.196275.6%answered 47%79 ms98 ms$0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell
Jev 1.13 Closed · Efficient

Jev 1.13 · TypeSafe · decision model

Parameters: not disclosed by maker

Accuracy 46.3% (95%: 43.4%–49.2%)

Math 64.1%

Its run record (Method) →

—46.3%43.4%–49.2%lmnopqrstuv521 / 1,12565 right of 65 kept0 mistakes · 1,060 skipped0.76463.36chosen-answer Brier 0.195915.8%answered 91%167 ms203 ms$0.026$25.64
d1.1 Closed · Efficient

d1.1 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 46.0% (95%: 43.2%–49.0%)

Math 65.2%

Its run record (Method) →

—46.0%43.2%–49.0%mnopqrstuvw518 / 1,12548 right of 48 kept0 mistakes · 1,077 skipped0.74863.14chosen-answer Brier 0.197790.6%answered 28%176 ms272 ms$0.006$6.02
StartLux-Decision-v1.1-4B-Instant Open · 4.0B

StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

Parameters: 4B dense

Accuracy 45.9% (95%: 43.0%–48.8%)

Math 55.8%

Its run record (Method) →

—45.9%43.0%–48.8%nopqrstuvx516 / 1,12521 right of 21 kept0 mistakes · 1,104 skipped0.72062.63chosen-answer Brier 0.206116.0%answered 91%73 ms84 ms$0.005$4.62estimate · DGX Spark
Dexy-Decision-9B-V2 Open · 9.0B

Dexy-Decision-9B-V2 · Patrick-18 · decision model

Parameters: 9B dense

Base model: vllm-sr/d3-flash

Accuracy 45.3% (95%: 42.4%–48.3%)

Math 58.8%

Its run record (Method) →

—45.3%42.4%–48.3%opqrstuvwxy510 / 1,12556 right of 56 kept0 mistakes · 1,069 skipped0.75961.81chosen-answer Brier 0.217112.4%answered 93%47 ms53 ms$0.034$33.81estimate · RTX PRO 6000 Blackwell
Mercury Decide Closed · Efficient

Mercury Decide · Inception · decision model

Parameters: not disclosed by maker

Accuracy 45.1% (95%: 42.2%–48.0%)

Math 53.5%

Its run record (Method) →

—45.1%42.2%–48.0%pqrstuvwxy507 / 1,12584 right of 91 kept7 wrong · 1,034 skipped0.74661.18chosen-answer Brier 0.227220.5%answered 88%301 ms360 ms$0.008$7.52
Perplexity Decider Flash v1.3 Open · 4.0B

Perplexity Decider Flash v1.3 · Perplexity · decision model

Parameters: 4B dense

Accuracy 44.7% (95%: 41.8%–47.6%)

Math 55.9%

Its run record (Method) →

—44.7%41.8%–47.6%pqrstuvwxy503 / 1,12517 right of 17 kept0 mistakes · 1,108 skipped0.75762.14chosen-answer Brier 0.204224.4%answered 85%33 ms41 ms$0.026$26.44estimate · RTX PRO 6000 Blackwell
Microsoft-Decision-1 Closed · Efficient

Microsoft-Decision-1 · Microsoft · decision model

Parameters: not disclosed by maker

Base model: Qwen3.5-9B

Accuracy 44.4% (95%: 41.6%–47.4%)

Math 59.4%

Its run record (Method) →

—44.4%41.6%–47.4%pqrstuvwxy500 / 1,12520 right of 20 kept0 mistakes · 1,105 skipped0.71261.53chosen-answer Brier 0.213919.8%answered 88%315 ms362 ms$0.013$13.42
Valen-Decision-1.5-9B Open · 9.0B

Valen-Decision-1.5-9B · Valen-Team · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 44.4% (95%: 41.5%–47.3%)

Math 55.3%

Its run record (Method) →

—44.4%41.5%–47.3%pqrstuvwxy499 / 1,12535 right of 37 kept2 wrong · 1,088 skipped0.70161.42chosen-answer Brier 0.215113.6%answered 92%60 ms67 ms$0.042$42.42estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen1 Open · 9.0B

KnowLine-9B-Gen1 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 44.3% (95%: 41.4%–47.2%)

Math 59.8%

Its run record (Method) →

—44.3%41.4%–47.2%pqrstuvwxy498 / 1,12530 right of 30 kept0 mistakes · 1,095 skipped0.72561.97chosen-answer Brier 0.20326.7%answered 96%50 ms61 ms$0.036$36.27estimate · RTX PRO 6000 Blackwell
vera-medium-omni Open · 4.2B

vera-medium-omni · scar-ai · decision model

Parameters: 4.2B dense

Base model: Qwen3.5-4B

Accuracy 44.1% (95%: 41.2%–47.0%)

Math 56.8%

Its run record (Method) →

—44.1%41.2%–47.0%qrstuvwxy496 / 1,1256 right of 6 kept0 mistakes · 1,119 skipped0.73961.90chosen-answer Brier 0.202813.6%answered 92%32 ms37 ms$0.026$25.82estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen2 Open · 9.0B

KnowLine-9B-Gen2 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Accuracy 43.9% (95%: 41.0%–46.8%)

Math 60.1%

Its run record (Method) →

—43.9%41.0%–46.8%qrstuvwxy494 / 1,125No answers at this threshold0.73061.90chosen-answer Brier 0.201119.5%answered 88%52 ms65 ms$0.040$39.96estimate · RTX PRO 6000 Blackwell
Cerno 9B v1.2 Open · 9.0B

Cerno 9B v1.2 · Apertix AI · decision model

Parameters: 9B dense

Accuracy 43.3% (95%: 40.4%–46.2%)

Math 56.0%

Its run record (Method) →

—43.3%40.4%–46.2%rstuvwxy487 / 1,12526 right of 27 kept1 wrong · 1,098 skipped0.75260.41chosen-answer Brier 0.22476.0%answered 97%209 ms269 ms$0.013$13.35estimate · DGX Spark
Perplexity Decider Flash v1.2 Open · 4.0B

Perplexity Decider Flash v1.2 · Perplexity · decision model

Parameters: 4B dense

Accuracy 43.1% (95%: 40.2%–46.0%)

Math 55.2%

Its run record (Method) →

—43.1%40.2%–46.0%qrstuvwxy485 / 1,12535 right of 35 kept0 mistakes · 1,090 skipped0.72961.34chosen-answer Brier 0.204328.9%answered 82%45 ms55 ms$0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell
Open-Jev-27B-v1.1 Open · 27B

Open-Jev-27B-v1.1 · Zefan Cai · decision model

Parameters: 27B dense

Base model: Qwen3.8-27B + adapter

Accuracy 42.6% (95%: 39.7%–45.5%)

Math 54.6%

Its run record (Method) →

—42.6%39.7%–45.5%rstuvwxy479 / 1,125No answers at this threshold0.73359.78chosen-answer Brier 0.230228.2%answered 82%413 ms576 ms$0.294$294estimate · RTX PRO 6000 Blackwell
KnowLine 4B Gen5.5 Open · 4.0B

KnowLine 4B Gen5.5 · PelaAI · decision model

Parameters: 4B dense

Accuracy 42.3% (95%: 39.5%–45.2%)

Math 56.0%

Its run record (Method) →

—42.3%39.5%–45.2%stuvwxy476 / 1,12517 right of 18 kept1 wrong · 1,107 skipped0.70859.66chosen-answer Brier 0.22988.3%answered 95%96 ms115 ms$0.006$5.89estimate · DGX Spark
Drex 2.0 Open · 35B MoE · 3.0B active

Drex 2.0 · Nace.AI · decision model

Parameters: 35B MoE · 3B active

Accuracy 41.9% (95%: 39.0%–44.8%)

Math 59.0%

Its run record (Method) →

—41.9%39.0%–44.8%stuvwxy471 / 1,12533 right of 33 kept0 mistakes · 1,092 skipped0.71760.73chosen-answer Brier 0.204037.7%answered 75%345 ms504 ms$0.018$18.41
Clef Open · 27B

Clef · Cloudflare · decision model

Parameters: 27B dense

Accuracy 40.8% (95%: 38.0%–43.7%)

Math 51.1%

Its run record (Method) →

—40.8%38.0%–43.7%uvwxy459 / 1,1256 right of 6 kept0 mistakes · 1,119 skipped0.69758.55chosen-answer Brier 0.23693.6%answered 98%196 ms222 ms$0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell
d1 Closed · Efficient

d1 · Liquid AI · decision model

Parameters: not disclosed by maker

Accuracy 40.8% (95%: 38.0%–43.7%)

Math 55.6%

Its run record (Method) →

—40.8%38.0%–43.7%tuvwxy459 / 1,12555 right of 56 kept1 wrong · 1,069 skipped0.72359.76chosen-answer Brier 0.212811.3%answered 93%188 ms290 ms$0.006$6.02
OneJev-27B Open · 27B

OneJev-27B · OmniJev · decision model

Parameters: 27B dense

Accuracy 40.6% (95%: 37.8%–43.5%)

Math 53.8%

Its run record (Method) →

—40.6%37.8%–43.5%tuvwxy457 / 1,1254 right of 4 kept0 mistakes · 1,121 skipped0.72660.27chosen-answer Brier 0.20091.2%answered 99%108 ms136 ms$0.086$86.07estimate · RTX PRO 6000 Blackwell
Chinese Decision Pointer 4B Open · 4.0B

Chinese Decision Pointer 4B · quincyqiang · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Accuracy 40.4% (95%: 37.5%–43.3%)

Math 50.4%

Its run record (Method) →

—40.4%37.5%–43.3%vwxy454 / 1,12539 right of 39 kept0 mistakes · 1,086 skipped0.72458.90chosen-answer Brier 0.225614.6%answered 91%38 ms48 ms$0.030$29.51estimate · RTX PRO 6000 Blackwell
Verdin-4B-v1.1 Open · 4.0B

Verdin-4B-v1.1 · victorbennett · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Accuracy 39.7% (95%: 36.9%–42.6%)

Math 51.5%

Its run record (Method) →

—39.7%36.9%–42.6%wy447 / 1,12535 right of 35 kept0 mistakes · 1,090 skipped0.72959.26chosen-answer Brier 0.212212.6%answered 92%45 ms51 ms$0.033$33.20estimate · RTX PRO 6000 Blackwell
Winnow-12B Open · 12B

Winnow-12B · EldanRing · decision model

Parameters: 12B dense

Accuracy 39.7% (95%: 36.9%–42.6%)

Math 46.9%

Its run record (Method) →

—39.7%36.9%–42.6%xy447 / 1,12532 right of 32 kept0 mistakes · 1,093 skipped0.72758.43chosen-answer Brier 0.228729.7%answered 80%66 ms91 ms$0.052$51.64estimate · RTX PRO 6000 Blackwell
Cygnet Open · 12B

Cygnet · Blockbrain Labs · decision model

Parameters: 11.95B dense

Base model: Gemma 4 12B IT

Accuracy 39.3% (95%: 36.5%–42.2%)

Math 44.3%

Its run record (Method) →

—39.3%36.5%–42.2%xy442 / 1,125No answers at this threshold0.72058.97chosen-answer Brier 0.213511.3%answered 93%40 ms48 ms$0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell
Clef-Flash Open · 9.0B

Clef-Flash · Cloudflare · decision model

Parameters: 9B dense

Accuracy 38.9% (95%: 36.1%–41.8%)

Math 49.8%

Its run record (Method) →

—38.9%36.1%–41.8%yz438 / 1,125No answers at this threshold0.65957.70chosen-answer Brier 0.235314.4%answered 91%110 ms137 ms$0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell
GLiDE 2B Open · 1.9B

GLiDE 2B · Fastino · decision model

Parameters: 1.88B dense

Accuracy 32.3% (95%: 29.6%–35.1%)

Math 36.8%

Its run record (Method) →

—32.3%29.6%–35.1%zA363 / 1,12514 right of 14 kept0 mistakes · 1,111 skipped0.63254.39chosen-answer Brier 0.234932.9%answered 75%34 ms42 ms$0.002$2.26estimate · DGX Spark
Metask-Jev-4B Open · 4.5B

Metask-Jev-4B · Metask Lab · decision model

Parameters: 4.54B dense

Accuracy 32.1% (95%: 29.4%–34.9%)

Math 39.2%

Its run record (Method) →

—32.1%29.4%–34.9%zA361 / 1,125No answers at this threshold0.66056.30chosen-answer Brier 0.19501.5%answered 99%123 ms152 ms$0.091$90.99estimate · RTX PRO 6000 Blackwell
KnowLine-2B-Gen1 Open · 2.0B

KnowLine-2B-Gen1 · PelaAI · decision model

Parameters: 2B dense

Base model: Qwen3.5-2B

Accuracy 31.7% (95%: 29.1%–34.5%)

Math 35.3%

Its run record (Method) →

—31.7%29.1%–34.5%A357 / 1,1258 right of 8 kept0 mistakes · 1,117 skipped0.60155.15chosen-answer Brier 0.214315.7%answered 89%59 ms70 ms$0.004$3.78estimate · DGX Spark
StartLux-Decision-v1.1-2B-Instant Open · 2.0B

StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

Parameters: 2B dense

Accuracy 30.8% (95%: 28.1%–33.5%)

Math 38.7%

Its run record (Method) →

—30.8%28.1%–33.5%AB346 / 1,1252 right of 2 kept0 mistakes · 1,123 skipped0.64055.51chosen-answer Brier 0.197531.4%answered 76%40 ms45 ms$0.003$2.52estimate · DGX Spark
d1-2B Open · 1.9B

d1-2B · Liquid AI · decision model

Parameters: 1.92B dense

Base model: LFM3-VL-2B

Accuracy 30.7% (95%: 28.0%–33.4%)

Math 37.1%

Its run record (Method) →

—30.7%28.0%–33.4%AB345 / 1,1253 right of 3 kept0 mistakes · 1,122 skipped0.62055.05chosen-answer Brier 0.205886.5%answered 22%17 ms23 ms$0.015$14.76estimate · RTX PRO 6000 Blackwell
GLiDE 0.8B Open · 0.8B

GLiDE 0.8B · Fastino · decision model

Parameters: 0.75B dense

Accuracy 29.2% (95%: 26.6%–31.9%)

Math 32.1%

Its run record (Method) →

—29.2%26.6%–31.9%ABC328 / 1,1259 right of 9 kept0 mistakes · 1,116 skipped0.61753.53chosen-answer Brier 0.221026.1%answered 80%23 ms29 ms$0.002$1.58estimate · DGX Spark
OneJev1.6-0.8B Open · 0.8B

OneJev1.6-0.8B · OmniJev · decision model

Parameters: 0.8B dense

Accuracy 27.0% (95%: 24.5%–29.7%)

Math 28.2%

Its run record (Method) →

—27.0%24.5%–29.7%ABC304 / 1,1252 right of 2 kept0 mistakes · 1,123 skipped0.54552.72chosen-answer Brier 0.21594.6%answered 97%16 ms19 ms$0.001$1.16estimate · DGX Spark
KnowLine-0.8B-Gen1 Open · 0.8B

KnowLine-0.8B-Gen1 · PelaAI · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Accuracy 26.7% (95%: 24.2%–29.3%)

Math 28.2%

Its run record (Method) →

—26.7%24.2%–29.3%ABC300 / 1,1252 right of 2 kept0 mistakes · 1,123 skipped0.57353.45chosen-answer Brier 0.19761.2%answered 99%31 ms39 ms$0.002$2.00estimate · DGX Spark
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B

StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Accuracy 24.7% (95%: 22.3%–27.3%)

Math 29.7%

Its run record (Method) →

—24.7%22.3%–27.3%BC278 / 1,125No answers at this threshold0.64153.22chosen-answer Brier 0.182721.4%answered 83%27 ms31 ms$0.002$1.79estimate · DGX Spark
GLiClass v4 Edge Open · 0.4B

GLiClass v4 Edge · decision model

Parameters: 0.4B dense

Base model: DeBERTa-v3-large

Accuracy 23.6% (95%: 21.2%–26.1%)

Math 22.7%

Its run record (Method) →

—23.6%21.2%–26.1%BC265 / 1,125No answers at this threshold0.52751.99chosen-answer Brier 0.195812.4%answered 90%15 ms28 ms$0.001$1.31estimate · DGX Spark
GLiDE 0.4B Open · 0.4B

GLiDE 0.4B · Fastino · decision model

Parameters: 0.40B dense

Accuracy 23.2% (95%: 20.8%–25.8%)

Math 20.6%

Its run record (Method) →

—23.2%20.8%–25.8%C261 / 1,125No answers at this threshold0.55652.19chosen-answer Brier 0.188376.1%answered 30%13 ms16 ms$0.001$0.95estimate · DGX Spark
Strands Decider 2B Open · 2.0B

Strands Decider 2B · Strands Agents · decision model

Parameters: 2B dense

Accuracy 22.4% (95%: 20.1%–24.9%)

Math 28.7%

Its run record (Method) →

—22.4%20.1%–24.9%C252 / 1,125No answers at this threshold0.56851.86chosen-answer Brier 0.18688.3%answered 94%36 ms43 ms$0.028$27.67estimate · RTX PRO 6000 Blackwell

Tie groups and “not measured”

estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.

Decision Score (AA formula): Artificial Analysis's formula applied to our questions. Not their Index. 50 × (1 + accuracy − chosen-answer Brier), the chosen-answer Brier over the probability of the chosen answer, on decision models' standard runs only. Chat models and the “I don't know” runs are excluded.