Logic and Math · Confidence

Confidence on Logic and Math

Logic and Math pooled: 3,288 questions; only setups that finished both. How to read this page · Method

Answers kept at the ≥99% confidence threshold

Share of the N questions whose chosen option had probability ≥99% in the five-option (standard) run: right (family colour) and wrong (red); beside it the counts right / wrong, whether any was wrong, and the AUROC of the chosen option's probability (right against wrong answers, 0.5 = no signal) · more kept with no mistake is better

  1. Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell

    Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 385 answers, 385 right · 0 wrong · AUROC 0.844

    Logic 54.8% · Math 72.5%

    Its run record (Method) →

    385 / 0✓ 0 mistakes0.844
  2. Copernicus-31B

    Copernicus-31B · Poindexter Labs · decision model

    Parameters: 31B dense

    Kept at ≥99%: 384 answers, 384 right · 0 wrong · AUROC 0.873

    Logic 58.7% · Math 77.3%

    Its run record (Method) →

    384 / 0✓ 0 mistakes0.873
  3. Jev 1.13

    Jev 1.13 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 313 answers, 313 right · 0 wrong · AUROC 0.817

    Logic 46.3% · Math 64.1%

    Its run record (Method) →

    313 / 0✓ 0 mistakes0.817
  4. Jev 1.14

    Jev 1.14 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 263 answers, 263 right · 0 wrong · AUROC 0.810

    Logic 48.2% · Math 65.1%

    Its run record (Method) →

    263 / 0✓ 0 mistakes0.810
  5. Gemini Decision 1 20261111

    Gemini Decision 1 20261111 · Google · decision model

    Parameters: not disclosed by maker (closed weights)

    Kept at ≥99%: 257 answers, 257 right · 0 wrong · AUROC 0.838

    Logic 50.0% · Math 71.3%

    Its run record (Method) →

    257 / 0✓ 0 mistakes0.838
  6. d1-ultra

    d1-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

    Kept at ≥99%: 233 answers, 233 right · 0 wrong · AUROC 0.852

    Logic 54.9% · Math 74.1%

    Its run record (Method) →

    233 / 0✓ 0 mistakes0.852
  7. d1.1

    d1.1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 231 answers, 231 right · 0 wrong · AUROC 0.811

    Logic 46.0% · Math 65.2%

    Its run record (Method) →

    231 / 0✓ 0 mistakes0.811
  8. REAX 1.3

    REAX 1.3 · REAX · decision model

    Parameters: 57.36B dense

    Kept at ≥99%: 229 answers, 229 right · 0 wrong · AUROC 0.878

    Logic 59.3% · Math 78.0%

    Its run record (Method) →

    229 / 0✓ 0 mistakes0.878
  9. GLiDE API

    GLiDE API · Fastino · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 213 answers, 213 right · 0 wrong · AUROC 0.852

    Logic 54.5% · Math 70.8%

    Its run record (Method) →

    213 / 0✓ 0 mistakes0.852
  10. Drex 2.0

    Drex 2.0 · Nace.AI · decision model

    Parameters: 35B MoE · 3B active

    Kept at ≥99%: 207 answers, 207 right · 0 wrong · AUROC 0.789

    Logic 41.9% · Math 59.0%

    Its run record (Method) →

    207 / 0✓ 0 mistakes0.789
  11. GLiDE v1.2

    GLiDE v1.2 · Fastino · decision model

    Parameters: 27B dense

    Kept at ≥99%: 176 answers, 176 right · 0 wrong · AUROC 0.843

    Logic 55.2% · Math 69.9%

    Its run record (Method) →

    176 / 0✓ 0 mistakes0.843
  12. d1.2

    d1.2 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 169 answers, 169 right · 0 wrong · AUROC 0.812

    Logic 49.2% · Math 65.5%

    Its run record (Method) →

    169 / 0✓ 0 mistakes0.812
  13. KnowLine-9B-Gen1

    KnowLine-9B-Gen1 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 153 answers, 153 right · 0 wrong · AUROC 0.782

    Logic 44.3% · Math 59.8%

    Its run record (Method) →

    153 / 0✓ 0 mistakes0.782
  14. d1.2-ultra

    d1.2-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 128 answers, 128 right · 0 wrong · AUROC 0.856

    Logic 51.6% · Math 69.9%

    Its run record (Method) →

    128 / 0✓ 0 mistakes0.856
  15. StartLux-Decision-v1.1-4B-Instant

    StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

    Parameters: 4B dense

    Kept at ≥99%: 117 answers, 117 right · 0 wrong · AUROC 0.763

    Logic 45.9% · Math 55.8%

    Its run record (Method) →

    117 / 0✓ 0 mistakes0.763
  16. GLiDE 9B

    GLiDE 9B · Fastino · decision model

    Parameters: 7.94B dense

    Kept at ≥99%: 105 answers, 105 right · 0 wrong · AUROC 0.786

    Logic 47.8% · Math 57.9%

    Its run record (Method) →

    105 / 0✓ 0 mistakes0.786
  17. Jeffan-9B-1211

    Jeffan-9B-1211 · tattrongvu · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 74 answers, 74 right · 0 wrong · AUROC 0.798

    Logic 48.2% · Math 59.8%

    Its run record (Method) →

    74 / 0✓ 0 mistakes0.798
  18. StartLux-Decision-v1.1-9B-Instant

    StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

    Parameters: 9B dense

    Kept at ≥99%: 63 answers, 63 right · 0 wrong · AUROC 0.779

    Logic 46.6% · Math 58.4%

    Its run record (Method) →

    63 / 0✓ 0 mistakes0.779
  19. Turion Jev 31B

    Turion Jev 31B · Turion.AI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 50 answers, 50 right · 0 wrong · AUROC 0.844

    Logic 52.3% · Math 69.0%

    Its run record (Method) →

    50 / 0✓ 0 mistakes0.844
  20. Microsoft-Decision-1

    Microsoft-Decision-1 · Microsoft · decision model

    Parameters: not disclosed by maker

    Base model: Qwen3.5-9B

    Kept at ≥99%: 47 answers, 47 right · 0 wrong · AUROC 0.769

    Logic 44.4% · Math 59.4%

    Its run record (Method) →

    47 / 0✓ 0 mistakes0.769
  21. d3-ultra

    d3-ultra · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Kept at ≥99%: 39 answers, 39 right · 0 wrong · AUROC 0.835

    Logic 54.6% · Math 71.5%

    Its run record (Method) →

    39 / 0✓ 0 mistakes0.835
  22. vera-medium-omni

    vera-medium-omni · scar-ai · decision model

    Parameters: 4.2B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 39 answers, 39 right · 0 wrong · AUROC 0.777

    Logic 44.1% · Math 56.8%

    Its run record (Method) →

    39 / 0✓ 0 mistakes0.777
  23. OneJev-27B

    OneJev-27B · OmniJev · decision model

    Parameters: 27B dense

    Kept at ≥99%: 30 answers, 30 right · 0 wrong · AUROC 0.770

    Logic 40.6% · Math 53.8%

    Its run record (Method) →

    30 / 0✓ 0 mistakes0.770
  24. Logits Sigma-1 Duo

    Logits Sigma-1 Duo · Logits · decision model

    Parameters: 57.36B dense

    Kept at ≥99%: 23 answers, 23 right · 0 wrong · AUROC 0.868

    Logic 57.8% · Math 75.2%

    Its run record (Method) →

    23 / 0✓ 0 mistakes0.868
  25. Clef

    Clef · Cloudflare · decision model

    Parameters: 27B dense

    Kept at ≥99%: 18 answers, 18 right · 0 wrong · AUROC 0.747

    Logic 40.8% · Math 51.1%

    Its run record (Method) →

    18 / 0✓ 0 mistakes0.747
  26. KnowLine-2B-Gen1

    KnowLine-2B-Gen1 · PelaAI · decision model

    Parameters: 2B dense

    Base model: Qwen3.5-2B

    Kept at ≥99%: 16 answers, 16 right · 0 wrong · AUROC 0.626

    Logic 31.7% · Math 35.3%

    Its run record (Method) →

    16 / 0✓ 0 mistakes0.626
  27. GLiDE 0.8B

    GLiDE 0.8B · Fastino · decision model

    Parameters: 0.75B dense

    Kept at ≥99%: 13 answers, 13 right · 0 wrong · AUROC 0.602

    Logic 29.2% · Math 32.1%

    Its run record (Method) →

    13 / 0✓ 0 mistakes0.602
  28. KnowLine-12B-Gen1

    KnowLine-12B-Gen1 · PelaAI · decision model

    Parameters: 12B dense

    Base model: Gemma 4 12B

    Kept at ≥99%: 13 answers, 13 right · 0 wrong · AUROC 0.810

    Logic 50.6% · Math 66.3%

    Its run record (Method) →

    13 / 0✓ 0 mistakes0.810
  29. d1-2B

    d1-2B · Liquid AI · decision model

    Parameters: 1.92B dense

    Base model: LFM3-VL-2B

    Kept at ≥99%: 7 answers, 7 right · 0 wrong · AUROC 0.648

    Logic 30.7% · Math 37.1%

    Its run record (Method) →

    7 / 0✓ 0 mistakes0.648
  30. StartLux-Decision-v1.1-2B-Instant

    StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

    Parameters: 2B dense

    Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.675

    Logic 30.8% · Math 38.7%

    Its run record (Method) →

    4 / 0✓ 0 mistakes0.675
  31. KnowLine-0.8B-Gen1

    KnowLine-0.8B-Gen1 · PelaAI · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.590

    Logic 26.7% · Math 28.2%

    Its run record (Method) →

    2 / 0✓ 0 mistakes0.590
  32. OneJev1.6-0.8B

    OneJev1.6-0.8B · OmniJev · decision model

    Parameters: 0.8B dense

    Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.576

    Logic 27.0% · Math 28.2%

    Its run record (Method) →

    2 / 0✓ 0 mistakes0.576
  33. Clef-Flash

    Clef-Flash · Cloudflare · decision model

    Parameters: 9B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.707

    Logic 38.9% · Math 49.8%

    Its run record (Method) →

    No answers at this threshold–0.707
  34. Cygnet

    Cygnet · Blockbrain Labs · decision model

    Parameters: 11.95B dense

    Base model: Gemma 4 12B IT

    Kept at ≥99%: No answers at this threshold · AUROC 0.729

    Logic 39.3% · Math 44.3%

    Its run record (Method) →

    No answers at this threshold–0.729
  35. GLiClass v4 Edge

    GLiClass v4 Edge · decision model

    Parameters: 0.4B dense

    Base model: DeBERTa-v3-large

    Kept at ≥99%: No answers at this threshold · AUROC 0.511

    Logic 23.6% · Math 22.7%

    Its run record (Method) →

    No answers at this threshold–0.511
  36. GLiDE 0.4B

    GLiDE 0.4B · Fastino · decision model

    Parameters: 0.40B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.520

    Logic 23.2% · Math 20.6%

    Its run record (Method) →

    No answers at this threshold–0.520
  37. KnowLine-9B-Gen2

    KnowLine-9B-Gen2 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: No answers at this threshold · AUROC 0.786

    Logic 43.9% · Math 60.1%

    Its run record (Method) →

    No answers at this threshold–0.786
  38. Metask-Jev-4B

    Metask-Jev-4B · Metask Lab · decision model

    Parameters: 4.54B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.659

    Logic 32.1% · Math 39.2%

    Its run record (Method) →

    No answers at this threshold–0.659
  39. Open-Jev-27B-v1.1

    Open-Jev-27B-v1.1 · Zefan Cai · decision model

    Parameters: 27B dense

    Base model: Qwen3.8-27B + adapter

    Kept at ≥99%: No answers at this threshold · AUROC 0.759

    Logic 42.6% · Math 54.6%

    Its run record (Method) →

    No answers at this threshold–0.759
  40. StartLux-Decision-v1.1-0.8B-Instant

    StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Kept at ≥99%: No answers at this threshold · AUROC 0.605

    Logic 24.7% · Math 29.7%

    Its run record (Method) →

    No answers at this threshold–0.605
  41. Strands Decider 2B

    Strands Decider 2B · Strands Agents · decision model

    Parameters: 2B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.573

    Logic 22.4% · Math 28.7%

    Its run record (Method) →

    No answers at this threshold–0.573
  42. celeris-1-decision

    celeris-1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: No answers at this threshold · AUROC 0.829

    Logic 47.3% · Math 70.0%

    Its run record (Method) →

    No answers at this threshold–0.829
  43. celeris-1.1-decision

    celeris-1.1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: No answers at this threshold · AUROC 0.830

    Logic 47.3% · Math 69.9%

    Its run record (Method) →

    No answers at this threshold–0.830
  44. Karpo Decision 31B

    Karpo Decision 31B · MachinePulse · decision model

    Parameters: 31B dense

    Kept at ≥99%: 1,009 answers, 1,006 right · 3 wrong · AUROC 0.882

    Logic 60.7% · Math 81.6%

    Its run record (Method) →

    1,006 / 33 wrong0.882
  45. Drex Max

    Drex Max · Nace.AI · decision model

    Parameters: 27B dense

    Kept at ≥99%: 884 answers, 866 right · 18 wrong · AUROC 0.841

    Logic 53.1% · Math 70.3%

    Its run record (Method) →

    866 / 1818 wrong0.841
  46. KnowLine-31B-Gen1.5

    KnowLine-31B-Gen1.5 · PelaAI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 554 answers, 553 right · 1 wrong · AUROC 0.870

    Logic 59.0% · Math 77.4%

    Its run record (Method) →

    553 / 11 wrong0.870
  47. KnowLine-31B-Gen2

    KnowLine-31B-Gen2 · PelaAI · decision model

    Parameters: 31B dense

    Base model: Gemma 4 31B

    Kept at ≥99%: 524 answers, 523 right · 1 wrong · AUROC 0.882

    Logic 58.9% · Math 80.3%

    Its run record (Method) →

    523 / 11 wrong0.882
  48. Mammoth-1

    Mammoth-1 · Base Labs · decision model

    Parameters: 27B dense

    Kept at ≥99%: 520 answers, 518 right · 2 wrong · AUROC 0.868

    Logic 54.5% · Math 74.0%

    Its run record (Method) →

    518 / 22 wrong0.868
  49. SubQ Snap 1.1

    SubQ Snap 1.1 · Subquadratic · decision model

    Parameters: not disclosed by maker

    Shown once with SubQ Snap Preview: the same answer to every question

    Kept at ≥99%: 433 answers, 430 right · 3 wrong · AUROC 0.827

    Logic 52.4% · Math 68.6%

    Its run record (Method) →

    430 / 33 wrong0.827
  50. Perplexity Decider 27B v1.2 API

    Perplexity Decider 27B v1.2 API · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 411 answers, 409 right · 2 wrong · AUROC 0.849

    Logic 53.3% · Math 69.8%

    Its run record (Method) →

    409 / 22 wrong0.849
  51. Drex Max v1.1

    Drex Max v1.1 · Nace.AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 405 answers, 404 right · 1 wrong · AUROC 0.846

    Logic 54.8% · Math 70.1%

    Its run record (Method) →

    404 / 11 wrong0.846
  52. Perplexity Decider 27B v1.2 API · with web

    Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 405 answers, 404 right · 1 wrong · AUROC 0.854

    Logic 52.5% · Math 69.3%

    Its run record (Method) →

    404 / 11 wrong0.854
  53. Mercury Decide

    Mercury Decide · Inception · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 372 answers, 356 right · 16 wrong · AUROC 0.777

    Logic 45.1% · Math 53.5%

    Its run record (Method) →

    356 / 1616 wrong0.777
  54. Sieve 31B v3

    Sieve 31B v3 · Sthānika AI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 296 answers, 294 right · 2 wrong · AUROC 0.865

    Logic 57.9% · Math 77.1%

    Its run record (Method) →

    294 / 22 wrong0.865
  55. GLiDE 27B

    GLiDE 27B · Fastino · decision model

    Parameters: 27B dense

    Kept at ≥99%: 274 answers, 272 right · 2 wrong · AUROC 0.852

    Logic 58.4% · Math 72.0%

    Its run record (Method) →

    272 / 22 wrong0.852
  56. Cerno 9B v1.2

    Cerno 9B v1.2 · Apertix AI · decision model

    Parameters: 9B dense

    Kept at ≥99%: 281 answers, 270 right · 11 wrong · AUROC 0.776

    Logic 43.3% · Math 56.0%

    Its run record (Method) →

    270 / 1111 wrong0.776
  57. Dexy-Decision-9B-V2

    Dexy-Decision-9B-V2 · Patrick-18 · decision model

    Parameters: 9B dense

    Base model: vllm-sr/d3-flash

    Kept at ≥99%: 266 answers, 263 right · 3 wrong · AUROC 0.795

    Logic 45.3% · Math 58.8%

    Its run record (Method) →

    263 / 33 wrong0.795
  58. Adjudex 9B v5

    Adjudex 9B v5 · newsbang · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 253 answers, 251 right · 2 wrong · AUROC 0.802

    Logic 46.4% · Math 62.1%

    Its run record (Method) →

    251 / 22 wrong0.802
  59. Elephant-1

    Elephant-1 · Base Labs · decision model

    Parameters: 9B dense

    Kept at ≥99%: 236 answers, 235 right · 1 wrong · AUROC 0.797

    Logic 46.4% · Math 61.3%

    Its run record (Method) →

    235 / 11 wrong0.797
  60. d3

    d3 · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Kept at ≥99%: 217 answers, 216 right · 1 wrong · AUROC 0.858

    Logic 54.5% · Math 70.4%

    Its run record (Method) →

    216 / 11 wrong0.858
  61. d1

    d1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 208 answers, 207 right · 1 wrong · AUROC 0.757

    Logic 40.8% · Math 55.6%

    Its run record (Method) →

    207 / 11 wrong0.757
  62. Chinese Decision Pointer 4B

    Chinese Decision Pointer 4B · quincyqiang · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 194 answers, 188 right · 6 wrong · AUROC 0.743

    Logic 40.4% · Math 50.4%

    Its run record (Method) →

    188 / 66 wrong0.743
  63. Perplexity Decider Flash v1.2

    Perplexity Decider Flash v1.2 · Perplexity · decision model

    Parameters: 4B dense

    Kept at ≥99%: 190 answers, 185 right · 5 wrong · AUROC 0.769

    Logic 43.1% · Math 55.2%

    Its run record (Method) →

    185 / 55 wrong0.769
  64. Valen-Decision-1.5-9B

    Valen-Decision-1.5-9B · Valen-Team · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 167 answers, 165 right · 2 wrong · AUROC 0.765

    Logic 44.4% · Math 55.3%

    Its run record (Method) →

    165 / 22 wrong0.765
  65. Verdin-4B-v1.1

    Verdin-4B-v1.1 · victorbennett · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 165 answers, 161 right · 4 wrong · AUROC 0.747

    Logic 39.7% · Math 51.5%

    Its run record (Method) →

    161 / 44 wrong0.747
  66. Perplexity Decider Flash v1.3

    Perplexity Decider Flash v1.3 · Perplexity · decision model

    Parameters: 4B dense

    Kept at ≥99%: 131 answers, 130 right · 1 wrong · AUROC 0.782

    Logic 44.7% · Math 55.9%

    Its run record (Method) →

    130 / 11 wrong0.782
  67. KnowLine 4B Gen5.5

    KnowLine 4B Gen5.5 · PelaAI · decision model

    Parameters: 4B dense

    Kept at ≥99%: 72 answers, 71 right · 1 wrong · AUROC 0.752

    Logic 42.3% · Math 56.0%

    Its run record (Method) →

    71 / 11 wrong0.752
  68. Winnow-12B

    Winnow-12B · EldanRing · decision model

    Parameters: 12B dense

    Kept at ≥99%: 53 answers, 52 right · 1 wrong · AUROC 0.737

    Logic 39.7% · Math 46.9%

    Its run record (Method) →

    52 / 11 wrong0.737
  69. GLiDE 2B

    GLiDE 2B · Fastino · decision model

    Parameters: 1.88B dense

    Kept at ≥99%: 30 answers, 29 right · 1 wrong · AUROC 0.637

    Logic 32.3% · Math 36.8%

    Its run record (Method) →

    29 / 11 wrong0.637

Perplexity Poindexter Labs TypeSafe Google Liquid AI REAX Fastino Nace.AI PelaAI StartLux tattrongvu Turion.AI Microsoft vLLM Semantic Router scar-ai OmniJev Logits Cloudflare Blockbrain Labs Other Metask Lab Zefan Cai Strands Agents Celeris MachinePulse Base Labs Subquadratic Inception Sthānika AI Apertix AI Patrick-18 newsbang quincyqiang Valen-Team victorbennett EldanRing Kept but wrong

Accuracy by confidence cut-off

SetupWebAll answersaccuracy · N≥50%accuracy of the kept · share kept≥70%accuracy of the kept · share kept≥90%accuracy of the kept · share kept≥99%accuracy of the kept · share kept
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—66.4%3,28878.1%75.1% kept (2,469)88.0%56.4% kept (1,853)95.7%38.0% kept (1,249)100.0%11.7% kept (385)
Copernicus-31B Open · 31B—71.0%3,28889.0%64.6% kept (2,123)96.0%48.3% kept (1,588)99.3%31.5% kept (1,036)100.0%11.7% kept (384)
Jev 1.13 Closed · Efficient—58.0%3,28877.2%58.2% kept (1,913)89.6%35.8% kept (1,178)97.6%20.3% kept (669)100.0%9.5% kept (313)
Jev 1.14 Closed · Efficient—59.3%3,28879.8%54.3% kept (1,784)91.0%34.0% kept (1,117)97.2%19.5% kept (640)100.0%8.0% kept (263)
Gemini Decision 1 20261111 Closed · Efficient—64.1%3,28884.1%59.2% kept (1,948)92.5%42.8% kept (1,407)98.1%27.6% kept (906)100.0%7.8% kept (257)
d1-ultra Closed · Efficient—67.5%3,28884.9%64.9% kept (2,134)92.4%49.0% kept (1,610)98.2%32.2% kept (1,058)100.0%7.1% kept (233)
d1.1 Closed · Efficient—58.6%3,28879.6%53.7% kept (1,767)90.9%35.0% kept (1,150)97.0%20.4% kept (670)100.0%7.0% kept (231)
REAX 1.3 Open · 57B—71.6%3,28887.1%70.6% kept (2,320)96.0%50.3% kept (1,655)99.5%33.4% kept (1,098)100.0%7.0% kept (229)
GLiDE API Closed · Efficient—65.2%3,28879.2%71.0% kept (2,336)90.3%50.6% kept (1,664)98.3%30.3% kept (996)100.0%6.5% kept (213)
Drex 2.0 Open · 35B MoE · 3.0B active—53.1%3,28879.7%42.5% kept (1,397)90.7%27.2% kept (893)97.0%16.2% kept (532)100.0%6.3% kept (207)
GLiDE v1.2 Open · 27B—64.8%3,28878.2%70.3% kept (2,312)91.8%48.6% kept (1,597)98.8%28.0% kept (922)100.0%5.4% kept (176)
d1.2 Closed · Undisclosed—59.9%3,28881.2%52.3% kept (1,718)93.0%34.0% kept (1,118)98.4%19.5% kept (640)100.0%5.1% kept (169)
KnowLine-9B-Gen1 Open · 9.0B—54.5%3,28875.0%49.1% kept (1,615)88.6%29.8% kept (981)98.0%16.4% kept (539)100.0%4.7% kept (153)
d1.2-ultra Closed · Undisclosed—63.6%3,28884.7%59.1% kept (1,944)93.4%41.6% kept (1,369)99.1%25.0% kept (822)100.0%3.9% kept (128)
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—52.4%3,28870.2%53.6% kept (1,762)81.4%32.8% kept (1,078)94.7%16.5% kept (543)100.0%3.6% kept (117)
GLiDE 9B Open · 7.9B—54.5%3,28869.6%61.2% kept (2,011)85.4%35.9% kept (1,182)95.3%17.4% kept (571)100.0%3.2% kept (105)
Jeffan-9B-1211 Open · 9.0B—55.8%3,28876.0%53.9% kept (1,771)89.5%32.9% kept (1,082)98.4%15.2% kept (501)100.0%2.3% kept (74)
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—54.4%3,28874.0%51.4% kept (1,689)85.0%31.3% kept (1,029)97.0%15.4% kept (505)100.0%1.9% kept (63)
Turion Jev 31B Open · 31B—63.3%3,28884.0%58.2% kept (1,913)92.7%41.7% kept (1,372)99.4%21.0% kept (692)100.0%1.5% kept (50)
Microsoft-Decision-1 Closed · Efficient—54.3%3,28870.4%57.1% kept (1,876)86.3%30.5% kept (1,003)97.2%11.8% kept (389)100.0%1.4% kept (47)
d3-ultra Open · 26B—65.7%3,28873.5%81.9% kept (2,694)82.7%62.6% kept (2,057)96.3%37.4% kept (1,231)100.0%1.2% kept (39)
vera-medium-omni Open · 4.2B—52.5%3,28874.9%46.3% kept (1,523)88.0%23.3% kept (766)99.1%9.8% kept (322)100.0%1.2% kept (39)
OneJev-27B Open · 27B—49.3%3,28880.3%32.3% kept (1,062)92.2%18.7% kept (616)98.6%8.5% kept (281)100.0%0.9% kept (30)
Logits Sigma-1 Duo Open · 57B—69.3%3,28885.2%68.0% kept (2,236)96.8%45.1% kept (1,483)99.5%22.6% kept (742)100.0%0.7% kept (23)
Clef Open · 27B—47.6%3,28861.4%59.6% kept (1,959)75.3%33.0% kept (1,086)89.6%14.7% kept (483)100.0%0.5% kept (18)
KnowLine-2B-Gen1 Open · 2.0B—34.1%3,28851.8%24.8% kept (816)67.8%8.3% kept (273)88.1%2.6% kept (84)100.0%0.5% kept (16)
GLiDE 0.8B Open · 0.8B—31.1%3,28839.8%37.4% kept (1,230)49.1%14.3% kept (470)68.9%3.1% kept (103)100.0%0.4% kept (13)
KnowLine-12B-Gen1 Open · 12B—60.9%3,28881.1%55.3% kept (1,817)91.7%36.4% kept (1,197)99.1%16.2% kept (534)100.0%0.4% kept (13)
d1-2B Open · 1.9B—34.9%3,28856.6%21.7% kept (712)74.2%6.7% kept (221)88.2%2.1% kept (68)100.0%0.2% kept (7)
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—35.9%3,28866.0%18.4% kept (606)81.0%7.1% kept (232)94.4%2.2% kept (72)100.0%0.1% kept (4)
KnowLine-0.8B-Gen1 Open · 0.8B—27.7%3,28843.1%13.7% kept (450)57.9%3.3% kept (107)88.9%0.5% kept (18)100.0%0.1% kept (2)
OneJev1.6-0.8B Open · 0.8B—27.8%3,28839.9%15.5% kept (509)50.0%4.0% kept (130)71.4%0.6% kept (21)100.0%0.1% kept (2)
Clef-Flash Open · 9.0B—46.1%3,28863.4%43.4% kept (1,427)74.6%22.4% kept (735)89.5%7.0% kept (229)none kept
Cygnet Open · 12B—42.6%3,28863.2%42.4% kept (1,394)79.1%16.0% kept (527)94.7%4.0% kept (132)none kept
GLiClass v4 Edge Open · 0.4B—23.0%3,28831.9%5.5% kept (182)33.3%1.2% kept (39)57.1%0.2% kept (7)none kept
GLiDE 0.4B Open · 0.4B—21.5%3,28826.9%5.5% kept (182)36.6%1.2% kept (41)100.0%0.1% kept (3)none kept
KnowLine-9B-Gen2 Open · 9.0B—54.6%3,28878.4%45.4% kept (1,492)91.4%25.4% kept (836)99.6%8.2% kept (269)none kept
Metask-Jev-4B Open · 4.5B—36.7%3,28864.0%17.2% kept (567)78.6%8.1% kept (266)89.1%1.7% kept (55)none kept
Open-Jev-27B-v1.1 Open · 27B—50.5%3,28895.7%8.6% kept (282)100.0%0.9% kept (29)none keptnone kept
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—28.0%3,28846.1%10.1% kept (332)62.7%3.6% kept (118)91.3%0.7% kept (23)none kept
Strands Decider 2B Open · 2.0B—26.5%3,28844.4%9.8% kept (322)65.1%1.3% kept (43)100.0%0.1% kept (3)none kept
celeris-1-decision Closed · Efficient—62.2%3,28875.8%69.1% kept (2,271)87.8%48.8% kept (1,606)95.4%32.0% kept (1,053)none kept
celeris-1.1-decision Closed · Efficient—62.1%3,28876.2%68.8% kept (2,262)87.9%48.6% kept (1,598)95.7%32.2% kept (1,060)none kept
Karpo Decision 31B Open · 31B—74.4%3,28885.1%79.4% kept (2,610)93.0%63.7% kept (2,095)97.7%48.1% kept (1,580)99.7%30.7% kept (1,009)
Drex Max Open · 27B—64.4%3,28872.4%81.7% kept (2,686)80.8%64.5% kept (2,120)91.5%47.3% kept (1,556)98.0%26.9% kept (884)
KnowLine-31B-Gen1.5 Open · 31B—71.1%3,28886.5%69.3% kept (2,279)93.9%53.0% kept (1,744)98.5%35.2% kept (1,156)99.8%16.8% kept (554)
KnowLine-31B-Gen2 Open · 31B—73.0%3,28887.1%72.9% kept (2,396)93.9%56.8% kept (1,868)99.0%38.8% kept (1,275)99.8%15.9% kept (524)
Mammoth-1 Open · 27B—67.3%3,28882.3%71.0% kept (2,334)92.4%53.0% kept (1,744)97.9%37.3% kept (1,228)99.6%15.8% kept (520)
SubQ Snap 1.1 Closed · Efficient—63.0%3,28873.0%76.7% kept (2,522)82.6%58.1% kept (1,911)92.7%38.6% kept (1,268)99.3%13.2% kept (433)
Perplexity Decider 27B v1.2 API Open · 27B—64.1%3,28881.5%64.6% kept (2,125)92.6%46.9% kept (1,541)97.8%31.4% kept (1,031)99.5%12.5% kept (411)
Drex Max v1.1 Closed · Efficient (provisional)—64.8%3,28877.5%72.3% kept (2,377)88.6%53.3% kept (1,752)97.0%35.3% kept (1,162)99.8%12.3% kept (405)
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web63.6%3,28880.6%65.2% kept (2,145)91.8%47.0% kept (1,545)97.4%31.3% kept (1,028)99.8%12.3% kept (405)
Mercury Decide Closed · Efficient—50.6%3,28864.0%65.1% kept (2,142)73.9%45.8% kept (1,505)83.8%28.6% kept (941)95.7%11.3% kept (372)
Sieve 31B v3 Open · 31B—70.5%3,28886.8%67.7% kept (2,227)94.3%51.4% kept (1,691)98.7%34.4% kept (1,131)99.3%9.0% kept (296)
GLiDE 27B Open · 27B—67.3%3,28880.6%72.4% kept (2,381)90.6%53.0% kept (1,742)98.5%32.5% kept (1,068)99.3%8.3% kept (274)
Cerno 9B v1.2 Open · 9.0B—51.7%3,28862.1%70.1% kept (2,306)73.5%47.7% kept (1,569)86.8%27.2% kept (895)96.1%8.5% kept (281)
Dexy-Decision-9B-V2 Open · 9.0B—54.2%3,28866.2%69.4% kept (2,281)78.0%47.1% kept (1,549)91.9%26.2% kept (863)98.9%8.1% kept (266)
Adjudex 9B v5 Open · 9.0B—56.7%3,28873.2%60.4% kept (1,985)85.5%38.9% kept (1,279)96.2%22.3% kept (732)99.2%7.7% kept (253)
Elephant-1 Open · 9.0B—56.2%3,28873.2%59.5% kept (1,957)86.1%38.2% kept (1,256)95.6%22.0% kept (724)99.6%7.2% kept (236)
d3 Open · 26B—65.0%3,28882.6%65.4% kept (2,150)94.1%46.2% kept (1,518)98.6%28.2% kept (927)99.5%6.6% kept (217)
d1 Closed · Efficient—50.5%3,28869.8%50.3% kept (1,653)82.8%28.5% kept (938)93.6%14.7% kept (484)99.5%6.3% kept (208)
Chinese Decision Pointer 4B Open · 4.0B—47.0%3,28859.2%60.7% kept (1,996)72.9%34.5% kept (1,133)85.5%17.6% kept (578)96.9%5.9% kept (194)
Perplexity Decider Flash v1.2 Open · 4.0B—51.1%3,28868.4%55.4% kept (1,820)79.9%36.8% kept (1,211)89.0%20.5% kept (674)97.4%5.8% kept (190)
Valen-Decision-1.5-9B Open · 9.0B—51.6%3,28871.4%49.0% kept (1,611)84.7%27.3% kept (897)94.0%14.2% kept (466)98.8%5.1% kept (167)
Verdin-4B-v1.1 Open · 4.0B—47.5%3,28866.3%47.5% kept (1,563)79.5%26.8% kept (881)90.5%13.8% kept (454)97.6%5.0% kept (165)
Perplexity Decider Flash v1.3 Open · 4.0B—52.1%3,28866.8%62.0% kept (2,039)77.7%41.7% kept (1,371)90.0%22.1% kept (728)99.2%4.0% kept (131)
KnowLine 4B Gen5.5 Open · 4.0B—51.3%3,28863.8%62.0% kept (2,038)76.6%37.0% kept (1,215)91.5%17.8% kept (585)98.6%2.2% kept (72)
Winnow-12B Open · 12B—44.4%3,28862.2%46.6% kept (1,531)73.1%31.1% kept (1,022)83.8%17.2% kept (567)98.1%1.6% kept (53)
GLiDE 2B Open · 1.9B—35.2%3,28845.7%44.7% kept (1,469)55.7%19.5% kept (641)75.7%5.5% kept (181)96.7%0.9% kept (30)

Answering only where the chosen option's probability reaches the cut-off: the accuracy of the answers kept and, under it, the share of the N questions kept.

Confidence by topic

SetupWebAllkept right / wrong · AUROCLogic · Numerical reasoningkept R / W · AUROCLogic · Verbal reasoningkept R / W · AUROCLogic · Formal logickept R / W · AUROCMath · Basic conceptskept R / W · AUROCMath · Digits and place valuekept R / W · AUROCMath · Number baseskept R / W · AUROCMath · Division and divisibilitykept R / W · AUROCMath · Prime factorisation and GCD/LCMkept R / W · AUROCMath · Rational numberskept R / W · AUROCMath · Decimalskept R / W · AUROCMath · Orderingkept R / W · AUROCMath · Simple inequalitieskept R / W · AUROCMath · Absolute valuekept R / W · AUROCMath · Exponentskept R / W · AUROCMath · Radicalskept R / W · AUROCMath · Factorisationkept R / W · AUROC
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—385 / 00.84422 / 00.7525 / 00.76435 / 00.87826 / 00.8288 / 00.7760 / 00.5778 / 00.78515 / 00.84517 / 00.79036 / 00.93527 / 00.83925 / 00.90022 / 00.93335 / 00.84549 / 00.87155 / 00.834
Copernicus-31B Open · 31B—384 / 00.87315 / 00.74612 / 00.77539 / 00.93527 / 00.90113 / 00.8651 / 00.8009 / 00.88117 / 00.89220 / 00.88142 / 00.98012 / 00.88425 / 00.87619 / 00.93729 / 00.88248 / 00.91456 / 00.875
Jev 1.13 Closed · Efficient—313 / 00.8177 / 00.7085 / 00.71453 / 00.87517 / 00.8368 / 00.7560 / 00.6719 / 00.7618 / 00.7979 / 00.83523 / 00.89420 / 00.80130 / 00.86817 / 00.83124 / 00.82141 / 00.85042 / 00.818
Jev 1.14 Closed · Efficient—263 / 00.8105 / 00.6718 / 00.71148 / 00.85616 / 00.8547 / 00.7910 / 00.6868 / 00.7005 / 00.7788 / 00.81018 / 00.91014 / 00.78825 / 00.86218 / 00.82621 / 00.78127 / 00.84935 / 00.834
Gemini Decision 1 20261111 Closed · Efficient—257 / 00.8385 / 00.7660 / 00.69938 / 00.88515 / 00.8626 / 00.7942 / 00.7290 / 00.7895 / 00.85410 / 00.85118 / 00.88213 / 00.79723 / 00.83315 / 00.79931 / 00.85938 / 00.90738 / 00.858
d1-ultra Closed · Efficient—233 / 00.85212 / 00.7593 / 00.7155 / 00.84623 / 00.8658 / 00.7550 / 00.6505 / 00.8139 / 00.89613 / 00.88712 / 00.91013 / 00.94118 / 00.90213 / 00.89320 / 00.84530 / 00.88549 / 00.888
d1.1 Closed · Efficient—231 / 00.8114 / 00.6766 / 00.71238 / 00.90613 / 00.8115 / 00.7680 / 00.7144 / 00.7213 / 00.8257 / 00.79616 / 00.84814 / 00.85419 / 00.87015 / 00.84923 / 00.81530 / 00.79834 / 00.835
REAX 1.3 Open · 57B—229 / 00.87812 / 00.75011 / 00.80428 / 00.92111 / 00.8887 / 00.8261 / 00.8187 / 00.8699 / 00.88611 / 00.91730 / 00.95510 / 00.94614 / 00.8817 / 00.93522 / 00.88725 / 00.91324 / 00.900
GLiDE API Closed · Efficient—213 / 00.8527 / 00.7777 / 00.76538 / 00.86617 / 00.8745 / 00.7461 / 00.62511 / 00.7784 / 00.8534 / 00.84020 / 00.98713 / 00.87229 / 00.88511 / 00.92415 / 00.89617 / 00.81114 / 00.881
Drex 2.0 Open · 35B MoE · 3.0B active—207 / 00.7893 / 00.6520 / 00.69030 / 00.8158 / 00.8196 / 00.8170 / 00.5330 / 00.7405 / 00.79813 / 00.84120 / 00.9059 / 00.76816 / 00.83714 / 00.76622 / 00.76235 / 00.83626 / 00.803
GLiDE v1.2 Open · 27B—176 / 00.8433 / 00.7326 / 00.73936 / 00.90612 / 00.8573 / 00.7400 / 00.5285 / 00.7774 / 00.8345 / 00.83820 / 00.96314 / 00.86522 / 00.9127 / 00.94815 / 00.87314 / 00.84910 / 00.877
d1.2 Closed · Undisclosed—169 / 00.8123 / 00.6762 / 00.71923 / 00.8748 / 00.8335 / 00.8000 / 00.5573 / 00.7083 / 00.8409 / 00.79420 / 00.8968 / 00.86212 / 00.84310 / 00.79724 / 00.84320 / 00.83819 / 00.845
KnowLine-9B-Gen1 Open · 9.0B—153 / 00.7823 / 00.6380 / 00.69227 / 00.84313 / 00.8165 / 00.7120 / 00.5956 / 00.6746 / 00.8035 / 00.76019 / 00.8736 / 00.79914 / 00.7927 / 00.78017 / 00.80311 / 00.82114 / 00.817
d1.2-ultra Closed · Undisclosed—128 / 00.8564 / 00.7414 / 00.7725 / 00.8376 / 00.8915 / 00.8391 / 00.8894 / 00.7744 / 00.8515 / 00.80118 / 00.96010 / 00.8558 / 00.91210 / 00.90215 / 00.85911 / 00.85518 / 00.868
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—117 / 00.7631 / 00.6332 / 00.67218 / 00.85311 / 00.8181 / 00.7070 / 00.5242 / 00.6762 / 00.7437 / 00.74818 / 00.8146 / 00.80811 / 00.8066 / 00.74412 / 00.76812 / 00.7788 / 00.775
GLiDE 9B Open · 7.9B—105 / 00.7863 / 00.6374 / 00.72925 / 00.9117 / 00.8002 / 00.7370 / 00.7313 / 00.7001 / 00.7164 / 00.75215 / 00.8053 / 00.77410 / 00.8734 / 00.8537 / 00.7768 / 00.8519 / 00.765
Jeffan-9B-1211 Open · 9.0B—74 / 00.7982 / 00.6635 / 00.7043 / 00.8664 / 00.8654 / 00.6770 / 00.7693 / 00.7323 / 00.8443 / 00.74515 / 00.8114 / 00.7976 / 00.8882 / 00.8255 / 00.8047 / 00.8088 / 00.768
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—63 / 00.7790 / 00.6753 / 00.71311 / 00.8543 / 00.7771 / 00.6780 / 00.3941 / 00.7031 / 00.7303 / 00.73511 / 00.8873 / 00.7746 / 00.8035 / 00.8304 / 00.7752 / 00.7889 / 00.779
Turion Jev 31B Open · 31B—50 / 00.8441 / 00.7020 / 00.7566 / 00.8755 / 00.8681 / 00.7700 / 00.7571 / 00.8090 / 00.9043 / 00.8173 / 00.9261 / 00.8546 / 00.8231 / 00.9408 / 00.8535 / 00.8669 / 00.852
Microsoft-Decision-1 Closed · Efficient—47 / 00.7691 / 00.6282 / 00.68317 / 00.8235 / 00.7890 / 00.6960 / 00.6151 / 00.6700 / 00.7470 / 00.7504 / 00.8484 / 00.8065 / 00.8371 / 00.8012 / 00.7683 / 00.8242 / 00.794
d3-ultra Open · 26B—39 / 00.8359 / 00.70710 / 00.7619 / 00.8531 / 00.8421 / 00.7780 / 00.7640 / 00.8200 / 00.7310 / 00.8273 / 00.9420 / 00.8191 / 00.9012 / 00.8481 / 00.8711 / 00.8501 / 00.874
vera-medium-omni Open · 4.2B—39 / 00.7770 / 00.6590 / 00.7106 / 00.8762 / 00.7823 / 00.7040 / 00.6361 / 00.6951 / 00.8361 / 00.70613 / 00.8600 / 00.8482 / 00.8371 / 00.7974 / 00.7701 / 00.8094 / 00.784
OneJev-27B Open · 27B—30 / 00.7700 / 00.6330 / 00.7124 / 00.8452 / 00.7680 / 00.6450 / 00.8061 / 00.7500 / 00.7671 / 00.7782 / 00.8841 / 00.7581 / 00.8113 / 00.7874 / 00.7576 / 00.7855 / 00.789
Logits Sigma-1 Duo Open · 57B—23 / 00.8681 / 00.7381 / 00.7646 / 00.8802 / 00.9000 / 00.8410 / 00.6670 / 00.8471 / 00.8880 / 00.8722 / 00.9500 / 00.9191 / 00.9101 / 00.9394 / 00.8951 / 00.8713 / 00.911
Clef Open · 27B—18 / 00.7472 / 00.6070 / 00.7044 / 00.8462 / 00.7511 / 00.7080 / 00.8752 / 00.7230 / 00.7200 / 00.7983 / 00.7780 / 00.7931 / 00.7440 / 00.8782 / 00.7480 / 00.8021 / 00.774
KnowLine-2B-Gen1 Open · 2.0B—16 / 00.6260 / 00.5010 / 00.5718 / 00.7560 / 00.5700 / 00.5780 / 00.5330 / 00.5080 / 00.5400 / 00.6550 / 00.6671 / 00.7794 / 00.7361 / 00.6991 / 00.6440 / 00.6561 / 00.587
GLiDE 0.8B Open · 0.8B—13 / 00.6020 / 00.5570 / 00.6089 / 00.7001 / 00.5380 / 00.4980 / 00.4670 / 00.6100 / 00.5790 / 00.5250 / 00.7101 / 00.6040 / 00.6010 / 00.6611 / 00.6730 / 00.5641 / 00.616
KnowLine-12B-Gen1 Open · 12B—13 / 00.8101 / 00.7011 / 00.7114 / 00.9372 / 00.8380 / 00.7360 / 00.8140 / 00.7520 / 00.8400 / 00.8335 / 00.9150 / 00.8000 / 00.8240 / 00.9160 / 00.7750 / 00.8570 / 00.794
d1-2B Open · 1.9B—7 / 00.6480 / 00.5490 / 00.6063 / 00.7870 / 00.6340 / 00.6290 / 00.3180 / 00.5721 / 00.5820 / 00.5890 / 00.7211 / 00.7781 / 00.6250 / 00.7181 / 00.7090 / 00.6570 / 00.677
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—4 / 00.6750 / 00.5390 / 00.6522 / 00.7670 / 00.6610 / 00.6010 / 00.3330 / 00.5600 / 00.6650 / 00.6550 / 00.7880 / 00.8131 / 00.7550 / 00.7030 / 00.6140 / 00.6581 / 00.679
KnowLine-0.8B-Gen1 Open · 0.8B—2 / 00.5900 / 00.5200 / 00.5312 / 00.7460 / 00.4860 / 00.6030 / 00.6670 / 00.6280 / 00.5730 / 00.4900 / 00.6830 / 00.7920 / 00.6600 / 00.6530 / 00.6030 / 00.5570 / 00.565
OneJev1.6-0.8B Open · 0.8B—2 / 00.5760 / 00.5080 / 00.5382 / 00.6080 / 00.5480 / 00.6010 / 00.5330 / 00.5190 / 00.6780 / 00.4740 / 00.5430 / 00.7260 / 00.6900 / 00.6190 / 00.6050 / 00.6090 / 00.559
Clef-Flash Open · 9.0B—No answers at this threshold0.7070 / 00.5370 / 00.6700 / 00.7900 / 00.7110 / 00.6420 / 00.3610 / 00.6140 / 00.7360 / 00.6700 / 00.7660 / 00.7890 / 00.7440 / 00.8230 / 00.7220 / 00.8040 / 00.737
Cygnet Open · 12B—No answers at this threshold0.7290 / 00.6040 / 00.6760 / 00.8480 / 00.7300 / 00.6170 / 00.6000 / 00.6900 / 00.6390 / 00.7260 / 00.8000 / 00.7880 / 00.7960 / 00.6970 / 00.6800 / 00.7680 / 00.761
GLiClass v4 Edge Open · 0.4B—No answers at this threshold0.5110 / 00.5160 / 00.5470 / 00.5840 / 00.5240 / 00.5540 / 00.5500 / 00.4760 / 00.4740 / 00.5050 / 00.4120 / 00.5220 / 00.4120 / 00.4600 / 00.5660 / 00.5430 / 00.518
GLiDE 0.4B Open · 0.4B—No answers at this threshold0.5200 / 00.5310 / 00.5450 / 00.6390 / 00.4960 / 00.5280 / 00.8750 / 00.5090 / 00.6060 / 00.4580 / 00.3700 / 00.6320 / 00.4500 / 00.4780 / 00.4930 / 00.4980 / 00.515
KnowLine-9B-Gen2 Open · 9.0B—No answers at this threshold0.7860 / 00.6720 / 00.6630 / 00.8590 / 00.7880 / 00.7110 / 00.7120 / 00.6710 / 00.8010 / 00.8190 / 00.8340 / 00.8410 / 00.8470 / 00.8240 / 00.7740 / 00.8060 / 00.817
Metask-Jev-4B Open · 4.5B—No answers at this threshold0.6590 / 00.5910 / 00.6040 / 00.8300 / 00.6140 / 00.5440 / 00.9330 / 00.6170 / 00.7370 / 00.6130 / 00.6330 / 00.6190 / 00.6660 / 00.7270 / 00.6600 / 00.6240 / 00.664
Open-Jev-27B-v1.1 Open · 27B—No answers at this threshold0.7590 / 00.6230 / 00.7160 / 00.8610 / 00.8160 / 00.6070 / 00.6250 / 00.6740 / 00.7310 / 00.7510 / 00.7620 / 00.7450 / 00.8470 / 00.8180 / 00.7760 / 00.7270 / 00.749
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—No answers at this threshold0.6050 / 00.5800 / 00.6150 / 00.6940 / 00.6110 / 00.4600 / 00.6670 / 00.5460 / 00.4400 / 00.6120 / 00.5670 / 00.6930 / 00.6720 / 00.6760 / 00.5510 / 00.5030 / 00.580
Strands Decider 2B Open · 2.0B—No answers at this threshold0.5730 / 00.5440 / 00.5620 / 00.6340 / 00.5100 / 00.4580 / 00.6150 / 00.5340 / 00.5890 / 00.5830 / 00.6350 / 00.6150 / 00.5740 / 00.6320 / 00.5560 / 00.5420 / 00.627
celeris-1-decision Closed · Efficient—No answers at this threshold0.8290 / 00.7060 / 00.6790 / 00.8670 / 00.8570 / 00.7870 / 00.8070 / 00.7940 / 00.8840 / 00.8960 / 00.9520 / 00.8400 / 00.8810 / 00.8360 / 00.8380 / 00.8690 / 00.849
celeris-1.1-decision Closed · Efficient—No answers at this threshold0.8300 / 00.7090 / 00.6790 / 00.8530 / 00.8630 / 00.7900 / 00.8750 / 00.7900 / 00.8630 / 00.8960 / 00.9440 / 00.8530 / 00.8770 / 00.8120 / 00.8520 / 00.8670 / 00.855
Karpo Decision 31B Open · 31B—1006 / 30.88258 / 00.80533 / 20.75878 / 10.87785 / 00.89934 / 00.8013 / 00.83334 / 00.88038 / 00.93350 / 00.89083 / 00.95245 / 00.87966 / 00.91553 / 00.95091 / 00.902114 / 00.893141 / 00.904
Drex Max Open · 27B—866 / 180.84134 / 00.72939 / 50.75874 / 30.83867 / 10.83425 / 00.8033 / 00.75032 / 00.79928 / 00.86736 / 40.79463 / 00.92342 / 20.87775 / 00.89444 / 00.90883 / 00.87696 / 20.855125 / 10.847
KnowLine-31B-Gen1.5 Open · 31B—553 / 10.87025 / 00.76514 / 00.76450 / 00.90438 / 00.87415 / 00.8501 / 00.75014 / 00.88122 / 00.90332 / 00.90060 / 00.95618 / 00.91235 / 00.87233 / 00.90455 / 10.88564 / 00.91477 / 00.888
KnowLine-31B-Gen2 Open · 31B—523 / 10.88226 / 00.7925 / 00.78523 / 00.90738 / 00.91615 / 00.8622 / 00.74214 / 00.84724 / 00.90237 / 00.92170 / 00.95612 / 00.87128 / 00.88228 / 00.91656 / 10.86368 / 00.90877 / 00.886
Mammoth-1 Open · 27B—518 / 20.86821 / 00.70313 / 00.79772 / 20.90527 / 00.8859 / 00.7661 / 00.78811 / 00.82114 / 00.87630 / 00.89741 / 00.90917 / 00.89347 / 00.93031 / 00.90453 / 00.90059 / 00.89672 / 00.903
SubQ Snap 1.1 Closed · Efficient—430 / 30.82717 / 00.73618 / 10.74335 / 10.85728 / 00.81912 / 00.7841 / 00.72215 / 00.78010 / 00.80819 / 00.75447 / 00.92935 / 00.82931 / 00.83228 / 00.93137 / 10.80245 / 00.87152 / 00.848
Perplexity Decider 27B v1.2 API Open · 27B—409 / 20.8499 / 00.73012 / 00.78357 / 20.79122 / 00.8616 / 00.8181 / 00.8068 / 00.77210 / 00.82316 / 00.81934 / 00.96134 / 00.90143 / 00.90333 / 00.93232 / 00.84047 / 00.87545 / 00.863
Drex Max v1.1 Closed · Efficient (provisional)—404 / 10.8467 / 00.72924 / 00.77158 / 10.87723 / 00.8398 / 00.8501 / 00.47114 / 00.7999 / 00.86115 / 00.81728 / 00.90331 / 00.84047 / 00.91030 / 00.91936 / 00.86639 / 00.85334 / 00.893
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web404 / 10.8548 / 00.72812 / 00.78758 / 10.86023 / 00.8509 / 00.8421 / 00.5716 / 00.82714 / 00.79918 / 00.85428 / 00.92736 / 00.86943 / 00.88731 / 00.94629 / 00.85141 / 00.88547 / 00.884
Mercury Decide Closed · Efficient—356 / 160.77714 / 20.67028 / 10.74442 / 40.77525 / 00.8119 / 00.6921 / 00.7298 / 00.7486 / 00.68711 / 30.71826 / 10.85923 / 20.80131 / 10.80818 / 00.82122 / 10.75144 / 10.78548 / 00.822
Sieve 31B v3 Open · 31B—294 / 20.8659 / 00.76416 / 20.74554 / 00.86124 / 00.8716 / 00.8480 / 00.9037 / 00.8568 / 00.89810 / 00.86326 / 00.95615 / 00.91822 / 00.90714 / 00.93924 / 00.89531 / 00.87928 / 00.848
GLiDE 27B Open · 27B—272 / 20.8526 / 00.75417 / 00.80051 / 10.88817 / 10.8586 / 00.7870 / 00.5698 / 00.7615 / 00.8467 / 00.87022 / 00.94820 / 00.93836 / 00.87318 / 00.90521 / 00.86120 / 00.84818 / 00.875
Cerno 9B v1.2 Open · 9.0B—270 / 110.7768 / 10.6453 / 00.74515 / 00.88124 / 00.8069 / 00.6771 / 10.5618 / 00.6796 / 00.79711 / 00.75131 / 00.84317 / 00.74722 / 20.80613 / 00.81231 / 20.72033 / 30.83638 / 20.776
Dexy-Decision-9B-V2 Open · 9.0B—263 / 30.7957 / 00.7129 / 00.68940 / 00.91018 / 10.7785 / 00.6840 / 00.6067 / 00.7317 / 10.7458 / 00.75929 / 10.83910 / 00.82918 / 00.82212 / 00.85333 / 00.82424 / 00.81436 / 00.820
Adjudex 9B v5 Open · 9.0B—251 / 20.8024 / 00.7062 / 00.73048 / 10.86918 / 00.8154 / 00.7010 / 00.7296 / 00.6848 / 00.7649 / 00.79731 / 00.86712 / 10.81421 / 00.82016 / 00.84023 / 00.81321 / 00.81228 / 00.806
Elephant-1 Open · 9.0B—235 / 10.7975 / 00.6719 / 00.72539 / 00.92514 / 00.8215 / 00.7110 / 00.4866 / 00.6846 / 00.82310 / 00.73028 / 00.83310 / 10.78515 / 00.86814 / 00.87925 / 00.77824 / 00.83425 / 00.774
d3 Open · 26B—216 / 10.8583 / 00.75511 / 00.76140 / 00.86313 / 10.8675 / 00.7911 / 00.8066 / 00.8117 / 00.9078 / 00.84721 / 00.95515 / 00.90425 / 00.90513 / 00.89321 / 00.88613 / 00.87514 / 00.872
d1 Closed · Efficient—207 / 10.7573 / 00.6305 / 00.68347 / 10.82814 / 00.7655 / 00.6530 / 00.5304 / 00.7003 / 00.7896 / 00.7529 / 00.76111 / 00.72621 / 00.80211 / 00.73820 / 00.76923 / 00.77725 / 00.784
Chinese Decision Pointer 4B Open · 4.0B—188 / 60.7431 / 00.6448 / 00.69930 / 00.86614 / 20.7935 / 00.6680 / 00.4245 / 00.7192 / 00.7975 / 20.65516 / 00.7358 / 10.77219 / 00.7327 / 00.76816 / 00.77124 / 10.72928 / 00.726
Perplexity Decider Flash v1.2 Open · 4.0B—185 / 50.7693 / 00.6421 / 00.69331 / 00.8797 / 10.7814 / 00.6880 / 00.3002 / 00.6943 / 00.7809 / 00.73226 / 00.88410 / 10.80721 / 00.85411 / 10.77315 / 10.73019 / 00.80623 / 10.753
Valen-Decision-1.5-9B Open · 9.0B—165 / 20.7655 / 10.5985 / 10.70925 / 00.78516 / 00.8373 / 00.6630 / 00.6677 / 00.7784 / 00.8495 / 00.75317 / 00.8769 / 00.76512 / 00.7777 / 00.83413 / 00.78216 / 00.79521 / 00.776
Verdin-4B-v1.1 Open · 4.0B—161 / 40.7472 / 00.6485 / 00.67228 / 00.88414 / 10.7315 / 00.7190 / 00.5484 / 00.6776 / 00.7438 / 10.76316 / 00.8248 / 00.77414 / 00.7999 / 20.77217 / 00.76112 / 00.77213 / 00.696
Perplexity Decider Flash v1.3 Open · 4.0B—130 / 10.7824 / 00.6631 / 00.73112 / 00.8606 / 00.7655 / 00.6950 / 00.4502 / 00.7013 / 00.8369 / 00.78823 / 00.9137 / 00.83012 / 00.7977 / 00.79211 / 00.76415 / 00.79213 / 10.765
KnowLine 4B Gen5.5 Open · 4.0B—71 / 10.7522 / 00.6343 / 10.64112 / 00.8813 / 00.7912 / 00.6110 / 00.5001 / 00.6881 / 00.8144 / 00.69911 / 00.8451 / 00.8427 / 00.8131 / 00.7427 / 00.7439 / 00.8037 / 00.747
Winnow-12B Open · 12B—52 / 10.7371 / 00.6086 / 00.67025 / 00.8355 / 00.7421 / 00.6380 / 00.6860 / 00.6701 / 00.6420 / 10.7431 / 00.7871 / 00.7425 / 00.7313 / 00.7851 / 00.7461 / 00.7581 / 00.774
GLiDE 2B Open · 1.9B—29 / 10.6371 / 00.5251 / 00.63712 / 00.7873 / 10.6250 / 00.5690 / 00.5191 / 00.5540 / 00.5901 / 00.6761 / 00.6461 / 00.7283 / 00.6632 / 00.5961 / 00.6171 / 00.6161 / 00.639

Per topic: answers kept at ≥99% (right / wrong) and, under them, the AUROC of the chosen option's probability (– where a topic has only right or only wrong answers).

SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 3,276 of 3,288); shown once.

Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.

Every setup, every measure

SetupWebAccuracy% of N (N ≤ 3,288) · Wilson 95%Tie groupHolm · McNemarRight / NquestionsKept at ≥99%right / wrongAUROCright vs wrongDecision Score (AA formula)0–100 · chosen-answer BrierNon-hallucinationof answers not right, % “I don't know” · higher is betterMedian latencymsp90ms, nearest rankCost$ per 1,000 qper 1M q
Karpo Decision 31B Open · 31B

Karpo Decision 31B · MachinePulse · decision model

Parameters: 31B dense

Kept at ≥99%: 1,009 answers, 1,006 right · 3 wrong · AUROC 0.882

Logic 60.7% · Math 81.6%

Its run record (Method) →

—74.4%72.9%–75.9%a2,447 / 3,2881,006 right of 1,009 kept3 wrong · 2,279 skipped0.88281.13chosen-answer Brier 0.12155.7%answered 98%162 ms198 ms$0.118$118estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen2 Open · 31B

KnowLine-31B-Gen2 · PelaAI · decision model

Parameters: 31B dense

Base model: Gemma 4 31B

Kept at ≥99%: 524 answers, 523 right · 1 wrong · AUROC 0.882

Logic 58.9% · Math 80.3%

Its run record (Method) →

—73.0%71.4%–74.5%ab2,399 / 3,288523 right of 524 kept1 wrong · 2,764 skipped0.88280.27chosen-answer Brier 0.12420.1%answered 100%116 ms177 ms$0.093$92.98estimate · RTX PRO 6000 Blackwell
REAX 1.3 Open · 57B

REAX 1.3 · REAX · decision model

Parameters: 57.36B dense

Kept at ≥99%: 229 answers, 229 right · 0 wrong · AUROC 0.878

Logic 59.3% · Math 78.0%

Its run record (Method) →

—71.6%70.0%–73.1%b2,354 / 3,288229 right of 229 kept0 mistakes · 3,059 skipped0.87879.30chosen-answer Brier 0.13003.8%answered 99%200 ms260 ms$0.148$148estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen1.5 Open · 31B

KnowLine-31B-Gen1.5 · PelaAI · decision model

Parameters: 31B dense

Kept at ≥99%: 554 answers, 553 right · 1 wrong · AUROC 0.870

Logic 59.0% · Math 77.4%

Its run record (Method) →

—71.1%69.5%–72.6%bc2,338 / 3,288553 right of 554 kept1 wrong · 2,734 skipped0.87078.92chosen-answer Brier 0.13270.7%answered 100%811 ms1,111 ms$0.052$51.85estimate · DGX Spark
Copernicus-31B Open · 31B

Copernicus-31B · Poindexter Labs · decision model

Parameters: 31B dense

Kept at ≥99%: 384 answers, 384 right · 0 wrong · AUROC 0.873

Logic 58.7% · Math 77.3%

Its run record (Method) →

—71.0%69.4%–72.5%bc2,333 / 3,288384 right of 384 kept0 mistakes · 2,904 skipped0.87378.76chosen-answer Brier 0.13441.8%answered 99%128 ms218 ms$0.102$102estimate · RTX PRO 6000 Blackwell
Sieve 31B v3 Open · 31B

Sieve 31B v3 · Sthānika AI · decision model

Parameters: 31B dense

Kept at ≥99%: 296 answers, 294 right · 2 wrong · AUROC 0.865

Logic 57.9% · Math 77.1%

Its run record (Method) →

—70.5%68.9%–72.1%bcd2,319 / 3,288294 right of 296 kept2 wrong · 2,992 skipped0.86578.49chosen-answer Brier 0.13543.8%answered 99%676 ms1,173 ms$0.047$47.29estimate · DGX Spark
Logits Sigma-1 Duo Open · 57B

Logits Sigma-1 Duo · Logits · decision model

Parameters: 57.36B dense

Kept at ≥99%: 23 answers, 23 right · 0 wrong · AUROC 0.868

Logic 57.8% · Math 75.2%

Its run record (Method) →

—69.3%67.7%–70.8%cde2,277 / 3,28823 right of 23 kept0 mistakes · 3,265 skipped0.86877.60chosen-answer Brier 0.14054.4%answered 99%207 ms324 ms$0.165$165estimate · RTX PRO 6000 Blackwell
d1-ultra Closed · Efficient

d1-ultra · Liquid AI · decision model

Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

Kept at ≥99%: 233 answers, 233 right · 0 wrong · AUROC 0.852

Logic 54.9% · Math 74.1%

Its run record (Method) →

—67.5%65.9%–69.1%defg2,221 / 3,288233 right of 233 kept0 mistakes · 3,055 skipped0.85276.50chosen-answer Brier 0.145589.1%answered 52%180 ms426 ms$0.008$8.37
GLiDE 27B Open · 27B

GLiDE 27B · Fastino · decision model

Parameters: 27B dense

Kept at ≥99%: 274 answers, 272 right · 2 wrong · AUROC 0.852

Logic 58.4% · Math 72.0%

Its run record (Method) →

—67.3%65.7%–68.9%efh2,214 / 3,288272 right of 274 kept2 wrong · 3,014 skipped0.85276.29chosen-answer Brier 0.147531.9%answered 89%450 ms587 ms$0.029$28.55estimate · DGX Spark
Mammoth-1 Open · 27B

Mammoth-1 · Base Labs · decision model

Parameters: 27B dense

Kept at ≥99%: 520 answers, 518 right · 2 wrong · AUROC 0.868

Logic 54.5% · Math 74.0%

Its run record (Method) →

—67.3%65.7%–68.9%efghi2,213 / 3,288518 right of 520 kept2 wrong · 2,768 skipped0.86876.68chosen-answer Brier 0.139471.4%answered 69%150 ms232 ms$0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B

Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 385 answers, 385 right · 0 wrong · AUROC 0.844

Logic 54.8% · Math 72.5%

Its run record (Method) →

—66.4%64.8%–68.0%fghij2,184 / 3,288385 right of 385 kept0 mistakes · 2,903 skipped0.84475.44chosen-answer Brier 0.155413.5%answered 95%97 ms125 ms$0.071$71.10estimate · RTX PRO 6000 Blackwell
d3-ultra Open · 26B

d3-ultra · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Kept at ≥99%: 39 answers, 39 right · 0 wrong · AUROC 0.835

Logic 54.6% · Math 71.5%

Its run record (Method) →

—65.7%64.1%–67.3%fghijk2,161 / 3,28839 right of 39 kept0 mistakes · 3,249 skipped0.83574.31chosen-answer Brier 0.17117.5%answered 97%129 ms168 ms$0.096$95.71estimate · RTX PRO 6000 Blackwell
GLiDE API Closed · Efficient

GLiDE API · Fastino · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 213 answers, 213 right · 0 wrong · AUROC 0.852

Logic 54.5% · Math 70.8%

Its run record (Method) →

—65.2%63.6%–66.8%fghijkl2,144 / 3,288213 right of 213 kept0 mistakes · 3,075 skipped0.85275.04chosen-answer Brier 0.151349.2%answered 80%266 ms284 ms$0.036$36.42
d3 Open · 26B

d3 · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Kept at ≥99%: 217 answers, 216 right · 1 wrong · AUROC 0.858

Logic 54.5% · Math 70.4%

Its run record (Method) →

—65.0%63.3%–66.6%fghijkl2,136 / 3,288216 right of 217 kept1 wrong · 3,071 skipped0.85875.16chosen-answer Brier 0.14649.3%answered 97%57 ms86 ms$0.046$46.28estimate · RTX PRO 6000 Blackwell
Drex Max v1.1 Closed · Efficient (provisional)

Drex Max v1.1 · Nace.AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 405 answers, 404 right · 1 wrong · AUROC 0.846

Logic 54.8% · Math 70.1%

Its run record (Method) →

—64.8%63.2%–66.5%fghijkl2,132 / 3,288404 right of 405 kept1 wrong · 2,883 skipped0.84674.61chosen-answer Brier 0.156248.3%answered 79%327 ms518 ms$0.016$16.13
GLiDE v1.2 Open · 27B

GLiDE v1.2 · Fastino · decision model

Parameters: 27B dense

Kept at ≥99%: 176 answers, 176 right · 0 wrong · AUROC 0.843

Logic 55.2% · Math 69.9%

Its run record (Method) →

—64.8%63.2%–66.5%gijkl2,132 / 3,288176 right of 176 kept0 mistakes · 3,112 skipped0.84374.68chosen-answer Brier 0.154842.3%answered 83%81 ms136 ms$0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell
Drex Max Open · 27B

Drex Max · Nace.AI · decision model

Parameters: 27B dense

Kept at ≥99%: 884 answers, 866 right · 18 wrong · AUROC 0.841

Logic 53.1% · Math 70.3%

Its run record (Method) →

—64.4%62.8%–66.0%gijkl2,118 / 3,288866 right of 884 kept18 wrong · 2,404 skipped0.84173.20chosen-answer Brier 0.18025.7%answered 98%81 ms130 ms$0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.2 API Open · 27B

Perplexity Decider 27B v1.2 API · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 411 answers, 409 right · 2 wrong · AUROC 0.849

Logic 53.3% · Math 69.8%

Its run record (Method) →

—64.1%62.5%–65.8%jklm2,109 / 3,288409 right of 411 kept2 wrong · 2,877 skipped0.84974.56chosen-answer Brier 0.150210.3%answered 96%230 ms439 ms$0.005$5.38
Gemini Decision 1 20261111 Closed · Efficient

Gemini Decision 1 20261111 · Google · decision model

Parameters: not disclosed by maker (closed weights)

Kept at ≥99%: 257 answers, 257 right · 0 wrong · AUROC 0.838

Logic 50.0% · Math 71.3%

Its run record (Method) →

—64.1%62.4%–65.7%hijklm2,106 / 3,288257 right of 257 kept0 mistakes · 3,031 skipped0.83874.23chosen-answer Brier 0.15593.0%answered 99%227 ms347 ms$0.027$27.07
d1.2-ultra Closed · Undisclosed

d1.2-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 128 answers, 128 right · 0 wrong · AUROC 0.856

Logic 51.6% · Math 69.9%

Its run record (Method) →

—63.6%61.9%–65.2%jklm2,091 / 3,288128 right of 128 kept0 mistakes · 3,160 skipped0.85674.31chosen-answer Brier 0.149890.2%answered 48%217 ms347 ms$0.008$8.37
Perplexity Decider 27B v1.2 API · with web Open · 27B

Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 405 answers, 404 right · 1 wrong · AUROC 0.854

Logic 52.5% · Math 69.3%

Its run record (Method) →

with web63.6%61.9%–65.2%klm2,090 / 3,288404 right of 405 kept1 wrong · 2,883 skipped0.85474.28chosen-answer Brier 0.150125.8%answered 90%430 ms707 ms$1.025$1,025
Turion Jev 31B Open · 31B

Turion Jev 31B · Turion.AI · decision model

Parameters: 31B dense

Kept at ≥99%: 50 answers, 50 right · 0 wrong · AUROC 0.844

Logic 52.3% · Math 69.0%

Its run record (Method) →

—63.3%61.6%–64.9%jklmn2,081 / 3,28850 right of 50 kept0 mistakes · 3,238 skipped0.84473.88chosen-answer Brier 0.15531.9%answered 99%301 ms448 ms$0.020$20.01estimate · DGX Spark
SubQ Snap 1.1 Closed · Efficient

SubQ Snap 1.1 · Subquadratic · decision model

Parameters: not disclosed by maker

Shown once with SubQ Snap Preview: the same answer to every question

Kept at ≥99%: 433 answers, 430 right · 3 wrong · AUROC 0.827

Logic 52.4% · Math 68.6%

Its run record (Method) →

—63.0%61.4%–64.7%klmn2,072 / 3,288430 right of 433 kept3 wrong · 2,855 skipped0.82772.66chosen-answer Brier 0.177058.3%answered 73%382 ms391 ms$0.011$10.75
celeris-1-decision Closed · Efficient

celeris-1-decision · Celeris · decision model

Parameters: not disclosed by maker

Kept at ≥99%: No answers at this threshold · AUROC 0.829

Logic 47.3% · Math 70.0%

Its run record (Method) →

—62.2%60.6%–63.9%lmno2,046 / 3,288No answers at this threshold0.82972.82chosen-answer Brier 0.165987.2%answered 50%388 ms392 ms$0.011$11.25
celeris-1.1-decision Closed · Efficient

celeris-1.1-decision · Celeris · decision model

Parameters: not disclosed by maker

Kept at ≥99%: No answers at this threshold · AUROC 0.830

Logic 47.3% · Math 69.9%

Its run record (Method) →

—62.1%60.5%–63.8%lmnop2,043 / 3,288No answers at this threshold0.83072.78chosen-answer Brier 0.165786.7%answered 51%381 ms388 ms$0.001$1.13
KnowLine-12B-Gen1 Open · 12B

KnowLine-12B-Gen1 · PelaAI · decision model

Parameters: 12B dense

Base model: Gemma 4 12B

Kept at ≥99%: 13 answers, 13 right · 0 wrong · AUROC 0.810

Logic 50.6% · Math 66.3%

Its run record (Method) →

—60.9%59.2%–62.5%mnopq2,002 / 3,28813 right of 13 kept0 mistakes · 3,275 skipped0.81071.86chosen-answer Brier 0.17164.5%answered 98%51 ms76 ms$0.041$41.44estimate · RTX PRO 6000 Blackwell
d1.2 Closed · Undisclosed

d1.2 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 169 answers, 169 right · 0 wrong · AUROC 0.812

Logic 49.2% · Math 65.5%

Its run record (Method) →

—59.9%58.2%–61.6%nopqr1,970 / 3,288169 right of 169 kept0 mistakes · 3,119 skipped0.81271.40chosen-answer Brier 0.171266.9%answered 65%208 ms390 ms$0.004$3.98
Jev 1.14 Closed · Efficient

Jev 1.14 · TypeSafe · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 263 answers, 263 right · 0 wrong · AUROC 0.810

Logic 48.2% · Math 65.1%

Its run record (Method) →

—59.3%57.6%–61.0%opqrs1,951 / 3,288263 right of 263 kept0 mistakes · 3,025 skipped0.81071.00chosen-answer Brier 0.173415.4%answered 93%164 ms188 ms$0.021$21.27
d1.1 Closed · Efficient

d1.1 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 231 answers, 231 right · 0 wrong · AUROC 0.811

Logic 46.0% · Math 65.2%

Its run record (Method) →

—58.6%56.9%–60.3%pqrs1,928 / 3,288231 right of 231 kept0 mistakes · 3,057 skipped0.81170.69chosen-answer Brier 0.172686.0%answered 46%174 ms285 ms$0.004$3.98
Jev 1.13 Closed · Efficient

Jev 1.13 · TypeSafe · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 313 answers, 313 right · 0 wrong · AUROC 0.817

Logic 46.3% · Math 64.1%

Its run record (Method) →

—58.0%56.3%–59.7%qrst1,907 / 3,288313 right of 313 kept0 mistakes · 2,975 skipped0.81770.41chosen-answer Brier 0.171810.2%answered 96%166 ms205 ms$0.021$21.27
Adjudex 9B v5 Open · 9.0B

Adjudex 9B v5 · newsbang · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 253 answers, 251 right · 2 wrong · AUROC 0.802

Logic 46.4% · Math 62.1%

Its run record (Method) →

—56.7%55.0%–58.4%rstu1,865 / 3,288251 right of 253 kept2 wrong · 3,035 skipped0.80269.27chosen-answer Brier 0.18187.0%answered 97%38 ms45 ms$0.028$28.19estimate · RTX PRO 6000 Blackwell
Elephant-1 Open · 9.0B

Elephant-1 · Base Labs · decision model

Parameters: 9B dense

Kept at ≥99%: 236 answers, 235 right · 1 wrong · AUROC 0.797

Logic 46.4% · Math 61.3%

Its run record (Method) →

—56.2%54.5%–57.9%stu1,849 / 3,288235 right of 236 kept1 wrong · 3,052 skipped0.79768.96chosen-answer Brier 0.183169.4%answered 60%80 ms94 ms$0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell
Jeffan-9B-1211 Open · 9.0B

Jeffan-9B-1211 · tattrongvu · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 74 answers, 74 right · 0 wrong · AUROC 0.798

Logic 48.2% · Math 59.8%

Its run record (Method) →

—55.8%54.1%–57.5%stu1,836 / 3,28874 right of 74 kept0 mistakes · 3,214 skipped0.79868.92chosen-answer Brier 0.179913.5%answered 94%39 ms53 ms$0.032$32.40estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen2 Open · 9.0B

KnowLine-9B-Gen2 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: No answers at this threshold · AUROC 0.786

Logic 43.9% · Math 60.1%

Its run record (Method) →

—54.6%52.9%–56.3%tuv1,794 / 3,288No answers at this threshold0.78667.99chosen-answer Brier 0.185810.4%answered 95%42 ms57 ms$0.034$34.08estimate · RTX PRO 6000 Blackwell
GLiDE 9B Open · 7.9B

GLiDE 9B · Fastino · decision model

Parameters: 7.94B dense

Kept at ≥99%: 105 answers, 105 right · 0 wrong · AUROC 0.786

Logic 47.8% · Math 57.9%

Its run record (Method) →

—54.5%52.8%–56.2%uv1,791 / 3,288105 right of 105 kept0 mistakes · 3,183 skipped0.78667.72chosen-answer Brier 0.190226.9%answered 86%177 ms222 ms$0.011$11.45estimate · DGX Spark
KnowLine-9B-Gen1 Open · 9.0B

KnowLine-9B-Gen1 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 153 answers, 153 right · 0 wrong · AUROC 0.782

Logic 44.3% · Math 59.8%

Its run record (Method) →

—54.5%52.8%–56.2%uv1,791 / 3,288153 right of 153 kept0 mistakes · 3,135 skipped0.78267.96chosen-answer Brier 0.18543.4%answered 98%38 ms55 ms$0.030$29.87estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-9B-Instant Open · 9.0B

StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

Parameters: 9B dense

Kept at ≥99%: 63 answers, 63 right · 0 wrong · AUROC 0.779

Logic 46.6% · Math 58.4%

Its run record (Method) →

—54.4%52.7%–56.1%uv1,788 / 3,28863 right of 63 kept0 mistakes · 3,225 skipped0.77967.73chosen-answer Brier 0.18939.6%answered 95%142 ms165 ms$0.009$8.92estimate · DGX Spark
Microsoft-Decision-1 Closed · Efficient

Microsoft-Decision-1 · Microsoft · decision model

Parameters: not disclosed by maker

Base model: Qwen3.5-9B

Kept at ≥99%: 47 answers, 47 right · 0 wrong · AUROC 0.769

Logic 44.4% · Math 59.4%

Its run record (Method) →

—54.3%52.6%–56.0%uvw1,784 / 3,28847 right of 47 kept0 mistakes · 3,241 skipped0.76967.42chosen-answer Brier 0.194111.4%answered 94%312 ms360 ms$0.009$9.07
Dexy-Decision-9B-V2 Open · 9.0B

Dexy-Decision-9B-V2 · Patrick-18 · decision model

Parameters: 9B dense

Base model: vllm-sr/d3-flash

Kept at ≥99%: 266 answers, 263 right · 3 wrong · AUROC 0.795

Logic 45.3% · Math 58.8%

Its run record (Method) →

—54.2%52.5%–55.9%uvw1,782 / 3,288263 right of 266 kept3 wrong · 3,022 skipped0.79566.98chosen-answer Brier 0.20236.1%answered 97%46 ms51 ms$0.033$32.61estimate · RTX PRO 6000 Blackwell
Drex 2.0 Open · 35B MoE · 3.0B active

Drex 2.0 · Nace.AI · decision model

Parameters: 35B MoE · 3B active

Kept at ≥99%: 207 answers, 207 right · 0 wrong · AUROC 0.789

Logic 41.9% · Math 59.0%

Its run record (Method) →

—53.1%51.4%–54.8%uvwx1,747 / 3,288207 right of 207 kept0 mistakes · 3,081 skipped0.78967.48chosen-answer Brier 0.181828.1%answered 85%339 ms472 ms$0.012$12.17
vera-medium-omni Open · 4.2B

vera-medium-omni · scar-ai · decision model

Parameters: 4.2B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 39 answers, 39 right · 0 wrong · AUROC 0.777

Logic 44.1% · Math 56.8%

Its run record (Method) →

—52.5%50.8%–54.2%vwxy1,725 / 3,28839 right of 39 kept0 mistakes · 3,249 skipped0.77766.69chosen-answer Brier 0.19097.9%answered 96%31 ms36 ms$0.024$24.19estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-4B-Instant Open · 4.0B

StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

Parameters: 4B dense

Kept at ≥99%: 117 answers, 117 right · 0 wrong · AUROC 0.763

Logic 45.9% · Math 55.8%

Its run record (Method) →

—52.4%50.7%–54.1%vwxy1,724 / 3,288117 right of 117 kept0 mistakes · 3,171 skipped0.76366.33chosen-answer Brier 0.197814.4%answered 93%68 ms80 ms$0.004$4.28estimate · DGX Spark
Perplexity Decider Flash v1.3 Open · 4.0B

Perplexity Decider Flash v1.3 · Perplexity · decision model

Parameters: 4B dense

Kept at ≥99%: 131 answers, 130 right · 1 wrong · AUROC 0.782

Logic 44.7% · Math 55.9%

Its run record (Method) →

—52.1%50.4%–53.8%vwxy1,713 / 3,288130 right of 131 kept1 wrong · 3,157 skipped0.78265.93chosen-answer Brier 0.202415.0%answered 92%29 ms35 ms$0.024$23.77estimate · RTX PRO 6000 Blackwell
Cerno 9B v1.2 Open · 9.0B

Cerno 9B v1.2 · Apertix AI · decision model

Parameters: 9B dense

Kept at ≥99%: 281 answers, 270 right · 11 wrong · AUROC 0.776

Logic 43.3% · Math 56.0%

Its run record (Method) →

—51.7%50.0%–53.4%vwxy1,699 / 3,288270 right of 281 kept11 wrong · 3,007 skipped0.77664.84chosen-answer Brier 0.21993.1%answered 99%163 ms240 ms$0.011$10.82estimate · DGX Spark
Valen-Decision-1.5-9B Open · 9.0B

Valen-Decision-1.5-9B · Valen-Team · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 167 answers, 165 right · 2 wrong · AUROC 0.765

Logic 44.4% · Math 55.3%

Its run record (Method) →

—51.6%49.9%–53.3%vwxy1,696 / 3,288165 right of 167 kept2 wrong · 3,121 skipped0.76565.99chosen-answer Brier 0.19607.1%answered 96%59 ms65 ms$0.041$41.44estimate · RTX PRO 6000 Blackwell
KnowLine 4B Gen5.5 Open · 4.0B

KnowLine 4B Gen5.5 · PelaAI · decision model

Parameters: 4B dense

Kept at ≥99%: 72 answers, 71 right · 1 wrong · AUROC 0.752

Logic 42.3% · Math 56.0%

Its run record (Method) →

—51.3%49.6%–53.0%vwxyz1,687 / 3,28871 right of 72 kept1 wrong · 3,216 skipped0.75265.07chosen-answer Brier 0.21164.4%answered 98%76 ms104 ms$0.005$5.02estimate · DGX Spark
Perplexity Decider Flash v1.2 Open · 4.0B

Perplexity Decider Flash v1.2 · Perplexity · decision model

Parameters: 4B dense

Kept at ≥99%: 190 answers, 185 right · 5 wrong · AUROC 0.769

Logic 43.1% · Math 55.2%

Its run record (Method) →

—51.1%49.4%–52.8%vwxyz1,680 / 3,288185 right of 190 kept5 wrong · 3,098 skipped0.76965.47chosen-answer Brier 0.201618.9%answered 90%41 ms51 ms$0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell
Mercury Decide Closed · Efficient

Mercury Decide · Inception · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 372 answers, 356 right · 16 wrong · AUROC 0.777

Logic 45.1% · Math 53.5%

Its run record (Method) →

—50.6%48.9%–52.3%wxyzA1,664 / 3,288356 right of 372 kept16 wrong · 2,916 skipped0.77764.36chosen-answer Brier 0.218910.1%answered 95%293 ms349 ms$0.005$5.46
d1 Closed · Efficient

d1 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 208 answers, 207 right · 1 wrong · AUROC 0.757

Logic 40.8% · Math 55.6%

Its run record (Method) →

—50.5%48.8%–52.3%wxyzA1,662 / 3,288207 right of 208 kept1 wrong · 3,080 skipped0.75765.29chosen-answer Brier 0.19986.4%answered 97%185 ms292 ms$0.004$3.98
Open-Jev-27B-v1.1 Open · 27B

Open-Jev-27B-v1.1 · Zefan Cai · decision model

Parameters: 27B dense

Base model: Qwen3.8-27B + adapter

Kept at ≥99%: No answers at this threshold · AUROC 0.759

Logic 42.6% · Math 54.6%

Its run record (Method) →

—50.5%48.7%–52.2%xyzA1,659 / 3,288No answers at this threshold0.75962.91chosen-answer Brier 0.246514.5%answered 92%250 ms482 ms$0.212$212estimate · RTX PRO 6000 Blackwell
OneJev-27B Open · 27B

OneJev-27B · OmniJev · decision model

Parameters: 27B dense

Kept at ≥99%: 30 answers, 30 right · 0 wrong · AUROC 0.770

Logic 40.6% · Math 53.8%

Its run record (Method) →

—49.3%47.6%–51.0%yzAB1,620 / 3,28830 right of 30 kept0 mistakes · 3,258 skipped0.77065.05chosen-answer Brier 0.19171.6%answered 99%80 ms134 ms$0.069$69.42estimate · RTX PRO 6000 Blackwell
Clef Open · 27B

Clef · Cloudflare · decision model

Parameters: 27B dense

Kept at ≥99%: 18 answers, 18 right · 0 wrong · AUROC 0.747

Logic 40.8% · Math 51.1%

Its run record (Method) →

—47.6%45.9%–49.3%zABC1,564 / 3,28818 right of 18 kept0 mistakes · 3,270 skipped0.74762.92chosen-answer Brier 0.21722.4%answered 99%185 ms216 ms$0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell
Verdin-4B-v1.1 Open · 4.0B

Verdin-4B-v1.1 · victorbennett · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 165 answers, 161 right · 4 wrong · AUROC 0.747

Logic 39.7% · Math 51.5%

Its run record (Method) →

—47.5%45.8%–49.2%ABC1,561 / 3,288161 right of 165 kept4 wrong · 3,123 skipped0.74763.44chosen-answer Brier 0.20598.9%answered 95%46 ms52 ms$0.033$33.45estimate · RTX PRO 6000 Blackwell
Chinese Decision Pointer 4B Open · 4.0B

Chinese Decision Pointer 4B · quincyqiang · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 194 answers, 188 right · 6 wrong · AUROC 0.743

Logic 40.4% · Math 50.4%

Its run record (Method) →

—47.0%45.3%–48.7%ABC1,545 / 3,288188 right of 194 kept6 wrong · 3,094 skipped0.74362.33chosen-answer Brier 0.22337.8%answered 96%30 ms42 ms$0.025$25.03estimate · RTX PRO 6000 Blackwell
Clef-Flash Open · 9.0B

Clef-Flash · Cloudflare · decision model

Parameters: 9B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.707

Logic 38.9% · Math 49.8%

Its run record (Method) →

—46.1%44.4%–47.8%BCD1,515 / 3,288No answers at this threshold0.70762.13chosen-answer Brier 0.21816.9%answered 96%103 ms131 ms$0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell
Winnow-12B Open · 12B

Winnow-12B · EldanRing · decision model

Parameters: 12B dense

Kept at ≥99%: 53 answers, 52 right · 1 wrong · AUROC 0.737

Logic 39.7% · Math 46.9%

Its run record (Method) →

—44.4%42.7%–46.1%CD1,461 / 3,28852 right of 53 kept1 wrong · 3,235 skipped0.73761.44chosen-answer Brier 0.215517.1%answered 90%55 ms72 ms$0.043$42.91estimate · RTX PRO 6000 Blackwell
Cygnet Open · 12B

Cygnet · Blockbrain Labs · decision model

Parameters: 11.95B dense

Base model: Gemma 4 12B IT

Kept at ≥99%: No answers at this threshold · AUROC 0.729

Logic 39.3% · Math 44.3%

Its run record (Method) →

—42.6%40.9%–44.3%D1,401 / 3,288No answers at this threshold0.72960.88chosen-answer Brier 0.20867.3%answered 96%38 ms44 ms$0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell
Metask-Jev-4B Open · 4.5B

Metask-Jev-4B · Metask Lab · decision model

Parameters: 4.54B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.659

Logic 32.1% · Math 39.2%

Its run record (Method) →

—36.7%35.1%–38.4%E1,208 / 3,288No answers at this threshold0.65957.83chosen-answer Brier 0.21082.0%answered 99%117 ms146 ms$0.086$85.83estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-2B-Instant Open · 2.0B

StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

Parameters: 2B dense

Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.675

Logic 30.8% · Math 38.7%

Its run record (Method) →

—35.9%34.3%–37.6%E1,182 / 3,2884 right of 4 kept0 mistakes · 3,284 skipped0.67557.77chosen-answer Brier 0.204116.4%answered 89%36 ms42 ms$0.002$2.36estimate · DGX Spark
GLiDE 2B Open · 1.9B

GLiDE 2B · Fastino · decision model

Parameters: 1.88B dense

Kept at ≥99%: 30 answers, 29 right · 1 wrong · AUROC 0.637

Logic 32.3% · Math 36.8%

Its run record (Method) →

—35.2%33.6%–36.9%E1,158 / 3,28829 right of 30 kept1 wrong · 3,258 skipped0.63755.25chosen-answer Brier 0.247223.1%answered 84%28 ms37 ms$0.002$1.94estimate · DGX Spark
d1-2B Open · 1.9B

d1-2B · Liquid AI · decision model

Parameters: 1.92B dense

Base model: LFM3-VL-2B

Kept at ≥99%: 7 answers, 7 right · 0 wrong · AUROC 0.648

Logic 30.7% · Math 37.1%

Its run record (Method) →

—34.9%33.3%–36.5%EF1,147 / 3,2887 right of 7 kept0 mistakes · 3,281 skipped0.64856.81chosen-answer Brier 0.212782.5%answered 30%17 ms21 ms$0.014$14.30estimate · RTX PRO 6000 Blackwell
KnowLine-2B-Gen1 Open · 2.0B

KnowLine-2B-Gen1 · PelaAI · decision model

Parameters: 2B dense

Base model: Qwen3.5-2B

Kept at ≥99%: 16 answers, 16 right · 0 wrong · AUROC 0.626

Logic 31.7% · Math 35.3%

Its run record (Method) →

—34.1%32.5%–35.7%EF1,120 / 3,28816 right of 16 kept0 mistakes · 3,272 skipped0.62656.04chosen-answer Brier 0.21996.8%answered 95%51 ms63 ms$0.003$3.40estimate · DGX Spark
GLiDE 0.8B Open · 0.8B

GLiDE 0.8B · Fastino · decision model

Parameters: 0.75B dense

Kept at ≥99%: 13 answers, 13 right · 0 wrong · AUROC 0.602

Logic 29.2% · Math 32.1%

Its run record (Method) →

—31.1%29.5%–32.7%FG1,022 / 3,28813 right of 13 kept0 mistakes · 3,275 skipped0.60253.32chosen-answer Brier 0.244422.7%answered 83%20 ms25 ms$0.001$1.38estimate · DGX Spark
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B

StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Kept at ≥99%: No answers at this threshold · AUROC 0.605

Logic 24.7% · Math 29.7%

Its run record (Method) →

—28.0%26.5%–29.5%GH920 / 3,288No answers at this threshold0.60554.00chosen-answer Brier 0.199914.9%answered 89%24 ms29 ms$0.002$1.62estimate · DGX Spark
OneJev1.6-0.8B Open · 0.8B

OneJev1.6-0.8B · OmniJev · decision model

Parameters: 0.8B dense

Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.576

Logic 27.0% · Math 28.2%

Its run record (Method) →

—27.8%26.3%–29.4%GH915 / 3,2882 right of 2 kept0 mistakes · 3,286 skipped0.57653.31chosen-answer Brier 0.21202.0%answered 99%15 ms17 ms$0.001$1.11estimate · DGX Spark
KnowLine-0.8B-Gen1 Open · 0.8B

KnowLine-0.8B-Gen1 · PelaAI · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.590

Logic 26.7% · Math 28.2%

Its run record (Method) →

—27.7%26.2%–29.2%GH910 / 3,2882 right of 2 kept0 mistakes · 3,286 skipped0.59053.55chosen-answer Brier 0.20570.6%answered 100%26 ms35 ms$0.002$1.80estimate · DGX Spark
Strands Decider 2B Open · 2.0B

Strands Decider 2B · Strands Agents · decision model

Parameters: 2B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.573

Logic 22.4% · Math 28.7%

Its run record (Method) →

—26.5%25.0%–28.1%HI872 / 3,288No answers at this threshold0.57353.20chosen-answer Brier 0.20125.0%answered 96%35 ms42 ms$0.027$26.72estimate · RTX PRO 6000 Blackwell
GLiClass v4 Edge Open · 0.4B

GLiClass v4 Edge · decision model

Parameters: 0.4B dense

Base model: DeBERTa-v3-large

Kept at ≥99%: No answers at this threshold · AUROC 0.511

Logic 23.6% · Math 22.7%

Its run record (Method) →

—23.0%21.6%–24.4%IJ755 / 3,288No answers at this threshold0.51152.00chosen-answer Brier 0.189615.6%answered 87%12 ms23 ms$0.001$1.01estimate · DGX Spark
GLiDE 0.4B Open · 0.4B

GLiDE 0.4B · Fastino · decision model

Parameters: 0.40B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.520

Logic 23.2% · Math 20.6%

Its run record (Method) →

—21.5%20.1%–22.9%J706 / 3,288No answers at this threshold0.52051.39chosen-answer Brier 0.186973.9%answered 32%11 ms14 ms$0.001$0.83estimate · DGX Spark

Tie groups and “not measured”

estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.

Decision Score (AA formula): Artificial Analysis's formula applied to our questions. Not their Index. 50 × (1 + accuracy − chosen-answer Brier), the chosen-answer Brier over the probability of the chosen answer, on decision models' standard runs only. Chat models and the “I don't know” runs are excluded.