Logic · Confidence

Confidence on Logic

1,125 questions, asked in English. How to read this page · Method

Answers kept at the ≥99% confidence threshold

Share of the N questions whose chosen option had probability ≥99% in the five-option (standard) run: right (family colour) and wrong (red); beside it the counts right / wrong, whether any was wrong, and the AUROC of the chosen option's probability (right against wrong answers, 0.5 = no signal) · more kept with no mistake is better

  1. KnowLine-31B-Gen1.5

    KnowLine-31B-Gen1.5 · PelaAI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 89 answers, 89 right · 0 wrong · AUROC 0.804

    Math 77.4%

    Its run record (Method) →

    89 / 0✓ 0 mistakes0.804
  2. Copernicus-31B

    Copernicus-31B · Poindexter Labs · decision model

    Parameters: 31B dense

    Kept at ≥99%: 66 answers, 66 right · 0 wrong · AUROC 0.801

    Math 77.3%

    Its run record (Method) →

    66 / 0✓ 0 mistakes0.801
  3. Jev 1.13

    Jev 1.13 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 65 answers, 65 right · 0 wrong · AUROC 0.764

    Math 64.1%

    Its run record (Method) →

    65 / 0✓ 0 mistakes0.764
  4. Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell

    Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 62 answers, 62 right · 0 wrong · AUROC 0.803

    Math 72.5%

    Its run record (Method) →

    62 / 0✓ 0 mistakes0.803
  5. Jev 1.14

    Jev 1.14 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 61 answers, 61 right · 0 wrong · AUROC 0.750

    Math 65.1%

    Its run record (Method) →

    61 / 0✓ 0 mistakes0.750
  6. Dexy-Decision-9B-V2

    Dexy-Decision-9B-V2 · Patrick-18 · decision model

    Parameters: 9B dense

    Base model: vllm-sr/d3-flash

    Kept at ≥99%: 56 answers, 56 right · 0 wrong · AUROC 0.759

    Math 58.8%

    Its run record (Method) →

    56 / 0✓ 0 mistakes0.759
  7. KnowLine-31B-Gen2

    KnowLine-31B-Gen2 · PelaAI · decision model

    Parameters: 31B dense

    Base model: Gemma 4 31B

    Kept at ≥99%: 54 answers, 54 right · 0 wrong · AUROC 0.825

    Math 80.3%

    Its run record (Method) →

    54 / 0✓ 0 mistakes0.825
  8. d3

    d3 · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Kept at ≥99%: 54 answers, 54 right · 0 wrong · AUROC 0.806

    Math 70.4%

    Its run record (Method) →

    54 / 0✓ 0 mistakes0.806
  9. Elephant-1

    Elephant-1 · Base Labs · decision model

    Parameters: 9B dense

    Kept at ≥99%: 53 answers, 53 right · 0 wrong · AUROC 0.768

    Math 61.3%

    Its run record (Method) →

    53 / 0✓ 0 mistakes0.768
  10. GLiDE API

    GLiDE API · Fastino · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 52 answers, 52 right · 0 wrong · AUROC 0.814

    Math 70.8%

    Its run record (Method) →

    52 / 0✓ 0 mistakes0.814
  11. REAX 1.3

    REAX 1.3 · REAX · decision model

    Parameters: 57.36B dense

    Kept at ≥99%: 51 answers, 51 right · 0 wrong · AUROC 0.816

    Math 78.0%

    Its run record (Method) →

    51 / 0✓ 0 mistakes0.816
  12. d1.1

    d1.1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 48 answers, 48 right · 0 wrong · AUROC 0.748

    Math 65.2%

    Its run record (Method) →

    48 / 0✓ 0 mistakes0.748
  13. GLiDE v1.2

    GLiDE v1.2 · Fastino · decision model

    Parameters: 27B dense

    Kept at ≥99%: 45 answers, 45 right · 0 wrong · AUROC 0.793

    Math 69.9%

    Its run record (Method) →

    45 / 0✓ 0 mistakes0.793
  14. Gemini Decision 1 20261111

    Gemini Decision 1 20261111 · Google · decision model

    Parameters: not disclosed by maker (closed weights)

    Kept at ≥99%: 43 answers, 43 right · 0 wrong · AUROC 0.768

    Math 71.3%

    Its run record (Method) →

    43 / 0✓ 0 mistakes0.768
  15. Chinese Decision Pointer 4B

    Chinese Decision Pointer 4B · quincyqiang · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 39 answers, 39 right · 0 wrong · AUROC 0.724

    Math 50.4%

    Its run record (Method) →

    39 / 0✓ 0 mistakes0.724
  16. Perplexity Decider Flash v1.2

    Perplexity Decider Flash v1.2 · Perplexity · decision model

    Parameters: 4B dense

    Kept at ≥99%: 35 answers, 35 right · 0 wrong · AUROC 0.729

    Math 55.2%

    Its run record (Method) →

    35 / 0✓ 0 mistakes0.729
  17. Verdin-4B-v1.1

    Verdin-4B-v1.1 · victorbennett · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 35 answers, 35 right · 0 wrong · AUROC 0.729

    Math 51.5%

    Its run record (Method) →

    35 / 0✓ 0 mistakes0.729
  18. Drex 2.0

    Drex 2.0 · Nace.AI · decision model

    Parameters: 35B MoE · 3B active

    Kept at ≥99%: 33 answers, 33 right · 0 wrong · AUROC 0.717

    Math 59.0%

    Its run record (Method) →

    33 / 0✓ 0 mistakes0.717
  19. GLiDE 9B

    GLiDE 9B · Fastino · decision model

    Parameters: 7.94B dense

    Kept at ≥99%: 32 answers, 32 right · 0 wrong · AUROC 0.749

    Math 57.9%

    Its run record (Method) →

    32 / 0✓ 0 mistakes0.749
  20. Winnow-12B

    Winnow-12B · EldanRing · decision model

    Parameters: 12B dense

    Kept at ≥99%: 32 answers, 32 right · 0 wrong · AUROC 0.727

    Math 46.9%

    Its run record (Method) →

    32 / 0✓ 0 mistakes0.727
  21. KnowLine-9B-Gen1

    KnowLine-9B-Gen1 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 30 answers, 30 right · 0 wrong · AUROC 0.725

    Math 59.8%

    Its run record (Method) →

    30 / 0✓ 0 mistakes0.725
  22. d1.2

    d1.2 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 28 answers, 28 right · 0 wrong · AUROC 0.742

    Math 65.5%

    Its run record (Method) →

    28 / 0✓ 0 mistakes0.742
  23. d3-ultra

    d3-ultra · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Kept at ≥99%: 28 answers, 28 right · 0 wrong · AUROC 0.778

    Math 71.5%

    Its run record (Method) →

    28 / 0✓ 0 mistakes0.778
  24. StartLux-Decision-v1.1-4B-Instant

    StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

    Parameters: 4B dense

    Kept at ≥99%: 21 answers, 21 right · 0 wrong · AUROC 0.720

    Math 55.8%

    Its run record (Method) →

    21 / 0✓ 0 mistakes0.720
  25. Microsoft-Decision-1

    Microsoft-Decision-1 · Microsoft · decision model

    Parameters: not disclosed by maker

    Base model: Qwen3.5-9B

    Kept at ≥99%: 20 answers, 20 right · 0 wrong · AUROC 0.712

    Math 59.4%

    Its run record (Method) →

    20 / 0✓ 0 mistakes0.712
  26. d1-ultra

    d1-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

    Kept at ≥99%: 20 answers, 20 right · 0 wrong · AUROC 0.783

    Math 74.1%

    Its run record (Method) →

    20 / 0✓ 0 mistakes0.783
  27. Perplexity Decider Flash v1.3

    Perplexity Decider Flash v1.3 · Perplexity · decision model

    Parameters: 4B dense

    Kept at ≥99%: 17 answers, 17 right · 0 wrong · AUROC 0.757

    Math 55.9%

    Its run record (Method) →

    17 / 0✓ 0 mistakes0.757
  28. GLiDE 2B

    GLiDE 2B · Fastino · decision model

    Parameters: 1.88B dense

    Kept at ≥99%: 14 answers, 14 right · 0 wrong · AUROC 0.632

    Math 36.8%

    Its run record (Method) →

    14 / 0✓ 0 mistakes0.632
  29. StartLux-Decision-v1.1-9B-Instant

    StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

    Parameters: 9B dense

    Kept at ≥99%: 14 answers, 14 right · 0 wrong · AUROC 0.762

    Math 58.4%

    Its run record (Method) →

    14 / 0✓ 0 mistakes0.762
  30. d1.2-ultra

    d1.2-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 13 answers, 13 right · 0 wrong · AUROC 0.800

    Math 69.9%

    Its run record (Method) →

    13 / 0✓ 0 mistakes0.800
  31. Jeffan-9B-1211

    Jeffan-9B-1211 · tattrongvu · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 10 answers, 10 right · 0 wrong · AUROC 0.753

    Math 59.8%

    Its run record (Method) →

    10 / 0✓ 0 mistakes0.753
  32. GLiDE 0.8B

    GLiDE 0.8B · Fastino · decision model

    Parameters: 0.75B dense

    Kept at ≥99%: 9 answers, 9 right · 0 wrong · AUROC 0.617

    Math 32.1%

    Its run record (Method) →

    9 / 0✓ 0 mistakes0.617
  33. KnowLine-2B-Gen1

    KnowLine-2B-Gen1 · PelaAI · decision model

    Parameters: 2B dense

    Base model: Qwen3.5-2B

    Kept at ≥99%: 8 answers, 8 right · 0 wrong · AUROC 0.601

    Math 35.3%

    Its run record (Method) →

    8 / 0✓ 0 mistakes0.601
  34. Logits Sigma-1 Duo

    Logits Sigma-1 Duo · Logits · decision model

    Parameters: 57.36B dense

    Kept at ≥99%: 8 answers, 8 right · 0 wrong · AUROC 0.796

    Math 75.2%

    Its run record (Method) →

    8 / 0✓ 0 mistakes0.796
  35. Turion Jev 31B

    Turion Jev 31B · Turion.AI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 7 answers, 7 right · 0 wrong · AUROC 0.782

    Math 69.0%

    Its run record (Method) →

    7 / 0✓ 0 mistakes0.782
  36. Clef

    Clef · Cloudflare · decision model

    Parameters: 27B dense

    Kept at ≥99%: 6 answers, 6 right · 0 wrong · AUROC 0.697

    Math 51.1%

    Its run record (Method) →

    6 / 0✓ 0 mistakes0.697
  37. KnowLine-12B-Gen1

    KnowLine-12B-Gen1 · PelaAI · decision model

    Parameters: 12B dense

    Base model: Gemma 4 12B

    Kept at ≥99%: 6 answers, 6 right · 0 wrong · AUROC 0.774

    Math 66.3%

    Its run record (Method) →

    6 / 0✓ 0 mistakes0.774
  38. vera-medium-omni

    vera-medium-omni · scar-ai · decision model

    Parameters: 4.2B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 6 answers, 6 right · 0 wrong · AUROC 0.739

    Math 56.8%

    Its run record (Method) →

    6 / 0✓ 0 mistakes0.739
  39. OneJev-27B

    OneJev-27B · OmniJev · decision model

    Parameters: 27B dense

    Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.726

    Math 53.8%

    Its run record (Method) →

    4 / 0✓ 0 mistakes0.726
  40. d1-2B

    d1-2B · Liquid AI · decision model

    Parameters: 1.92B dense

    Base model: LFM3-VL-2B

    Kept at ≥99%: 3 answers, 3 right · 0 wrong · AUROC 0.620

    Math 37.1%

    Its run record (Method) →

    3 / 0✓ 0 mistakes0.620
  41. KnowLine-0.8B-Gen1

    KnowLine-0.8B-Gen1 · PelaAI · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.573

    Math 28.2%

    Its run record (Method) →

    2 / 0✓ 0 mistakes0.573
  42. OneJev1.6-0.8B

    OneJev1.6-0.8B · OmniJev · decision model

    Parameters: 0.8B dense

    Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.545

    Math 28.2%

    Its run record (Method) →

    2 / 0✓ 0 mistakes0.545
  43. StartLux-Decision-v1.1-2B-Instant

    StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

    Parameters: 2B dense

    Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.640

    Math 38.7%

    Its run record (Method) →

    2 / 0✓ 0 mistakes0.640
  44. Clef-Flash

    Clef-Flash · Cloudflare · decision model

    Parameters: 9B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.659

    Math 49.8%

    Its run record (Method) →

    No answers at this threshold–0.659
  45. Cygnet

    Cygnet · Blockbrain Labs · decision model

    Parameters: 11.95B dense

    Base model: Gemma 4 12B IT

    Kept at ≥99%: No answers at this threshold · AUROC 0.720

    Math 44.3%

    Its run record (Method) →

    No answers at this threshold–0.720
  46. GLiClass v4 Edge

    GLiClass v4 Edge · decision model

    Parameters: 0.4B dense

    Base model: DeBERTa-v3-large

    Kept at ≥99%: No answers at this threshold · AUROC 0.527

    Math 22.7%

    Its run record (Method) →

    No answers at this threshold–0.527
  47. GLiDE 0.4B

    GLiDE 0.4B · Fastino · decision model

    Parameters: 0.40B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.556

    Math 20.6%

    Its run record (Method) →

    No answers at this threshold–0.556
  48. KnowLine-9B-Gen2

    KnowLine-9B-Gen2 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: No answers at this threshold · AUROC 0.730

    Math 60.1%

    Its run record (Method) →

    No answers at this threshold–0.730
  49. Metask-Jev-4B

    Metask-Jev-4B · Metask Lab · decision model

    Parameters: 4.54B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.660

    Math 39.2%

    Its run record (Method) →

    No answers at this threshold–0.660
  50. Open-Jev-27B-v1.1

    Open-Jev-27B-v1.1 · Zefan Cai · decision model

    Parameters: 27B dense

    Base model: Qwen3.8-27B + adapter

    Kept at ≥99%: No answers at this threshold · AUROC 0.733

    Math 54.6%

    Its run record (Method) →

    No answers at this threshold–0.733
  51. StartLux-Decision-v1.1-0.8B-Instant

    StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Kept at ≥99%: No answers at this threshold · AUROC 0.641

    Math 29.7%

    Its run record (Method) →

    No answers at this threshold–0.641
  52. Strands Decider 2B

    Strands Decider 2B · Strands Agents · decision model

    Parameters: 2B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.568

    Math 28.7%

    Its run record (Method) →

    No answers at this threshold–0.568
  53. celeris-1-decision

    celeris-1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: No answers at this threshold · AUROC 0.735

    Math 70.0%

    Its run record (Method) →

    No answers at this threshold–0.735
  54. celeris-1.1-decision

    celeris-1.1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: No answers at this threshold · AUROC 0.733

    Math 69.9%

    Its run record (Method) →

    No answers at this threshold–0.733
  55. Karpo Decision 31B

    Karpo Decision 31B · MachinePulse · decision model

    Parameters: 31B dense

    Kept at ≥99%: 172 answers, 169 right · 3 wrong · AUROC 0.816

    Math 81.6%

    Its run record (Method) →

    169 / 33 wrong0.816
  56. Drex Max

    Drex Max · Nace.AI · decision model

    Parameters: 27B dense

    Kept at ≥99%: 155 answers, 147 right · 8 wrong · AUROC 0.786

    Math 70.3%

    Its run record (Method) →

    147 / 88 wrong0.786
  57. Mammoth-1

    Mammoth-1 · Base Labs · decision model

    Parameters: 27B dense

    Kept at ≥99%: 108 answers, 106 right · 2 wrong · AUROC 0.795

    Math 74.0%

    Its run record (Method) →

    106 / 22 wrong0.795
  58. Drex Max v1.1

    Drex Max v1.1 · Nace.AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 90 answers, 89 right · 1 wrong · AUROC 0.793

    Math 70.1%

    Its run record (Method) →

    89 / 11 wrong0.793
  59. Mercury Decide

    Mercury Decide · Inception · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 91 answers, 84 right · 7 wrong · AUROC 0.746

    Math 53.5%

    Its run record (Method) →

    84 / 77 wrong0.746
  60. Sieve 31B v3

    Sieve 31B v3 · Sthānika AI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 81 answers, 79 right · 2 wrong · AUROC 0.797

    Math 77.1%

    Its run record (Method) →

    79 / 22 wrong0.797
  61. Perplexity Decider 27B v1.2 API · with web

    Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 79 answers, 78 right · 1 wrong · AUROC 0.800

    Math 69.3%

    Its run record (Method) →

    78 / 11 wrong0.800
  62. Perplexity Decider 27B v1.2 API

    Perplexity Decider 27B v1.2 API · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 80 answers, 78 right · 2 wrong · AUROC 0.798

    Math 69.8%

    Its run record (Method) →

    78 / 22 wrong0.798
  63. GLiDE 27B

    GLiDE 27B · Fastino · decision model

    Parameters: 27B dense

    Kept at ≥99%: 75 answers, 74 right · 1 wrong · AUROC 0.822

    Math 72.0%

    Its run record (Method) →

    74 / 11 wrong0.822
  64. SubQ Snap 1.1

    SubQ Snap 1.1 · Subquadratic · decision model

    Parameters: not disclosed by maker

    Shown once with SubQ Snap Preview: the same answer to every question

    Kept at ≥99%: 72 answers, 70 right · 2 wrong · AUROC 0.787

    Math 68.6%

    Its run record (Method) →

    70 / 22 wrong0.787
  65. d1

    d1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 56 answers, 55 right · 1 wrong · AUROC 0.723

    Math 55.6%

    Its run record (Method) →

    55 / 11 wrong0.723
  66. Adjudex 9B v5

    Adjudex 9B v5 · newsbang · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 55 answers, 54 right · 1 wrong · AUROC 0.771

    Math 62.1%

    Its run record (Method) →

    54 / 11 wrong0.771
  67. Valen-Decision-1.5-9B

    Valen-Decision-1.5-9B · Valen-Team · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 37 answers, 35 right · 2 wrong · AUROC 0.701

    Math 55.3%

    Its run record (Method) →

    35 / 22 wrong0.701
  68. Cerno 9B v1.2

    Cerno 9B v1.2 · Apertix AI · decision model

    Parameters: 9B dense

    Kept at ≥99%: 27 answers, 26 right · 1 wrong · AUROC 0.752

    Math 56.0%

    Its run record (Method) →

    26 / 11 wrong0.752
  69. KnowLine 4B Gen5.5

    KnowLine 4B Gen5.5 · PelaAI · decision model

    Parameters: 4B dense

    Kept at ≥99%: 18 answers, 17 right · 1 wrong · AUROC 0.708

    Math 56.0%

    Its run record (Method) →

    17 / 11 wrong0.708

PelaAI Poindexter Labs TypeSafe Perplexity Patrick-18 vLLM Semantic Router Base Labs Fastino REAX Liquid AI Google quincyqiang victorbennett Nace.AI EldanRing StartLux Microsoft tattrongvu Logits Turion.AI Cloudflare scar-ai OmniJev Blockbrain Labs Other Metask Lab Zefan Cai Strands Agents Celeris MachinePulse Inception Sthānika AI Subquadratic newsbang Valen-Team Apertix AI Kept but wrong

Accuracy by confidence cut-off

SetupWebAll answersaccuracy · N≥50%accuracy of the kept · share kept≥70%accuracy of the kept · share kept≥90%accuracy of the kept · share kept≥99%accuracy of the kept · share kept
KnowLine-31B-Gen1.5 Open · 31B—59.0%1,12578.3%54.6% kept (614)89.4%37.1% kept (417)96.1%20.7% kept (233)100.0%7.9% kept (89)
Copernicus-31B Open · 31B—58.7%1,12582.3%47.6% kept (536)91.9%30.6% kept (344)99.0%17.2% kept (193)100.0%5.9% kept (66)
Jev 1.13 Closed · Efficient—46.3%1,12568.4%44.7% kept (503)83.5%22.6% kept (254)94.4%11.2% kept (126)100.0%5.8% kept (65)
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—54.8%1,12570.5%63.6% kept (715)82.3%41.7% kept (469)93.2%22.3% kept (251)100.0%5.5% kept (62)
Jev 1.14 Closed · Efficient—48.2%1,12572.6%39.9% kept (449)84.2%21.4% kept (241)92.6%10.8% kept (121)100.0%5.4% kept (61)
Dexy-Decision-9B-V2 Open · 9.0B—45.3%1,12558.8%58.8% kept (662)74.4%34.4% kept (387)91.7%15.0% kept (169)100.0%5.0% kept (56)
KnowLine-31B-Gen2 Open · 31B—58.9%1,12579.2%57.8% kept (650)89.1%38.2% kept (430)97.4%20.9% kept (235)100.0%4.8% kept (54)
d3 Open · 26B—54.5%1,12575.4%53.1% kept (597)90.5%31.7% kept (357)95.8%17.1% kept (192)100.0%4.8% kept (54)
Elephant-1 Open · 9.0B—46.4%1,12566.5%47.8% kept (538)84.2%27.0% kept (304)94.9%14.0% kept (157)100.0%4.7% kept (53)
GLiDE API Closed · Efficient—54.5%1,12572.9%58.5% kept (658)85.0%36.1% kept (406)96.8%16.7% kept (188)100.0%4.6% kept (52)
REAX 1.3 Open · 57B—59.3%1,12577.6%58.3% kept (656)90.8%34.7% kept (390)98.2%19.6% kept (221)100.0%4.5% kept (51)
d1.1 Closed · Efficient—46.0%1,12570.3%39.6% kept (445)84.6%22.5% kept (253)94.7%11.7% kept (132)100.0%4.3% kept (48)
GLiDE v1.2 Open · 27B—55.2%1,12571.2%59.6% kept (671)89.2%33.6% kept (378)97.7%15.7% kept (177)100.0%4.0% kept (45)
Gemini Decision 1 20261111 Closed · Efficient—50.0%1,12579.4%34.5% kept (388)91.5%18.9% kept (213)97.5%10.8% kept (121)100.0%3.8% kept (43)
Chinese Decision Pointer 4B Open · 4.0B—40.4%1,12554.7%49.4% kept (556)68.8%25.6% kept (288)83.1%12.1% kept (136)100.0%3.5% kept (39)
Perplexity Decider Flash v1.2 Open · 4.0B—43.1%1,12565.9%36.3% kept (408)82.4%18.7% kept (210)92.3%10.4% kept (117)100.0%3.1% kept (35)
Verdin-4B-v1.1 Open · 4.0B—39.7%1,12559.2%39.8% kept (448)73.6%20.9% kept (235)87.0%9.6% kept (108)100.0%3.1% kept (35)
Drex 2.0 Open · 35B MoE · 3.0B active—41.9%1,12576.8%22.2% kept (250)82.2%12.0% kept (135)93.1%7.7% kept (87)100.0%2.9% kept (33)
GLiDE 9B Open · 7.9B—47.8%1,12566.4%47.4% kept (533)87.9%22.8% kept (256)97.5%10.7% kept (120)100.0%2.8% kept (32)
Winnow-12B Open · 12B—39.7%1,12557.3%45.2% kept (508)65.5%32.4% kept (365)74.5%19.2% kept (216)100.0%2.8% kept (32)
KnowLine-9B-Gen1 Open · 9.0B—44.3%1,12568.7%31.2% kept (351)85.3%16.4% kept (184)97.9%8.4% kept (95)100.0%2.7% kept (30)
d1.2 Closed · Undisclosed—49.2%1,12573.8%35.6% kept (400)87.5%19.9% kept (224)95.3%11.4% kept (128)100.0%2.5% kept (28)
d3-ultra Open · 26B—54.6%1,12562.7%75.7% kept (852)73.6%50.9% kept (573)93.0%25.3% kept (285)100.0%2.5% kept (28)
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—45.9%1,12568.4%36.8% kept (414)82.7%18.0% kept (202)97.8%8.3% kept (93)100.0%1.9% kept (21)
Microsoft-Decision-1 Closed · Efficient—44.4%1,12561.9%43.9% kept (494)78.7%19.6% kept (221)94.4%8.0% kept (90)100.0%1.8% kept (20)
d1-ultra Closed · Efficient—54.9%1,12576.4%48.5% kept (546)86.2%29.7% kept (334)96.6%15.6% kept (176)100.0%1.8% kept (20)
Perplexity Decider Flash v1.3 Open · 4.0B—44.7%1,12563.9%47.6% kept (535)77.7%27.6% kept (310)92.2%11.5% kept (129)100.0%1.5% kept (17)
GLiDE 2B Open · 1.9B—32.3%1,12546.3%34.8% kept (391)54.7%15.1% kept (170)79.7%5.2% kept (59)100.0%1.2% kept (14)
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—46.6%1,12571.5%38.3% kept (431)84.6%19.6% kept (221)98.9%8.2% kept (92)100.0%1.2% kept (14)
d1.2-ultra Closed · Undisclosed—51.6%1,12577.1%44.7% kept (503)88.6%28.2% kept (317)96.9%14.4% kept (162)100.0%1.2% kept (13)
Jeffan-9B-1211 Open · 9.0B—48.2%1,12571.1%41.5% kept (467)86.4%22.9% kept (258)98.1%9.6% kept (108)100.0%0.9% kept (10)
GLiDE 0.8B Open · 0.8B—29.2%1,12546.1%24.1% kept (271)54.7%9.4% kept (106)62.5%3.6% kept (40)100.0%0.8% kept (9)
KnowLine-2B-Gen1 Open · 2.0B—31.7%1,12551.2%18.4% kept (207)69.1%7.2% kept (81)87.5%2.8% kept (32)100.0%0.7% kept (8)
Logits Sigma-1 Duo Open · 57B—57.8%1,12575.4%57.5% kept (647)92.0%31.0% kept (349)97.6%14.6% kept (164)100.0%0.7% kept (8)
Turion Jev 31B Open · 31B—52.3%1,12581.5%37.9% kept (426)92.5%23.6% kept (266)97.7%11.6% kept (130)100.0%0.6% kept (7)
Clef Open · 27B—40.8%1,12553.0%54.5% kept (613)65.5%25.8% kept (290)85.3%9.1% kept (102)100.0%0.5% kept (6)
KnowLine-12B-Gen1 Open · 12B—50.6%1,12574.3%44.6% kept (502)90.5%25.3% kept (285)99.1%9.7% kept (109)100.0%0.5% kept (6)
vera-medium-omni Open · 4.2B—44.1%1,12572.7%29.3% kept (330)85.6%11.1% kept (125)100.0%4.9% kept (55)100.0%0.5% kept (6)
OneJev-27B Open · 27B—40.6%1,12574.4%18.4% kept (207)91.1%10.0% kept (112)100.0%4.6% kept (52)100.0%0.4% kept (4)
d1-2B Open · 1.9B—30.7%1,12552.0%15.2% kept (171)73.3%5.3% kept (60)100.0%1.7% kept (19)100.0%0.3% kept (3)
KnowLine-0.8B-Gen1 Open · 0.8B—26.7%1,12547.4%10.1% kept (114)63.9%3.2% kept (36)92.9%1.2% kept (14)100.0%0.2% kept (2)
OneJev1.6-0.8B Open · 0.8B—27.0%1,12536.3%15.9% kept (179)44.6%5.0% kept (56)83.3%0.5% kept (6)100.0%0.2% kept (2)
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—30.8%1,12559.7%13.2% kept (149)73.8%5.4% kept (61)100.0%2.0% kept (22)100.0%0.2% kept (2)
Clef-Flash Open · 9.0B—38.9%1,12552.4%40.9% kept (460)59.7%20.5% kept (231)77.9%6.0% kept (68)none kept
Cygnet Open · 12B—39.3%1,12558.1%43.3% kept (487)71.2%18.2% kept (205)89.5%5.1% kept (57)none kept
GLiClass v4 Edge Open · 0.4B—23.6%1,12535.6%9.2% kept (104)30.8%2.3% kept (26)50.0%0.5% kept (6)none kept
GLiDE 0.4B Open · 0.4B—23.2%1,12533.3%5.6% kept (63)40.0%1.8% kept (20)100.0%0.3% kept (3)none kept
KnowLine-9B-Gen2 Open · 9.0B—43.9%1,12570.9%32.1% kept (361)86.9%14.9% kept (168)100.0%4.1% kept (46)none kept
Metask-Jev-4B Open · 4.5B—32.1%1,12566.9%12.4% kept (139)81.0%7.0% kept (79)90.5%1.9% kept (21)none kept
Open-Jev-27B-v1.1 Open · 27B—42.6%1,12595.5%5.9% kept (66)100.0%0.4% kept (4)none keptnone kept
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—24.7%1,12546.6%9.2% kept (103)65.9%3.6% kept (41)90.0%0.9% kept (10)none kept
Strands Decider 2B Open · 2.0B—22.4%1,12538.8%6.0% kept (67)58.8%1.5% kept (17)100.0%0.2% kept (2)none kept
celeris-1-decision Closed · Efficient—47.3%1,12561.9%53.7% kept (604)76.5%30.6% kept (344)90.5%15.9% kept (179)none kept
celeris-1.1-decision Closed · Efficient—47.3%1,12562.9%52.7% kept (593)76.8%29.9% kept (336)90.8%16.4% kept (184)none kept
Karpo Decision 31B Open · 31B—60.7%1,12575.3%65.2% kept (734)89.4%44.5% kept (501)96.3%28.8% kept (324)98.3%15.3% kept (172)
Drex Max Open · 27B—53.1%1,12563.7%72.1% kept (811)74.1%49.7% kept (559)86.5%30.8% kept (347)94.8%13.8% kept (155)
Mammoth-1 Open · 27B—54.5%1,12574.1%55.3% kept (622)86.3%36.3% kept (408)95.4%21.3% kept (240)98.1%9.6% kept (108)
Drex Max v1.1 Closed · Efficient (provisional)—54.8%1,12571.8%58.0% kept (652)86.2%35.6% kept (400)95.5%19.9% kept (224)98.9%8.0% kept (90)
Mercury Decide Closed · Efficient—45.1%1,12561.0%53.6% kept (603)70.3%38.0% kept (428)79.9%22.1% kept (249)92.3%8.1% kept (91)
Sieve 31B v3 Open · 31B—57.9%1,12578.1%53.2% kept (599)89.3%35.7% kept (402)96.3%21.6% kept (243)97.5%7.2% kept (81)
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web52.5%1,12575.0%49.3% kept (555)88.5%29.3% kept (330)95.7%16.7% kept (188)98.7%7.0% kept (79)
Perplexity Decider 27B v1.2 API Open · 27B—53.3%1,12577.0%47.6% kept (535)90.7%28.8% kept (324)95.9%17.2% kept (193)97.5%7.1% kept (80)
GLiDE 27B Open · 27B—58.4%1,12576.8%60.3% kept (678)88.6%38.3% kept (431)97.7%18.9% kept (213)98.7%6.7% kept (75)
SubQ Snap 1.1 Closed · Efficient—52.4%1,12565.0%66.5% kept (748)77.9%42.2% kept (475)88.7%23.6% kept (266)97.2%6.4% kept (72)
d1 Closed · Efficient—40.8%1,12559.9%41.0% kept (461)74.2%21.7% kept (244)87.9%10.3% kept (116)98.2%5.0% kept (56)
Adjudex 9B v5 Open · 9.0B—46.4%1,12568.9%44.4% kept (499)83.2%25.3% kept (285)95.8%12.7% kept (143)98.2%4.9% kept (55)
Valen-Decision-1.5-9B Open · 9.0B—44.4%1,12563.2%37.7% kept (424)77.9%17.7% kept (199)90.2%8.2% kept (92)94.6%3.3% kept (37)
Cerno 9B v1.2 Open · 9.0B—43.3%1,12556.1%59.4% kept (668)70.8%33.5% kept (377)89.6%14.6% kept (164)96.3%2.4% kept (27)
KnowLine 4B Gen5.5 Open · 4.0B—42.3%1,12555.4%51.6% kept (581)68.8%27.4% kept (308)86.4%11.1% kept (125)94.4%1.6% kept (18)

Answering only where the chosen option's probability reaches the cut-off: the accuracy of the answers kept and, under it, the share of the N questions kept.

Confidence by topic

SetupWebAllkept right / wrong · AUROCNumerical reasoningkept R / W · AUROCVerbal reasoningkept R / W · AUROCFormal logickept R / W · AUROC
KnowLine-31B-Gen1.5 Open · 31B—89 / 00.80425 / 00.76514 / 00.76450 / 00.904
Copernicus-31B Open · 31B—66 / 00.80115 / 00.74612 / 00.77539 / 00.935
Jev 1.13 Closed · Efficient—65 / 00.7647 / 00.7085 / 00.71453 / 00.875
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—62 / 00.80322 / 00.7525 / 00.76435 / 00.878
Jev 1.14 Closed · Efficient—61 / 00.7505 / 00.6718 / 00.71148 / 00.856
Dexy-Decision-9B-V2 Open · 9.0B—56 / 00.7597 / 00.7129 / 00.68940 / 00.910
KnowLine-31B-Gen2 Open · 31B—54 / 00.82526 / 00.7925 / 00.78523 / 00.907
d3 Open · 26B—54 / 00.8063 / 00.75511 / 00.76140 / 00.863
Elephant-1 Open · 9.0B—53 / 00.7685 / 00.6719 / 00.72539 / 00.925
GLiDE API Closed · Efficient—52 / 00.8147 / 00.7777 / 00.76538 / 00.866
REAX 1.3 Open · 57B—51 / 00.81612 / 00.75011 / 00.80428 / 00.921
d1.1 Closed · Efficient—48 / 00.7484 / 00.6766 / 00.71238 / 00.906
GLiDE v1.2 Open · 27B—45 / 00.7933 / 00.7326 / 00.73936 / 00.906
Gemini Decision 1 20261111 Closed · Efficient—43 / 00.7685 / 00.7660 / 00.69938 / 00.885
Chinese Decision Pointer 4B Open · 4.0B—39 / 00.7241 / 00.6448 / 00.69930 / 00.866
Perplexity Decider Flash v1.2 Open · 4.0B—35 / 00.7293 / 00.6421 / 00.69331 / 00.879
Verdin-4B-v1.1 Open · 4.0B—35 / 00.7292 / 00.6485 / 00.67228 / 00.884
Drex 2.0 Open · 35B MoE · 3.0B active—33 / 00.7173 / 00.6520 / 00.69030 / 00.815
GLiDE 9B Open · 7.9B—32 / 00.7493 / 00.6374 / 00.72925 / 00.911
Winnow-12B Open · 12B—32 / 00.7271 / 00.6086 / 00.67025 / 00.835
KnowLine-9B-Gen1 Open · 9.0B—30 / 00.7253 / 00.6380 / 00.69227 / 00.843
d1.2 Closed · Undisclosed—28 / 00.7423 / 00.6762 / 00.71923 / 00.874
d3-ultra Open · 26B—28 / 00.7789 / 00.70710 / 00.7619 / 00.853
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—21 / 00.7201 / 00.6332 / 00.67218 / 00.853
Microsoft-Decision-1 Closed · Efficient—20 / 00.7121 / 00.6282 / 00.68317 / 00.823
d1-ultra Closed · Efficient—20 / 00.78312 / 00.7593 / 00.7155 / 00.846
Perplexity Decider Flash v1.3 Open · 4.0B—17 / 00.7574 / 00.6631 / 00.73112 / 00.860
GLiDE 2B Open · 1.9B—14 / 00.6321 / 00.5251 / 00.63712 / 00.787
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—14 / 00.7620 / 00.6753 / 00.71311 / 00.854
d1.2-ultra Closed · Undisclosed—13 / 00.8004 / 00.7414 / 00.7725 / 00.837
Jeffan-9B-1211 Open · 9.0B—10 / 00.7532 / 00.6635 / 00.7043 / 00.866
GLiDE 0.8B Open · 0.8B—9 / 00.6170 / 00.5570 / 00.6089 / 00.700
KnowLine-2B-Gen1 Open · 2.0B—8 / 00.6010 / 00.5010 / 00.5718 / 00.756
Logits Sigma-1 Duo Open · 57B—8 / 00.7961 / 00.7381 / 00.7646 / 00.880
Turion Jev 31B Open · 31B—7 / 00.7821 / 00.7020 / 00.7566 / 00.875
Clef Open · 27B—6 / 00.6972 / 00.6070 / 00.7044 / 00.846
KnowLine-12B-Gen1 Open · 12B—6 / 00.7741 / 00.7011 / 00.7114 / 00.937
vera-medium-omni Open · 4.2B—6 / 00.7390 / 00.6590 / 00.7106 / 00.876
OneJev-27B Open · 27B—4 / 00.7260 / 00.6330 / 00.7124 / 00.845
d1-2B Open · 1.9B—3 / 00.6200 / 00.5490 / 00.6063 / 00.787
KnowLine-0.8B-Gen1 Open · 0.8B—2 / 00.5730 / 00.5200 / 00.5312 / 00.746
OneJev1.6-0.8B Open · 0.8B—2 / 00.5450 / 00.5080 / 00.5382 / 00.608
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—2 / 00.6400 / 00.5390 / 00.6522 / 00.767
Clef-Flash Open · 9.0B—No answers at this threshold0.6590 / 00.5370 / 00.6700 / 00.790
Cygnet Open · 12B—No answers at this threshold0.7200 / 00.6040 / 00.6760 / 00.848
GLiClass v4 Edge Open · 0.4B—No answers at this threshold0.5270 / 00.5160 / 00.5470 / 00.584
GLiDE 0.4B Open · 0.4B—No answers at this threshold0.5560 / 00.5310 / 00.5450 / 00.639
KnowLine-9B-Gen2 Open · 9.0B—No answers at this threshold0.7300 / 00.6720 / 00.6630 / 00.859
Metask-Jev-4B Open · 4.5B—No answers at this threshold0.6600 / 00.5910 / 00.6040 / 00.830
Open-Jev-27B-v1.1 Open · 27B—No answers at this threshold0.7330 / 00.6230 / 00.7160 / 00.861
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—No answers at this threshold0.6410 / 00.5800 / 00.6150 / 00.694
Strands Decider 2B Open · 2.0B—No answers at this threshold0.5680 / 00.5440 / 00.5620 / 00.634
celeris-1-decision Closed · Efficient—No answers at this threshold0.7350 / 00.7060 / 00.6790 / 00.867
celeris-1.1-decision Closed · Efficient—No answers at this threshold0.7330 / 00.7090 / 00.6790 / 00.853
Karpo Decision 31B Open · 31B—169 / 30.81658 / 00.80533 / 20.75878 / 10.877
Drex Max Open · 27B—147 / 80.78634 / 00.72939 / 50.75874 / 30.838
Mammoth-1 Open · 27B—106 / 20.79521 / 00.70313 / 00.79772 / 20.905
Drex Max v1.1 Closed · Efficient (provisional)—89 / 10.7937 / 00.72924 / 00.77158 / 10.877
Mercury Decide Closed · Efficient—84 / 70.74614 / 20.67028 / 10.74442 / 40.775
Sieve 31B v3 Open · 31B—79 / 20.7979 / 00.76416 / 20.74554 / 00.861
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web78 / 10.8008 / 00.72812 / 00.78758 / 10.860
Perplexity Decider 27B v1.2 API Open · 27B—78 / 20.7989 / 00.73012 / 00.78357 / 20.791
GLiDE 27B Open · 27B—74 / 10.8226 / 00.75417 / 00.80051 / 10.888
SubQ Snap 1.1 Closed · Efficient—70 / 20.78717 / 00.73618 / 10.74335 / 10.857
d1 Closed · Efficient—55 / 10.7233 / 00.6305 / 00.68347 / 10.828
Adjudex 9B v5 Open · 9.0B—54 / 10.7714 / 00.7062 / 00.73048 / 10.869
Valen-Decision-1.5-9B Open · 9.0B—35 / 20.7015 / 10.5985 / 10.70925 / 00.785
Cerno 9B v1.2 Open · 9.0B—26 / 10.7528 / 10.6453 / 00.74515 / 00.881
KnowLine 4B Gen5.5 Open · 4.0B—17 / 10.7082 / 00.6343 / 10.64112 / 00.881

Per topic: answers kept at ≥99% (right / wrong) and, under them, the AUROC of the chosen option's probability (– where a topic has only right or only wrong answers).

SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 1,125 of 1,125); shown once.

Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.

Every setup, every measure

SetupWebAccuracy% of N (N ≤ 1,125) · Wilson 95%Tie groupHolm · McNemarRight / NquestionsKept at ≥99%right / wrongAUROCright vs wrongDecision Score (AA formula)0–100 · chosen-answer BrierNon-hallucinationof answers not right, % “I don't know” · higher is betterMedian latencymsp90ms, nearest rankCost$ per 1,000 qper 1M q
Karpo Decision 31B Open · 31B

Karpo Decision 31B · MachinePulse · decision model

Parameters: 31B dense

Kept at ≥99%: 172 answers, 169 right · 3 wrong · AUROC 0.816

Math 81.6%

Its run record (Method) →

—60.7%57.8%–63.5%ab683 / 1,125169 right of 172 kept3 wrong · 953 skipped0.81671.80chosen-answer Brier 0.17116.6%answered 97%173 ms211 ms$0.128$128estimate · RTX PRO 6000 Blackwell
REAX 1.3 Open · 57B

REAX 1.3 · REAX · decision model

Parameters: 57.36B dense

Kept at ≥99%: 51 answers, 51 right · 0 wrong · AUROC 0.816

Math 78.0%

Its run record (Method) →

—59.3%56.4%–62.1%ac667 / 1,12551 right of 51 kept0 mistakes · 1,074 skipped0.81671.07chosen-answer Brier 0.17145.3%answered 98%215 ms278 ms$0.160$160estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen1.5 Open · 31B

KnowLine-31B-Gen1.5 · PelaAI · decision model

Parameters: 31B dense

Kept at ≥99%: 89 answers, 89 right · 0 wrong · AUROC 0.804

Math 77.4%

Its run record (Method) →

—59.0%56.1%–61.9%abcd664 / 1,12589 right of 89 kept0 mistakes · 1,036 skipped0.80470.72chosen-answer Brier 0.17570.4%answered 100%983 ms1,279 ms$0.061$61.43estimate · DGX Spark
KnowLine-31B-Gen2 Open · 31B

KnowLine-31B-Gen2 · PelaAI · decision model

Parameters: 31B dense

Base model: Gemma 4 31B

Kept at ≥99%: 54 answers, 54 right · 0 wrong · AUROC 0.825

Math 80.3%

Its run record (Method) →

—58.9%56.0%–61.8%abcd663 / 1,12554 right of 54 kept0 mistakes · 1,071 skipped0.82571.09chosen-answer Brier 0.16750.0%answered 100%151 ms204 ms$0.112$112estimate · RTX PRO 6000 Blackwell
Copernicus-31B Open · 31B

Copernicus-31B · Poindexter Labs · decision model

Parameters: 31B dense

Kept at ≥99%: 66 answers, 66 right · 0 wrong · AUROC 0.801

Math 77.3%

Its run record (Method) →

—58.7%55.8%–61.5%abcd660 / 1,12566 right of 66 kept0 mistakes · 1,059 skipped0.80170.48chosen-answer Brier 0.17711.9%answered 99%185 ms255 ms$0.136$136estimate · RTX PRO 6000 Blackwell
GLiDE 27B Open · 27B

GLiDE 27B · Fastino · decision model

Parameters: 27B dense

Kept at ≥99%: 75 answers, 74 right · 1 wrong · AUROC 0.822

Math 72.0%

Its run record (Method) →

—58.4%55.5%–61.2%abcde657 / 1,12574 right of 75 kept1 wrong · 1,050 skipped0.82270.67chosen-answer Brier 0.170744.3%answered 79%544 ms655 ms$0.033$33.16estimate · DGX Spark
Sieve 31B v3 Open · 31B

Sieve 31B v3 · Sthānika AI · decision model

Parameters: 31B dense

Kept at ≥99%: 81 answers, 79 right · 2 wrong · AUROC 0.797

Math 77.1%

Its run record (Method) →

—57.9%55.0%–60.7%abcdf651 / 1,12579 right of 81 kept2 wrong · 1,044 skipped0.79770.03chosen-answer Brier 0.17815.5%answered 98%1,050 ms1,335 ms$0.063$62.95estimate · DGX Spark
Logits Sigma-1 Duo Open · 57B

Logits Sigma-1 Duo · Logits · decision model

Parameters: 57.36B dense

Kept at ≥99%: 8 answers, 8 right · 0 wrong · AUROC 0.796

Math 75.2%

Its run record (Method) →

—57.8%54.9%–60.6%abcdefg650 / 1,1258 right of 8 kept0 mistakes · 1,117 skipped0.79669.90chosen-answer Brier 0.17987.1%answered 97%285 ms399 ms$0.211$211estimate · RTX PRO 6000 Blackwell
GLiDE v1.2 Open · 27B

GLiDE v1.2 · Fastino · decision model

Parameters: 27B dense

Kept at ≥99%: 45 answers, 45 right · 0 wrong · AUROC 0.793

Math 69.9%

Its run record (Method) →

—55.2%52.3%–58.1%abcdefgh621 / 1,12545 right of 45 kept0 mistakes · 1,080 skipped0.79368.39chosen-answer Brier 0.184250.2%answered 73%111 ms140 ms$0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell
d1-ultra Closed · Efficient

d1-ultra · Liquid AI · decision model

Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

Kept at ≥99%: 20 answers, 20 right · 0 wrong · AUROC 0.783

Math 74.1%

Its run record (Method) →

—54.9%52.0%–57.8%abcdefgh618 / 1,12520 right of 20 kept0 mistakes · 1,105 skipped0.78368.09chosen-answer Brier 0.187691.0%answered 35%182 ms425 ms$0.013$12.64
Drex Max v1.1 Closed · Efficient (provisional)

Drex Max v1.1 · Nace.AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 90 answers, 89 right · 1 wrong · AUROC 0.793

Math 70.1%

Its run record (Method) →

—54.8%51.8%–57.6%abcdefghi616 / 1,12589 right of 90 kept1 wrong · 1,035 skipped0.79368.07chosen-answer Brier 0.186154.1%answered 69%329 ms1,540 ms$0.022$22.37
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B

Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 62 answers, 62 right · 0 wrong · AUROC 0.803

Math 72.5%

Its run record (Method) →

—54.8%51.8%–57.6%abcdefgh616 / 1,12562 right of 62 kept0 mistakes · 1,063 skipped0.80367.98chosen-answer Brier 0.188118.1%answered 91%108 ms131 ms$0.081$81.16estimate · RTX PRO 6000 Blackwell
d3-ultra Open · 26B

d3-ultra · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Kept at ≥99%: 28 answers, 28 right · 0 wrong · AUROC 0.778

Math 71.5%

Its run record (Method) →

—54.6%51.7%–57.5%bdefghij614 / 1,12528 right of 28 kept0 mistakes · 1,097 skipped0.77866.46chosen-answer Brier 0.216610.1%answered 95%156 ms185 ms$0.109$109estimate · RTX PRO 6000 Blackwell
GLiDE API Closed · Efficient

GLiDE API · Fastino · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 52 answers, 52 right · 0 wrong · AUROC 0.814

Math 70.8%

Its run record (Method) →

—54.5%51.6%–57.4%cdefghi613 / 1,12552 right of 52 kept0 mistakes · 1,073 skipped0.81468.27chosen-answer Brier 0.179460.0%answered 67%271 ms291 ms$0.052$52.03
Mammoth-1 Open · 27B

Mammoth-1 · Base Labs · decision model

Parameters: 27B dense

Kept at ≥99%: 108 answers, 106 right · 2 wrong · AUROC 0.795

Math 74.0%

Its run record (Method) →

—54.5%51.6%–57.4%cdefghij613 / 1,125106 right of 108 kept2 wrong · 1,017 skipped0.79568.07chosen-answer Brier 0.183572.3%answered 58%221 ms244 ms$0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell
d3 Open · 26B

d3 · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Kept at ≥99%: 54 answers, 54 right · 0 wrong · AUROC 0.806

Math 70.4%

Its run record (Method) →

—54.5%51.6%–57.4%cdefgh613 / 1,12554 right of 54 kept0 mistakes · 1,071 skipped0.80668.34chosen-answer Brier 0.178114.8%answered 93%76 ms98 ms$0.057$57.18estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.2 API Open · 27B

Perplexity Decider 27B v1.2 API · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 80 answers, 78 right · 2 wrong · AUROC 0.798

Math 69.8%

Its run record (Method) →

—53.3%50.4%–56.2%defghijk600 / 1,12578 right of 80 kept2 wrong · 1,045 skipped0.79867.64chosen-answer Brier 0.180514.4%answered 93%238 ms552 ms$0.007$7.46
Drex Max Open · 27B

Drex Max · Nace.AI · decision model

Parameters: 27B dense

Kept at ≥99%: 155 answers, 147 right · 8 wrong · AUROC 0.786

Math 70.3%

Its run record (Method) →

—53.1%50.1%–56.0%defghijk597 / 1,125147 right of 155 kept8 wrong · 970 skipped0.78665.68chosen-answer Brier 0.21707.3%answered 97%106 ms143 ms$0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.2 API · with web Open · 27B

Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 79 answers, 78 right · 1 wrong · AUROC 0.800

Math 69.3%

Its run record (Method) →

with web52.5%49.6%–55.4%fghijkl591 / 1,12578 right of 79 kept1 wrong · 1,046 skipped0.80067.24chosen-answer Brier 0.180637.7%answered 80%448 ms827 ms$1.030$1,030
SubQ Snap 1.1 Closed · Efficient

SubQ Snap 1.1 · Subquadratic · decision model

Parameters: not disclosed by maker

Shown once with SubQ Snap Preview: the same answer to every question

Kept at ≥99%: 72 answers, 70 right · 2 wrong · AUROC 0.787

Math 68.6%

Its run record (Method) →

—52.4%49.4%–55.3%fghijklm589 / 1,12570 right of 72 kept2 wrong · 1,053 skipped0.78765.94chosen-answer Brier 0.204765.7%answered 61%384 ms393 ms$0.015$14.92
Turion Jev 31B Open · 31B

Turion Jev 31B · Turion.AI · decision model

Parameters: 31B dense

Kept at ≥99%: 7 answers, 7 right · 0 wrong · AUROC 0.782

Math 69.0%

Its run record (Method) →

—52.3%49.3%–55.2%eghijklmn588 / 1,1257 right of 7 kept0 mistakes · 1,118 skipped0.78266.89chosen-answer Brier 0.18491.5%answered 99%385 ms533 ms$0.025$24.91estimate · DGX Spark
d1.2-ultra Closed · Undisclosed

d1.2-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 13 answers, 13 right · 0 wrong · AUROC 0.800

Math 69.9%

Its run record (Method) →

—51.6%48.6%–54.5%hijklmno580 / 1,12513 right of 13 kept0 mistakes · 1,112 skipped0.80066.77chosen-answer Brier 0.180294.5%answered 28%220 ms341 ms$0.013$12.64
KnowLine-12B-Gen1 Open · 12B

KnowLine-12B-Gen1 · PelaAI · decision model

Parameters: 12B dense

Base model: Gemma 4 12B

Kept at ≥99%: 6 answers, 6 right · 0 wrong · AUROC 0.774

Math 66.3%

Its run record (Method) →

—50.6%47.7%–53.5%hijklmnop569 / 1,1256 right of 6 kept0 mistakes · 1,119 skipped0.77465.90chosen-answer Brier 0.18788.8%answered 95%68 ms88 ms$0.052$51.64estimate · RTX PRO 6000 Blackwell
Gemini Decision 1 20261111 Closed · Efficient

Gemini Decision 1 20261111 · Google · decision model

Parameters: not disclosed by maker (closed weights)

Kept at ≥99%: 43 answers, 43 right · 0 wrong · AUROC 0.768

Math 71.3%

Its run record (Method) →

—50.0%47.1%–53.0%hijklmnopq563 / 1,12543 right of 43 kept0 mistakes · 1,082 skipped0.76865.37chosen-answer Brier 0.19302.0%answered 99%228 ms344 ms$0.037$36.77
d1.2 Closed · Undisclosed

d1.2 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 28 answers, 28 right · 0 wrong · AUROC 0.742

Math 65.5%

Its run record (Method) →

—49.2%46.2%–52.1%hijklmnopqr553 / 1,12528 right of 28 kept0 mistakes · 1,097 skipped0.74264.51chosen-answer Brier 0.201378.8%answered 43%222 ms390 ms$0.006$6.02
Jeffan-9B-1211 Open · 9.0B

Jeffan-9B-1211 · tattrongvu · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 10 answers, 10 right · 0 wrong · AUROC 0.753

Math 59.8%

Its run record (Method) →

—48.2%45.3%–51.1%jklmnopqrs542 / 1,12510 right of 10 kept0 mistakes · 1,115 skipped0.75364.33chosen-answer Brier 0.195223.4%answered 87%52 ms66 ms$0.039$39.35estimate · RTX PRO 6000 Blackwell
Jev 1.14 Closed · Efficient

Jev 1.14 · TypeSafe · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 61 answers, 61 right · 0 wrong · AUROC 0.750

Math 65.1%

Its run record (Method) →

—48.2%45.3%–51.1%ijklmnopqrs542 / 1,12561 right of 61 kept0 mistakes · 1,064 skipped0.75064.14chosen-answer Brier 0.199024.5%answered 86%166 ms193 ms$0.026$25.64
GLiDE 9B Open · 7.9B

GLiDE 9B · Fastino · decision model

Parameters: 7.94B dense

Kept at ≥99%: 32 answers, 32 right · 0 wrong · AUROC 0.749

Math 57.9%

Its run record (Method) →

—47.8%44.9%–50.7%klmnopqrs538 / 1,12532 right of 32 kept0 mistakes · 1,093 skipped0.74963.95chosen-answer Brier 0.199330.4%answered 82%208 ms251 ms$0.013$13.29estimate · DGX Spark
celeris-1-decision Closed · Efficient

celeris-1-decision · Celeris · decision model

Parameters: not disclosed by maker

Kept at ≥99%: No answers at this threshold · AUROC 0.735

Math 70.0%

Its run record (Method) →

—47.3%44.4%–50.2%klmnopqrstu532 / 1,125No answers at this threshold0.73562.94chosen-answer Brier 0.214192.3%answered 28%390 ms394 ms$0.015$15.40
celeris-1.1-decision Closed · Efficient

celeris-1.1-decision · Celeris · decision model

Parameters: not disclosed by maker

Kept at ≥99%: No answers at this threshold · AUROC 0.733

Math 69.9%

Its run record (Method) →

—47.3%44.4%–50.2%klmnopqrst532 / 1,125No answers at this threshold0.73362.92chosen-answer Brier 0.214491.0%answered 29%383 ms390 ms$0.002$1.54
StartLux-Decision-v1.1-9B-Instant Open · 9.0B

StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

Parameters: 9B dense

Kept at ≥99%: 14 answers, 14 right · 0 wrong · AUROC 0.762

Math 58.4%

Its run record (Method) →

—46.6%43.7%–49.5%lmnopqrstuv524 / 1,12514 right of 14 kept0 mistakes · 1,111 skipped0.76263.55chosen-answer Brier 0.194717.4%answered 90%160 ms175 ms$0.010$10.04estimate · DGX Spark
Adjudex 9B v5 Open · 9.0B

Adjudex 9B v5 · newsbang · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 55 answers, 54 right · 1 wrong · AUROC 0.771

Math 62.1%

Its run record (Method) →

—46.4%43.5%–49.3%lmnopqrstuv522 / 1,12554 right of 55 kept1 wrong · 1,070 skipped0.77163.48chosen-answer Brier 0.194512.2%answered 93%41 ms46 ms$0.031$31.36estimate · RTX PRO 6000 Blackwell
Elephant-1 Open · 9.0B

Elephant-1 · Base Labs · decision model

Parameters: 9B dense

Kept at ≥99%: 53 answers, 53 right · 0 wrong · AUROC 0.768

Math 61.3%

Its run record (Method) →

—46.4%43.5%–49.3%lmnopqrstuv522 / 1,12553 right of 53 kept0 mistakes · 1,072 skipped0.76863.39chosen-answer Brier 0.196275.6%answered 47%79 ms98 ms$0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell
Jev 1.13 Closed · Efficient

Jev 1.13 · TypeSafe · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 65 answers, 65 right · 0 wrong · AUROC 0.764

Math 64.1%

Its run record (Method) →

—46.3%43.4%–49.2%lmnopqrstuv521 / 1,12565 right of 65 kept0 mistakes · 1,060 skipped0.76463.36chosen-answer Brier 0.195915.8%answered 91%167 ms203 ms$0.026$25.64
d1.1 Closed · Efficient

d1.1 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 48 answers, 48 right · 0 wrong · AUROC 0.748

Math 65.2%

Its run record (Method) →

—46.0%43.2%–49.0%mnopqrstuvw518 / 1,12548 right of 48 kept0 mistakes · 1,077 skipped0.74863.14chosen-answer Brier 0.197790.6%answered 28%176 ms272 ms$0.006$6.02
StartLux-Decision-v1.1-4B-Instant Open · 4.0B

StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

Parameters: 4B dense

Kept at ≥99%: 21 answers, 21 right · 0 wrong · AUROC 0.720

Math 55.8%

Its run record (Method) →

—45.9%43.0%–48.8%nopqrstuvx516 / 1,12521 right of 21 kept0 mistakes · 1,104 skipped0.72062.63chosen-answer Brier 0.206116.0%answered 91%73 ms84 ms$0.005$4.62estimate · DGX Spark
Dexy-Decision-9B-V2 Open · 9.0B

Dexy-Decision-9B-V2 · Patrick-18 · decision model

Parameters: 9B dense

Base model: vllm-sr/d3-flash

Kept at ≥99%: 56 answers, 56 right · 0 wrong · AUROC 0.759

Math 58.8%

Its run record (Method) →

—45.3%42.4%–48.3%opqrstuvwxy510 / 1,12556 right of 56 kept0 mistakes · 1,069 skipped0.75961.81chosen-answer Brier 0.217112.4%answered 93%47 ms53 ms$0.034$33.81estimate · RTX PRO 6000 Blackwell
Mercury Decide Closed · Efficient

Mercury Decide · Inception · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 91 answers, 84 right · 7 wrong · AUROC 0.746

Math 53.5%

Its run record (Method) →

—45.1%42.2%–48.0%pqrstuvwxy507 / 1,12584 right of 91 kept7 wrong · 1,034 skipped0.74661.18chosen-answer Brier 0.227220.5%answered 88%301 ms360 ms$0.008$7.52
Perplexity Decider Flash v1.3 Open · 4.0B

Perplexity Decider Flash v1.3 · Perplexity · decision model

Parameters: 4B dense

Kept at ≥99%: 17 answers, 17 right · 0 wrong · AUROC 0.757

Math 55.9%

Its run record (Method) →

—44.7%41.8%–47.6%pqrstuvwxy503 / 1,12517 right of 17 kept0 mistakes · 1,108 skipped0.75762.14chosen-answer Brier 0.204224.4%answered 85%33 ms41 ms$0.026$26.44estimate · RTX PRO 6000 Blackwell
Microsoft-Decision-1 Closed · Efficient

Microsoft-Decision-1 · Microsoft · decision model

Parameters: not disclosed by maker

Base model: Qwen3.5-9B

Kept at ≥99%: 20 answers, 20 right · 0 wrong · AUROC 0.712

Math 59.4%

Its run record (Method) →

—44.4%41.6%–47.4%pqrstuvwxy500 / 1,12520 right of 20 kept0 mistakes · 1,105 skipped0.71261.53chosen-answer Brier 0.213919.8%answered 88%315 ms362 ms$0.013$13.42
Valen-Decision-1.5-9B Open · 9.0B

Valen-Decision-1.5-9B · Valen-Team · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 37 answers, 35 right · 2 wrong · AUROC 0.701

Math 55.3%

Its run record (Method) →

—44.4%41.5%–47.3%pqrstuvwxy499 / 1,12535 right of 37 kept2 wrong · 1,088 skipped0.70161.42chosen-answer Brier 0.215113.6%answered 92%60 ms67 ms$0.042$42.42estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen1 Open · 9.0B

KnowLine-9B-Gen1 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 30 answers, 30 right · 0 wrong · AUROC 0.725

Math 59.8%

Its run record (Method) →

—44.3%41.4%–47.2%pqrstuvwxy498 / 1,12530 right of 30 kept0 mistakes · 1,095 skipped0.72561.97chosen-answer Brier 0.20326.7%answered 96%50 ms61 ms$0.036$36.27estimate · RTX PRO 6000 Blackwell
vera-medium-omni Open · 4.2B

vera-medium-omni · scar-ai · decision model

Parameters: 4.2B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 6 answers, 6 right · 0 wrong · AUROC 0.739

Math 56.8%

Its run record (Method) →

—44.1%41.2%–47.0%qrstuvwxy496 / 1,1256 right of 6 kept0 mistakes · 1,119 skipped0.73961.90chosen-answer Brier 0.202813.6%answered 92%32 ms37 ms$0.026$25.82estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen2 Open · 9.0B

KnowLine-9B-Gen2 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: No answers at this threshold · AUROC 0.730

Math 60.1%

Its run record (Method) →

—43.9%41.0%–46.8%qrstuvwxy494 / 1,125No answers at this threshold0.73061.90chosen-answer Brier 0.201119.5%answered 88%52 ms65 ms$0.040$39.96estimate · RTX PRO 6000 Blackwell
Cerno 9B v1.2 Open · 9.0B

Cerno 9B v1.2 · Apertix AI · decision model

Parameters: 9B dense

Kept at ≥99%: 27 answers, 26 right · 1 wrong · AUROC 0.752

Math 56.0%

Its run record (Method) →

—43.3%40.4%–46.2%rstuvwxy487 / 1,12526 right of 27 kept1 wrong · 1,098 skipped0.75260.41chosen-answer Brier 0.22476.0%answered 97%209 ms269 ms$0.013$13.35estimate · DGX Spark
Perplexity Decider Flash v1.2 Open · 4.0B

Perplexity Decider Flash v1.2 · Perplexity · decision model

Parameters: 4B dense

Kept at ≥99%: 35 answers, 35 right · 0 wrong · AUROC 0.729

Math 55.2%

Its run record (Method) →

—43.1%40.2%–46.0%qrstuvwxy485 / 1,12535 right of 35 kept0 mistakes · 1,090 skipped0.72961.34chosen-answer Brier 0.204328.9%answered 82%45 ms55 ms$0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell
Open-Jev-27B-v1.1 Open · 27B

Open-Jev-27B-v1.1 · Zefan Cai · decision model

Parameters: 27B dense

Base model: Qwen3.8-27B + adapter

Kept at ≥99%: No answers at this threshold · AUROC 0.733

Math 54.6%

Its run record (Method) →

—42.6%39.7%–45.5%rstuvwxy479 / 1,125No answers at this threshold0.73359.78chosen-answer Brier 0.230228.2%answered 82%413 ms576 ms$0.294$294estimate · RTX PRO 6000 Blackwell
KnowLine 4B Gen5.5 Open · 4.0B

KnowLine 4B Gen5.5 · PelaAI · decision model

Parameters: 4B dense

Kept at ≥99%: 18 answers, 17 right · 1 wrong · AUROC 0.708

Math 56.0%

Its run record (Method) →

—42.3%39.5%–45.2%stuvwxy476 / 1,12517 right of 18 kept1 wrong · 1,107 skipped0.70859.66chosen-answer Brier 0.22988.3%answered 95%96 ms115 ms$0.006$5.89estimate · DGX Spark
Drex 2.0 Open · 35B MoE · 3.0B active

Drex 2.0 · Nace.AI · decision model

Parameters: 35B MoE · 3B active

Kept at ≥99%: 33 answers, 33 right · 0 wrong · AUROC 0.717

Math 59.0%

Its run record (Method) →

—41.9%39.0%–44.8%stuvwxy471 / 1,12533 right of 33 kept0 mistakes · 1,092 skipped0.71760.73chosen-answer Brier 0.204037.7%answered 75%345 ms504 ms$0.018$18.41
Clef Open · 27B

Clef · Cloudflare · decision model

Parameters: 27B dense

Kept at ≥99%: 6 answers, 6 right · 0 wrong · AUROC 0.697

Math 51.1%

Its run record (Method) →

—40.8%38.0%–43.7%uvwxy459 / 1,1256 right of 6 kept0 mistakes · 1,119 skipped0.69758.55chosen-answer Brier 0.23693.6%answered 98%196 ms222 ms$0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell
d1 Closed · Efficient

d1 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 56 answers, 55 right · 1 wrong · AUROC 0.723

Math 55.6%

Its run record (Method) →

—40.8%38.0%–43.7%tuvwxy459 / 1,12555 right of 56 kept1 wrong · 1,069 skipped0.72359.76chosen-answer Brier 0.212811.3%answered 93%188 ms290 ms$0.006$6.02
OneJev-27B Open · 27B

OneJev-27B · OmniJev · decision model

Parameters: 27B dense

Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.726

Math 53.8%

Its run record (Method) →

—40.6%37.8%–43.5%tuvwxy457 / 1,1254 right of 4 kept0 mistakes · 1,121 skipped0.72660.27chosen-answer Brier 0.20091.2%answered 99%108 ms136 ms$0.086$86.07estimate · RTX PRO 6000 Blackwell
Chinese Decision Pointer 4B Open · 4.0B

Chinese Decision Pointer 4B · quincyqiang · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 39 answers, 39 right · 0 wrong · AUROC 0.724

Math 50.4%

Its run record (Method) →

—40.4%37.5%–43.3%vwxy454 / 1,12539 right of 39 kept0 mistakes · 1,086 skipped0.72458.90chosen-answer Brier 0.225614.6%answered 91%38 ms48 ms$0.030$29.51estimate · RTX PRO 6000 Blackwell
Verdin-4B-v1.1 Open · 4.0B

Verdin-4B-v1.1 · victorbennett · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 35 answers, 35 right · 0 wrong · AUROC 0.729

Math 51.5%

Its run record (Method) →

—39.7%36.9%–42.6%wy447 / 1,12535 right of 35 kept0 mistakes · 1,090 skipped0.72959.26chosen-answer Brier 0.212212.6%answered 92%45 ms51 ms$0.033$33.20estimate · RTX PRO 6000 Blackwell
Winnow-12B Open · 12B

Winnow-12B · EldanRing · decision model

Parameters: 12B dense

Kept at ≥99%: 32 answers, 32 right · 0 wrong · AUROC 0.727

Math 46.9%

Its run record (Method) →

—39.7%36.9%–42.6%xy447 / 1,12532 right of 32 kept0 mistakes · 1,093 skipped0.72758.43chosen-answer Brier 0.228729.7%answered 80%66 ms91 ms$0.052$51.64estimate · RTX PRO 6000 Blackwell
Cygnet Open · 12B

Cygnet · Blockbrain Labs · decision model

Parameters: 11.95B dense

Base model: Gemma 4 12B IT

Kept at ≥99%: No answers at this threshold · AUROC 0.720

Math 44.3%

Its run record (Method) →

—39.3%36.5%–42.2%xy442 / 1,125No answers at this threshold0.72058.97chosen-answer Brier 0.213511.3%answered 93%40 ms48 ms$0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell
Clef-Flash Open · 9.0B

Clef-Flash · Cloudflare · decision model

Parameters: 9B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.659

Math 49.8%

Its run record (Method) →

—38.9%36.1%–41.8%yz438 / 1,125No answers at this threshold0.65957.70chosen-answer Brier 0.235314.4%answered 91%110 ms137 ms$0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell
GLiDE 2B Open · 1.9B

GLiDE 2B · Fastino · decision model

Parameters: 1.88B dense

Kept at ≥99%: 14 answers, 14 right · 0 wrong · AUROC 0.632

Math 36.8%

Its run record (Method) →

—32.3%29.6%–35.1%zA363 / 1,12514 right of 14 kept0 mistakes · 1,111 skipped0.63254.39chosen-answer Brier 0.234932.9%answered 75%34 ms42 ms$0.002$2.26estimate · DGX Spark
Metask-Jev-4B Open · 4.5B

Metask-Jev-4B · Metask Lab · decision model

Parameters: 4.54B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.660

Math 39.2%

Its run record (Method) →

—32.1%29.4%–34.9%zA361 / 1,125No answers at this threshold0.66056.30chosen-answer Brier 0.19501.5%answered 99%123 ms152 ms$0.091$90.99estimate · RTX PRO 6000 Blackwell
KnowLine-2B-Gen1 Open · 2.0B

KnowLine-2B-Gen1 · PelaAI · decision model

Parameters: 2B dense

Base model: Qwen3.5-2B

Kept at ≥99%: 8 answers, 8 right · 0 wrong · AUROC 0.601

Math 35.3%

Its run record (Method) →

—31.7%29.1%–34.5%A357 / 1,1258 right of 8 kept0 mistakes · 1,117 skipped0.60155.15chosen-answer Brier 0.214315.7%answered 89%59 ms70 ms$0.004$3.78estimate · DGX Spark
StartLux-Decision-v1.1-2B-Instant Open · 2.0B

StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

Parameters: 2B dense

Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.640

Math 38.7%

Its run record (Method) →

—30.8%28.1%–33.5%AB346 / 1,1252 right of 2 kept0 mistakes · 1,123 skipped0.64055.51chosen-answer Brier 0.197531.4%answered 76%40 ms45 ms$0.003$2.52estimate · DGX Spark
d1-2B Open · 1.9B

d1-2B · Liquid AI · decision model

Parameters: 1.92B dense

Base model: LFM3-VL-2B

Kept at ≥99%: 3 answers, 3 right · 0 wrong · AUROC 0.620

Math 37.1%

Its run record (Method) →

—30.7%28.0%–33.4%AB345 / 1,1253 right of 3 kept0 mistakes · 1,122 skipped0.62055.05chosen-answer Brier 0.205886.5%answered 22%17 ms23 ms$0.015$14.76estimate · RTX PRO 6000 Blackwell
GLiDE 0.8B Open · 0.8B

GLiDE 0.8B · Fastino · decision model

Parameters: 0.75B dense

Kept at ≥99%: 9 answers, 9 right · 0 wrong · AUROC 0.617

Math 32.1%

Its run record (Method) →

—29.2%26.6%–31.9%ABC328 / 1,1259 right of 9 kept0 mistakes · 1,116 skipped0.61753.53chosen-answer Brier 0.221026.1%answered 80%23 ms29 ms$0.002$1.58estimate · DGX Spark
OneJev1.6-0.8B Open · 0.8B

OneJev1.6-0.8B · OmniJev · decision model

Parameters: 0.8B dense

Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.545

Math 28.2%

Its run record (Method) →

—27.0%24.5%–29.7%ABC304 / 1,1252 right of 2 kept0 mistakes · 1,123 skipped0.54552.72chosen-answer Brier 0.21594.6%answered 97%16 ms19 ms$0.001$1.16estimate · DGX Spark
KnowLine-0.8B-Gen1 Open · 0.8B

KnowLine-0.8B-Gen1 · PelaAI · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.573

Math 28.2%

Its run record (Method) →

—26.7%24.2%–29.3%ABC300 / 1,1252 right of 2 kept0 mistakes · 1,123 skipped0.57353.45chosen-answer Brier 0.19761.2%answered 99%31 ms39 ms$0.002$2.00estimate · DGX Spark
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B

StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Kept at ≥99%: No answers at this threshold · AUROC 0.641

Math 29.7%

Its run record (Method) →

—24.7%22.3%–27.3%BC278 / 1,125No answers at this threshold0.64153.22chosen-answer Brier 0.182721.4%answered 83%27 ms31 ms$0.002$1.79estimate · DGX Spark
GLiClass v4 Edge Open · 0.4B

GLiClass v4 Edge · decision model

Parameters: 0.4B dense

Base model: DeBERTa-v3-large

Kept at ≥99%: No answers at this threshold · AUROC 0.527

Math 22.7%

Its run record (Method) →

—23.6%21.2%–26.1%BC265 / 1,125No answers at this threshold0.52751.99chosen-answer Brier 0.195812.4%answered 90%15 ms28 ms$0.001$1.31estimate · DGX Spark
GLiDE 0.4B Open · 0.4B

GLiDE 0.4B · Fastino · decision model

Parameters: 0.40B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.556

Math 20.6%

Its run record (Method) →

—23.2%20.8%–25.8%C261 / 1,125No answers at this threshold0.55652.19chosen-answer Brier 0.188376.1%answered 30%13 ms16 ms$0.001$0.95estimate · DGX Spark
Strands Decider 2B Open · 2.0B

Strands Decider 2B · Strands Agents · decision model

Parameters: 2B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.568

Math 28.7%

Its run record (Method) →

—22.4%20.1%–24.9%C252 / 1,125No answers at this threshold0.56851.86chosen-answer Brier 0.18688.3%answered 94%36 ms43 ms$0.028$27.67estimate · RTX PRO 6000 Blackwell

Tie groups and “not measured”

estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.

Decision Score (AA formula): Artificial Analysis's formula applied to our questions. Not their Index. 50 × (1 + accuracy − chosen-answer Brier), the chosen-answer Brier over the probability of the chosen answer, on decision models' standard runs only. Chat models and the “I don't know” runs are excluded.