Math · Confidence

Confidence on Math

2,163 questions, asked in English. How to read this page · Method

Answers kept at the ≥99% confidence threshold

Share of the N questions whose chosen option had probability ≥99% in the five-option (standard) run: right (family colour) and wrong (red); beside it the counts right / wrong, whether any was wrong, and the AUROC of the chosen option's probability (right against wrong answers, 0.5 = no signal) · more kept with no mistake is better

  1. Karpo Decision 31B

    Karpo Decision 31B · MachinePulse · decision model

    Parameters: 31B dense

    Kept at ≥99%: 837 answers, 837 right · 0 wrong · AUROC 0.903

    Logic 60.7%

    Its run record (Method) →

    837 / 0✓ 0 mistakes0.903
  2. Mammoth-1

    Mammoth-1 · Base Labs · decision model

    Parameters: 27B dense

    Kept at ≥99%: 412 answers, 412 right · 0 wrong · AUROC 0.895

    Logic 54.5%

    Its run record (Method) →

    412 / 0✓ 0 mistakes0.895
  3. Perplexity Decider 27B v1.2 API

    Perplexity Decider 27B v1.2 API · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 331 answers, 331 right · 0 wrong · AUROC 0.869

    Logic 53.3%

    Its run record (Method) →

    331 / 0✓ 0 mistakes0.869
  4. Perplexity Decider 27B v1.2 API · with web

    Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 326 answers, 326 right · 0 wrong · AUROC 0.873

    Logic 52.5%

    Its run record (Method) →

    326 / 0✓ 0 mistakes0.873
  5. Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell

    Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

    Parameters: 27B dense

    Kept at ≥99%: 323 answers, 323 right · 0 wrong · AUROC 0.856

    Logic 54.8%

    Its run record (Method) →

    323 / 0✓ 0 mistakes0.856
  6. Copernicus-31B

    Copernicus-31B · Poindexter Labs · decision model

    Parameters: 31B dense

    Kept at ≥99%: 318 answers, 318 right · 0 wrong · AUROC 0.899

    Logic 58.7%

    Its run record (Method) →

    318 / 0✓ 0 mistakes0.899
  7. Drex Max v1.1

    Drex Max v1.1 · Nace.AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 315 answers, 315 right · 0 wrong · AUROC 0.867

    Logic 54.8%

    Its run record (Method) →

    315 / 0✓ 0 mistakes0.867
  8. Jev 1.13

    Jev 1.13 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 248 answers, 248 right · 0 wrong · AUROC 0.834

    Logic 46.3%

    Its run record (Method) →

    248 / 0✓ 0 mistakes0.834
  9. Sieve 31B v3

    Sieve 31B v3 · Sthānika AI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 215 answers, 215 right · 0 wrong · AUROC 0.891

    Logic 57.9%

    Its run record (Method) →

    215 / 0✓ 0 mistakes0.891
  10. Gemini Decision 1 20261111

    Gemini Decision 1 20261111 · Google · decision model

    Parameters: not disclosed by maker (closed weights)

    Kept at ≥99%: 214 answers, 214 right · 0 wrong · AUROC 0.859

    Logic 50.0%

    Its run record (Method) →

    214 / 0✓ 0 mistakes0.859
  11. d1-ultra

    d1-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

    Kept at ≥99%: 213 answers, 213 right · 0 wrong · AUROC 0.876

    Logic 54.9%

    Its run record (Method) →

    213 / 0✓ 0 mistakes0.876
  12. Jev 1.14

    Jev 1.14 · TypeSafe · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 202 answers, 202 right · 0 wrong · AUROC 0.829

    Logic 48.2%

    Its run record (Method) →

    202 / 0✓ 0 mistakes0.829
  13. d1.1

    d1.1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 183 answers, 183 right · 0 wrong · AUROC 0.829

    Logic 46.0%

    Its run record (Method) →

    183 / 0✓ 0 mistakes0.829
  14. REAX 1.3

    REAX 1.3 · REAX · decision model

    Parameters: 57.36B dense

    Kept at ≥99%: 178 answers, 178 right · 0 wrong · AUROC 0.903

    Logic 59.3%

    Its run record (Method) →

    178 / 0✓ 0 mistakes0.903
  15. Drex 2.0

    Drex 2.0 · Nace.AI · decision model

    Parameters: 35B MoE · 3B active

    Kept at ≥99%: 174 answers, 174 right · 0 wrong · AUROC 0.813

    Logic 41.9%

    Its run record (Method) →

    174 / 0✓ 0 mistakes0.813
  16. GLiDE API

    GLiDE API · Fastino · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 161 answers, 161 right · 0 wrong · AUROC 0.865

    Logic 54.5%

    Its run record (Method) →

    161 / 0✓ 0 mistakes0.865
  17. d1

    d1 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 152 answers, 152 right · 0 wrong · AUROC 0.767

    Logic 40.8%

    Its run record (Method) →

    152 / 0✓ 0 mistakes0.767
  18. d1.2

    d1.2 · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 141 answers, 141 right · 0 wrong · AUROC 0.838

    Logic 49.2%

    Its run record (Method) →

    141 / 0✓ 0 mistakes0.838
  19. GLiDE v1.2

    GLiDE v1.2 · Fastino · decision model

    Parameters: 27B dense

    Kept at ≥99%: 131 answers, 131 right · 0 wrong · AUROC 0.862

    Logic 55.2%

    Its run record (Method) →

    131 / 0✓ 0 mistakes0.862
  20. Valen-Decision-1.5-9B

    Valen-Decision-1.5-9B · Valen-Team · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 130 answers, 130 right · 0 wrong · AUROC 0.790

    Logic 44.4%

    Its run record (Method) →

    130 / 0✓ 0 mistakes0.790
  21. KnowLine-9B-Gen1

    KnowLine-9B-Gen1 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 123 answers, 123 right · 0 wrong · AUROC 0.799

    Logic 44.3%

    Its run record (Method) →

    123 / 0✓ 0 mistakes0.799
  22. d1.2-ultra

    d1.2-ultra · Liquid AI · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 115 answers, 115 right · 0 wrong · AUROC 0.876

    Logic 51.6%

    Its run record (Method) →

    115 / 0✓ 0 mistakes0.876
  23. StartLux-Decision-v1.1-4B-Instant

    StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

    Parameters: 4B dense

    Kept at ≥99%: 96 answers, 96 right · 0 wrong · AUROC 0.779

    Logic 45.9%

    Its run record (Method) →

    96 / 0✓ 0 mistakes0.779
  24. GLiDE 9B

    GLiDE 9B · Fastino · decision model

    Parameters: 7.94B dense

    Kept at ≥99%: 73 answers, 73 right · 0 wrong · AUROC 0.798

    Logic 47.8%

    Its run record (Method) →

    73 / 0✓ 0 mistakes0.798
  25. Jeffan-9B-1211

    Jeffan-9B-1211 · tattrongvu · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 64 answers, 64 right · 0 wrong · AUROC 0.814

    Logic 48.2%

    Its run record (Method) →

    64 / 0✓ 0 mistakes0.814
  26. KnowLine 4B Gen5.5

    KnowLine 4B Gen5.5 · PelaAI · decision model

    Parameters: 4B dense

    Kept at ≥99%: 54 answers, 54 right · 0 wrong · AUROC 0.765

    Logic 42.3%

    Its run record (Method) →

    54 / 0✓ 0 mistakes0.765
  27. StartLux-Decision-v1.1-9B-Instant

    StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

    Parameters: 9B dense

    Kept at ≥99%: 49 answers, 49 right · 0 wrong · AUROC 0.781

    Logic 46.6%

    Its run record (Method) →

    49 / 0✓ 0 mistakes0.781
  28. Turion Jev 31B

    Turion Jev 31B · Turion.AI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 43 answers, 43 right · 0 wrong · AUROC 0.864

    Logic 52.3%

    Its run record (Method) →

    43 / 0✓ 0 mistakes0.864
  29. vera-medium-omni

    vera-medium-omni · scar-ai · decision model

    Parameters: 4.2B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 33 answers, 33 right · 0 wrong · AUROC 0.789

    Logic 44.1%

    Its run record (Method) →

    33 / 0✓ 0 mistakes0.789
  30. Microsoft-Decision-1

    Microsoft-Decision-1 · Microsoft · decision model

    Parameters: not disclosed by maker

    Base model: Qwen3.5-9B

    Kept at ≥99%: 27 answers, 27 right · 0 wrong · AUROC 0.787

    Logic 44.4%

    Its run record (Method) →

    27 / 0✓ 0 mistakes0.787
  31. OneJev-27B

    OneJev-27B · OmniJev · decision model

    Parameters: 27B dense

    Kept at ≥99%: 26 answers, 26 right · 0 wrong · AUROC 0.783

    Logic 40.6%

    Its run record (Method) →

    26 / 0✓ 0 mistakes0.783
  32. Logits Sigma-1 Duo

    Logits Sigma-1 Duo · Logits · decision model

    Parameters: 57.36B dense

    Kept at ≥99%: 15 answers, 15 right · 0 wrong · AUROC 0.899

    Logic 57.8%

    Its run record (Method) →

    15 / 0✓ 0 mistakes0.899
  33. Clef

    Clef · Cloudflare · decision model

    Parameters: 27B dense

    Kept at ≥99%: 12 answers, 12 right · 0 wrong · AUROC 0.767

    Logic 40.8%

    Its run record (Method) →

    12 / 0✓ 0 mistakes0.767
  34. d3-ultra

    d3-ultra · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Kept at ≥99%: 11 answers, 11 right · 0 wrong · AUROC 0.857

    Logic 54.6%

    Its run record (Method) →

    11 / 0✓ 0 mistakes0.857
  35. KnowLine-2B-Gen1

    KnowLine-2B-Gen1 · PelaAI · decision model

    Parameters: 2B dense

    Base model: Qwen3.5-2B

    Kept at ≥99%: 8 answers, 8 right · 0 wrong · AUROC 0.636

    Logic 31.7%

    Its run record (Method) →

    8 / 0✓ 0 mistakes0.636
  36. KnowLine-12B-Gen1

    KnowLine-12B-Gen1 · PelaAI · decision model

    Parameters: 12B dense

    Base model: Gemma 4 12B

    Kept at ≥99%: 7 answers, 7 right · 0 wrong · AUROC 0.819

    Logic 50.6%

    Its run record (Method) →

    7 / 0✓ 0 mistakes0.819
  37. GLiDE 0.8B

    GLiDE 0.8B · Fastino · decision model

    Parameters: 0.75B dense

    Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.589

    Logic 29.2%

    Its run record (Method) →

    4 / 0✓ 0 mistakes0.589
  38. d1-2B

    d1-2B · Liquid AI · decision model

    Parameters: 1.92B dense

    Base model: LFM3-VL-2B

    Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.656

    Logic 30.7%

    Its run record (Method) →

    4 / 0✓ 0 mistakes0.656
  39. StartLux-Decision-v1.1-2B-Instant

    StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

    Parameters: 2B dense

    Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.685

    Logic 30.8%

    Its run record (Method) →

    2 / 0✓ 0 mistakes0.685
  40. Clef-Flash

    Clef-Flash · Cloudflare · decision model

    Parameters: 9B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.730

    Logic 38.9%

    Its run record (Method) →

    No answers at this threshold–0.730
  41. Cygnet

    Cygnet · Blockbrain Labs · decision model

    Parameters: 11.95B dense

    Base model: Gemma 4 12B IT

    Kept at ≥99%: No answers at this threshold · AUROC 0.735

    Logic 39.3%

    Its run record (Method) →

    No answers at this threshold–0.735
  42. GLiClass v4 Edge

    GLiClass v4 Edge · decision model

    Parameters: 0.4B dense

    Base model: DeBERTa-v3-large

    Kept at ≥99%: No answers at this threshold · AUROC 0.501

    Logic 23.6%

    Its run record (Method) →

    No answers at this threshold–0.501
  43. GLiDE 0.4B

    GLiDE 0.4B · Fastino · decision model

    Parameters: 0.40B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.500

    Logic 23.2%

    Its run record (Method) →

    No answers at this threshold–0.500
  44. KnowLine-0.8B-Gen1

    KnowLine-0.8B-Gen1 · PelaAI · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Kept at ≥99%: No answers at this threshold · AUROC 0.596

    Logic 26.7%

    Its run record (Method) →

    No answers at this threshold–0.596
  45. KnowLine-9B-Gen2

    KnowLine-9B-Gen2 · PelaAI · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: No answers at this threshold · AUROC 0.803

    Logic 43.9%

    Its run record (Method) →

    No answers at this threshold–0.803
  46. Metask-Jev-4B

    Metask-Jev-4B · Metask Lab · decision model

    Parameters: 4.54B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.651

    Logic 32.1%

    Its run record (Method) →

    No answers at this threshold–0.651
  47. OneJev1.6-0.8B

    OneJev1.6-0.8B · OmniJev · decision model

    Parameters: 0.8B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.592

    Logic 27.0%

    Its run record (Method) →

    No answers at this threshold–0.592
  48. Open-Jev-27B-v1.1

    Open-Jev-27B-v1.1 · Zefan Cai · decision model

    Parameters: 27B dense

    Base model: Qwen3.8-27B + adapter

    Kept at ≥99%: No answers at this threshold · AUROC 0.764

    Logic 42.6%

    Its run record (Method) →

    No answers at this threshold–0.764
  49. StartLux-Decision-v1.1-0.8B-Instant

    StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

    Parameters: 0.8B dense

    Base model: Qwen3.5-0.8B

    Kept at ≥99%: No answers at this threshold · AUROC 0.583

    Logic 24.7%

    Its run record (Method) →

    No answers at this threshold–0.583
  50. Strands Decider 2B

    Strands Decider 2B · Strands Agents · decision model

    Parameters: 2B dense

    Kept at ≥99%: No answers at this threshold · AUROC 0.569

    Logic 22.4%

    Its run record (Method) →

    No answers at this threshold–0.569
  51. celeris-1-decision

    celeris-1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: No answers at this threshold · AUROC 0.858

    Logic 47.3%

    Its run record (Method) →

    No answers at this threshold–0.858
  52. celeris-1.1-decision

    celeris-1.1-decision · Celeris · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: No answers at this threshold · AUROC 0.859

    Logic 47.3%

    Its run record (Method) →

    No answers at this threshold–0.859
  53. Drex Max

    Drex Max · Nace.AI · decision model

    Parameters: 27B dense

    Kept at ≥99%: 729 answers, 719 right · 10 wrong · AUROC 0.861

    Logic 53.1%

    Its run record (Method) →

    719 / 1010 wrong0.861
  54. KnowLine-31B-Gen2

    KnowLine-31B-Gen2 · PelaAI · decision model

    Parameters: 31B dense

    Base model: Gemma 4 31B

    Kept at ≥99%: 470 answers, 469 right · 1 wrong · AUROC 0.898

    Logic 58.9%

    Its run record (Method) →

    469 / 11 wrong0.898
  55. KnowLine-31B-Gen1.5

    KnowLine-31B-Gen1.5 · PelaAI · decision model

    Parameters: 31B dense

    Kept at ≥99%: 465 answers, 464 right · 1 wrong · AUROC 0.895

    Logic 59.0%

    Its run record (Method) →

    464 / 11 wrong0.895
  56. SubQ Snap 1.1

    SubQ Snap 1.1 · Subquadratic · decision model

    Parameters: not disclosed by maker

    Shown once with SubQ Snap Preview: the same answer to every question

    Kept at ≥99%: 361 answers, 360 right · 1 wrong · AUROC 0.839

    Logic 52.4%

    Its run record (Method) →

    360 / 11 wrong0.839
  57. Mercury Decide

    Mercury Decide · Inception · decision model

    Parameters: not disclosed by maker

    Kept at ≥99%: 281 answers, 272 right · 9 wrong · AUROC 0.789

    Logic 45.1%

    Its run record (Method) →

    272 / 99 wrong0.789
  58. Cerno 9B v1.2

    Cerno 9B v1.2 · Apertix AI · decision model

    Parameters: 9B dense

    Kept at ≥99%: 254 answers, 244 right · 10 wrong · AUROC 0.780

    Logic 43.3%

    Its run record (Method) →

    244 / 1010 wrong0.780
  59. Dexy-Decision-9B-V2

    Dexy-Decision-9B-V2 · Patrick-18 · decision model

    Parameters: 9B dense

    Base model: vllm-sr/d3-flash

    Kept at ≥99%: 210 answers, 207 right · 3 wrong · AUROC 0.804

    Logic 45.3%

    Its run record (Method) →

    207 / 33 wrong0.804
  60. GLiDE 27B

    GLiDE 27B · Fastino · decision model

    Parameters: 27B dense

    Kept at ≥99%: 199 answers, 198 right · 1 wrong · AUROC 0.864

    Logic 58.4%

    Its run record (Method) →

    198 / 11 wrong0.864
  61. Adjudex 9B v5

    Adjudex 9B v5 · newsbang · decision model

    Parameters: 9B dense

    Base model: Qwen3.5-9B

    Kept at ≥99%: 198 answers, 197 right · 1 wrong · AUROC 0.806

    Logic 46.4%

    Its run record (Method) →

    197 / 11 wrong0.806
  62. Elephant-1

    Elephant-1 · Base Labs · decision model

    Parameters: 9B dense

    Kept at ≥99%: 183 answers, 182 right · 1 wrong · AUROC 0.803

    Logic 46.4%

    Its run record (Method) →

    182 / 11 wrong0.803
  63. d3

    d3 · vLLM Semantic Router · decision model

    Parameters: 26.09B dense

    Kept at ≥99%: 163 answers, 162 right · 1 wrong · AUROC 0.878

    Logic 54.5%

    Its run record (Method) →

    162 / 11 wrong0.878
  64. Perplexity Decider Flash v1.2

    Perplexity Decider Flash v1.2 · Perplexity · decision model

    Parameters: 4B dense

    Kept at ≥99%: 155 answers, 150 right · 5 wrong · AUROC 0.781

    Logic 43.1%

    Its run record (Method) →

    150 / 55 wrong0.781
  65. Chinese Decision Pointer 4B

    Chinese Decision Pointer 4B · quincyqiang · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 155 answers, 149 right · 6 wrong · AUROC 0.746

    Logic 40.4%

    Its run record (Method) →

    149 / 66 wrong0.746
  66. Verdin-4B-v1.1

    Verdin-4B-v1.1 · victorbennett · decision model

    Parameters: 4B dense

    Base model: Qwen3.5-4B

    Kept at ≥99%: 130 answers, 126 right · 4 wrong · AUROC 0.752

    Logic 39.7%

    Its run record (Method) →

    126 / 44 wrong0.752
  67. Perplexity Decider Flash v1.3

    Perplexity Decider Flash v1.3 · Perplexity · decision model

    Parameters: 4B dense

    Kept at ≥99%: 114 answers, 113 right · 1 wrong · AUROC 0.787

    Logic 44.7%

    Its run record (Method) →

    113 / 11 wrong0.787
  68. Winnow-12B

    Winnow-12B · EldanRing · decision model

    Parameters: 12B dense

    Kept at ≥99%: 21 answers, 20 right · 1 wrong · AUROC 0.743

    Logic 39.7%

    Its run record (Method) →

    20 / 11 wrong0.743
  69. GLiDE 2B

    GLiDE 2B · Fastino · decision model

    Parameters: 1.88B dense

    Kept at ≥99%: 16 answers, 15 right · 1 wrong · AUROC 0.635

    Logic 32.3%

    Its run record (Method) →

    15 / 11 wrong0.635

MachinePulse Base Labs Perplexity Poindexter Labs Nace.AI TypeSafe Sthānika AI Google Liquid AI REAX Fastino Valen-Team PelaAI StartLux tattrongvu Turion.AI scar-ai Microsoft OmniJev Logits Cloudflare vLLM Semantic Router Blockbrain Labs Other Metask Lab Zefan Cai Strands Agents Celeris Subquadratic Inception Apertix AI Patrick-18 newsbang quincyqiang victorbennett EldanRing Kept but wrong

Accuracy by confidence cut-off

SetupWebAll answersaccuracy · N≥50%accuracy of the kept · share kept≥70%accuracy of the kept · share kept≥90%accuracy of the kept · share kept≥99%accuracy of the kept · share kept
Karpo Decision 31B Open · 31B—81.6%2,16388.9%86.7% kept (1,876)94.2%73.7% kept (1,594)98.0%58.1% kept (1,256)100.0%38.7% kept (837)
Mammoth-1 Open · 27B—74.0%2,16385.3%79.1% kept (1,712)94.3%61.8% kept (1,336)98.5%45.7% kept (988)100.0%19.0% kept (412)
Perplexity Decider 27B v1.2 API Open · 27B—69.8%2,16383.0%73.5% kept (1,590)93.1%56.3% kept (1,217)98.2%38.7% kept (838)100.0%15.3% kept (331)
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web69.3%2,16382.5%73.5% kept (1,590)92.8%56.2% kept (1,215)97.7%38.8% kept (840)100.0%15.1% kept (326)
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—72.5%2,16381.2%81.1% kept (1,754)90.0%64.0% kept (1,384)96.3%46.1% kept (998)100.0%14.9% kept (323)
Copernicus-31B Open · 31B—77.3%2,16391.3%73.4% kept (1,587)97.2%57.5% kept (1,244)99.4%39.0% kept (843)100.0%14.7% kept (318)
Drex Max v1.1 Closed · Efficient (provisional)—70.1%2,16379.7%79.8% kept (1,725)89.3%62.5% kept (1,352)97.3%43.4% kept (938)100.0%14.6% kept (315)
Jev 1.13 Closed · Efficient—64.1%2,16380.3%65.2% kept (1,410)91.3%42.7% kept (924)98.3%25.1% kept (543)100.0%11.5% kept (248)
Sieve 31B v3 Open · 31B—77.1%2,16390.0%75.3% kept (1,628)95.9%59.6% kept (1,289)99.3%41.1% kept (888)100.0%9.9% kept (215)
Gemini Decision 1 20261111 Closed · Efficient—71.3%2,16385.3%72.1% kept (1,560)92.7%55.2% kept (1,194)98.2%36.3% kept (785)100.0%9.9% kept (214)
d1-ultra Closed · Efficient—74.1%2,16387.8%73.4% kept (1,588)94.0%59.0% kept (1,276)98.5%40.8% kept (882)100.0%9.8% kept (213)
Jev 1.14 Closed · Efficient—65.1%2,16382.2%61.7% kept (1,335)92.8%40.5% kept (876)98.3%24.0% kept (519)100.0%9.3% kept (202)
d1.1 Closed · Efficient—65.2%2,16382.7%61.1% kept (1,322)92.6%41.5% kept (897)97.6%24.9% kept (538)100.0%8.5% kept (183)
REAX 1.3 Open · 57B—78.0%2,16390.9%76.9% kept (1,664)97.5%58.5% kept (1,265)99.9%40.5% kept (877)100.0%8.2% kept (178)
Drex 2.0 Open · 35B MoE · 3.0B active—59.0%2,16380.4%53.0% kept (1,147)92.2%35.0% kept (758)97.8%20.6% kept (445)100.0%8.0% kept (174)
GLiDE API Closed · Efficient—70.8%2,16381.6%77.6% kept (1,678)92.0%58.2% kept (1,258)98.6%37.4% kept (808)100.0%7.4% kept (161)
d1 Closed · Efficient—55.6%2,16373.6%55.1% kept (1,192)85.9%32.1% kept (694)95.4%17.0% kept (368)100.0%7.0% kept (152)
d1.2 Closed · Undisclosed—65.5%2,16383.5%60.9% kept (1,318)94.4%41.3% kept (894)99.2%23.7% kept (512)100.0%6.5% kept (141)
GLiDE v1.2 Open · 27B—69.9%2,16381.1%75.9% kept (1,641)92.6%56.4% kept (1,219)99.1%34.4% kept (745)100.0%6.1% kept (131)
Valen-Decision-1.5-9B Open · 9.0B—55.3%2,16374.3%54.9% kept (1,187)86.7%32.3% kept (698)94.9%17.3% kept (374)100.0%6.0% kept (130)
KnowLine-9B-Gen1 Open · 9.0B—59.8%2,16376.7%58.4% kept (1,264)89.3%36.8% kept (797)98.0%20.5% kept (444)100.0%5.7% kept (123)
d1.2-ultra Closed · Undisclosed—69.9%2,16387.4%66.6% kept (1,441)94.8%48.6% kept (1,052)99.7%30.5% kept (660)100.0%5.3% kept (115)
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—55.8%2,16370.8%62.3% kept (1,348)81.1%40.5% kept (876)94.0%20.8% kept (450)100.0%4.4% kept (96)
GLiDE 9B Open · 7.9B—57.9%2,16370.8%68.3% kept (1,478)84.7%42.8% kept (926)94.7%20.9% kept (451)100.0%3.4% kept (73)
Jeffan-9B-1211 Open · 9.0B—59.8%2,16377.8%60.3% kept (1,304)90.4%38.1% kept (824)98.5%18.2% kept (393)100.0%3.0% kept (64)
KnowLine 4B Gen5.5 Open · 4.0B—56.0%2,16367.1%67.4% kept (1,457)79.3%41.9% kept (907)92.8%21.3% kept (460)100.0%2.5% kept (54)
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—58.4%2,16374.9%58.2% kept (1,258)85.1%37.4% kept (808)96.6%19.1% kept (413)100.0%2.3% kept (49)
Turion Jev 31B Open · 31B—69.0%2,16384.7%68.7% kept (1,487)92.8%51.1% kept (1,106)99.8%26.0% kept (562)100.0%2.0% kept (43)
vera-medium-omni Open · 4.2B—56.8%2,16375.4%55.2% kept (1,193)88.5%29.6% kept (641)98.9%12.3% kept (267)100.0%1.5% kept (33)
Microsoft-Decision-1 Closed · Efficient—59.4%2,16373.4%63.9% kept (1,382)88.5%36.2% kept (782)98.0%13.8% kept (299)100.0%1.2% kept (27)
OneJev-27B Open · 27B—53.8%2,16381.8%39.5% kept (855)92.5%23.3% kept (504)98.3%10.6% kept (229)100.0%1.2% kept (26)
Logits Sigma-1 Duo Open · 57B—75.2%2,16389.2%73.5% kept (1,589)98.3%52.4% kept (1,134)100.0%26.7% kept (578)100.0%0.7% kept (15)
Clef Open · 27B—51.1%2,16365.2%62.2% kept (1,346)78.9%36.8% kept (796)90.8%17.6% kept (381)100.0%0.6% kept (12)
d3-ultra Open · 26B—71.5%2,16378.5%85.2% kept (1,842)86.2%68.6% kept (1,484)97.4%43.7% kept (946)100.0%0.5% kept (11)
KnowLine-2B-Gen1 Open · 2.0B—35.3%2,16352.1%28.2% kept (609)67.2%8.9% kept (192)88.5%2.4% kept (52)100.0%0.4% kept (8)
KnowLine-12B-Gen1 Open · 12B—66.3%2,16383.7%60.8% kept (1,315)92.1%42.2% kept (912)99.1%19.6% kept (425)100.0%0.3% kept (7)
GLiDE 0.8B Open · 0.8B—32.1%2,16338.1%44.3% kept (959)47.5%16.8% kept (364)73.0%2.9% kept (63)100.0%0.2% kept (4)
d1-2B Open · 1.9B—37.1%2,16358.0%25.0% kept (541)74.5%7.4% kept (161)83.7%2.3% kept (49)100.0%0.2% kept (4)
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—38.7%2,16368.1%21.1% kept (457)83.6%7.9% kept (171)92.0%2.3% kept (50)100.0%0.1% kept (2)
Clef-Flash Open · 9.0B—49.8%2,16368.7%44.7% kept (967)81.3%23.3% kept (504)94.4%7.4% kept (161)none kept
Cygnet Open · 12B—44.3%2,16365.9%41.9% kept (907)84.2%14.9% kept (322)98.7%3.5% kept (75)none kept
GLiClass v4 Edge Open · 0.4B—22.7%2,16326.9%3.6% kept (78)38.5%0.6% kept (13)100.0%0.0% kept (1)none kept
GLiDE 0.4B Open · 0.4B—20.6%2,16323.5%5.5% kept (119)33.3%1.0% kept (21)none keptnone kept
KnowLine-0.8B-Gen1 Open · 0.8B—28.2%2,16341.7%15.5% kept (336)54.9%3.3% kept (71)75.0%0.2% kept (4)none kept
KnowLine-9B-Gen2 Open · 9.0B—60.1%2,16380.7%52.3% kept (1,131)92.5%30.9% kept (668)99.6%10.3% kept (223)none kept
Metask-Jev-4B Open · 4.5B—39.2%2,16363.1%19.8% kept (428)77.5%8.6% kept (187)88.2%1.6% kept (34)none kept
OneJev1.6-0.8B Open · 0.8B—28.2%2,16341.8%15.3% kept (330)54.1%3.4% kept (74)66.7%0.7% kept (15)none kept
Open-Jev-27B-v1.1 Open · 27B—54.6%2,16395.8%10.0% kept (216)100.0%1.2% kept (25)none keptnone kept
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—29.7%2,16345.9%10.6% kept (229)61.0%3.6% kept (77)92.3%0.6% kept (13)none kept
Strands Decider 2B Open · 2.0B—28.7%2,16345.9%11.8% kept (255)69.2%1.2% kept (26)100.0%0.0% kept (1)none kept
celeris-1-decision Closed · Efficient—70.0%2,16380.9%77.1% kept (1,667)90.9%58.3% kept (1,262)96.5%40.4% kept (874)none kept
celeris-1.1-decision Closed · Efficient—69.9%2,16380.9%77.2% kept (1,669)90.8%58.3% kept (1,262)96.7%40.5% kept (876)none kept
Drex Max Open · 27B—70.3%2,16376.2%86.7% kept (1,875)83.3%72.2% kept (1,561)92.9%55.9% kept (1,209)98.6%33.7% kept (729)
KnowLine-31B-Gen2 Open · 31B—80.3%2,16390.0%80.7% kept (1,746)95.3%66.5% kept (1,438)99.3%48.1% kept (1,040)99.8%21.7% kept (470)
KnowLine-31B-Gen1.5 Open · 31B—77.4%2,16389.5%77.0% kept (1,665)95.3%61.3% kept (1,327)99.1%42.7% kept (923)99.8%21.5% kept (465)
SubQ Snap 1.1 Closed · Efficient—68.6%2,16376.4%82.0% kept (1,774)84.1%66.4% kept (1,436)93.7%46.3% kept (1,002)99.7%16.7% kept (361)
Mercury Decide Closed · Efficient—53.5%2,16365.1%71.2% kept (1,539)75.3%49.8% kept (1,077)85.3%32.0% kept (692)96.8%13.0% kept (281)
Cerno 9B v1.2 Open · 9.0B—56.0%2,16364.6%75.7% kept (1,638)74.3%55.1% kept (1,192)86.2%33.8% kept (731)96.1%11.7% kept (254)
Dexy-Decision-9B-V2 Open · 9.0B—58.8%2,16369.2%74.8% kept (1,619)79.2%53.7% kept (1,162)91.9%32.1% kept (694)98.6%9.7% kept (210)
GLiDE 27B Open · 27B—72.0%2,16382.1%78.7% kept (1,703)91.2%60.6% kept (1,311)98.7%39.5% kept (855)99.5%9.2% kept (199)
Adjudex 9B v5 Open · 9.0B—62.1%2,16374.7%68.7% kept (1,486)86.2%46.0% kept (994)96.3%27.2% kept (589)99.5%9.2% kept (198)
Elephant-1 Open · 9.0B—61.3%2,16375.7%65.6% kept (1,419)86.7%44.0% kept (952)95.8%26.2% kept (567)99.5%8.5% kept (183)
d3 Open · 26B—70.4%2,16385.3%71.8% kept (1,553)95.2%53.7% kept (1,161)99.3%34.0% kept (735)99.4%7.5% kept (163)
Perplexity Decider Flash v1.2 Open · 4.0B—55.2%2,16369.1%65.3% kept (1,412)79.3%46.3% kept (1,001)88.3%25.8% kept (557)96.8%7.2% kept (155)
Chinese Decision Pointer 4B Open · 4.0B—50.4%2,16360.9%66.6% kept (1,440)74.3%39.1% kept (845)86.2%20.4% kept (442)96.1%7.2% kept (155)
Verdin-4B-v1.1 Open · 4.0B—51.5%2,16369.1%51.5% kept (1,115)81.6%29.9% kept (646)91.6%16.0% kept (346)96.9%6.0% kept (130)
Perplexity Decider Flash v1.3 Open · 4.0B—55.9%2,16367.9%69.5% kept (1,504)77.7%49.1% kept (1,061)89.5%27.7% kept (599)99.1%5.3% kept (114)
Winnow-12B Open · 12B—46.9%2,16364.6%47.3% kept (1,023)77.3%30.4% kept (657)89.5%16.2% kept (351)95.2%1.0% kept (21)
GLiDE 2B Open · 1.9B—36.8%2,16345.5%49.8% kept (1,078)56.1%21.8% kept (471)73.8%5.6% kept (122)93.8%0.7% kept (16)

Answering only where the chosen option's probability reaches the cut-off: the accuracy of the answers kept and, under it, the share of the N questions kept.

Confidence by topic

SetupWebAllkept right / wrong · AUROCBasic conceptskept R / W · AUROCDigits and place valuekept R / W · AUROCNumber baseskept R / W · AUROCDivision and divisibilitykept R / W · AUROCPrime factorisation and GCD/LCMkept R / W · AUROCRational numberskept R / W · AUROCDecimalskept R / W · AUROCOrderingkept R / W · AUROCSimple inequalitieskept R / W · AUROCAbsolute valuekept R / W · AUROCExponentskept R / W · AUROCRadicalskept R / W · AUROCFactorisationkept R / W · AUROC
Karpo Decision 31B Open · 31B—837 / 00.90385 / 00.89934 / 00.8013 / 00.83334 / 00.88038 / 00.93350 / 00.89083 / 00.95245 / 00.87966 / 00.91553 / 00.95091 / 00.902114 / 00.893141 / 00.904
Mammoth-1 Open · 27B—412 / 00.89527 / 00.8859 / 00.7661 / 00.78811 / 00.82114 / 00.87630 / 00.89741 / 00.90917 / 00.89347 / 00.93031 / 00.90453 / 00.90059 / 00.89672 / 00.903
Perplexity Decider 27B v1.2 API Open · 27B—331 / 00.86922 / 00.8616 / 00.8181 / 00.8068 / 00.77210 / 00.82316 / 00.81934 / 00.96134 / 00.90143 / 00.90333 / 00.93232 / 00.84047 / 00.87545 / 00.863
Perplexity Decider 27B v1.2 API · with web Open · 27Bwith web326 / 00.87323 / 00.8509 / 00.8421 / 00.5716 / 00.82714 / 00.79918 / 00.85428 / 00.92736 / 00.86943 / 00.88731 / 00.94629 / 00.85141 / 00.88547 / 00.884
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B—323 / 00.85626 / 00.8288 / 00.7760 / 00.5778 / 00.78515 / 00.84517 / 00.79036 / 00.93527 / 00.83925 / 00.90022 / 00.93335 / 00.84549 / 00.87155 / 00.834
Copernicus-31B Open · 31B—318 / 00.89927 / 00.90113 / 00.8651 / 00.8009 / 00.88117 / 00.89220 / 00.88142 / 00.98012 / 00.88425 / 00.87619 / 00.93729 / 00.88248 / 00.91456 / 00.875
Drex Max v1.1 Closed · Efficient (provisional)—315 / 00.86723 / 00.8398 / 00.8501 / 00.47114 / 00.7999 / 00.86115 / 00.81728 / 00.90331 / 00.84047 / 00.91030 / 00.91936 / 00.86639 / 00.85334 / 00.893
Jev 1.13 Closed · Efficient—248 / 00.83417 / 00.8368 / 00.7560 / 00.6719 / 00.7618 / 00.7979 / 00.83523 / 00.89420 / 00.80130 / 00.86817 / 00.83124 / 00.82141 / 00.85042 / 00.818
Sieve 31B v3 Open · 31B—215 / 00.89124 / 00.8716 / 00.8480 / 00.9037 / 00.8568 / 00.89810 / 00.86326 / 00.95615 / 00.91822 / 00.90714 / 00.93924 / 00.89531 / 00.87928 / 00.848
Gemini Decision 1 20261111 Closed · Efficient—214 / 00.85915 / 00.8626 / 00.7942 / 00.7290 / 00.7895 / 00.85410 / 00.85118 / 00.88213 / 00.79723 / 00.83315 / 00.79931 / 00.85938 / 00.90738 / 00.858
d1-ultra Closed · Efficient—213 / 00.87623 / 00.8658 / 00.7550 / 00.6505 / 00.8139 / 00.89613 / 00.88712 / 00.91013 / 00.94118 / 00.90213 / 00.89320 / 00.84530 / 00.88549 / 00.888
Jev 1.14 Closed · Efficient—202 / 00.82916 / 00.8547 / 00.7910 / 00.6868 / 00.7005 / 00.7788 / 00.81018 / 00.91014 / 00.78825 / 00.86218 / 00.82621 / 00.78127 / 00.84935 / 00.834
d1.1 Closed · Efficient—183 / 00.82913 / 00.8115 / 00.7680 / 00.7144 / 00.7213 / 00.8257 / 00.79616 / 00.84814 / 00.85419 / 00.87015 / 00.84923 / 00.81530 / 00.79834 / 00.835
REAX 1.3 Open · 57B—178 / 00.90311 / 00.8887 / 00.8261 / 00.8187 / 00.8699 / 00.88611 / 00.91730 / 00.95510 / 00.94614 / 00.8817 / 00.93522 / 00.88725 / 00.91324 / 00.900
Drex 2.0 Open · 35B MoE · 3.0B active—174 / 00.8138 / 00.8196 / 00.8170 / 00.5330 / 00.7405 / 00.79813 / 00.84120 / 00.9059 / 00.76816 / 00.83714 / 00.76622 / 00.76235 / 00.83626 / 00.803
GLiDE API Closed · Efficient—161 / 00.86517 / 00.8745 / 00.7461 / 00.62511 / 00.7784 / 00.8534 / 00.84020 / 00.98713 / 00.87229 / 00.88511 / 00.92415 / 00.89617 / 00.81114 / 00.881
d1 Closed · Efficient—152 / 00.76714 / 00.7655 / 00.6530 / 00.5304 / 00.7003 / 00.7896 / 00.7529 / 00.76111 / 00.72621 / 00.80211 / 00.73820 / 00.76923 / 00.77725 / 00.784
d1.2 Closed · Undisclosed—141 / 00.8388 / 00.8335 / 00.8000 / 00.5573 / 00.7083 / 00.8409 / 00.79420 / 00.8968 / 00.86212 / 00.84310 / 00.79724 / 00.84320 / 00.83819 / 00.845
GLiDE v1.2 Open · 27B—131 / 00.86212 / 00.8573 / 00.7400 / 00.5285 / 00.7774 / 00.8345 / 00.83820 / 00.96314 / 00.86522 / 00.9127 / 00.94815 / 00.87314 / 00.84910 / 00.877
Valen-Decision-1.5-9B Open · 9.0B—130 / 00.79016 / 00.8373 / 00.6630 / 00.6677 / 00.7784 / 00.8495 / 00.75317 / 00.8769 / 00.76512 / 00.7777 / 00.83413 / 00.78216 / 00.79521 / 00.776
KnowLine-9B-Gen1 Open · 9.0B—123 / 00.79913 / 00.8165 / 00.7120 / 00.5956 / 00.6746 / 00.8035 / 00.76019 / 00.8736 / 00.79914 / 00.7927 / 00.78017 / 00.80311 / 00.82114 / 00.817
d1.2-ultra Closed · Undisclosed—115 / 00.8766 / 00.8915 / 00.8391 / 00.8894 / 00.7744 / 00.8515 / 00.80118 / 00.96010 / 00.8558 / 00.91210 / 00.90215 / 00.85911 / 00.85518 / 00.868
StartLux-Decision-v1.1-4B-Instant Open · 4.0B—96 / 00.77911 / 00.8181 / 00.7070 / 00.5242 / 00.6762 / 00.7437 / 00.74818 / 00.8146 / 00.80811 / 00.8066 / 00.74412 / 00.76812 / 00.7788 / 00.775
GLiDE 9B Open · 7.9B—73 / 00.7987 / 00.8002 / 00.7370 / 00.7313 / 00.7001 / 00.7164 / 00.75215 / 00.8053 / 00.77410 / 00.8734 / 00.8537 / 00.7768 / 00.8519 / 00.765
Jeffan-9B-1211 Open · 9.0B—64 / 00.8144 / 00.8654 / 00.6770 / 00.7693 / 00.7323 / 00.8443 / 00.74515 / 00.8114 / 00.7976 / 00.8882 / 00.8255 / 00.8047 / 00.8088 / 00.768
KnowLine 4B Gen5.5 Open · 4.0B—54 / 00.7653 / 00.7912 / 00.6110 / 00.5001 / 00.6881 / 00.8144 / 00.69911 / 00.8451 / 00.8427 / 00.8131 / 00.7427 / 00.7439 / 00.8037 / 00.747
StartLux-Decision-v1.1-9B-Instant Open · 9.0B—49 / 00.7813 / 00.7771 / 00.6780 / 00.3941 / 00.7031 / 00.7303 / 00.73511 / 00.8873 / 00.7746 / 00.8035 / 00.8304 / 00.7752 / 00.7889 / 00.779
Turion Jev 31B Open · 31B—43 / 00.8645 / 00.8681 / 00.7700 / 00.7571 / 00.8090 / 00.9043 / 00.8173 / 00.9261 / 00.8546 / 00.8231 / 00.9408 / 00.8535 / 00.8669 / 00.852
vera-medium-omni Open · 4.2B—33 / 00.7892 / 00.7823 / 00.7040 / 00.6361 / 00.6951 / 00.8361 / 00.70613 / 00.8600 / 00.8482 / 00.8371 / 00.7974 / 00.7701 / 00.8094 / 00.784
Microsoft-Decision-1 Closed · Efficient—27 / 00.7875 / 00.7890 / 00.6960 / 00.6151 / 00.6700 / 00.7470 / 00.7504 / 00.8484 / 00.8065 / 00.8371 / 00.8012 / 00.7683 / 00.8242 / 00.794
OneJev-27B Open · 27B—26 / 00.7832 / 00.7680 / 00.6450 / 00.8061 / 00.7500 / 00.7671 / 00.7782 / 00.8841 / 00.7581 / 00.8113 / 00.7874 / 00.7576 / 00.7855 / 00.789
Logits Sigma-1 Duo Open · 57B—15 / 00.8992 / 00.9000 / 00.8410 / 00.6670 / 00.8471 / 00.8880 / 00.8722 / 00.9500 / 00.9191 / 00.9101 / 00.9394 / 00.8951 / 00.8713 / 00.911
Clef Open · 27B—12 / 00.7672 / 00.7511 / 00.7080 / 00.8752 / 00.7230 / 00.7200 / 00.7983 / 00.7780 / 00.7931 / 00.7440 / 00.8782 / 00.7480 / 00.8021 / 00.774
d3-ultra Open · 26B—11 / 00.8571 / 00.8421 / 00.7780 / 00.7640 / 00.8200 / 00.7310 / 00.8273 / 00.9420 / 00.8191 / 00.9012 / 00.8481 / 00.8711 / 00.8501 / 00.874
KnowLine-2B-Gen1 Open · 2.0B—8 / 00.6360 / 00.5700 / 00.5780 / 00.5330 / 00.5080 / 00.5400 / 00.6550 / 00.6671 / 00.7794 / 00.7361 / 00.6991 / 00.6440 / 00.6561 / 00.587
KnowLine-12B-Gen1 Open · 12B—7 / 00.8192 / 00.8380 / 00.7360 / 00.8140 / 00.7520 / 00.8400 / 00.8335 / 00.9150 / 00.8000 / 00.8240 / 00.9160 / 00.7750 / 00.8570 / 00.794
GLiDE 0.8B Open · 0.8B—4 / 00.5891 / 00.5380 / 00.4980 / 00.4670 / 00.6100 / 00.5790 / 00.5250 / 00.7101 / 00.6040 / 00.6010 / 00.6611 / 00.6730 / 00.5641 / 00.616
d1-2B Open · 1.9B—4 / 00.6560 / 00.6340 / 00.6290 / 00.3180 / 00.5721 / 00.5820 / 00.5890 / 00.7211 / 00.7781 / 00.6250 / 00.7181 / 00.7090 / 00.6570 / 00.677
StartLux-Decision-v1.1-2B-Instant Open · 2.0B—2 / 00.6850 / 00.6610 / 00.6010 / 00.3330 / 00.5600 / 00.6650 / 00.6550 / 00.7880 / 00.8131 / 00.7550 / 00.7030 / 00.6140 / 00.6581 / 00.679
Clef-Flash Open · 9.0B—No answers at this threshold0.7300 / 00.7110 / 00.6420 / 00.3610 / 00.6140 / 00.7360 / 00.6700 / 00.7660 / 00.7890 / 00.7440 / 00.8230 / 00.7220 / 00.8040 / 00.737
Cygnet Open · 12B—No answers at this threshold0.7350 / 00.7300 / 00.6170 / 00.6000 / 00.6900 / 00.6390 / 00.7260 / 00.8000 / 00.7880 / 00.7960 / 00.6970 / 00.6800 / 00.7680 / 00.761
GLiClass v4 Edge Open · 0.4B—No answers at this threshold0.5010 / 00.5240 / 00.5540 / 00.5500 / 00.4760 / 00.4740 / 00.5050 / 00.4120 / 00.5220 / 00.4120 / 00.4600 / 00.5660 / 00.5430 / 00.518
GLiDE 0.4B Open · 0.4B—No answers at this threshold0.5000 / 00.4960 / 00.5280 / 00.8750 / 00.5090 / 00.6060 / 00.4580 / 00.3700 / 00.6320 / 00.4500 / 00.4780 / 00.4930 / 00.4980 / 00.515
KnowLine-0.8B-Gen1 Open · 0.8B—No answers at this threshold0.5960 / 00.4860 / 00.6030 / 00.6670 / 00.6280 / 00.5730 / 00.4900 / 00.6830 / 00.7920 / 00.6600 / 00.6530 / 00.6030 / 00.5570 / 00.565
KnowLine-9B-Gen2 Open · 9.0B—No answers at this threshold0.8030 / 00.7880 / 00.7110 / 00.7120 / 00.6710 / 00.8010 / 00.8190 / 00.8340 / 00.8410 / 00.8470 / 00.8240 / 00.7740 / 00.8060 / 00.817
Metask-Jev-4B Open · 4.5B—No answers at this threshold0.6510 / 00.6140 / 00.5440 / 00.9330 / 00.6170 / 00.7370 / 00.6130 / 00.6330 / 00.6190 / 00.6660 / 00.7270 / 00.6600 / 00.6240 / 00.664
OneJev1.6-0.8B Open · 0.8B—No answers at this threshold0.5920 / 00.5480 / 00.6010 / 00.5330 / 00.5190 / 00.6780 / 00.4740 / 00.5430 / 00.7260 / 00.6900 / 00.6190 / 00.6050 / 00.6090 / 00.559
Open-Jev-27B-v1.1 Open · 27B—No answers at this threshold0.7640 / 00.8160 / 00.6070 / 00.6250 / 00.6740 / 00.7310 / 00.7510 / 00.7620 / 00.7450 / 00.8470 / 00.8180 / 00.7760 / 00.7270 / 00.749
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B—No answers at this threshold0.5830 / 00.6110 / 00.4600 / 00.6670 / 00.5460 / 00.4400 / 00.6120 / 00.5670 / 00.6930 / 00.6720 / 00.6760 / 00.5510 / 00.5030 / 00.580
Strands Decider 2B Open · 2.0B—No answers at this threshold0.5690 / 00.5100 / 00.4580 / 00.6150 / 00.5340 / 00.5890 / 00.5830 / 00.6350 / 00.6150 / 00.5740 / 00.6320 / 00.5560 / 00.5420 / 00.627
celeris-1-decision Closed · Efficient—No answers at this threshold0.8580 / 00.8570 / 00.7870 / 00.8070 / 00.7940 / 00.8840 / 00.8960 / 00.9520 / 00.8400 / 00.8810 / 00.8360 / 00.8380 / 00.8690 / 00.849
celeris-1.1-decision Closed · Efficient—No answers at this threshold0.8590 / 00.8630 / 00.7900 / 00.8750 / 00.7900 / 00.8630 / 00.8960 / 00.9440 / 00.8530 / 00.8770 / 00.8120 / 00.8520 / 00.8670 / 00.855
Drex Max Open · 27B—719 / 100.86167 / 10.83425 / 00.8033 / 00.75032 / 00.79928 / 00.86736 / 40.79463 / 00.92342 / 20.87775 / 00.89444 / 00.90883 / 00.87696 / 20.855125 / 10.847
KnowLine-31B-Gen2 Open · 31B—469 / 10.89838 / 00.91615 / 00.8622 / 00.74214 / 00.84724 / 00.90237 / 00.92170 / 00.95612 / 00.87128 / 00.88228 / 00.91656 / 10.86368 / 00.90877 / 00.886
KnowLine-31B-Gen1.5 Open · 31B—464 / 10.89538 / 00.87415 / 00.8501 / 00.75014 / 00.88122 / 00.90332 / 00.90060 / 00.95618 / 00.91235 / 00.87233 / 00.90455 / 10.88564 / 00.91477 / 00.888
SubQ Snap 1.1 Closed · Efficient—360 / 10.83928 / 00.81912 / 00.7841 / 00.72215 / 00.78010 / 00.80819 / 00.75447 / 00.92935 / 00.82931 / 00.83228 / 00.93137 / 10.80245 / 00.87152 / 00.848
Mercury Decide Closed · Efficient—272 / 90.78925 / 00.8119 / 00.6921 / 00.7298 / 00.7486 / 00.68711 / 30.71826 / 10.85923 / 20.80131 / 10.80818 / 00.82122 / 10.75144 / 10.78548 / 00.822
Cerno 9B v1.2 Open · 9.0B—244 / 100.78024 / 00.8069 / 00.6771 / 10.5618 / 00.6796 / 00.79711 / 00.75131 / 00.84317 / 00.74722 / 20.80613 / 00.81231 / 20.72033 / 30.83638 / 20.776
Dexy-Decision-9B-V2 Open · 9.0B—207 / 30.80418 / 10.7785 / 00.6840 / 00.6067 / 00.7317 / 10.7458 / 00.75929 / 10.83910 / 00.82918 / 00.82212 / 00.85333 / 00.82424 / 00.81436 / 00.820
GLiDE 27B Open · 27B—198 / 10.86417 / 10.8586 / 00.7870 / 00.5698 / 00.7615 / 00.8467 / 00.87022 / 00.94820 / 00.93836 / 00.87318 / 00.90521 / 00.86120 / 00.84818 / 00.875
Adjudex 9B v5 Open · 9.0B—197 / 10.80618 / 00.8154 / 00.7010 / 00.7296 / 00.6848 / 00.7649 / 00.79731 / 00.86712 / 10.81421 / 00.82016 / 00.84023 / 00.81321 / 00.81228 / 00.806
Elephant-1 Open · 9.0B—182 / 10.80314 / 00.8215 / 00.7110 / 00.4866 / 00.6846 / 00.82310 / 00.73028 / 00.83310 / 10.78515 / 00.86814 / 00.87925 / 00.77824 / 00.83425 / 00.774
d3 Open · 26B—162 / 10.87813 / 10.8675 / 00.7911 / 00.8066 / 00.8117 / 00.9078 / 00.84721 / 00.95515 / 00.90425 / 00.90513 / 00.89321 / 00.88613 / 00.87514 / 00.872
Perplexity Decider Flash v1.2 Open · 4.0B—150 / 50.7817 / 10.7814 / 00.6880 / 00.3002 / 00.6943 / 00.7809 / 00.73226 / 00.88410 / 10.80721 / 00.85411 / 10.77315 / 10.73019 / 00.80623 / 10.753
Chinese Decision Pointer 4B Open · 4.0B—149 / 60.74614 / 20.7935 / 00.6680 / 00.4245 / 00.7192 / 00.7975 / 20.65516 / 00.7358 / 10.77219 / 00.7327 / 00.76816 / 00.77124 / 10.72928 / 00.726
Verdin-4B-v1.1 Open · 4.0B—126 / 40.75214 / 10.7315 / 00.7190 / 00.5484 / 00.6776 / 00.7438 / 10.76316 / 00.8248 / 00.77414 / 00.7999 / 20.77217 / 00.76112 / 00.77213 / 00.696
Perplexity Decider Flash v1.3 Open · 4.0B—113 / 10.7876 / 00.7655 / 00.6950 / 00.4502 / 00.7013 / 00.8369 / 00.78823 / 00.9137 / 00.83012 / 00.7977 / 00.79211 / 00.76415 / 00.79213 / 10.765
Winnow-12B Open · 12B—20 / 10.7435 / 00.7421 / 00.6380 / 00.6860 / 00.6701 / 00.6420 / 10.7431 / 00.7871 / 00.7425 / 00.7313 / 00.7851 / 00.7461 / 00.7581 / 00.774
GLiDE 2B Open · 1.9B—15 / 10.6353 / 10.6250 / 00.5690 / 00.5191 / 00.5540 / 00.5901 / 00.6761 / 00.6461 / 00.7283 / 00.6632 / 00.5961 / 00.6171 / 00.6161 / 00.639

Per topic: answers kept at ≥99% (right / wrong) and, under them, the AUROC of the chosen option's probability (– where a topic has only right or only wrong answers).

SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 2,151 of 2,163); shown once.

Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.

Every setup, every measure

SetupWebAccuracy% of N (N ≤ 2,163) · Wilson 95%Tie groupHolm · McNemarRight / NquestionsKept at ≥99%right / wrongAUROCright vs wrongDecision Score (AA formula)0–100 · chosen-answer BrierNon-hallucinationof answers not right, % “I don't know” · higher is betterMedian latencymsp90ms, nearest rankCost$ per 1,000 qper 1M q
Karpo Decision 31B Open · 31B

Karpo Decision 31B · MachinePulse · decision model

Parameters: 31B dense

Kept at ≥99%: 837 answers, 837 right · 0 wrong · AUROC 0.903

Logic 60.7%

Its run record (Method) →

—81.6%79.9%–83.1%a1,764 / 2,163837 right of 837 kept0 mistakes · 1,326 skipped0.90385.99chosen-answer Brier 0.09584.8%answered 99%155 ms186 ms$0.113$113estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen2 Open · 31B

KnowLine-31B-Gen2 · PelaAI · decision model

Parameters: 31B dense

Base model: Gemma 4 31B

Kept at ≥99%: 470 answers, 469 right · 1 wrong · AUROC 0.898

Logic 58.9%

Its run record (Method) →

—80.3%78.5%–81.9%ab1,736 / 2,163469 right of 470 kept1 wrong · 1,693 skipped0.89885.05chosen-answer Brier 0.10160.2%answered 100%114 ms120 ms$0.083$83.14estimate · RTX PRO 6000 Blackwell
REAX 1.3 Open · 57B

REAX 1.3 · REAX · decision model

Parameters: 57.36B dense

Kept at ≥99%: 178 answers, 178 right · 0 wrong · AUROC 0.903

Logic 59.3%

Its run record (Method) →

—78.0%76.2%–79.7%bc1,687 / 2,163178 right of 178 kept0 mistakes · 1,985 skipped0.90383.57chosen-answer Brier 0.10842.3%answered 99%189 ms232 ms$0.142$142estimate · RTX PRO 6000 Blackwell
KnowLine-31B-Gen1.5 Open · 31B

KnowLine-31B-Gen1.5 · PelaAI · decision model

Parameters: 31B dense

Kept at ≥99%: 465 answers, 464 right · 1 wrong · AUROC 0.895

Logic 59.0%

Its run record (Method) →

—77.4%75.6%–79.1%cd1,674 / 2,163464 right of 465 kept1 wrong · 1,698 skipped0.89583.18chosen-answer Brier 0.11031.1%answered 100%764 ms867 ms$0.047$46.87estimate · DGX Spark
Copernicus-31B Open · 31B

Copernicus-31B · Poindexter Labs · decision model

Parameters: 31B dense

Kept at ≥99%: 318 answers, 318 right · 0 wrong · AUROC 0.899

Logic 58.7%

Its run record (Method) →

—77.3%75.5%–79.1%cd1,673 / 2,163318 right of 318 kept0 mistakes · 1,845 skipped0.89983.06chosen-answer Brier 0.11221.7%answered 100%118 ms142 ms$0.084$84.42estimate · RTX PRO 6000 Blackwell
Sieve 31B v3 Open · 31B

Sieve 31B v3 · Sthānika AI · decision model

Parameters: 31B dense

Kept at ≥99%: 215 answers, 215 right · 0 wrong · AUROC 0.891

Logic 57.9%

Its run record (Method) →

—77.1%75.3%–78.8%cd1,668 / 2,163215 right of 215 kept0 mistakes · 1,948 skipped0.89182.89chosen-answer Brier 0.11332.1%answered 100%637 ms735 ms$0.039$39.14estimate · DGX Spark
Logits Sigma-1 Duo Open · 57B

Logits Sigma-1 Duo · Logits · decision model

Parameters: 57.36B dense

Kept at ≥99%: 15 answers, 15 right · 0 wrong · AUROC 0.899

Logic 57.8%

Its run record (Method) →

—75.2%73.4%–77.0%de1,627 / 2,16315 right of 15 kept0 mistakes · 2,148 skipped0.89981.60chosen-answer Brier 0.12012.1%answered 99%196 ms221 ms$0.141$141estimate · RTX PRO 6000 Blackwell
d1-ultra Closed · Efficient

d1-ultra · Liquid AI · decision model

Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count)

Kept at ≥99%: 213 answers, 213 right · 0 wrong · AUROC 0.876

Logic 54.9%

Its run record (Method) →

—74.1%72.2%–75.9%defg1,603 / 2,163213 right of 213 kept0 mistakes · 1,950 skipped0.87680.88chosen-answer Brier 0.123587.4%answered 61%179 ms426 ms$0.006$6.14
Mammoth-1 Open · 27B

Mammoth-1 · Base Labs · decision model

Parameters: 27B dense

Kept at ≥99%: 412 answers, 412 right · 0 wrong · AUROC 0.895

Logic 54.5%

Its run record (Method) →

—74.0%72.1%–75.8%defh1,600 / 2,163412 right of 412 kept0 mistakes · 1,751 skipped0.89581.16chosen-answer Brier 0.116470.6%answered 75%140 ms173 ms$0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell
Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B

Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 323 answers, 323 right · 0 wrong · AUROC 0.856

Logic 54.8%

Its run record (Method) →

—72.5%70.6%–74.3%efghi1,568 / 2,163323 right of 323 kept0 mistakes · 1,840 skipped0.85679.32chosen-answer Brier 0.13859.4%answered 97%91 ms103 ms$0.066$65.87estimate · RTX PRO 6000 Blackwell
GLiDE 27B Open · 27B

GLiDE 27B · Fastino · decision model

Parameters: 27B dense

Kept at ≥99%: 199 answers, 198 right · 1 wrong · AUROC 0.864

Logic 58.4%

Its run record (Method) →

—72.0%70.1%–73.8%efghij1,557 / 2,163198 right of 199 kept1 wrong · 1,964 skipped0.86479.22chosen-answer Brier 0.135421.6%answered 94%425 ms461 ms$0.026$26.16estimate · DGX Spark
d3-ultra Open · 26B

d3-ultra · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Kept at ≥99%: 11 answers, 11 right · 0 wrong · AUROC 0.857

Logic 54.6%

Its run record (Method) →

—71.5%69.6%–73.4%fghij1,547 / 2,16311 right of 11 kept0 mistakes · 2,152 skipped0.85778.39chosen-answer Brier 0.14745.3%answered 98%118 ms148 ms$0.089$88.58estimate · RTX PRO 6000 Blackwell
Gemini Decision 1 20261111 Closed · Efficient

Gemini Decision 1 20261111 · Google · decision model

Parameters: not disclosed by maker (closed weights)

Kept at ≥99%: 214 answers, 214 right · 0 wrong · AUROC 0.859

Logic 50.0%

Its run record (Method) →

—71.3%69.4%–73.2%efghij1,543 / 2,163214 right of 214 kept0 mistakes · 1,949 skipped0.85978.84chosen-answer Brier 0.13663.9%answered 99%226 ms351 ms$0.022$22.02
GLiDE API Closed · Efficient

GLiDE API · Fastino · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 161 answers, 161 right · 0 wrong · AUROC 0.865

Logic 54.5%

Its run record (Method) →

—70.8%68.8%–72.7%fghij1,531 / 2,163161 right of 161 kept0 mistakes · 2,002 skipped0.86578.56chosen-answer Brier 0.136740.6%answered 86%264 ms279 ms$0.028$28.30
d3 Open · 26B

d3 · vLLM Semantic Router · decision model

Parameters: 26.09B dense

Kept at ≥99%: 163 answers, 162 right · 1 wrong · AUROC 0.878

Logic 54.5%

Its run record (Method) →

—70.4%68.5%–72.3%gij1,523 / 2,163162 right of 163 kept1 wrong · 2,000 skipped0.87878.71chosen-answer Brier 0.12994.7%answered 99%55 ms62 ms$0.041$40.61estimate · RTX PRO 6000 Blackwell
Drex Max Open · 27B

Drex Max · Nace.AI · decision model

Parameters: 27B dense

Kept at ≥99%: 729 answers, 719 right · 10 wrong · AUROC 0.861

Logic 53.1%

Its run record (Method) →

—70.3%68.4%–72.2%gijk1,521 / 2,163719 right of 729 kept10 wrong · 1,434 skipped0.86177.10chosen-answer Brier 0.16114.5%answered 99%76 ms89 ms$0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell
Drex Max v1.1 Closed · Efficient (provisional)

Drex Max v1.1 · Nace.AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 315 answers, 315 right · 0 wrong · AUROC 0.867

Logic 54.8%

Its run record (Method) →

—70.1%68.1%–72.0%ijk1,516 / 2,163315 right of 315 kept0 mistakes · 1,848 skipped0.86778.01chosen-answer Brier 0.140743.5%answered 84%326 ms473 ms$0.013$12.88
celeris-1-decision Closed · Efficient

celeris-1-decision · Celeris · decision model

Parameters: not disclosed by maker

Kept at ≥99%: No answers at this threshold · AUROC 0.858

Logic 47.3%

Its run record (Method) →

—70.0%68.0%–71.9%hijk1,514 / 2,163No answers at this threshold0.85877.96chosen-answer Brier 0.140782.7%answered 62%387 ms391 ms$0.009$9.10
GLiDE v1.2 Open · 27B

GLiDE v1.2 · Fastino · decision model

Parameters: 27B dense

Kept at ≥99%: 131 answers, 131 right · 0 wrong · AUROC 0.862

Logic 55.2%

Its run record (Method) →

—69.9%67.9%–71.8%ijk1,511 / 2,163131 right of 131 kept0 mistakes · 2,032 skipped0.86277.95chosen-answer Brier 0.139635.8%answered 88%79 ms84 ms$0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell
celeris-1.1-decision Closed · Efficient

celeris-1.1-decision · Celeris · decision model

Parameters: not disclosed by maker

Kept at ≥99%: No answers at this threshold · AUROC 0.859

Logic 47.3%

Its run record (Method) →

—69.9%67.9%–71.8%ijkl1,511 / 2,163No answers at this threshold0.85977.91chosen-answer Brier 0.140382.9%answered 62%381 ms386 ms$0.001$0.91
d1.2-ultra Closed · Undisclosed

d1.2-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 115 answers, 115 right · 0 wrong · AUROC 0.876

Logic 51.6%

Its run record (Method) →

—69.9%67.9%–71.8%ijk1,511 / 2,163115 right of 115 kept0 mistakes · 2,048 skipped0.87678.23chosen-answer Brier 0.134086.7%answered 58%215 ms347 ms$0.006$6.14
Perplexity Decider 27B v1.2 API Open · 27B

Perplexity Decider 27B v1.2 API · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 331 answers, 331 right · 0 wrong · AUROC 0.869

Logic 53.3%

Its run record (Method) →

—69.8%67.8%–71.7%ijkl1,509 / 2,163331 right of 331 kept0 mistakes · 1,832 skipped0.86978.16chosen-answer Brier 0.13457.0%answered 98%226 ms368 ms$0.004$4.29
Perplexity Decider 27B v1.2 API · with web Open · 27B

Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

Parameters: 27B dense

Kept at ≥99%: 326 answers, 326 right · 0 wrong · AUROC 0.873

Logic 52.5%

Its run record (Method) →

with web69.3%67.3%–71.2%jklm1,499 / 2,163326 right of 326 kept0 mistakes · 1,837 skipped0.87377.94chosen-answer Brier 0.134215.4%answered 95%422 ms641 ms$1.023$1,023
Turion Jev 31B Open · 31B

Turion Jev 31B · Turion.AI · decision model

Parameters: 31B dense

Kept at ≥99%: 43 answers, 43 right · 0 wrong · AUROC 0.864

Logic 52.3%

Its run record (Method) →

—69.0%67.0%–70.9%ijklmn1,493 / 2,16343 right of 43 kept0 mistakes · 2,120 skipped0.86477.52chosen-answer Brier 0.13992.2%answered 99%278 ms329 ms$0.017$17.46estimate · DGX Spark
SubQ Snap 1.1 Closed · Efficient

SubQ Snap 1.1 · Subquadratic · decision model

Parameters: not disclosed by maker

Shown once with SubQ Snap Preview: the same answer to every question

Kept at ≥99%: 361 answers, 360 right · 1 wrong · AUROC 0.839

Logic 52.4%

Its run record (Method) →

—68.6%66.6%–70.5%jklmn1,483 / 2,163360 right of 361 kept1 wrong · 1,802 skipped0.83976.15chosen-answer Brier 0.162652.5%answered 80%381 ms390 ms$0.009$8.59
KnowLine-12B-Gen1 Open · 12B

KnowLine-12B-Gen1 · PelaAI · decision model

Parameters: 12B dense

Base model: Gemma 4 12B

Kept at ≥99%: 7 answers, 7 right · 0 wrong · AUROC 0.819

Logic 50.6%

Its run record (Method) →

—66.3%64.2%–68.2%klmno1,433 / 2,1637 right of 7 kept0 mistakes · 2,156 skipped0.81974.96chosen-answer Brier 0.16321.1%answered 100%49 ms55 ms$0.036$36.13estimate · RTX PRO 6000 Blackwell
d1.2 Closed · Undisclosed

d1.2 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 141 answers, 141 right · 0 wrong · AUROC 0.838

Logic 49.2%

Its run record (Method) →

—65.5%63.5%–67.5%lmnop1,417 / 2,163141 right of 141 kept0 mistakes · 2,022 skipped0.83874.98chosen-answer Brier 0.155656.3%answered 76%204 ms386 ms$0.003$2.92
d1.1 Closed · Efficient

d1.1 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 183 answers, 183 right · 0 wrong · AUROC 0.829

Logic 46.0%

Its run record (Method) →

—65.2%63.2%–67.2%mnop1,410 / 2,163183 right of 183 kept0 mistakes · 1,980 skipped0.82974.62chosen-answer Brier 0.159582.5%answered 56%172 ms290 ms$0.003$2.92
Jev 1.14 Closed · Efficient

Jev 1.14 · TypeSafe · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 202 answers, 202 right · 0 wrong · AUROC 0.829

Logic 48.2%

Its run record (Method) →

—65.1%63.1%–67.1%nop1,409 / 2,163202 right of 202 kept0 mistakes · 1,961 skipped0.82974.56chosen-answer Brier 0.16027.7%answered 97%163 ms185 ms$0.019$18.99
Jev 1.13 Closed · Efficient

Jev 1.13 · TypeSafe · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 248 answers, 248 right · 0 wrong · AUROC 0.834

Logic 46.3%

Its run record (Method) →

—64.1%62.0%–66.1%opq1,386 / 2,163248 right of 248 kept0 mistakes · 1,915 skipped0.83474.08chosen-answer Brier 0.15925.7%answered 98%166 ms206 ms$0.019$18.99
Adjudex 9B v5 Open · 9.0B

Adjudex 9B v5 · newsbang · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 198 answers, 197 right · 1 wrong · AUROC 0.806

Logic 46.4%

Its run record (Method) →

—62.1%60.0%–64.1%opqr1,343 / 2,163197 right of 198 kept1 wrong · 1,965 skipped0.80672.28chosen-answer Brier 0.17523.1%answered 99%36 ms44 ms$0.027$26.54estimate · RTX PRO 6000 Blackwell
Elephant-1 Open · 9.0B

Elephant-1 · Base Labs · decision model

Parameters: 9B dense

Kept at ≥99%: 183 answers, 182 right · 1 wrong · AUROC 0.803

Logic 46.4%

Its run record (Method) →

—61.3%59.3%–63.4%pqr1,327 / 2,163182 right of 183 kept1 wrong · 1,980 skipped0.80371.86chosen-answer Brier 0.176365.0%answered 67%80 ms92 ms$0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen2 Open · 9.0B

KnowLine-9B-Gen2 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: No answers at this threshold · AUROC 0.803

Logic 43.9%

Its run record (Method) →

—60.1%58.0%–62.1%qrs1,300 / 2,163No answers at this threshold0.80371.15chosen-answer Brier 0.17793.2%answered 99%41 ms46 ms$0.031$31.02estimate · RTX PRO 6000 Blackwell
Jeffan-9B-1211 Open · 9.0B

Jeffan-9B-1211 · tattrongvu · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 64 answers, 64 right · 0 wrong · AUROC 0.814

Logic 48.2%

Its run record (Method) →

—59.8%57.7%–61.9%qrs1,294 / 2,16364 right of 64 kept0 mistakes · 2,099 skipped0.81471.32chosen-answer Brier 0.17196.6%answered 97%37 ms51 ms$0.029$28.78estimate · RTX PRO 6000 Blackwell
KnowLine-9B-Gen1 Open · 9.0B

KnowLine-9B-Gen1 · PelaAI · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 123 answers, 123 right · 0 wrong · AUROC 0.799

Logic 44.3%

Its run record (Method) →

—59.8%57.7%–61.8%qrs1,293 / 2,163123 right of 123 kept0 mistakes · 2,040 skipped0.79971.08chosen-answer Brier 0.17620.9%answered 100%37 ms43 ms$0.027$26.54estimate · RTX PRO 6000 Blackwell
Microsoft-Decision-1 Closed · Efficient

Microsoft-Decision-1 · Microsoft · decision model

Parameters: not disclosed by maker

Base model: Qwen3.5-9B

Kept at ≥99%: 27 answers, 27 right · 0 wrong · AUROC 0.787

Logic 44.4%

Its run record (Method) →

—59.4%57.3%–61.4%rst1,284 / 2,16327 right of 27 kept0 mistakes · 2,136 skipped0.78770.49chosen-answer Brier 0.18395.1%answered 98%310 ms359 ms$0.007$6.81
Drex 2.0 Open · 35B MoE · 3.0B active

Drex 2.0 · Nace.AI · decision model

Parameters: 35B MoE · 3B active

Kept at ≥99%: 174 answers, 174 right · 0 wrong · AUROC 0.813

Logic 41.9%

Its run record (Method) →

—59.0%56.9%–61.0%rstu1,276 / 2,163174 right of 174 kept0 mistakes · 1,989 skipped0.81370.98chosen-answer Brier 0.170320.7%answered 91%334 ms432 ms$0.009$8.92
Dexy-Decision-9B-V2 Open · 9.0B

Dexy-Decision-9B-V2 · Patrick-18 · decision model

Parameters: 9B dense

Base model: vllm-sr/d3-flash

Kept at ≥99%: 210 answers, 207 right · 3 wrong · AUROC 0.804

Logic 45.3%

Its run record (Method) →

—58.8%56.7%–60.9%rstu1,272 / 2,163207 right of 210 kept3 wrong · 1,953 skipped0.80469.67chosen-answer Brier 0.19461.7%answered 99%45 ms50 ms$0.032$31.98estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-9B-Instant Open · 9.0B

StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

Parameters: 9B dense

Kept at ≥99%: 49 answers, 49 right · 0 wrong · AUROC 0.781

Logic 46.6%

Its run record (Method) →

—58.4%56.3%–60.5%rstuv1,264 / 2,16349 right of 49 kept0 mistakes · 2,114 skipped0.78169.90chosen-answer Brier 0.18644.2%answered 98%140 ms146 ms$0.008$8.34estimate · DGX Spark
GLiDE 9B Open · 7.9B

GLiDE 9B · Fastino · decision model

Parameters: 7.94B dense

Kept at ≥99%: 73 answers, 73 right · 0 wrong · AUROC 0.798

Logic 47.8%

Its run record (Method) →

—57.9%55.8%–60.0%rstuvw1,253 / 2,16373 right of 73 kept0 mistakes · 2,090 skipped0.79869.69chosen-answer Brier 0.185524.6%answered 89%174 ms182 ms$0.010$10.49estimate · DGX Spark
vera-medium-omni Open · 4.2B

vera-medium-omni · scar-ai · decision model

Parameters: 4.2B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 33 answers, 33 right · 0 wrong · AUROC 0.789

Logic 44.1%

Its run record (Method) →

—56.8%54.7%–58.9%stuvw1,229 / 2,16333 right of 33 kept0 mistakes · 2,130 skipped0.78969.18chosen-answer Brier 0.18463.8%answered 98%31 ms35 ms$0.023$23.34estimate · RTX PRO 6000 Blackwell
Cerno 9B v1.2 Open · 9.0B

Cerno 9B v1.2 · Apertix AI · decision model

Parameters: 9B dense

Kept at ≥99%: 254 answers, 244 right · 10 wrong · AUROC 0.780

Logic 43.3%

Its run record (Method) →

—56.0%53.9%–58.1%stuvw1,212 / 2,163244 right of 254 kept10 wrong · 1,909 skipped0.78067.15chosen-answer Brier 0.21741.2%answered 99%153 ms175 ms$0.010$9.51estimate · DGX Spark
KnowLine 4B Gen5.5 Open · 4.0B

KnowLine 4B Gen5.5 · PelaAI · decision model

Parameters: 4B dense

Kept at ≥99%: 54 answers, 54 right · 0 wrong · AUROC 0.765

Logic 42.3%

Its run record (Method) →

—56.0%53.9%–58.1%stuvw1,211 / 2,16354 right of 54 kept0 mistakes · 2,109 skipped0.76567.89chosen-answer Brier 0.20211.6%answered 99%74 ms84 ms$0.005$4.56estimate · DGX Spark
Perplexity Decider Flash v1.3 Open · 4.0B

Perplexity Decider Flash v1.3 · Perplexity · decision model

Parameters: 4B dense

Kept at ≥99%: 114 answers, 113 right · 1 wrong · AUROC 0.787

Logic 44.7%

Its run record (Method) →

—55.9%53.8%–58.0%stuvw1,210 / 2,163113 right of 114 kept1 wrong · 2,049 skipped0.78767.89chosen-answer Brier 0.20158.4%answered 96%28 ms31 ms$0.022$22.38estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-4B-Instant Open · 4.0B

StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

Parameters: 4B dense

Kept at ≥99%: 96 answers, 96 right · 0 wrong · AUROC 0.779

Logic 45.9%

Its run record (Method) →

—55.8%53.7%–57.9%stuvwx1,208 / 2,16396 right of 96 kept0 mistakes · 2,067 skipped0.77968.25chosen-answer Brier 0.193413.4%answered 94%67 ms69 ms$0.004$4.10estimate · DGX Spark
d1 Closed · Efficient

d1 · Liquid AI · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 152 answers, 152 right · 0 wrong · AUROC 0.767

Logic 40.8%

Its run record (Method) →

—55.6%53.5%–57.7%stuvwxy1,203 / 2,163152 right of 152 kept0 mistakes · 2,011 skipped0.76768.16chosen-answer Brier 0.19302.9%answered 99%184 ms297 ms$0.003$2.92
Valen-Decision-1.5-9B Open · 9.0B

Valen-Decision-1.5-9B · Valen-Team · decision model

Parameters: 9B dense

Base model: Qwen3.5-9B

Kept at ≥99%: 130 answers, 130 right · 0 wrong · AUROC 0.790

Logic 44.4%

Its run record (Method) →

—55.3%53.2%–57.4%tuvwxy1,197 / 2,163130 right of 130 kept0 mistakes · 2,033 skipped0.79068.37chosen-answer Brier 0.18602.6%answered 99%58 ms64 ms$0.041$40.93estimate · RTX PRO 6000 Blackwell
Perplexity Decider Flash v1.2 Open · 4.0B

Perplexity Decider Flash v1.2 · Perplexity · decision model

Parameters: 4B dense

Kept at ≥99%: 155 answers, 150 right · 5 wrong · AUROC 0.781

Logic 43.1%

Its run record (Method) →

—55.2%53.1%–57.3%tuvwxy1,195 / 2,163150 right of 155 kept5 wrong · 2,008 skipped0.78167.61chosen-answer Brier 0.200212.0%answered 94%40 ms47 ms$0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell
Open-Jev-27B-v1.1 Open · 27B

Open-Jev-27B-v1.1 · Zefan Cai · decision model

Parameters: 27B dense

Base model: Qwen3.8-27B + adapter

Kept at ≥99%: No answers at this threshold · AUROC 0.764

Logic 42.6%

Its run record (Method) →

—54.6%52.4%–56.6%uvwxyz1,180 / 2,163No answers at this threshold0.76464.53chosen-answer Brier 0.25495.0%answered 98%233 ms272 ms$0.169$169estimate · RTX PRO 6000 Blackwell
OneJev-27B Open · 27B

OneJev-27B · OmniJev · decision model

Parameters: 27B dense

Kept at ≥99%: 26 answers, 26 right · 0 wrong · AUROC 0.783

Logic 40.6%

Its run record (Method) →

—53.8%51.7%–55.9%vwxyzA1,163 / 2,16326 right of 26 kept0 mistakes · 2,137 skipped0.78367.54chosen-answer Brier 0.18691.8%answered 99%79 ms106 ms$0.061$60.76estimate · RTX PRO 6000 Blackwell
Mercury Decide Closed · Efficient

Mercury Decide · Inception · decision model

Parameters: not disclosed by maker

Kept at ≥99%: 281 answers, 272 right · 9 wrong · AUROC 0.789

Logic 45.1%

Its run record (Method) →

—53.5%51.4%–55.6%wxyzA1,157 / 2,163272 right of 281 kept9 wrong · 1,882 skipped0.78966.02chosen-answer Brier 0.21463.2%answered 99%288 ms342 ms$0.004$4.38
Verdin-4B-v1.1 Open · 4.0B

Verdin-4B-v1.1 · victorbennett · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 130 answers, 126 right · 4 wrong · AUROC 0.752

Logic 39.7%

Its run record (Method) →

—51.5%49.4%–53.6%yzAB1,114 / 2,163126 right of 130 kept4 wrong · 2,033 skipped0.75265.62chosen-answer Brier 0.20266.4%answered 97%47 ms52 ms$0.034$33.58estimate · RTX PRO 6000 Blackwell
Clef Open · 27B

Clef · Cloudflare · decision model

Parameters: 27B dense

Kept at ≥99%: 12 answers, 12 right · 0 wrong · AUROC 0.767

Logic 40.8%

Its run record (Method) →

—51.1%49.0%–53.2%xyzAB1,105 / 2,16312 right of 12 kept0 mistakes · 2,151 skipped0.76765.20chosen-answer Brier 0.20701.6%answered 99%181 ms210 ms$0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell
Chinese Decision Pointer 4B Open · 4.0B

Chinese Decision Pointer 4B · quincyqiang · decision model

Parameters: 4B dense

Base model: Qwen3.5-4B

Kept at ≥99%: 155 answers, 149 right · 6 wrong · AUROC 0.746

Logic 40.4%

Its run record (Method) →

—50.4%48.3%–52.5%zAB1,091 / 2,163149 right of 155 kept6 wrong · 2,008 skipped0.74664.11chosen-answer Brier 0.22213.4%answered 98%28 ms32 ms$0.023$22.70estimate · RTX PRO 6000 Blackwell
Clef-Flash Open · 9.0B

Clef-Flash · Cloudflare · decision model

Parameters: 9B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.730

Logic 38.9%

Its run record (Method) →

—49.8%47.7%–51.9%AB1,077 / 2,163No answers at this threshold0.73064.44chosen-answer Brier 0.20922.0%answered 99%99 ms127 ms$0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell
Winnow-12B Open · 12B

Winnow-12B · EldanRing · decision model

Parameters: 12B dense

Kept at ≥99%: 21 answers, 20 right · 1 wrong · AUROC 0.743

Logic 39.7%

Its run record (Method) →

—46.9%44.8%–49.0%BC1,014 / 2,16320 right of 21 kept1 wrong · 2,142 skipped0.74363.00chosen-answer Brier 0.20879.2%answered 95%52 ms61 ms$0.038$38.37estimate · RTX PRO 6000 Blackwell
Cygnet Open · 12B

Cygnet · Blockbrain Labs · decision model

Parameters: 11.95B dense

Base model: Gemma 4 12B IT

Kept at ≥99%: No answers at this threshold · AUROC 0.735

Logic 39.3%

Its run record (Method) →

—44.3%42.3%–46.4%C959 / 2,163No answers at this threshold0.73561.87chosen-answer Brier 0.20604.9%answered 97%37 ms41 ms$0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell
Metask-Jev-4B Open · 4.5B

Metask-Jev-4B · Metask Lab · decision model

Parameters: 4.54B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.651

Logic 32.1%

Its run record (Method) →

—39.2%37.1%–41.2%D847 / 2,163No answers at this threshold0.65158.63chosen-answer Brier 0.21902.2%answered 99%113 ms144 ms$0.083$83.14estimate · RTX PRO 6000 Blackwell
StartLux-Decision-v1.1-2B-Instant Open · 2.0B

StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

Parameters: 2B dense

Kept at ≥99%: 2 answers, 2 right · 0 wrong · AUROC 0.685

Logic 30.8%

Its run record (Method) →

—38.7%36.6%–40.7%D836 / 2,1632 right of 2 kept0 mistakes · 2,161 skipped0.68558.95chosen-answer Brier 0.20757.2%answered 95%36 ms37 ms$0.002$2.27estimate · DGX Spark
d1-2B Open · 1.9B

d1-2B · Liquid AI · decision model

Parameters: 1.92B dense

Base model: LFM3-VL-2B

Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.656

Logic 30.7%

Its run record (Method) →

—37.1%35.1%–39.1%D802 / 2,1634 right of 4 kept0 mistakes · 2,159 skipped0.65657.72chosen-answer Brier 0.216380.2%answered 34%16 ms21 ms$0.014$14.07estimate · RTX PRO 6000 Blackwell
GLiDE 2B Open · 1.9B

GLiDE 2B · Fastino · decision model

Parameters: 1.88B dense

Kept at ≥99%: 16 answers, 15 right · 1 wrong · AUROC 0.635

Logic 32.3%

Its run record (Method) →

—36.8%34.7%–38.8%D795 / 2,16315 right of 16 kept1 wrong · 2,147 skipped0.63555.70chosen-answer Brier 0.253617.3%answered 89%28 ms30 ms$0.002$1.78estimate · DGX Spark
KnowLine-2B-Gen1 Open · 2.0B

KnowLine-2B-Gen1 · PelaAI · decision model

Parameters: 2B dense

Base model: Qwen3.5-2B

Kept at ≥99%: 8 answers, 8 right · 0 wrong · AUROC 0.636

Logic 31.7%

Its run record (Method) →

—35.3%33.3%–37.3%DE763 / 2,1638 right of 8 kept0 mistakes · 2,155 skipped0.63656.49chosen-answer Brier 0.22291.7%answered 99%50 ms54 ms$0.003$3.20estimate · DGX Spark
GLiDE 0.8B Open · 0.8B

GLiDE 0.8B · Fastino · decision model

Parameters: 0.75B dense

Kept at ≥99%: 4 answers, 4 right · 0 wrong · AUROC 0.589

Logic 29.2%

Its run record (Method) →

—32.1%30.2%–34.1%EF694 / 2,1634 right of 4 kept0 mistakes · 2,159 skipped0.58953.21chosen-answer Brier 0.256620.8%answered 85%20 ms21 ms$0.001$1.28estimate · DGX Spark
StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B

StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Kept at ≥99%: No answers at this threshold · AUROC 0.583

Logic 24.7%

Its run record (Method) →

—29.7%27.8%–31.6%F642 / 2,163No answers at this threshold0.58354.40chosen-answer Brier 0.208811.3%answered 92%24 ms25 ms$0.002$1.53estimate · DGX Spark
Strands Decider 2B Open · 2.0B

Strands Decider 2B · Strands Agents · decision model

Parameters: 2B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.569

Logic 22.4%

Its run record (Method) →

—28.7%26.8%–30.6%F620 / 2,163No answers at this threshold0.56953.90chosen-answer Brier 0.20863.2%answered 98%35 ms41 ms$0.026$26.22estimate · RTX PRO 6000 Blackwell
OneJev1.6-0.8B Open · 0.8B

OneJev1.6-0.8B · OmniJev · decision model

Parameters: 0.8B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.592

Logic 27.0%

Its run record (Method) →

—28.2%26.4%–30.2%F611 / 2,163No answers at this threshold0.59253.62chosen-answer Brier 0.21000.6%answered 100%15 ms17 ms$0.001$1.09estimate · DGX Spark
KnowLine-0.8B-Gen1 Open · 0.8B

KnowLine-0.8B-Gen1 · PelaAI · decision model

Parameters: 0.8B dense

Base model: Qwen3.5-0.8B

Kept at ≥99%: No answers at this threshold · AUROC 0.596

Logic 26.7%

Its run record (Method) →

—28.2%26.3%–30.1%F610 / 2,163No answers at this threshold0.59653.60chosen-answer Brier 0.21000.3%answered 100%25 ms28 ms$0.002$1.69estimate · DGX Spark
GLiClass v4 Edge Open · 0.4B

GLiClass v4 Edge · decision model

Parameters: 0.4B dense

Base model: DeBERTa-v3-large

Kept at ≥99%: No answers at this threshold · AUROC 0.501

Logic 23.6%

Its run record (Method) →

—22.7%20.9%–24.5%G490 / 2,163No answers at this threshold0.50152.01chosen-answer Brier 0.186317.2%answered 86%12 ms13 ms$0.001$0.85estimate · DGX Spark
GLiDE 0.4B Open · 0.4B

GLiDE 0.4B · Fastino · decision model

Parameters: 0.40B dense

Kept at ≥99%: No answers at this threshold · AUROC 0.500

Logic 23.2%

Its run record (Method) →

—20.6%18.9%–22.3%G445 / 2,163No answers at this threshold0.50050.98chosen-answer Brier 0.186272.8%answered 33%11 ms12 ms$0.001$0.77estimate · DGX Spark

Tie groups and “not measured”

estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.

Decision Score (AA formula): Artificial Analysis's formula applied to our questions. Not their Index. 50 × (1 + accuracy − chosen-answer Brier), the chosen-answer Brier over the probability of the chosen answer, on decision models' standard runs only. Chat models and the “I don't know” runs are excluded.