Logic · Cost
Cost on Logic
1,125 questions, asked in English. How to read this page · Method
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino OmniJev Other Celeris StartLux PelaAI Perplexity Liquid AI Inception Apertix AI Microsoft Base Labs Subquadratic Cloudflare Nace.AI Turion.AI TypeSafe scar-ai Strands Agents quincyqiang newsbang victorbennett Patrick-18 Google tattrongvu Blockbrain Labs Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab MachinePulse Poindexter Labs REAX Logits Zefan Cai eCloud Technology MiniMax Z.ai OpenBMB Alibaba Xiaomi DeepSeek Meta Thinking Machines KRAFTON IBM Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 23.2% at $0.001 per 1,000 questions ($0.95 per 1M) (estimate)
OneJev1.6-0.8B: 27.0% at $0.001 per 1,000 questions ($1.16 per 1M) (estimate)
GLiClass v4 Edge: 23.6% at $0.001 per 1,000 questions ($1.31 per 1M) (estimate)
celeris-1.1-decision: 47.3% at $0.002 per 1,000 questions ($1.54 per 1M)
GLiDE 0.8B: 29.2% at $0.002 per 1,000 questions ($1.58 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 24.7% at $0.002 per 1,000 questions ($1.79 per 1M) (estimate)
KnowLine-0.8B-Gen1: 26.7% at $0.002 per 1,000 questions ($2.00 per 1M) (estimate)
GLiDE 2B: 32.3% at $0.002 per 1,000 questions ($2.26 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 30.8% at $0.003 per 1,000 questions ($2.52 per 1M) (estimate)
Perplexity Decider Flash v1.2: 43.1% at $0.004 per 1,000 questions ($3.76 per 1M)
KnowLine-2B-Gen1: 31.7% at $0.004 per 1,000 questions ($3.78 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 45.9% at $0.005 per 1,000 questions ($4.62 per 1M) (estimate)
KnowLine 4B Gen5.5: 42.3% at $0.006 per 1,000 questions ($5.89 per 1M) (estimate)
d1.2: 49.2% at $0.006 per 1,000 questions ($6.02 per 1M)
d1.1: 46.0% at $0.006 per 1,000 questions ($6.02 per 1M)
d1: 40.8% at $0.006 per 1,000 questions ($6.02 per 1M)
Perplexity Decider 27B v1.2 (API): 53.3% at $0.007 per 1,000 questions ($7.46 per 1M)
Mercury Decide: 45.1% at $0.008 per 1,000 questions ($7.52 per 1M)
StartLux-Decision-v1.1-9B-Instant: 46.6% at $0.010 per 1,000 questions ($10.04 per 1M) (estimate)
d1-ultra: 54.9% at $0.013 per 1,000 questions ($12.64 per 1M)
d1.2-ultra: 51.6% at $0.013 per 1,000 questions ($12.64 per 1M)
GLiDE 9B: 47.8% at $0.013 per 1,000 questions ($13.29 per 1M) (estimate)
Cerno 9B v1.2: 43.3% at $0.013 per 1,000 questions ($13.35 per 1M) (estimate)
Microsoft-Decision-1: 44.4% at $0.013 per 1,000 questions ($13.42 per 1M)
Elephant-1: 46.4% at $0.013 per 1,000 questions ($13.44 per 1M)
GLiDE (v1.2): 55.2% at $0.014 per 1,000 questions ($13.88 per 1M)
d1-2B: 30.7% at $0.015 per 1,000 questions ($14.76 per 1M) (estimate)
SubQ Snap 1.1: 52.4% at $0.015 per 1,000 questions ($14.92 per 1M)
celeris-1-decision: 47.3% at $0.015 per 1,000 questions ($15.40 per 1M)
Clef-Flash: 38.9% at $0.017 per 1,000 questions ($16.83 per 1M)
Drex 2.0: 41.9% at $0.018 per 1,000 questions ($18.41 per 1M)
Drex Max v1.1: 54.8% at $0.022 per 1,000 questions ($22.37 per 1M)
Drex Max: 53.1% at $0.023 per 1,000 questions ($22.55 per 1M)
Turion Jev 31B: 52.3% at $0.025 per 1,000 questions ($24.91 per 1M) (estimate)
Jev 1.14: 48.2% at $0.026 per 1,000 questions ($25.64 per 1M)
Jev 1.13: 46.3% at $0.026 per 1,000 questions ($25.64 per 1M)
vera-medium-omni: 44.1% at $0.026 per 1,000 questions ($25.82 per 1M) (estimate)
Perplexity Decider Flash v1.3: 44.7% at $0.026 per 1,000 questions ($26.44 per 1M) (estimate)
Strands Decider 2B: 22.4% at $0.028 per 1,000 questions ($27.67 per 1M) (estimate)
Chinese Decision Pointer 4B: 40.4% at $0.030 per 1,000 questions ($29.51 per 1M) (estimate)
Adjudex 9B v5: 46.4% at $0.031 per 1,000 questions ($31.36 per 1M) (estimate)
GLiDE 27B: 58.4% at $0.033 per 1,000 questions ($33.16 per 1M) (estimate)
Verdin-4B-v1.1: 39.7% at $0.033 per 1,000 questions ($33.20 per 1M) (estimate)
Dexy-Decision-9B-V2: 45.3% at $0.034 per 1,000 questions ($33.81 per 1M) (estimate)
KnowLine-9B-Gen1: 44.3% at $0.036 per 1,000 questions ($36.27 per 1M) (estimate)
Gemini Decision 1 (20261111): 50.0% at $0.037 per 1,000 questions ($36.77 per 1M)
Mammoth-1: 54.5% at $0.038 per 1,000 questions ($38.39 per 1M)
Jeffan-9B-1211: 48.2% at $0.039 per 1,000 questions ($39.35 per 1M) (estimate)
Cygnet: 39.3% at $0.040 per 1,000 questions ($39.71 per 1M)
KnowLine-9B-Gen2: 43.9% at $0.040 per 1,000 questions ($39.96 per 1M) (estimate)
Valen-Decision-1.5-9B: 44.4% at $0.042 per 1,000 questions ($42.42 per 1M) (estimate)
KnowLine-12B-Gen1: 50.6% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
Winnow-12B: 39.7% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
GLiDE (API): 54.5% at $0.052 per 1,000 questions ($52.03 per 1M)
d3: 54.5% at $0.057 per 1,000 questions ($57.18 per 1M) (estimate)
KnowLine-31B-Gen1.5: 59.0% at $0.061 per 1,000 questions ($61.43 per 1M) (estimate)
Sieve 31B v3: 57.9% at $0.063 per 1,000 questions ($62.95 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 54.8% at $0.081 per 1,000 questions ($81.16 per 1M) (estimate)
OneJev-27B: 40.6% at $0.086 per 1,000 questions ($86.07 per 1M) (estimate)
Metask-Jev-4B: 32.1% at $0.091 per 1,000 questions ($90.99 per 1M) (estimate)
Clef: 40.8% at $0.106 per 1,000 questions ($106 per 1M)
d3-ultra: 54.6% at $0.109 per 1,000 questions ($109 per 1M) (estimate)
KnowLine-31B-Gen2: 58.9% at $0.112 per 1,000 questions ($112 per 1M) (estimate)
Karpo Decision 31B: 60.7% at $0.128 per 1,000 questions ($128 per 1M) (estimate)
Copernicus-31B: 58.7% at $0.136 per 1,000 questions ($136 per 1M) (estimate)
REAX 1.3: 59.3% at $0.160 per 1,000 questions ($160 per 1M) (estimate)
Logits Sigma-1 Duo: 57.8% at $0.211 per 1,000 questions ($211 per 1M) (estimate)
Open-Jev-27B-v1.1: 42.6% at $0.294 per 1,000 questions ($294 per 1M) (estimate)
Gemma 4 E2B (on · chat): 74.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
Erk-14B (off · chat): 54.2% at $0.390 per 1,000 questions ($390 per 1M) (estimate)
LFM3-1.7B (off · chat): 85.5% at $0.487 per 1,000 questions ($487 per 1M) (estimate)
Gemma 4 E2B (chat): 72.9% at $0.513 per 1,000 questions ($513 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 76.8% at $0.574 per 1,000 questions ($574 per 1M) (estimate)
Gemma 4 E4B (on · chat): 82.6% at $0.650 per 1,000 questions ($650 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 94.0% at $0.803 per 1,000 questions ($803 per 1M)
Gemma 4 E4B (chat): 83.2% at $0.842 per 1,000 questions ($842 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 85.9% at $0.993 per 1,000 questions ($993 per 1M) (estimate)
LFM3-1.7B (on · chat): 84.2% at $1.017 per 1,000 questions ($1,017 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 52.5% at $1.030 per 1,000 questions ($1,030 per 1M)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 84.6% at $1.040 per 1,000 questions ($1,040 per 1M) (estimate)
GLM-5.3-Flash (high · chat): 93.2% at $1.057 per 1,000 questions ($1,057 per 1M)
Gemma 4 E2B (on · chat · 32k cap): 75.9% at $1.193 per 1,000 questions ($1,193 per 1M) (estimate)
Gemma 4 31B (chat): 93.3% at $1.244 per 1,000 questions ($1,244 per 1M) (estimate)
Gemma 4 12B Unified (chat): 87.7% at $1.292 per 1,000 questions ($1,292 per 1M) (estimate)
MiniCPM5-2B (off · chat): 83.6% at $1.327 per 1,000 questions ($1,327 per 1M) (estimate)
Qwen3.8 Flash (off · chat): 94.0% at $1.376 per 1,000 questions ($1,376 per 1M)
MiMo-V2.6-Flash (on · chat): 95.1% at $1.390 per 1,000 questions ($1,390 per 1M)
Qwen3.5-4B (off · chat): 90.1% at $1.420 per 1,000 questions ($1,420 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 85.0% at $1.503 per 1,000 questions ($1,503 per 1M) (estimate)
Qwen3.5-2B (off · chat): 64.8% at $1.605 per 1,000 questions ($1,605 per 1M) (estimate)
DeepSeek-V4.1-Flash (high · chat · 1017): 96.2% at $1.647 per 1,000 questions ($1,647 per 1M)
Qwen3-14B (off · chat): 65.3% at $1.772 per 1,000 questions ($1,772 per 1M) (estimate)
Muse Ember (max · chat): 95.5% at $1.820 per 1,000 questions ($1,820 per 1M)
LFM2.5-2.6B (on · chat): 75.4% at $2.480 per 1,000 questions ($2,480 per 1M) (estimate)
MiniCPM5-2B (on · chat): 88.9% at $3.335 per 1,000 questions ($3,335 per 1M) (estimate)
Qwen3.5-4B (on · chat): 92.4% at $3.525 per 1,000 questions ($3,525 per 1M) (estimate)
Inkling 2 Small (high · chat): 95.6% at $4.121 per 1,000 questions ($4,121 per 1M)
Qwen3.5-2B (on · chat): 70.3% at $4.330 per 1,000 questions ($4,330 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 60.2% at $4.664 per 1,000 questions ($4,664 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Qwen3.8-27B (low · chat): 94.0% at $4.884 per 1,000 questions ($4,884 per 1M) (estimate)
Inkling-Small (high · chat · Jul 2026): 95.4% at $5.489 per 1,000 questions ($5,489 per 1M)
Qwen4-Max (high · chat): 96.3% at $5.690 per 1,000 questions ($5,690 per 1M)
LFM2.5-24B-A2B (on · chat): 86.7% at $7.562 per 1,000 questions ($7,562 per 1M) (estimate)
Raon-1 nano (off · chat): 65.1% at $9.270 per 1,000 questions ($9,270 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Granite-4.2-3B (high · chat): 91.6% at $10.893 per 1,000 questions ($10,893 per 1M) (estimate)
Claude Sonnet 5.5 (chat): 95.5% at $11.480 per 1,000 questions ($11,480 per 1M)
Qwen3.5-397B-A17B (on · chat): 95.1% at $13.426 per 1,000 questions ($13,426 per 1M)
Raon-1 nano (on · chat): 70.9% at $15.330 per 1,000 questions ($15,330 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Claude Opus 5.5 (chat): 96.2% at $19.540 per 1,000 questions ($19,540 per 1M)
Fastino OmniJev Other Celeris StartLux PelaAI Perplexity Liquid AI Inception Apertix AI Microsoft Base Labs Subquadratic Cloudflare Nace.AI Turion.AI TypeSafe scar-ai Strands Agents quincyqiang newsbang victorbennett Patrick-18 Google tattrongvu Blockbrain Labs Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab MachinePulse Poindexter Labs REAX Logits Zefan Cai eCloud Technology MiniMax Z.ai OpenBMB Alibaba Xiaomi DeepSeek Meta Thinking Machines KRAFTON IBM Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | Decision | $0.001$0.95estimate · DGX Spark | $0.95 estimate | $0.001064 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | Decision | $0.001$1.16estimate · DGX Spark | $1.16 estimate | $0.001301 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Open · 0.4B | — | Decision | $0.001$1.31estimate · DGX Spark | $1.31 estimate | $0.001478 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1.1-decision Pareto Closed · Efficient | — | Decision | $0.002$1.54 | $1.54 | $0.001733 | — | provider price × reported usage |
| GLiDE 0.8B Open · 0.8B | — | Decision | $0.002$1.58estimate · DGX Spark | $1.58 estimate | $0.001773 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | Decision | $0.002$1.79estimate · DGX Spark | $1.79 estimate | $0.00201 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | Decision | $0.002$2.00estimate · DGX Spark | $2.00 estimate | $0.002246 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Open · 1.9B | — | Decision | $0.002$2.26estimate · DGX Spark | $2.26 estimate | $0.002542 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | Decision | $0.003$2.52estimate · DGX Spark | $2.52 estimate | $0.002838 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Open · 4.0B | — | Decision | $0.004$3.76 list | $3.76 | $0.004229 | $0.034$34.43estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | Decision | $0.004$3.78estimate · DGX Spark | $3.78 estimate | $0.004256 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | — | Decision | $0.005$4.62estimate · DGX Spark | $4.62 estimate | $0.005202 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | Decision | $0.006$5.89estimate · DGX Spark | $5.89 estimate | $0.006621 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1.2 Pareto Closed · Undisclosed | — | Decision | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | Decision | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1 Closed · Efficient | — | Decision | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | Decision | $0.007$7.46 | $7.46 | $0.00839 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | Decision | $0.008$7.52 | $7.52 | $0.008459 | — | provider price × reported usage |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | Decision | $0.010$10.04estimate · DGX Spark | $10.04 estimate | $0.01129 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-ultra Pareto Closed · Efficient | — | Decision | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
| d1.2-ultra Closed · Undisclosed | — | Decision | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
| GLiDE 9B Open · 7.9B | — | Decision | $0.013$13.29estimate · DGX Spark | $13.29 estimate | $0.01496 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cerno 9B v1.2 Open · 9.0B | — | Decision | $0.013$13.35estimate · DGX Spark | $13.35 estimate | $0.01502 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Microsoft-Decision-1 Closed · Efficient | — | Decision | $0.013$13.42 | $13.42 | $0.0151 | — | provider price × reported usage |
| Elephant-1 Open · 9.0B | — | Decision | $0.013$13.44 list | $13.44 | $0.01512 | $0.058$58.41estimate · RTX PRO 6000 Blackwell | maker's list price |
| GLiDE Pareto v1.2 Open · 27B | — | Decision | $0.014$13.88 list | $13.88 | $0.01561 | $0.082$81.77estimate · RTX PRO 6000 Blackwell | maker's list price |
| d1-2B Open · 1.9B | — | Decision | $0.015$14.76estimate · RTX PRO 6000 Blackwell | $14.76 estimate | $0.0166 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| SubQ Snap 1.1 Closed · Efficient | — | Decision | $0.015$14.92 | $14.92 | $0.01678 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | Decision | $0.015$15.40 | $15.40 | $0.01733 | — | provider price × reported usage |
| Clef-Flash Open · 9.0B | — | Decision | $0.017$16.83 list | $16.83 | $0.01894 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | Decision | $0.018$18.41 | $18.41 | $0.02072 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | Decision | $0.022$22.37 | $22.37 | $0.02517 | — | provider price × reported usage |
| Drex Max Open · 27B | — | Decision | $0.023$22.55 list | $22.55 | $0.02537 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | Decision | $0.025$24.91estimate · DGX Spark | $24.91 estimate | $0.02802 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Jev 1.14 Closed · Efficient | — | Decision | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | Decision | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| vera-medium-omni Open · 4.2B | — | Decision | $0.026$25.82estimate · RTX PRO 6000 Blackwell | $25.82 estimate | $0.02905 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | Decision | $0.026$26.44estimate · RTX PRO 6000 Blackwell | $26.44 estimate | $0.02974 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | Decision | $0.028$27.67estimate · RTX PRO 6000 Blackwell | $27.67 estimate | $0.03113 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | Decision | $0.030$29.51estimate · RTX PRO 6000 Blackwell | $29.51 estimate | $0.0332 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Adjudex 9B v5 Open · 9.0B | — | Decision | $0.031$31.36estimate · RTX PRO 6000 Blackwell | $31.36 estimate | $0.03528 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE 27B Pareto Open · 27B | — | Decision | $0.033$33.16estimate · DGX Spark | $33.16 estimate | $0.0373 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Verdin-4B-v1.1 Open · 4.0B | — | Decision | $0.033$33.20estimate · RTX PRO 6000 Blackwell | $33.20 estimate | $0.03735 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | Decision | $0.034$33.81estimate · RTX PRO 6000 Blackwell | $33.81 estimate | $0.03804 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | Decision | $0.036$36.27estimate · RTX PRO 6000 Blackwell | $36.27 estimate | $0.04081 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemini Decision 1 20261111 Closed · Efficient | — | Decision | $0.037$36.77 | $36.77 | $0.04137 | — | provider price × reported usage |
| Mammoth-1 Open · 27B | — | Decision | $0.038$38.39 list | $38.39 | $0.04319 | $0.151$151estimate · RTX PRO 6000 Blackwell | maker's list price |
| Jeffan-9B-1211 Open · 9.0B | — | Decision | $0.039$39.35estimate · RTX PRO 6000 Blackwell | $39.35 estimate | $0.04427 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | Decision | $0.040$39.71 list | $39.71 | $0.04468 | $0.031$30.74estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen2 Open · 9.0B | — | Decision | $0.040$39.96estimate · RTX PRO 6000 Blackwell | $39.96 estimate | $0.04496 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | Decision | $0.042$42.42estimate · RTX PRO 6000 Blackwell | $42.42 estimate | $0.04773 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Open · 12B | — | Decision | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | Decision | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE API Closed · Efficient | — | Decision | $0.052$52.03 | $52.03 | $0.05854 | — | provider price × reported usage |
| d3 Open · 26B | — | Decision | $0.057$57.18estimate · RTX PRO 6000 Blackwell | $57.18 estimate | $0.06433 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | Decision | $0.061$61.43estimate · DGX Spark | $61.43 estimate | $0.06911 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Sieve 31B v3 Open · 31B | — | Decision | $0.063$62.95estimate · DGX Spark | $62.95 estimate | $0.07082 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | Decision | $0.081$81.16estimate · RTX PRO 6000 Blackwell | $81.16 estimate | $0.0913 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| OneJev-27B Open · 27B | — | Decision | $0.086$86.07estimate · RTX PRO 6000 Blackwell | $86.07 estimate | $0.09683 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | Decision | $0.091$90.99estimate · RTX PRO 6000 Blackwell | $90.99 estimate | $0.1024 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | Decision | $0.106$106 list | $106.31 | $0.1196 | $0.141$141estimate · RTX PRO 6000 Blackwell | maker's list price |
| d3-ultra Open · 26B | — | Decision | $0.109$109estimate · RTX PRO 6000 Blackwell | $109.44 estimate | $0.1231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen2 Open · 31B | — | Decision | $0.112$112estimate · RTX PRO 6000 Blackwell | $111.90 estimate | $0.1259 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | Decision | $0.128$128estimate · RTX PRO 6000 Blackwell | $128.50 estimate | $0.1446 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | Decision | $0.136$136estimate · RTX PRO 6000 Blackwell | $135.87 estimate | $0.1529 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Open · 57B | — | Decision | $0.160$160estimate · RTX PRO 6000 Blackwell | $160.47 estimate | $0.1805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | Decision | $0.211$211estimate · RTX PRO 6000 Blackwell | $211.50 estimate | $0.2379 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | Decision | $0.294$294estimate · RTX PRO 6000 Blackwell | $294.50 estimate | $0.3313 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B Pareto on · chat Open · 5.1B | — | Chat | $0.363$363estimate · DGX Spark | $363.26 estimate | $0.4087 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | Chat | $0.390$390estimate · RTX PRO 6000 Blackwell | $389.79 estimate | $0.4385 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | Chat | $0.487$487estimate · DGX Spark | $486.75 estimate | $0.5476 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B chat Open · 5.1B | — | Chat | $0.513$513estimate · RTX PRO 6000 Blackwell | $513.37 estimate | $0.5775 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | Chat | $0.574$574estimate · RTX PRO 6000 Blackwell | $574.03 estimate | $0.6426 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat Open · 8.0B | — | Chat | $0.650$650estimate · DGX Spark | $649.54 estimate | $0.7307 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | Chat | $0.803$803 | $803.23 | $0.9036 | — | provider price × reported usage |
| Gemma 4 E4B chat Open · 8.0B | — | Chat | $0.842$842estimate · RTX PRO 6000 Blackwell | $842.30 estimate | $0.9476 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B | — | Chat | $0.993$993estimate · RTX PRO 6000 Blackwell | $992.52 estimate | $1.091 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B on · chat Open · 1.7B | — | Chat | $1.017$1,017estimate · RTX PRO 6000 Blackwell | $1,017.31 estimate | $1.127 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | Decision | $1.030$1,030 | $1,029.58 | $1.158 | — | provider price × reported usage |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | Chat | $1.040$1,040estimate · RTX PRO 6000 Blackwell | $1,039.65 estimate | $1.181 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active | — | Chat | $1.057$1,057 | $1,056.60 | $1.189 | — | provider price × reported usage |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | Chat | $1.193$1,193estimate · RTX PRO 6000 Blackwell | $1,192.74 estimate | $1.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B chat Open · 31B | — | Chat | $1.244$1,244estimate · RTX PRO 6000 Blackwell | $1,243.77 estimate | $1.399 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified chat Open · 12B | — | Chat | $1.292$1,292estimate · RTX PRO 6000 Blackwell | $1,291.73 estimate | $1.453 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | Chat | $1.327$1,327estimate · RTX PRO 6000 Blackwell | $1,326.98 estimate | $1.594 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8 Flash off · chat Closed · Undisclosed | — | Chat | $1.376$1,376 | $1,376.26 | $1.548 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | Chat | $1.390$1,390 | $1,390.14 | $1.564 | — | provider price × reported usage |
| Qwen3.5-4B off · chat Open · 4.7B | — | Chat | $1.420$1,420estimate · RTX PRO 6000 Blackwell | $1,420.22 estimate | $1.598 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | Chat | $1.503$1,503estimate · RTX PRO 6000 Blackwell | $1,502.61 estimate | $1.69 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | Chat | $1.605$1,605estimate · RTX PRO 6000 Blackwell | $1,604.67 estimate | $1.805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | Chat | $1.647$1,647 | $1,646.92 | $1.853 | — | provider price × reported usage |
| Qwen3-14B off · chat Open · 15B | — | Chat | $1.772$1,772estimate · RTX PRO 6000 Blackwell | $1,771.90 estimate | $1.993 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Muse Ember max · chat Closed · Efficient | — | Chat | $1.820$1,820 | $1,820.45 | $2.048 | — | provider price × reported usage |
| LFM2.5-2.6B on · chat Open · 2.7B | — | Chat | $2.480$2,480estimate · RTX PRO 6000 Blackwell | $2,479.55 estimate | $2.789 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B on · chat Open · 2.5B | — | Chat | $3.335$3,335estimate · RTX PRO 6000 Blackwell | $3,334.55 estimate | $3.828 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B on · chat Open · 4.7B | — | Chat | $3.525$3,525estimate · RTX PRO 6000 Blackwell | $3,524.73 estimate | $3.965 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | Chat | $4.121$4,121 | $4,121.38 | $4.637 | — | provider price × reported usage |
| Qwen3.5-2B on · chat Open · 2.0B | — | Chat | $4.330$4,330estimate · RTX PRO 6000 Blackwell | $4,330.14 estimate | $4.871 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | Chat | $4.664$4,664estimate · RTX PRO 6000 Blackwell† | $4,663.99 estimate† | $5.247 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B low · chat Open · 27B | — | Chat | $4.884$4,884estimate · RTX PRO 6000 Blackwell | $4,884.09 estimate | $5.495 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | Chat | $5.489$5,489 | $5,489.38 | $6.176 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | Chat | $5.690$5,690 | $5,689.67 | $6.401 | — | provider price × reported usage |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | Chat | $7.562$7,562estimate · RTX PRO 6000 Blackwell | $7,561.61 estimate | $8.507 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | Chat | $9.270$9,270estimate · RTX PRO 6000 Blackwell† | $9,270.18 estimate† | $10.43 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | Chat | $10.893$10,893estimate · RTX PRO 6000 Blackwell | $10,892.67 estimate | $12.25 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Sonnet 5.5 chat Closed · Workhorse | — | Chat | $11.480$11,480 API list-price equivalent | $11,479.90 | $12.91 | — | Claude subscription |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active | — | Chat | $13.426$13,426 | $13,425.63 | $15.1 | — | provider price × reported usage |
| Raon-1 nano on · chat Open · 3.1B | — | Chat | $15.330$15,330estimate · RTX PRO 6000 Blackwell† | $15,330.41 estimate† | $17.25 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Opus 5.5 chat Closed · Flagship | — | Chat | $19.540$19,540 API list-price equivalent | $19,540.15 | $21.98 | — | Claude subscription |
| GPT-6 Astra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Hydrogen chat | — | Chat | — | — | — | — | |
| Claude Fable 5.1 chat Closed · Flagship | — | Chat | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6 Luna chat Closed · Efficient | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 1,125 of 1,125); shown once.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Not ranked (only complete runs are): Spark-X2.6-4B (high · chat) (not running now, 20 of 1,125 stored, last answer 13 Nov 02:18 UTC); Grok 4.7 (chat) (paused for now at 576 of 1,125 (Ümit #8603, zeki #8637, 2026-11-15T11:37Z); cost so far $80.830$80,830 per 1,000 questions API list-price equivalent, over the 576 of 1,125 answered); Claude Fable 5.2 (chat) (not running now, 147 of 1,125 stored, last answer 12 Nov 11:32 UTC); Granite-4.2-3B (off · chat) (still running, 259 of 1,125); GPT-6 Luna (max · chat) (paused for now at 55 of 1,125 (zeki #4129, 2026-11-10T14:13Z)); DeepSeek-V4-Flash (chat) (sample: a random subset by design, 317 answers stored, not the full 1,125); Claude Haiku 5.5 (max · chat) (paused for now at 47 of 1,125 (zeki #4129, 2026-11-10T14:13Z)).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 96.7%95.5%–97.6% | a | 1,088 / 1,125 | Not measured | Not measured | 56.4%answered 97% | 6,590 ms | 14,330 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Math 99.6% | — | Chat | 96.7%95.5%–97.6% | ab | 1,088 / 1,125 | Not measured | Not measured | 45.5%answered 98% | 44,520 ms | 206,660 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 96.5%95.3%–97.5% | a | 1,086 / 1,125 | Not measured | Not measured | 63.2%answered 97% | 7,070 ms | 16,200 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $5.690$5,690 per 1,000 questionsper 1M questions Math 99.8% | — | Chat | 96.3%95.0%–97.2% | abc | 1,083 / 1,125 | Not measured | Not measured | 0.0%answered 100% | 11,570 ms | 38,640 ms | $5.690$5,690 |
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $19.540$19,540 API list-price equivalent per 1,000 questionsper 1M questions Math 99.8% | — | Chat | 96.2%94.9%–97.2% | abcde | 1,082 / 1,125 | Not measured | Not measured | 38.2%answered 98% | 5,830 ms | 12,240 ms | $19.540$19,540 API list-price equivalent |
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $1.647$1,647 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 96.2%94.9%–97.2% | abcde | 1,082 / 1,125 | Not measured | Not measured | 3.8%answered 100% | 6,680 ms | 40,040 ms | $1.647$1,647 |
| Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Math 99.6% | — | Chat | 96.1%94.8%–97.1% | abdf | 1,081 / 1,125 | Not measured | Not measured | 76.1%answered 95% | 8,210 ms | 24,090 ms | — |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $4.121$4,121 per 1,000 questionsper 1M questions Math 99.5% | — | Chat | 95.6%94.3%–96.7% | abcdefg | 1,076 / 1,125 | Not measured | Not measured | 37.3%answered 98% | 13,180 ms | 144,250 ms | $4.121$4,121 |
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $11.480$11,480 API list-price equivalent per 1,000 questionsper 1M questions Math 99.7% | — | Chat | 95.5%94.1%–96.5% | abcdefg | 1,074 / 1,125 | Not measured | Not measured | 41.3%answered 98% | 6,710 ms | 14,890 ms | $11.480$11,480 API list-price equivalent |
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $1.820$1,820 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 95.5%94.1%–96.5% | abcdefg | 1,074 / 1,125 | Not measured | Not measured | 22.6%answered 99% | 3,110 ms | 14,020 ms | $1.820$1,820 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $5.489$5,489 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 95.4%94.0%–96.5% | abcdefg | 1,073 / 1,125 | Not measured | Not measured | 50.0%answered 97% | 13,290 ms | 78,510 ms | $5.489$5,489 |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.7% | — | Chat | 95.2%93.8%–96.3% | abcdefgh | 1,071 / 1,125 | Not measured | Not measured | 8.3%answered 100% | 3,820 ms | 13,460 ms | $0.000$0.00 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $1.390$1,390 per 1,000 questionsper 1M questions Math 99.4% | — | Chat | 95.1%93.7%–96.2% | abcdefgh | 1,070 / 1,125 | Not measured | Not measured | 23.3%answered 98% | 24,740 ms | 179,930 ms | $1.390$1,390 |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active Qwen3.5-397B-A17B on · chat · Alibaba · chat model · reasoning on Parameters: 397B MoE · 17B active Cost $13.426$13,426 per 1,000 questionsper 1M questions Math running | — | Chat | 95.1%93.7%–96.2% | abcdefgh | 1,070 / 1,125 | Not measured | Not measured | 31.7%answered 98% | 84,650 ms | 167,520 ms | $13.426$13,426 |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $4.884$4,884estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 94.0%92.5%–95.3% | bcdefghi | 1,058 / 1,125 | Not measured | Not measured | 39.1%answered 97% | 18,710 ms | 101,030 ms | $4.884$4,884estimate · RTX PRO 6000 Blackwell |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 94.0%92.4%–95.2% | ceghi | 1,057 / 1,125 | Not measured | Not measured | 61.1%answered 95% | 5,640 ms | 10,330 ms | $0.000$0.00 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.803$803 per 1,000 questionsper 1M questions Math 98.2% | — | Chat | 94.0%92.4%–95.2% | bcdefghi | 1,057 / 1,125 | Not measured | Not measured | 38.3%answered 97% | 5,610 ms | 37,430 ms | $0.803$803 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $1.376$1,376 per 1,000 questionsper 1M questions Math 99.5% | — | Chat | 94.0%92.4%–95.2% | defghi | 1,057 / 1,125 | Not measured | Not measured | 34.6%answered 98% | 17,550 ms | 73,220 ms | $1.376$1,376 |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $1.244$1,244estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | Chat | 93.3%91.7%–94.6% | fghij | 1,050 / 1,125 | Not measured | Not measured | 16.3%answered 99% | 18,290 ms | 39,790 ms | $1.244$1,244estimate · RTX PRO 6000 Blackwell |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $1.057$1,057 per 1,000 questionsper 1M questions Math 98.8% | — | Chat | 93.2%91.5%–94.5% | ghij | 1,048 / 1,125 | Not measured | Not measured | 4.2%answered 100% | 12,050 ms | 82,830 ms | $1.057$1,057 |
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $3.525$3,525estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | Chat | 92.4%90.7%–93.8% | hij | 1,039 / 1,125 | Not measured | Not measured | 36.1%answered 97% | 142,520 ms | 246,280 ms | $3.525$3,525estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $10.893$10,893estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 91.6%89.9%–93.1% | ijk | 1,031 / 1,125 | Not measured | Not measured | 53.2%answered 94% | 221,350 ms | 1,229,550 ms | $10.893$10,893estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $1.420$1,420estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | Chat | 90.1%88.3%–91.7% | jkl | 1,014 / 1,125 | Not measured | Not measured | 33.6%answered 96% | 45,380 ms | 127,510 ms | $1.420$1,420estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $3.335$3,335estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 88.9%88.3%–89.6% min–max of 3 seeds | klm | 998 / 1,125seed 0 run | Not measured | Not measured | 51.5%answered 92% | 101,100 ms | 333,427 ms | $3.335$3,335estimate · RTX PRO 6000 Blackwell |
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $1.292$1,292estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.8% | — | Chat | 87.7%85.7%–89.5% | klmn | 987 / 1,125 | Not measured | Not measured | 15.5%answered 98% | 42,060 ms | 82,940 ms | $1.292$1,292estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $7.562$7,562estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.0% | — | Chat | 86.7%84.6%–88.5% | lmno | 975 / 1,125 | Not measured | Not measured | 37.8%answered 95% | 228,980 ms | 776,930 ms | $7.562$7,562estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.993$993estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | Chat | 85.9%85.5%–86.1% min–max of 3 seeds | lmno | 968 / 1,125seed 0 run | Not measured | Not measured | 55.9%answered 90% | 31,567 ms | 83,467 ms | $0.993$993estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.487$487estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.7% | — | Chat | 85.5%83.3%–87.4% | lmno | 962 / 1,125 | Not measured | Not measured | 54.2%answered 90% | 163,870 ms | 656,750 ms | $0.487$487estimate · DGX Spark |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $1.503$1,503estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | Chat | 85.0%82.8%–86.9% | mno | 956 / 1,125 | Not measured | Not measured | Not measured | 57,900 ms | 109,440 ms | $1.503$1,503estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $1.040$1,040estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | Chat | 84.6%84.4%–85.0% min–max of 3 seeds | mno | 956 / 1,125seed 0 run | Not measured | Not measured | Not measured | 27,950 ms | 107,497 ms | $1.040$1,040estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $1.017$1,017estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | Chat | 84.2%83.9%–84.4% min–max of 3 seeds | mno | 950 / 1,125seed 0 run | Not measured | Not measured | 43.6%answered 91% | 29,847 ms | 113,220 ms | $1.017$1,017estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $1.327$1,327estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.2% | — | Chat | 83.6%83.1%–83.9% min–max of 3 seeds | no | 942 / 1,125seed 0 run | Not measured | Not measured | 43.9%answered 88% | 46,453 ms | 111,607 ms | $1.327$1,327estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.842$842estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.1% | — | Chat | 83.2%80.9%–85.3% | no | 936 / 1,125 | Not measured | Not measured | 52.4%answered 88% | 32,690 ms | 64,070 ms | $0.842$842estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.650$650estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.2% | — | Chat | 82.6%80.3%–84.7% | o | 929 / 1,125 | Not measured | Not measured | 45.1%answered 91% | 284,080 ms | 539,980 ms | $0.650$650estimate · DGX Spark |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.574$574estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.7% | — | Chat | 76.8%76.0%–77.6% min–max of 3 seeds | p | 855 / 1,125seed 0 run | Not measured | Not measured | 32.6%answered 91% | 21,223 ms | 45,077 ms | $0.574$574estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $1.193$1,193estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.8% | — | Chat | 75.9%73.3%–78.3% | p | 854 / 1,125 | Not measured | Not measured | Not measured | 41,350 ms | 70,680 ms | $1.193$1,193estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $2.480$2,480estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.0% | — | Chat | 75.4%72.8%–77.8% | p | 848 / 1,125 | Not measured | Not measured | 44.3%answered 86% | 81,330 ms | 224,000 ms | $2.480$2,480estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.363$363estimate · DGX Spark per 1,000 questionsper 1M questions Math 96.9% | — | Chat | 74.6%72.0%–77.0% | p | 839 / 1,125 | Not measured | Not measured | 26.4%answered 93% | 173,220 ms | 297,090 ms | $0.363$363estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.513$513estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 96.8% | — | Chat | 72.9%70.2%–75.4% | p | 820 / 1,125 | Not measured | Not measured | 34.0%answered 90% | 21,390 ms | 35,160 ms | $0.513$513estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $15.330$15,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 97.0% | — | Chat | 70.9%68.2%–73.5% | pq | 798 / 1,125 | Not measured | Not measured | 10.5%answered 95% | 493,230 ms† | 1,396,520 ms† | $15.330$15,330estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $4.330$4,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.7% | — | Chat | 70.3%67.6%–72.9% | pq | 791 / 1,125 | Not measured | Not measured | 33.1%answered 91% | 186,330 ms | 272,210 ms | $4.330$4,330estimate · RTX PRO 6000 Blackwell |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $1.772$1,772estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 95.8% | — | Chat | 65.3%62.5%–68.1% | qr | 735 / 1,125 | Not measured | Not measured | 16.1%answered 94% | 16,560 ms | 178,680 ms | $1.772$1,772estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $9.270$9,270estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 89.6% | — | Chat | 65.1%62.2%–67.8% | qrs | 732 / 1,125 | Not measured | Not measured | 35.7%answered 87% | 276,720 ms† | 965,630 ms† | $9.270$9,270estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $1.605$1,605estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.6% | — | Chat | 64.8%62.0%–67.5% | qrs | 729 / 1,125 | Not measured | Not measured | 35.2%answered 86% | 71,290 ms | 113,470 ms | $1.605$1,605estimate · RTX PRO 6000 Blackwell |
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.128$128estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 81.6% | — | Decision | 60.7%57.8%–63.5% | rstu | 683 / 1,125 | 169 right of 172 kept3 wrong · 953 skipped | 0.816 | 6.6%answered 97% | 173 ms | 211 ms | $0.128$128estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $4.664$4,664estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 96.3% | — | Chat | 60.2%57.3%–63.0% | rstuvwxyzA | 677 / 1,125 | Not measured | Not measured | 58.3%answered 70% | 138,140 ms† | 403,380 ms† | $4.664$4,664estimate · RTX PRO 6000 Blackwell† |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.160$160estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 78.0% | — | Decision | 59.3%56.4%–62.1% | rstv | 667 / 1,125 | 51 right of 51 kept0 mistakes · 1,074 skipped | 0.816 | 5.3%answered 98% | 215 ms | 278 ms | $0.160$160estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.061$61.43estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.4% | — | Decision | 59.0%56.1%–61.9% | rstuvw | 664 / 1,125 | 89 right of 89 kept0 mistakes · 1,036 skipped | 0.804 | 0.4%answered 100% | 983 ms | 1,279 ms | $0.061$61.43estimate · DGX Spark |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.112$112estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 80.3% | — | Decision | 58.9%56.0%–61.8% | rstuvw | 663 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.825 | 0.0%answered 100% | 151 ms | 204 ms | $0.112$112estimate · RTX PRO 6000 Blackwell |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.136$136estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 77.3% | — | Decision | 58.7%55.8%–61.5% | rstuvwx | 660 / 1,125 | 66 right of 66 kept0 mistakes · 1,059 skipped | 0.801 | 1.9%answered 99% | 185 ms | 255 ms | $0.136$136estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.033$33.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 72.0% | — | Decision | 58.4%55.5%–61.2% | rstuvwxy | 657 / 1,125 | 74 right of 75 kept1 wrong · 1,050 skipped | 0.822 | 44.3%answered 79% | 544 ms | 655 ms | $0.033$33.16estimate · DGX Spark |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 93.7% | — | Chat | 58.0%55.0%–60.8% | stuvwxyzAB | 652 / 1,125 | Not measured | Not measured | 55.1%answered 72% | 3,860 ms | 5,820 ms | $0.000$0.00 |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.063$62.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.1% | — | Decision | 57.9%55.0%–60.7% | rstuvwxz | 651 / 1,125 | 79 right of 81 kept2 wrong · 1,044 skipped | 0.797 | 5.5%answered 98% | 1,050 ms | 1,335 ms | $0.063$62.95estimate · DGX Spark |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.211$211estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 75.2% | — | Decision | 57.8%54.9%–60.6% | rstuvwxyzA | 650 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.796 | 7.1%answered 97% | 285 ms | 399 ms | $0.211$211estimate · RTX PRO 6000 Blackwell |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 69.9% | — | Decision | 55.2%52.3%–58.1% | tuvwxyzABC | 621 / 1,125 | 45 right of 45 kept0 mistakes · 1,080 skipped | 0.793 | 50.2%answered 73% | 111 ms | 140 ms | $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 74.1% | — | Decision | 54.9%52.0%–57.8% | tuvwxyzABC | 618 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.783 | 91.0%answered 35% | 182 ms | 425 ms | $0.013$12.64 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.022$22.37 per 1,000 questionsper 1M questions Math 70.1% | — | Decision | 54.8%51.8%–57.6% | tuvwxyzABCD | 616 / 1,125 | 89 right of 90 kept1 wrong · 1,035 skipped | 0.793 | 54.1%answered 69% | 329 ms | 1,540 ms | $0.022$22.37 |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 72.5% | — | Decision | 54.8%51.8%–57.6% | tuvwxyzABCD | 616 / 1,125 | 62 right of 62 kept0 mistakes · 1,063 skipped | 0.803 | 18.1%answered 91% | 108 ms | 131 ms | $0.081$81.16estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.109$109estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 71.5% | — | Decision | 54.6%51.7%–57.5% | uwxyzABCDE | 614 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.778 | 10.1%answered 95% | 156 ms | 185 ms | $0.109$109estimate · RTX PRO 6000 Blackwell |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.052$52.03 per 1,000 questionsper 1M questions Math 70.8% | — | Decision | 54.5%51.6%–57.4% | vwxyzABCD | 613 / 1,125 | 52 right of 52 kept0 mistakes · 1,073 skipped | 0.814 | 60.0%answered 67% | 271 ms | 291 ms | $0.052$52.03 |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 74.0% | — | Decision | 54.5%51.6%–57.4% | vwxyzABCDE | 613 / 1,125 | 106 right of 108 kept2 wrong · 1,017 skipped | 0.795 | 72.3%answered 58% | 221 ms | 244 ms | $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.057$57.18estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.4% | — | Decision | 54.5%51.6%–57.4% | vwxyzABCD | 613 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.806 | 14.8%answered 93% | 76 ms | 98 ms | $0.057$57.18estimate · RTX PRO 6000 Blackwell |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.390$390estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 90.8% | — | Chat | 54.2%51.3%–57.1% | tuvwxyzABCDEFG | 610 / 1,125 | Not measured | Not measured | 30.7%answered 84% | 10,150 ms | 18,410 ms | $0.390$390estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.007$7.46 per 1,000 questionsper 1M questions Math 69.8% | — | Decision | 53.3%50.4%–56.2% | wxyzABCDEF | 600 / 1,125 | 78 right of 80 kept2 wrong · 1,045 skipped | 0.798 | 14.4%answered 93% | 238 ms | 552 ms | $0.007$7.46 |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.3% | — | Decision | 53.1%50.1%–56.0% | wxyzABCDEF | 597 / 1,125 | 147 right of 155 kept8 wrong · 970 skipped | 0.786 | 7.3%answered 97% | 106 ms | 143 ms | $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.030$1,030 per 1,000 questionsper 1M questions Math 69.3% | with web | Decision | 52.5%49.6%–55.4% | xyzABCDEFGH | 591 / 1,125 | 78 right of 79 kept1 wrong · 1,046 skipped | 0.800 | 37.7%answered 80% | 448 ms | 827 ms | $1.030$1,030 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.015$14.92 per 1,000 questionsper 1M questions Math 68.6% | — | Decision | 52.4%49.4%–55.3% | zABCDEFGHI | 589 / 1,125 | 70 right of 72 kept2 wrong · 1,053 skipped | 0.787 | 65.7%answered 61% | 384 ms | 393 ms | $0.015$14.92 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.025$24.91estimate · DGX Spark per 1,000 questionsper 1M questions Math 69.0% | — | Decision | 52.3%49.3%–55.2% | yABCDEFGHI | 588 / 1,125 | 7 right of 7 kept0 mistakes · 1,118 skipped | 0.782 | 1.5%answered 99% | 385 ms | 533 ms | $0.025$24.91estimate · DGX Spark |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 69.9% | — | Decision | 51.6%48.6%–54.5% | BCDEFGHIJ | 580 / 1,125 | 13 right of 13 kept0 mistakes · 1,112 skipped | 0.800 | 94.5%answered 28% | 220 ms | 341 ms | $0.013$12.64 |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 66.3% | — | Decision | 50.6%47.7%–53.5% | BCDEFGHIJK | 569 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.774 | 8.8%answered 95% | 68 ms | 88 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.037$36.77 per 1,000 questionsper 1M questions Math 71.3% | — | Decision | 50.0%47.1%–53.0% | CDEFGHIJKL | 563 / 1,125 | 43 right of 43 kept0 mistakes · 1,082 skipped | 0.768 | 2.0%answered 99% | 228 ms | 344 ms | $0.037$36.77 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.5% | — | Decision | 49.2%46.2%–52.1% | CDEFGHIJKLM | 553 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.742 | 78.8%answered 43% | 222 ms | 390 ms | $0.006$6.02 |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.039$39.35estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | Decision | 48.2%45.3%–51.1% | EFGHIJKLMN | 542 / 1,125 | 10 right of 10 kept0 mistakes · 1,115 skipped | 0.753 | 23.4%answered 87% | 52 ms | 66 ms | $0.039$39.35estimate · RTX PRO 6000 Blackwell |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 65.1% | — | Decision | 48.2%45.3%–51.1% | DEFGHIJKLMN | 542 / 1,125 | 61 right of 61 kept0 mistakes · 1,064 skipped | 0.750 | 24.5%answered 86% | 166 ms | 193 ms | $0.026$25.64 |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.013$13.29estimate · DGX Spark per 1,000 questionsper 1M questions Math 57.9% | — | Decision | 47.8%44.9%–50.7% | FGHIJKLMN | 538 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.749 | 30.4%answered 82% | 208 ms | 251 ms | $0.013$13.29estimate · DGX Spark |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.015$15.40 per 1,000 questionsper 1M questions Math 70.0% | — | Decision | 47.3%44.4%–50.2% | FGHIJKLMNO | 532 / 1,125 | No answers at this threshold | 0.735 | 92.3%answered 28% | 390 ms | 394 ms | $0.015$15.40 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.002$1.54 per 1,000 questionsper 1M questions Math 69.9% | — | Decision | 47.3%44.4%–50.2% | FGHIJKLMNO | 532 / 1,125 | No answers at this threshold | 0.733 | 91.0%answered 29% | 383 ms | 390 ms | $0.002$1.54 |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.010$10.04estimate · DGX Spark per 1,000 questionsper 1M questions Math 58.4% | — | Decision | 46.6%43.7%–49.5% | GHIJKLMNOP | 524 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.762 | 17.4%answered 90% | 160 ms | 175 ms | $0.010$10.04estimate · DGX Spark |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.36estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 62.1% | — | Decision | 46.4%43.5%–49.3% | GHIJKLMNOP | 522 / 1,125 | 54 right of 55 kept1 wrong · 1,070 skipped | 0.771 | 12.2%answered 93% | 41 ms | 46 ms | $0.031$31.36estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 61.3% | — | Decision | 46.4%43.5%–49.3% | GHIJKLMNOP | 522 / 1,125 | 53 right of 53 kept0 mistakes · 1,072 skipped | 0.768 | 75.6%answered 47% | 79 ms | 98 ms | $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 64.1% | — | Decision | 46.3%43.4%–49.2% | HIJKLMNOP | 521 / 1,125 | 65 right of 65 kept0 mistakes · 1,060 skipped | 0.764 | 15.8%answered 91% | 167 ms | 203 ms | $0.026$25.64 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.2% | — | Decision | 46.0%43.2%–49.0% | IJKLMNOPQR | 518 / 1,125 | 48 right of 48 kept0 mistakes · 1,077 skipped | 0.748 | 90.6%answered 28% | 176 ms | 272 ms | $0.006$6.02 |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.005$4.62estimate · DGX Spark per 1,000 questionsper 1M questions Math 55.8% | — | Decision | 45.9%43.0%–48.8% | IJKLMNOPQS | 516 / 1,125 | 21 right of 21 kept0 mistakes · 1,104 skipped | 0.720 | 16.0%answered 91% | 73 ms | 84 ms | $0.005$4.62estimate · DGX Spark |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.034$33.81estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 58.8% | — | Decision | 45.3%42.4%–48.3% | JKLMNOPQRST | 510 / 1,125 | 56 right of 56 kept0 mistakes · 1,069 skipped | 0.759 | 12.4%answered 93% | 47 ms | 53 ms | $0.034$33.81estimate · RTX PRO 6000 Blackwell |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.008$7.52 per 1,000 questionsper 1M questions Math 53.5% | — | Decision | 45.1%42.2%–48.0% | KLMNOPQRST | 507 / 1,125 | 84 right of 91 kept7 wrong · 1,034 skipped | 0.746 | 20.5%answered 88% | 301 ms | 360 ms | $0.008$7.52 |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.026$26.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.9% | — | Decision | 44.7%41.8%–47.6% | KLMNOPQRST | 503 / 1,125 | 17 right of 17 kept0 mistakes · 1,108 skipped | 0.757 | 24.4%answered 85% | 33 ms | 41 ms | $0.026$26.44estimate · RTX PRO 6000 Blackwell |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.013$13.42 per 1,000 questionsper 1M questions Math 59.4% | — | Decision | 44.4%41.6%–47.4% | KLMNOPQRST | 500 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.712 | 19.8%answered 88% | 315 ms | 362 ms | $0.013$13.42 |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.042$42.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.3% | — | Decision | 44.4%41.5%–47.3% | KLMNOPQRST | 499 / 1,125 | 35 right of 37 kept2 wrong · 1,088 skipped | 0.701 | 13.6%answered 92% | 60 ms | 67 ms | $0.042$42.42estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.036$36.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | Decision | 44.3%41.4%–47.2% | KLMNOPQRST | 498 / 1,125 | 30 right of 30 kept0 mistakes · 1,095 skipped | 0.725 | 6.7%answered 96% | 50 ms | 61 ms | $0.036$36.27estimate · RTX PRO 6000 Blackwell |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.026$25.82estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 56.8% | — | Decision | 44.1%41.2%–47.0% | LMNOPQRST | 496 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.739 | 13.6%answered 92% | 32 ms | 37 ms | $0.026$25.82estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.040$39.96estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 60.1% | — | Decision | 43.9%41.0%–46.8% | LMNOPQRST | 494 / 1,125 | No answers at this threshold | 0.730 | 19.5%answered 88% | 52 ms | 65 ms | $0.040$39.96estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.013$13.35estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | Decision | 43.3%40.4%–46.2% | LMNOPQRST | 487 / 1,125 | 26 right of 27 kept1 wrong · 1,098 skipped | 0.752 | 6.0%answered 97% | 209 ms | 269 ms | $0.013$13.35estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.2% | — | Decision | 43.1%40.2%–46.0% | LMNOPQRST | 485 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 28.9%answered 82% | 45 ms | 55 ms | $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.294$294estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 54.6% | — | Decision | 42.6%39.7%–45.5% | MNOPQRST | 479 / 1,125 | No answers at this threshold | 0.733 | 28.2%answered 82% | 413 ms | 576 ms | $0.294$294estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.006$5.89estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | Decision | 42.3%39.5%–45.2% | NOPQRST | 476 / 1,125 | 17 right of 18 kept1 wrong · 1,107 skipped | 0.708 | 8.3%answered 95% | 96 ms | 115 ms | $0.006$5.89estimate · DGX Spark |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.018$18.41 per 1,000 questionsper 1M questions Math 59.0% | — | Decision | 41.9%39.0%–44.8% | NOPQRST | 471 / 1,125 | 33 right of 33 kept0 mistakes · 1,092 skipped | 0.717 | 37.7%answered 75% | 345 ms | 504 ms | $0.018$18.41 |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.1% | — | Decision | 40.8%38.0%–43.7% | OPQRST | 459 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.697 | 3.6%answered 98% | 196 ms | 222 ms | $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 55.6% | — | Decision | 40.8%38.0%–43.7% | OPQRST | 459 / 1,125 | 55 right of 56 kept1 wrong · 1,069 skipped | 0.723 | 11.3%answered 93% | 188 ms | 290 ms | $0.006$6.02 |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.086$86.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 53.8% | — | Decision | 40.6%37.8%–43.5% | OPQRST | 457 / 1,125 | 4 right of 4 kept0 mistakes · 1,121 skipped | 0.726 | 1.2%answered 99% | 108 ms | 136 ms | $0.086$86.07estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.030$29.51estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 50.4% | — | Decision | 40.4%37.5%–43.3% | PQRST | 454 / 1,125 | 39 right of 39 kept0 mistakes · 1,086 skipped | 0.724 | 14.6%answered 91% | 38 ms | 48 ms | $0.030$29.51estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.5% | — | Decision | 39.7%36.9%–42.6% | RT | 447 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 12.6%answered 92% | 45 ms | 51 ms | $0.033$33.20estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 46.9% | — | Decision | 39.7%36.9%–42.6% | QRST | 447 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.727 | 29.7%answered 80% | 66 ms | 91 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 44.3% | — | Decision | 39.3%36.5%–42.2% | STU | 442 / 1,125 | No answers at this threshold | 0.720 | 11.3%answered 93% | 40 ms | 48 ms | $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 49.8% | — | Decision | 38.9%36.1%–41.8% | TU | 438 / 1,125 | No answers at this threshold | 0.659 | 14.4%answered 91% | 110 ms | 137 ms | $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$2.26estimate · DGX Spark per 1,000 questionsper 1M questions Math 36.8% | — | Decision | 32.3%29.6%–35.1% | UV | 363 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.632 | 32.9%answered 75% | 34 ms | 42 ms | $0.002$2.26estimate · DGX Spark |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.091$90.99estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 39.2% | — | Decision | 32.1%29.4%–34.9% | UV | 361 / 1,125 | No answers at this threshold | 0.660 | 1.5%answered 99% | 123 ms | 152 ms | $0.091$90.99estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.004$3.78estimate · DGX Spark per 1,000 questionsper 1M questions Math 35.3% | — | Decision | 31.7%29.1%–34.5% | V | 357 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.601 | 15.7%answered 89% | 59 ms | 70 ms | $0.004$3.78estimate · DGX Spark |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.003$2.52estimate · DGX Spark per 1,000 questionsper 1M questions Math 38.7% | — | Decision | 30.8%28.1%–33.5% | VW | 346 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.640 | 31.4%answered 76% | 40 ms | 45 ms | $0.003$2.52estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.015$14.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 37.1% | — | Decision | 30.7%28.0%–33.4% | VW | 345 / 1,125 | 3 right of 3 kept0 mistakes · 1,122 skipped | 0.620 | 86.5%answered 22% | 17 ms | 23 ms | $0.015$14.76estimate · RTX PRO 6000 Blackwell |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.002$1.58estimate · DGX Spark per 1,000 questionsper 1M questions Math 32.1% | — | Decision | 29.2%26.6%–31.9% | VWX | 328 / 1,125 | 9 right of 9 kept0 mistakes · 1,116 skipped | 0.617 | 26.1%answered 80% | 23 ms | 29 ms | $0.002$1.58estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | Decision | 27.0%24.5%–29.7% | VWX | 304 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.545 | 4.6%answered 97% | 16 ms | 19 ms | $0.001$1.16estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$2.00estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | Decision | 26.7%24.2%–29.3% | VWX | 300 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.573 | 1.2%answered 99% | 31 ms | 39 ms | $0.002$2.00estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.79estimate · DGX Spark per 1,000 questionsper 1M questions Math 29.7% | — | Decision | 24.7%22.3%–27.3% | WX | 278 / 1,125 | No answers at this threshold | 0.641 | 21.4%answered 83% | 27 ms | 31 ms | $0.002$1.79estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.31estimate · DGX Spark per 1,000 questionsper 1M questions Math 22.7% | — | Decision | 23.6%21.2%–26.1% | WX | 265 / 1,125 | No answers at this threshold | 0.527 | 12.4%answered 90% | 15 ms | 28 ms | $0.001$1.31estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 20.6% | — | Decision | 23.2%20.8%–25.8% | X | 261 / 1,125 | No answers at this threshold | 0.556 | 76.1%answered 30% | 13 ms | 16 ms | $0.001$0.95estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.028$27.67estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 28.7% | — | Decision | 22.4%20.1%–24.9% | X | 252 / 1,125 | No answers at this threshold | 0.568 | 8.3%answered 94% | 36 ms | 43 ms | $0.028$27.67estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino OmniJev Other StartLux PelaAI Perplexity Liquid AI scar-ai Strands Agents quincyqiang victorbennett Metask Lab Google OpenBMB Alibaba KRAFTON IBM
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 23.2% at $0.001 per 1,000 questions ($0.95 per 1M) (estimate)
OneJev1.6-0.8B: 27.0% at $0.001 per 1,000 questions ($1.16 per 1M) (estimate)
GLiClass v4 Edge: 23.6% at $0.001 per 1,000 questions ($1.31 per 1M) (estimate)
GLiDE 0.8B: 29.2% at $0.002 per 1,000 questions ($1.58 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 24.7% at $0.002 per 1,000 questions ($1.79 per 1M) (estimate)
KnowLine-0.8B-Gen1: 26.7% at $0.002 per 1,000 questions ($2.00 per 1M) (estimate)
GLiDE 2B: 32.3% at $0.002 per 1,000 questions ($2.26 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 30.8% at $0.003 per 1,000 questions ($2.52 per 1M) (estimate)
Perplexity Decider Flash v1.2: 43.1% at $0.004 per 1,000 questions ($3.76 per 1M)
KnowLine-2B-Gen1: 31.7% at $0.004 per 1,000 questions ($3.78 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 45.9% at $0.005 per 1,000 questions ($4.62 per 1M) (estimate)
KnowLine 4B Gen5.5: 42.3% at $0.006 per 1,000 questions ($5.89 per 1M) (estimate)
d1-2B: 30.7% at $0.015 per 1,000 questions ($14.76 per 1M) (estimate)
vera-medium-omni: 44.1% at $0.026 per 1,000 questions ($25.82 per 1M) (estimate)
Perplexity Decider Flash v1.3: 44.7% at $0.026 per 1,000 questions ($26.44 per 1M) (estimate)
Strands Decider 2B: 22.4% at $0.028 per 1,000 questions ($27.67 per 1M) (estimate)
Chinese Decision Pointer 4B: 40.4% at $0.030 per 1,000 questions ($29.51 per 1M) (estimate)
Verdin-4B-v1.1: 39.7% at $0.033 per 1,000 questions ($33.20 per 1M) (estimate)
Metask-Jev-4B: 32.1% at $0.091 per 1,000 questions ($90.99 per 1M) (estimate)
Gemma 4 E2B (on · chat): 74.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
LFM3-1.7B (off · chat): 85.5% at $0.487 per 1,000 questions ($487 per 1M) (estimate)
Gemma 4 E2B (chat): 72.9% at $0.513 per 1,000 questions ($513 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 76.8% at $0.574 per 1,000 questions ($574 per 1M) (estimate)
LFM3-1.7B (on · chat): 84.2% at $1.017 per 1,000 questions ($1,017 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 84.6% at $1.040 per 1,000 questions ($1,040 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 75.9% at $1.193 per 1,000 questions ($1,193 per 1M) (estimate)
MiniCPM5-2B (off · chat): 83.6% at $1.327 per 1,000 questions ($1,327 per 1M) (estimate)
Qwen3.5-4B (off · chat): 90.1% at $1.420 per 1,000 questions ($1,420 per 1M) (estimate)
Qwen3.5-2B (off · chat): 64.8% at $1.605 per 1,000 questions ($1,605 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 75.4% at $2.480 per 1,000 questions ($2,480 per 1M) (estimate)
MiniCPM5-2B (on · chat): 88.9% at $3.335 per 1,000 questions ($3,335 per 1M) (estimate)
Qwen3.5-4B (on · chat): 92.4% at $3.525 per 1,000 questions ($3,525 per 1M) (estimate)
Qwen3.5-2B (on · chat): 70.3% at $4.330 per 1,000 questions ($4,330 per 1M) (estimate)
Raon-1 nano (off · chat): 65.1% at $9.270 per 1,000 questions ($9,270 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Granite-4.2-3B (high · chat): 91.6% at $10.893 per 1,000 questions ($10,893 per 1M) (estimate)
Raon-1 nano (on · chat): 70.9% at $15.330 per 1,000 questions ($15,330 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Fastino OmniJev Other StartLux PelaAI Perplexity Liquid AI scar-ai Strands Agents quincyqiang victorbennett Metask Lab Google OpenBMB Alibaba KRAFTON IBM
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | Decision | $0.001$0.95estimate · DGX Spark | $0.95 estimate | $0.001064 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | Decision | $0.001$1.16estimate · DGX Spark | $1.16 estimate | $0.001301 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Open · 0.4B | — | Decision | $0.001$1.31estimate · DGX Spark | $1.31 estimate | $0.001478 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Pareto Open · 0.8B | — | Decision | $0.002$1.58estimate · DGX Spark | $1.58 estimate | $0.001773 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | Decision | $0.002$1.79estimate · DGX Spark | $1.79 estimate | $0.00201 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | Decision | $0.002$2.00estimate · DGX Spark | $2.00 estimate | $0.002246 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Pareto Open · 1.9B | — | Decision | $0.002$2.26estimate · DGX Spark | $2.26 estimate | $0.002542 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | Decision | $0.003$2.52estimate · DGX Spark | $2.52 estimate | $0.002838 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Pareto Open · 4.0B | — | Decision | $0.004$3.76 list | $3.76 | $0.004229 | $0.034$34.43estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | Decision | $0.004$3.78estimate · DGX Spark | $3.78 estimate | $0.004256 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Pareto Open · 4.0B | — | Decision | $0.005$4.62estimate · DGX Spark | $4.62 estimate | $0.005202 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | Decision | $0.006$5.89estimate · DGX Spark | $5.89 estimate | $0.006621 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-2B Open · 1.9B | — | Decision | $0.015$14.76estimate · RTX PRO 6000 Blackwell | $14.76 estimate | $0.0166 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Open · 4.2B | — | Decision | $0.026$25.82estimate · RTX PRO 6000 Blackwell | $25.82 estimate | $0.02905 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | Decision | $0.026$26.44estimate · RTX PRO 6000 Blackwell | $26.44 estimate | $0.02974 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | Decision | $0.028$27.67estimate · RTX PRO 6000 Blackwell | $27.67 estimate | $0.03113 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | Decision | $0.030$29.51estimate · RTX PRO 6000 Blackwell | $29.51 estimate | $0.0332 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | Decision | $0.033$33.20estimate · RTX PRO 6000 Blackwell | $33.20 estimate | $0.03735 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | Decision | $0.091$90.99estimate · RTX PRO 6000 Blackwell | $90.99 estimate | $0.1024 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B Pareto on · chat Open · 5.1B | — | Chat | $0.363$363estimate · DGX Spark | $363.26 estimate | $0.4087 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | Chat | $0.487$487estimate · DGX Spark | $486.75 estimate | $0.5476 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B chat Open · 5.1B | — | Chat | $0.513$513estimate · RTX PRO 6000 Blackwell | $513.37 estimate | $0.5775 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | Chat | $0.574$574estimate · RTX PRO 6000 Blackwell | $574.03 estimate | $0.6426 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B on · chat Open · 1.7B | — | Chat | $1.017$1,017estimate · RTX PRO 6000 Blackwell | $1,017.31 estimate | $1.127 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | Chat | $1.040$1,040estimate · RTX PRO 6000 Blackwell | $1,039.65 estimate | $1.181 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | Chat | $1.193$1,193estimate · RTX PRO 6000 Blackwell | $1,192.74 estimate | $1.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | Chat | $1.327$1,327estimate · RTX PRO 6000 Blackwell | $1,326.98 estimate | $1.594 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto off · chat Open · 4.7B | — | Chat | $1.420$1,420estimate · RTX PRO 6000 Blackwell | $1,420.22 estimate | $1.598 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | Chat | $1.605$1,605estimate · RTX PRO 6000 Blackwell | $1,604.67 estimate | $1.805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | Chat | $2.480$2,480estimate · RTX PRO 6000 Blackwell | $2,479.55 estimate | $2.789 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B on · chat Open · 2.5B | — | Chat | $3.335$3,335estimate · RTX PRO 6000 Blackwell | $3,334.55 estimate | $3.828 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto on · chat Open · 4.7B | — | Chat | $3.525$3,525estimate · RTX PRO 6000 Blackwell | $3,524.73 estimate | $3.965 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | Chat | $4.330$4,330estimate · RTX PRO 6000 Blackwell | $4,330.14 estimate | $4.871 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | Chat | $9.270$9,270estimate · RTX PRO 6000 Blackwell† | $9,270.18 estimate† | $10.43 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | Chat | $10.893$10,893estimate · RTX PRO 6000 Blackwell | $10,892.67 estimate | $12.25 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | Chat | $15.330$15,330estimate · RTX PRO 6000 Blackwell† | $15,330.41 estimate† | $17.25 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Spark-X2.6-4B (high · chat) (not running now, 20 of 1,125 stored, last answer 13 Nov 02:18 UTC); Granite-4.2-3B (off · chat) (still running, 259 of 1,125).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $3.525$3,525estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | Chat | 92.4%90.7%–93.8% | a | 1,039 / 1,125 | Not measured | Not measured | 36.1%answered 97% | 142,520 ms | 246,280 ms | $3.525$3,525estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $10.893$10,893estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 91.6%89.9%–93.1% | ab | 1,031 / 1,125 | Not measured | Not measured | 53.2%answered 94% | 221,350 ms | 1,229,550 ms | $10.893$10,893estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $1.420$1,420estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | Chat | 90.1%88.3%–91.7% | ab | 1,014 / 1,125 | Not measured | Not measured | 33.6%answered 96% | 45,380 ms | 127,510 ms | $1.420$1,420estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $3.335$3,335estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 88.9%88.3%–89.6% min–max of 3 seeds | bc | 998 / 1,125seed 0 run | Not measured | Not measured | 51.5%answered 92% | 101,100 ms | 333,427 ms | $3.335$3,335estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.487$487estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.7% | — | Chat | 85.5%83.3%–87.4% | cd | 962 / 1,125 | Not measured | Not measured | 54.2%answered 90% | 163,870 ms | 656,750 ms | $0.487$487estimate · DGX Spark |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $1.040$1,040estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | Chat | 84.6%84.4%–85.0% min–max of 3 seeds | cd | 956 / 1,125seed 0 run | Not measured | Not measured | Not measured | 27,950 ms | 107,497 ms | $1.040$1,040estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $1.017$1,017estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | Chat | 84.2%83.9%–84.4% min–max of 3 seeds | d | 950 / 1,125seed 0 run | Not measured | Not measured | 43.6%answered 91% | 29,847 ms | 113,220 ms | $1.017$1,017estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $1.327$1,327estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.2% | — | Chat | 83.6%83.1%–83.9% min–max of 3 seeds | d | 942 / 1,125seed 0 run | Not measured | Not measured | 43.9%answered 88% | 46,453 ms | 111,607 ms | $1.327$1,327estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.574$574estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.7% | — | Chat | 76.8%76.0%–77.6% min–max of 3 seeds | e | 855 / 1,125seed 0 run | Not measured | Not measured | 32.6%answered 91% | 21,223 ms | 45,077 ms | $0.574$574estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $1.193$1,193estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.8% | — | Chat | 75.9%73.3%–78.3% | e | 854 / 1,125 | Not measured | Not measured | Not measured | 41,350 ms | 70,680 ms | $1.193$1,193estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $2.480$2,480estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.0% | — | Chat | 75.4%72.8%–77.8% | ef | 848 / 1,125 | Not measured | Not measured | 44.3%answered 86% | 81,330 ms | 224,000 ms | $2.480$2,480estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.363$363estimate · DGX Spark per 1,000 questionsper 1M questions Math 96.9% | — | Chat | 74.6%72.0%–77.0% | ef | 839 / 1,125 | Not measured | Not measured | 26.4%answered 93% | 173,220 ms | 297,090 ms | $0.363$363estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.513$513estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 96.8% | — | Chat | 72.9%70.2%–75.4% | ef | 820 / 1,125 | Not measured | Not measured | 34.0%answered 90% | 21,390 ms | 35,160 ms | $0.513$513estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $15.330$15,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 97.0% | — | Chat | 70.9%68.2%–73.5% | ef | 798 / 1,125 | Not measured | Not measured | 10.5%answered 95% | 493,230 ms† | 1,396,520 ms† | $15.330$15,330estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $4.330$4,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.7% | — | Chat | 70.3%67.6%–72.9% | fg | 791 / 1,125 | Not measured | Not measured | 33.1%answered 91% | 186,330 ms | 272,210 ms | $4.330$4,330estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $9.270$9,270estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 89.6% | — | Chat | 65.1%62.2%–67.8% | gh | 732 / 1,125 | Not measured | Not measured | 35.7%answered 87% | 276,720 ms† | 965,630 ms† | $9.270$9,270estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $1.605$1,605estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.6% | — | Chat | 64.8%62.0%–67.5% | h | 729 / 1,125 | Not measured | Not measured | 35.2%answered 86% | 71,290 ms | 113,470 ms | $1.605$1,605estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.005$4.62estimate · DGX Spark per 1,000 questionsper 1M questions Math 55.8% | — | Decision | 45.9%43.0%–48.8% | i | 516 / 1,125 | 21 right of 21 kept0 mistakes · 1,104 skipped | 0.720 | 16.0%answered 91% | 73 ms | 84 ms | $0.005$4.62estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.026$26.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.9% | — | Decision | 44.7%41.8%–47.6% | ij | 503 / 1,125 | 17 right of 17 kept0 mistakes · 1,108 skipped | 0.757 | 24.4%answered 85% | 33 ms | 41 ms | $0.026$26.44estimate · RTX PRO 6000 Blackwell |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.026$25.82estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 56.8% | — | Decision | 44.1%41.2%–47.0% | ijk | 496 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.739 | 13.6%answered 92% | 32 ms | 37 ms | $0.026$25.82estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.2% | — | Decision | 43.1%40.2%–46.0% | ijk | 485 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 28.9%answered 82% | 45 ms | 55 ms | $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.006$5.89estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | Decision | 42.3%39.5%–45.2% | ijk | 476 / 1,125 | 17 right of 18 kept1 wrong · 1,107 skipped | 0.708 | 8.3%answered 95% | 96 ms | 115 ms | $0.006$5.89estimate · DGX Spark |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.030$29.51estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 50.4% | — | Decision | 40.4%37.5%–43.3% | jk | 454 / 1,125 | 39 right of 39 kept0 mistakes · 1,086 skipped | 0.724 | 14.6%answered 91% | 38 ms | 48 ms | $0.030$29.51estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.5% | — | Decision | 39.7%36.9%–42.6% | k | 447 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 12.6%answered 92% | 45 ms | 51 ms | $0.033$33.20estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$2.26estimate · DGX Spark per 1,000 questionsper 1M questions Math 36.8% | — | Decision | 32.3%29.6%–35.1% | l | 363 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.632 | 32.9%answered 75% | 34 ms | 42 ms | $0.002$2.26estimate · DGX Spark |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.091$90.99estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 39.2% | — | Decision | 32.1%29.4%–34.9% | l | 361 / 1,125 | No answers at this threshold | 0.660 | 1.5%answered 99% | 123 ms | 152 ms | $0.091$90.99estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.004$3.78estimate · DGX Spark per 1,000 questionsper 1M questions Math 35.3% | — | Decision | 31.7%29.1%–34.5% | l | 357 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.601 | 15.7%answered 89% | 59 ms | 70 ms | $0.004$3.78estimate · DGX Spark |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.003$2.52estimate · DGX Spark per 1,000 questionsper 1M questions Math 38.7% | — | Decision | 30.8%28.1%–33.5% | l | 346 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.640 | 31.4%answered 76% | 40 ms | 45 ms | $0.003$2.52estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.015$14.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 37.1% | — | Decision | 30.7%28.0%–33.4% | lm | 345 / 1,125 | 3 right of 3 kept0 mistakes · 1,122 skipped | 0.620 | 86.5%answered 22% | 17 ms | 23 ms | $0.015$14.76estimate · RTX PRO 6000 Blackwell |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.002$1.58estimate · DGX Spark per 1,000 questionsper 1M questions Math 32.1% | — | Decision | 29.2%26.6%–31.9% | lmn | 328 / 1,125 | 9 right of 9 kept0 mistakes · 1,116 skipped | 0.617 | 26.1%answered 80% | 23 ms | 29 ms | $0.002$1.58estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | Decision | 27.0%24.5%–29.7% | lmno | 304 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.545 | 4.6%answered 97% | 16 ms | 19 ms | $0.001$1.16estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$2.00estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | Decision | 26.7%24.2%–29.3% | lmno | 300 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.573 | 1.2%answered 99% | 31 ms | 39 ms | $0.002$2.00estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.79estimate · DGX Spark per 1,000 questionsper 1M questions Math 29.7% | — | Decision | 24.7%22.3%–27.3% | mno | 278 / 1,125 | No answers at this threshold | 0.641 | 21.4%answered 83% | 27 ms | 31 ms | $0.002$1.79estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.31estimate · DGX Spark per 1,000 questionsper 1M questions Math 22.7% | — | Decision | 23.6%21.2%–26.1% | no | 265 / 1,125 | No answers at this threshold | 0.527 | 12.4%answered 90% | 15 ms | 28 ms | $0.001$1.31estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 20.6% | — | Decision | 23.2%20.8%–25.8% | o | 261 / 1,125 | No answers at this threshold | 0.556 | 76.1%answered 30% | 13 ms | 16 ms | $0.001$0.95estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.028$27.67estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 28.7% | — | Decision | 22.4%20.1%–24.9% | o | 252 / 1,125 | No answers at this threshold | 0.568 | 8.3%answered 94% | 36 ms | 43 ms | $0.028$27.67estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
StartLux Fastino Apertix AI Base Labs Cloudflare newsbang Patrick-18 PelaAI tattrongvu Blockbrain Labs Valen-Team EldanRing eCloud Technology Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
StartLux-Decision-v1.1-9B-Instant: 46.6% at $0.010 per 1,000 questions ($10.04 per 1M) (estimate)
GLiDE 9B: 47.8% at $0.013 per 1,000 questions ($13.29 per 1M) (estimate)
Cerno 9B v1.2: 43.3% at $0.013 per 1,000 questions ($13.35 per 1M) (estimate)
Elephant-1: 46.4% at $0.013 per 1,000 questions ($13.44 per 1M)
Clef-Flash: 38.9% at $0.017 per 1,000 questions ($16.83 per 1M)
Adjudex 9B v5: 46.4% at $0.031 per 1,000 questions ($31.36 per 1M) (estimate)
Dexy-Decision-9B-V2: 45.3% at $0.034 per 1,000 questions ($33.81 per 1M) (estimate)
KnowLine-9B-Gen1: 44.3% at $0.036 per 1,000 questions ($36.27 per 1M) (estimate)
Jeffan-9B-1211: 48.2% at $0.039 per 1,000 questions ($39.35 per 1M) (estimate)
Cygnet: 39.3% at $0.040 per 1,000 questions ($39.71 per 1M)
KnowLine-9B-Gen2: 43.9% at $0.040 per 1,000 questions ($39.96 per 1M) (estimate)
Valen-Decision-1.5-9B: 44.4% at $0.042 per 1,000 questions ($42.42 per 1M) (estimate)
KnowLine-12B-Gen1: 50.6% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
Winnow-12B: 39.7% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
Erk-14B (off · chat): 54.2% at $0.390 per 1,000 questions ($390 per 1M) (estimate)
Gemma 4 E4B (on · chat): 82.6% at $0.650 per 1,000 questions ($650 per 1M) (estimate)
Gemma 4 E4B (chat): 83.2% at $0.842 per 1,000 questions ($842 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 85.9% at $0.993 per 1,000 questions ($993 per 1M) (estimate)
Gemma 4 12B Unified (chat): 87.7% at $1.292 per 1,000 questions ($1,292 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 85.0% at $1.503 per 1,000 questions ($1,503 per 1M) (estimate)
Qwen3-14B (off · chat): 65.3% at $1.772 per 1,000 questions ($1,772 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 60.2% at $4.664 per 1,000 questions ($4,664 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
StartLux Fastino Apertix AI Base Labs Cloudflare newsbang Patrick-18 PelaAI tattrongvu Blockbrain Labs Valen-Team EldanRing eCloud Technology Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| StartLux-Decision-v1.1-9B-Instant Pareto Open · 9.0B | — | Decision | $0.010$10.04estimate · DGX Spark | $10.04 estimate | $0.01129 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Pareto Open · 7.9B | — | Decision | $0.013$13.29estimate · DGX Spark | $13.29 estimate | $0.01496 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cerno 9B v1.2 Open · 9.0B | — | Decision | $0.013$13.35estimate · DGX Spark | $13.35 estimate | $0.01502 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Elephant-1 Open · 9.0B | — | Decision | $0.013$13.44 list | $13.44 | $0.01512 | $0.058$58.41estimate · RTX PRO 6000 Blackwell | maker's list price |
| Clef-Flash Open · 9.0B | — | Decision | $0.017$16.83 list | $16.83 | $0.01894 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Adjudex 9B v5 Open · 9.0B | — | Decision | $0.031$31.36estimate · RTX PRO 6000 Blackwell | $31.36 estimate | $0.03528 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | Decision | $0.034$33.81estimate · RTX PRO 6000 Blackwell | $33.81 estimate | $0.03804 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | Decision | $0.036$36.27estimate · RTX PRO 6000 Blackwell | $36.27 estimate | $0.04081 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Pareto Open · 9.0B | — | Decision | $0.039$39.35estimate · RTX PRO 6000 Blackwell | $39.35 estimate | $0.04427 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | Decision | $0.040$39.71 list | $39.71 | $0.04468 | $0.031$30.74estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen2 Open · 9.0B | — | Decision | $0.040$39.96estimate · RTX PRO 6000 Blackwell | $39.96 estimate | $0.04496 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | Decision | $0.042$42.42estimate · RTX PRO 6000 Blackwell | $42.42 estimate | $0.04773 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Pareto Open · 12B | — | Decision | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | Decision | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Erk-14B Pareto off · chat Open · 15B | — | Chat | $0.390$390estimate · RTX PRO 6000 Blackwell | $389.79 estimate | $0.4385 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat Open · 8.0B | — | Chat | $0.650$650estimate · DGX Spark | $649.54 estimate | $0.7307 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E4B Pareto chat Open · 8.0B | — | Chat | $0.842$842estimate · RTX PRO 6000 Blackwell | $842.30 estimate | $0.9476 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat · 32k cap · sampled Open · 8.0B | — | Chat | $0.993$993estimate · RTX PRO 6000 Blackwell | $992.52 estimate | $1.091 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified Pareto chat Open · 12B | — | Chat | $1.292$1,292estimate · RTX PRO 6000 Blackwell | $1,291.73 estimate | $1.453 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | Chat | $1.503$1,503estimate · RTX PRO 6000 Blackwell | $1,502.61 estimate | $1.69 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | Chat | $1.772$1,772estimate · RTX PRO 6000 Blackwell | $1,771.90 estimate | $1.993 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | Chat | $4.664$4,664estimate · RTX PRO 6000 Blackwell† | $4,663.99 estimate† | $5.247 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $1.292$1,292estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.8% | — | Chat | 87.7%85.7%–89.5% | a | 987 / 1,125 | Not measured | Not measured | 15.5%answered 98% | 42,060 ms | 82,940 ms | $1.292$1,292estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.993$993estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | Chat | 85.9%85.5%–86.1% min–max of 3 seeds | ab | 968 / 1,125seed 0 run | Not measured | Not measured | 55.9%answered 90% | 31,567 ms | 83,467 ms | $0.993$993estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $1.503$1,503estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | Chat | 85.0%82.8%–86.9% | ab | 956 / 1,125 | Not measured | Not measured | Not measured | 57,900 ms | 109,440 ms | $1.503$1,503estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.842$842estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.1% | — | Chat | 83.2%80.9%–85.3% | b | 936 / 1,125 | Not measured | Not measured | 52.4%answered 88% | 32,690 ms | 64,070 ms | $0.842$842estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.650$650estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.2% | — | Chat | 82.6%80.3%–84.7% | b | 929 / 1,125 | Not measured | Not measured | 45.1%answered 91% | 284,080 ms | 539,980 ms | $0.650$650estimate · DGX Spark |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $1.772$1,772estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 95.8% | — | Chat | 65.3%62.5%–68.1% | c | 735 / 1,125 | Not measured | Not measured | 16.1%answered 94% | 16,560 ms | 178,680 ms | $1.772$1,772estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $4.664$4,664estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 96.3% | — | Chat | 60.2%57.3%–63.0% | cd | 677 / 1,125 | Not measured | Not measured | 58.3%answered 70% | 138,140 ms† | 403,380 ms† | $4.664$4,664estimate · RTX PRO 6000 Blackwell† |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.390$390estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 90.8% | — | Chat | 54.2%51.3%–57.1% | de | 610 / 1,125 | Not measured | Not measured | 30.7%answered 84% | 10,150 ms | 18,410 ms | $0.390$390estimate · RTX PRO 6000 Blackwell |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 66.3% | — | Decision | 50.6%47.7%–53.5% | ef | 569 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.774 | 8.8%answered 95% | 68 ms | 88 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.039$39.35estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | Decision | 48.2%45.3%–51.1% | efg | 542 / 1,125 | 10 right of 10 kept0 mistakes · 1,115 skipped | 0.753 | 23.4%answered 87% | 52 ms | 66 ms | $0.039$39.35estimate · RTX PRO 6000 Blackwell |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.013$13.29estimate · DGX Spark per 1,000 questionsper 1M questions Math 57.9% | — | Decision | 47.8%44.9%–50.7% | efg | 538 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.749 | 30.4%answered 82% | 208 ms | 251 ms | $0.013$13.29estimate · DGX Spark |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.010$10.04estimate · DGX Spark per 1,000 questionsper 1M questions Math 58.4% | — | Decision | 46.6%43.7%–49.5% | fg | 524 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.762 | 17.4%answered 90% | 160 ms | 175 ms | $0.010$10.04estimate · DGX Spark |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.36estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 62.1% | — | Decision | 46.4%43.5%–49.3% | fg | 522 / 1,125 | 54 right of 55 kept1 wrong · 1,070 skipped | 0.771 | 12.2%answered 93% | 41 ms | 46 ms | $0.031$31.36estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 61.3% | — | Decision | 46.4%43.5%–49.3% | fg | 522 / 1,125 | 53 right of 53 kept0 mistakes · 1,072 skipped | 0.768 | 75.6%answered 47% | 79 ms | 98 ms | $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.034$33.81estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 58.8% | — | Decision | 45.3%42.4%–48.3% | fgh | 510 / 1,125 | 56 right of 56 kept0 mistakes · 1,069 skipped | 0.759 | 12.4%answered 93% | 47 ms | 53 ms | $0.034$33.81estimate · RTX PRO 6000 Blackwell |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.042$42.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.3% | — | Decision | 44.4%41.5%–47.3% | ghi | 499 / 1,125 | 35 right of 37 kept2 wrong · 1,088 skipped | 0.701 | 13.6%answered 92% | 60 ms | 67 ms | $0.042$42.42estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.036$36.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | Decision | 44.3%41.4%–47.2% | ghi | 498 / 1,125 | 30 right of 30 kept0 mistakes · 1,095 skipped | 0.725 | 6.7%answered 96% | 50 ms | 61 ms | $0.036$36.27estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.040$39.96estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 60.1% | — | Decision | 43.9%41.0%–46.8% | ghi | 494 / 1,125 | No answers at this threshold | 0.730 | 19.5%answered 88% | 52 ms | 65 ms | $0.040$39.96estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.013$13.35estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | Decision | 43.3%40.4%–46.2% | ghi | 487 / 1,125 | 26 right of 27 kept1 wrong · 1,098 skipped | 0.752 | 6.0%answered 97% | 209 ms | 269 ms | $0.013$13.35estimate · DGX Spark |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 46.9% | — | Decision | 39.7%36.9%–42.6% | hi | 447 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.727 | 29.7%answered 80% | 66 ms | 91 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 44.3% | — | Decision | 39.3%36.5%–42.2% | i | 442 / 1,125 | No answers at this threshold | 0.720 | 11.3%answered 93% | 40 ms | 48 ms | $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 49.8% | — | Decision | 38.9%36.1%–41.8% | i | 438 / 1,125 | No answers at this threshold | 0.659 | 14.4%answered 91% | 110 ms | 137 ms | $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router PelaAI Sthānika AI OmniJev Cloudflare MachinePulse Poindexter Labs Zefan Cai Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Perplexity Decider 27B v1.2 (API): 53.3% at $0.007 per 1,000 questions ($7.46 per 1M)
GLiDE (v1.2): 55.2% at $0.014 per 1,000 questions ($13.88 per 1M)
Drex 2.0: 41.9% at $0.018 per 1,000 questions ($18.41 per 1M)
Drex Max: 53.1% at $0.023 per 1,000 questions ($22.55 per 1M)
Turion Jev 31B: 52.3% at $0.025 per 1,000 questions ($24.91 per 1M) (estimate)
GLiDE 27B: 58.4% at $0.033 per 1,000 questions ($33.16 per 1M) (estimate)
Mammoth-1: 54.5% at $0.038 per 1,000 questions ($38.39 per 1M)
d3: 54.5% at $0.057 per 1,000 questions ($57.18 per 1M) (estimate)
KnowLine-31B-Gen1.5: 59.0% at $0.061 per 1,000 questions ($61.43 per 1M) (estimate)
Sieve 31B v3: 57.9% at $0.063 per 1,000 questions ($62.95 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 54.8% at $0.081 per 1,000 questions ($81.16 per 1M) (estimate)
OneJev-27B: 40.6% at $0.086 per 1,000 questions ($86.07 per 1M) (estimate)
Clef: 40.8% at $0.106 per 1,000 questions ($106 per 1M)
d3-ultra: 54.6% at $0.109 per 1,000 questions ($109 per 1M) (estimate)
KnowLine-31B-Gen2: 58.9% at $0.112 per 1,000 questions ($112 per 1M) (estimate)
Karpo Decision 31B: 60.7% at $0.128 per 1,000 questions ($128 per 1M) (estimate)
Copernicus-31B: 58.7% at $0.136 per 1,000 questions ($136 per 1M) (estimate)
Open-Jev-27B-v1.1: 42.6% at $0.294 per 1,000 questions ($294 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 52.5% at $1.030 per 1,000 questions ($1,030 per 1M)
Gemma 4 31B (chat): 93.3% at $1.244 per 1,000 questions ($1,244 per 1M) (estimate)
Qwen3.8-27B (low · chat): 94.0% at $4.884 per 1,000 questions ($4,884 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 86.7% at $7.562 per 1,000 questions ($7,562 per 1M) (estimate)
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router PelaAI Sthānika AI OmniJev Cloudflare MachinePulse Poindexter Labs Zefan Cai Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | Decision | $0.007$7.46 | $7.46 | $0.00839 | — | provider price × reported usage |
| GLiDE Pareto v1.2 Open · 27B | — | Decision | $0.014$13.88 list | $13.88 | $0.01561 | $0.082$81.77estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | Decision | $0.018$18.41 | $18.41 | $0.02072 | — | provider price × reported usage |
| Drex Max Open · 27B | — | Decision | $0.023$22.55 list | $22.55 | $0.02537 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | Decision | $0.025$24.91estimate · DGX Spark | $24.91 estimate | $0.02802 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 27B Pareto Open · 27B | — | Decision | $0.033$33.16estimate · DGX Spark | $33.16 estimate | $0.0373 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Mammoth-1 Open · 27B | — | Decision | $0.038$38.39 list | $38.39 | $0.04319 | $0.151$151estimate · RTX PRO 6000 Blackwell | maker's list price |
| d3 Open · 26B | — | Decision | $0.057$57.18estimate · RTX PRO 6000 Blackwell | $57.18 estimate | $0.06433 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | Decision | $0.061$61.43estimate · DGX Spark | $61.43 estimate | $0.06911 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Sieve 31B v3 Open · 31B | — | Decision | $0.063$62.95estimate · DGX Spark | $62.95 estimate | $0.07082 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | Decision | $0.081$81.16estimate · RTX PRO 6000 Blackwell | $81.16 estimate | $0.0913 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| OneJev-27B Open · 27B | — | Decision | $0.086$86.07estimate · RTX PRO 6000 Blackwell | $86.07 estimate | $0.09683 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | Decision | $0.106$106 list | $106.31 | $0.1196 | $0.141$141estimate · RTX PRO 6000 Blackwell | maker's list price |
| d3-ultra Open · 26B | — | Decision | $0.109$109estimate · RTX PRO 6000 Blackwell | $109.44 estimate | $0.1231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen2 Open · 31B | — | Decision | $0.112$112estimate · RTX PRO 6000 Blackwell | $111.90 estimate | $0.1259 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | Decision | $0.128$128estimate · RTX PRO 6000 Blackwell | $128.50 estimate | $0.1446 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | Decision | $0.136$136estimate · RTX PRO 6000 Blackwell | $135.87 estimate | $0.1529 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | Decision | $0.294$294estimate · RTX PRO 6000 Blackwell | $294.50 estimate | $0.3313 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | Decision | $1.030$1,030 | $1,029.58 | $1.158 | — | provider price × reported usage |
| Gemma 4 31B Pareto chat Open · 31B | — | Chat | $1.244$1,244estimate · RTX PRO 6000 Blackwell | $1,243.77 estimate | $1.399 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B Pareto low · chat Open · 27B | — | Chat | $4.884$4,884estimate · RTX PRO 6000 Blackwell | $4,884.09 estimate | $5.495 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | Chat | $7.562$7,562estimate · RTX PRO 6000 Blackwell | $7,561.61 estimate | $8.507 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $4.884$4,884estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 94.0%92.5%–95.3% | a | 1,058 / 1,125 | Not measured | Not measured | 39.1%answered 97% | 18,710 ms | 101,030 ms | $4.884$4,884estimate · RTX PRO 6000 Blackwell |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $1.244$1,244estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | Chat | 93.3%91.7%–94.6% | a | 1,050 / 1,125 | Not measured | Not measured | 16.3%answered 99% | 18,290 ms | 39,790 ms | $1.244$1,244estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $7.562$7,562estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.0% | — | Chat | 86.7%84.6%–88.5% | b | 975 / 1,125 | Not measured | Not measured | 37.8%answered 95% | 228,980 ms | 776,930 ms | $7.562$7,562estimate · RTX PRO 6000 Blackwell |
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.128$128estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 81.6% | — | Decision | 60.7%57.8%–63.5% | c | 683 / 1,125 | 169 right of 172 kept3 wrong · 953 skipped | 0.816 | 6.6%answered 97% | 173 ms | 211 ms | $0.128$128estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.061$61.43estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.4% | — | Decision | 59.0%56.1%–61.9% | cd | 664 / 1,125 | 89 right of 89 kept0 mistakes · 1,036 skipped | 0.804 | 0.4%answered 100% | 983 ms | 1,279 ms | $0.061$61.43estimate · DGX Spark |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.112$112estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 80.3% | — | Decision | 58.9%56.0%–61.8% | cd | 663 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.825 | 0.0%answered 100% | 151 ms | 204 ms | $0.112$112estimate · RTX PRO 6000 Blackwell |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.136$136estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 77.3% | — | Decision | 58.7%55.8%–61.5% | cd | 660 / 1,125 | 66 right of 66 kept0 mistakes · 1,059 skipped | 0.801 | 1.9%answered 99% | 185 ms | 255 ms | $0.136$136estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.033$33.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 72.0% | — | Decision | 58.4%55.5%–61.2% | cd | 657 / 1,125 | 74 right of 75 kept1 wrong · 1,050 skipped | 0.822 | 44.3%answered 79% | 544 ms | 655 ms | $0.033$33.16estimate · DGX Spark |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.063$62.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.1% | — | Decision | 57.9%55.0%–60.7% | cde | 651 / 1,125 | 79 right of 81 kept2 wrong · 1,044 skipped | 0.797 | 5.5%answered 98% | 1,050 ms | 1,335 ms | $0.063$62.95estimate · DGX Spark |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 69.9% | — | Decision | 55.2%52.3%–58.1% | def | 621 / 1,125 | 45 right of 45 kept0 mistakes · 1,080 skipped | 0.793 | 50.2%answered 73% | 111 ms | 140 ms | $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 72.5% | — | Decision | 54.8%51.8%–57.6% | def | 616 / 1,125 | 62 right of 62 kept0 mistakes · 1,063 skipped | 0.803 | 18.1%answered 91% | 108 ms | 131 ms | $0.081$81.16estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.109$109estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 71.5% | — | Decision | 54.6%51.7%–57.5% | def | 614 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.778 | 10.1%answered 95% | 156 ms | 185 ms | $0.109$109estimate · RTX PRO 6000 Blackwell |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 74.0% | — | Decision | 54.5%51.6%–57.4% | def | 613 / 1,125 | 106 right of 108 kept2 wrong · 1,017 skipped | 0.795 | 72.3%answered 58% | 221 ms | 244 ms | $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.057$57.18estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.4% | — | Decision | 54.5%51.6%–57.4% | def | 613 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.806 | 14.8%answered 93% | 76 ms | 98 ms | $0.057$57.18estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.007$7.46 per 1,000 questionsper 1M questions Math 69.8% | — | Decision | 53.3%50.4%–56.2% | ef | 600 / 1,125 | 78 right of 80 kept2 wrong · 1,045 skipped | 0.798 | 14.4%answered 93% | 238 ms | 552 ms | $0.007$7.46 |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.3% | — | Decision | 53.1%50.1%–56.0% | ef | 597 / 1,125 | 147 right of 155 kept8 wrong · 970 skipped | 0.786 | 7.3%answered 97% | 106 ms | 143 ms | $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.030$1,030 per 1,000 questionsper 1M questions Math 69.3% | with web | Decision | 52.5%49.6%–55.4% | f | 591 / 1,125 | 78 right of 79 kept1 wrong · 1,046 skipped | 0.800 | 37.7%answered 80% | 448 ms | 827 ms | $1.030$1,030 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.025$24.91estimate · DGX Spark per 1,000 questionsper 1M questions Math 69.0% | — | Decision | 52.3%49.3%–55.2% | f | 588 / 1,125 | 7 right of 7 kept0 mistakes · 1,118 skipped | 0.782 | 1.5%answered 99% | 385 ms | 533 ms | $0.025$24.91estimate · DGX Spark |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.294$294estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 54.6% | — | Decision | 42.6%39.7%–45.5% | g | 479 / 1,125 | No answers at this threshold | 0.733 | 28.2%answered 82% | 413 ms | 576 ms | $0.294$294estimate · RTX PRO 6000 Blackwell |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.018$18.41 per 1,000 questionsper 1M questions Math 59.0% | — | Decision | 41.9%39.0%–44.8% | g | 471 / 1,125 | 33 right of 33 kept0 mistakes · 1,092 skipped | 0.717 | 37.7%answered 75% | 345 ms | 504 ms | $0.018$18.41 |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.1% | — | Decision | 40.8%38.0%–43.7% | g | 459 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.697 | 3.6%answered 98% | 196 ms | 222 ms | $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.086$86.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 53.8% | — | Decision | 40.6%37.8%–43.5% | g | 457 / 1,125 | 4 right of 4 kept0 mistakes · 1,121 skipped | 0.726 | 1.2%answered 99% | 108 ms | 136 ms | $0.086$86.07estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
REAX Logits MiniMax Z.ai Xiaomi DeepSeek Thinking Machines Alibaba
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
REAX 1.3: 59.3% at $0.160 per 1,000 questions ($160 per 1M) (estimate)
Logits Sigma-1 Duo: 57.8% at $0.211 per 1,000 questions ($211 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 94.0% at $0.803 per 1,000 questions ($803 per 1M)
GLM-5.3-Flash (high · chat): 93.2% at $1.057 per 1,000 questions ($1,057 per 1M)
MiMo-V2.6-Flash (on · chat): 95.1% at $1.390 per 1,000 questions ($1,390 per 1M)
DeepSeek-V4.1-Flash (high · chat · 1017): 96.2% at $1.647 per 1,000 questions ($1,647 per 1M)
Inkling 2 Small (high · chat): 95.6% at $4.121 per 1,000 questions ($4,121 per 1M)
Inkling-Small (high · chat · Jul 2026): 95.4% at $5.489 per 1,000 questions ($5,489 per 1M)
Qwen3.5-397B-A17B (on · chat): 95.1% at $13.426 per 1,000 questions ($13,426 per 1M)
REAX Logits MiniMax Z.ai Xiaomi DeepSeek Thinking Machines Alibaba
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| REAX 1.3 Pareto Open · 57B | — | Decision | $0.160$160estimate · RTX PRO 6000 Blackwell | $160.47 estimate | $0.1805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | Decision | $0.211$211estimate · RTX PRO 6000 Blackwell | $211.50 estimate | $0.2379 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | Chat | $0.803$803 | $803.23 | $0.9036 | — | provider price × reported usage |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active | — | Chat | $1.057$1,057 | $1,056.60 | $1.189 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | Chat | $1.390$1,390 | $1,390.14 | $1.564 | — | provider price × reported usage |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | Chat | $1.647$1,647 | $1,646.92 | $1.853 | — | provider price × reported usage |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | Chat | $4.121$4,121 | $4,121.38 | $4.637 | — | provider price × reported usage |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | Chat | $5.489$5,489 | $5,489.38 | $6.176 | — | provider price × reported usage |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active | — | Chat | $13.426$13,426 | $13,425.63 | $15.1 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): DeepSeek-V4-Flash (chat) (sample: a random subset by design, 317 answers stored, not the full 1,125).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $1.647$1,647 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 96.2%94.9%–97.2% | a | 1,082 / 1,125 | Not measured | Not measured | 3.8%answered 100% | 6,680 ms | 40,040 ms | $1.647$1,647 |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $4.121$4,121 per 1,000 questionsper 1M questions Math 99.5% | — | Chat | 95.6%94.3%–96.7% | ab | 1,076 / 1,125 | Not measured | Not measured | 37.3%answered 98% | 13,180 ms | 144,250 ms | $4.121$4,121 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $5.489$5,489 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 95.4%94.0%–96.5% | ab | 1,073 / 1,125 | Not measured | Not measured | 50.0%answered 97% | 13,290 ms | 78,510 ms | $5.489$5,489 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $1.390$1,390 per 1,000 questionsper 1M questions Math 99.4% | — | Chat | 95.1%93.7%–96.2% | abc | 1,070 / 1,125 | Not measured | Not measured | 23.3%answered 98% | 24,740 ms | 179,930 ms | $1.390$1,390 |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active Qwen3.5-397B-A17B on · chat · Alibaba · chat model · reasoning on Parameters: 397B MoE · 17B active Cost $13.426$13,426 per 1,000 questionsper 1M questions Math running | — | Chat | 95.1%93.7%–96.2% | abc | 1,070 / 1,125 | Not measured | Not measured | 31.7%answered 98% | 84,650 ms | 167,520 ms | $13.426$13,426 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.803$803 per 1,000 questionsper 1M questions Math 98.2% | — | Chat | 94.0%92.4%–95.2% | bc | 1,057 / 1,125 | Not measured | Not measured | 38.3%answered 97% | 5,610 ms | 37,430 ms | $0.803$803 |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $1.057$1,057 per 1,000 questionsper 1M questions Math 98.8% | — | Chat | 93.2%91.5%–94.5% | c | 1,048 / 1,125 | Not measured | Not measured | 4.2%answered 100% | 12,050 ms | 82,830 ms | $1.057$1,057 |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.160$160estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 78.0% | — | Decision | 59.3%56.4%–62.1% | d | 667 / 1,125 | 51 right of 51 kept0 mistakes · 1,074 skipped | 0.816 | 5.3%answered 98% | 215 ms | 278 ms | $0.160$160estimate · RTX PRO 6000 Blackwell |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.211$211estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 75.2% | — | Decision | 57.8%54.9%–60.6% | d | 650 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.796 | 7.1%answered 97% | 285 ms | 399 ms | $0.211$211estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino Meta
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
celeris-1.1-decision: 47.3% at $0.002 per 1,000 questions ($1.54 per 1M)
d1.1: 46.0% at $0.006 per 1,000 questions ($6.02 per 1M)
d1: 40.8% at $0.006 per 1,000 questions ($6.02 per 1M)
Mercury Decide: 45.1% at $0.008 per 1,000 questions ($7.52 per 1M)
d1-ultra: 54.9% at $0.013 per 1,000 questions ($12.64 per 1M)
Microsoft-Decision-1: 44.4% at $0.013 per 1,000 questions ($13.42 per 1M)
SubQ Snap 1.1: 52.4% at $0.015 per 1,000 questions ($14.92 per 1M)
celeris-1-decision: 47.3% at $0.015 per 1,000 questions ($15.40 per 1M)
Drex Max v1.1: 54.8% at $0.022 per 1,000 questions ($22.37 per 1M)
Jev 1.14: 48.2% at $0.026 per 1,000 questions ($25.64 per 1M)
Jev 1.13: 46.3% at $0.026 per 1,000 questions ($25.64 per 1M)
Gemini Decision 1 (20261111): 50.0% at $0.037 per 1,000 questions ($36.77 per 1M)
GLiDE (API): 54.5% at $0.052 per 1,000 questions ($52.03 per 1M)
Muse Ember (max · chat): 95.5% at $1.820 per 1,000 questions ($1,820 per 1M)
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino Meta
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| celeris-1.1-decision Pareto Closed · Efficient | — | Decision | $0.002$1.54 | $1.54 | $0.001733 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | Decision | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1 Closed · Efficient | — | Decision | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | Decision | $0.008$7.52 | $7.52 | $0.008459 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | Decision | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | Decision | $0.013$13.42 | $13.42 | $0.0151 | — | provider price × reported usage |
| SubQ Snap 1.1 Closed · Efficient | — | Decision | $0.015$14.92 | $14.92 | $0.01678 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | Decision | $0.015$15.40 | $15.40 | $0.01733 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | Decision | $0.022$22.37 | $22.37 | $0.02517 | — | provider price × reported usage |
| Jev 1.14 Closed · Efficient | — | Decision | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | Decision | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | Decision | $0.037$36.77 | $36.77 | $0.04137 | — | provider price × reported usage |
| GLiDE API Closed · Efficient | — | Decision | $0.052$52.03 | $52.03 | $0.05854 | — | provider price × reported usage |
| Muse Ember Pareto max · chat Closed · Efficient | — | Chat | $1.820$1,820 | $1,820.45 | $2.048 | — | provider price × reported usage |
| GPT-6 Luna chat Closed · Efficient | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 1,125 of 1,125); shown once.
Not ranked (only complete runs are): GPT-6 Luna (max · chat) (paused for now at 55 of 1,125 (zeki #4129, 2026-11-10T14:13Z)); Claude Haiku 5.5 (max · chat) (paused for now at 47 of 1,125 (zeki #4129, 2026-11-10T14:13Z)).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $1.820$1,820 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 95.5%94.1%–96.5% | a | 1,074 / 1,125 | Not measured | Not measured | 22.6%answered 99% | 3,110 ms | 14,020 ms | $1.820$1,820 |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 93.7% | — | Chat | 58.0%55.0%–60.8% | b | 652 / 1,125 | Not measured | Not measured | 55.1%answered 72% | 3,860 ms | 5,820 ms | $0.000$0.00 |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 74.1% | — | Decision | 54.9%52.0%–57.8% | bc | 618 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.783 | 91.0%answered 35% | 182 ms | 425 ms | $0.013$12.64 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.022$22.37 per 1,000 questionsper 1M questions Math 70.1% | — | Decision | 54.8%51.8%–57.6% | bc | 616 / 1,125 | 89 right of 90 kept1 wrong · 1,035 skipped | 0.793 | 54.1%answered 69% | 329 ms | 1,540 ms | $0.022$22.37 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.052$52.03 per 1,000 questionsper 1M questions Math 70.8% | — | Decision | 54.5%51.6%–57.4% | bc | 613 / 1,125 | 52 right of 52 kept0 mistakes · 1,073 skipped | 0.814 | 60.0%answered 67% | 271 ms | 291 ms | $0.052$52.03 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.015$14.92 per 1,000 questionsper 1M questions Math 68.6% | — | Decision | 52.4%49.4%–55.3% | bcd | 589 / 1,125 | 70 right of 72 kept2 wrong · 1,053 skipped | 0.787 | 65.7%answered 61% | 384 ms | 393 ms | $0.015$14.92 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.037$36.77 per 1,000 questionsper 1M questions Math 71.3% | — | Decision | 50.0%47.1%–53.0% | cde | 563 / 1,125 | 43 right of 43 kept0 mistakes · 1,082 skipped | 0.768 | 2.0%answered 99% | 228 ms | 344 ms | $0.037$36.77 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 65.1% | — | Decision | 48.2%45.3%–51.1% | de | 542 / 1,125 | 61 right of 61 kept0 mistakes · 1,064 skipped | 0.750 | 24.5%answered 86% | 166 ms | 193 ms | $0.026$25.64 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.015$15.40 per 1,000 questionsper 1M questions Math 70.0% | — | Decision | 47.3%44.4%–50.2% | de | 532 / 1,125 | No answers at this threshold | 0.735 | 92.3%answered 28% | 390 ms | 394 ms | $0.015$15.40 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.002$1.54 per 1,000 questionsper 1M questions Math 69.9% | — | Decision | 47.3%44.4%–50.2% | de | 532 / 1,125 | No answers at this threshold | 0.733 | 91.0%answered 29% | 383 ms | 390 ms | $0.002$1.54 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 64.1% | — | Decision | 46.3%43.4%–49.2% | e | 521 / 1,125 | 65 right of 65 kept0 mistakes · 1,060 skipped | 0.764 | 15.8%answered 91% | 167 ms | 203 ms | $0.026$25.64 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.2% | — | Decision | 46.0%43.2%–49.0% | e | 518 / 1,125 | 48 right of 48 kept0 mistakes · 1,077 skipped | 0.748 | 90.6%answered 28% | 176 ms | 272 ms | $0.006$6.02 |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.008$7.52 per 1,000 questionsper 1M questions Math 53.5% | — | Decision | 45.1%42.2%–48.0% | ef | 507 / 1,125 | 84 right of 91 kept7 wrong · 1,034 skipped | 0.746 | 20.5%answered 88% | 301 ms | 360 ms | $0.008$7.52 |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.013$13.42 per 1,000 questionsper 1M questions Math 59.4% | — | Decision | 44.4%41.6%–47.4% | ef | 500 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.712 | 19.8%answered 88% | 315 ms | 362 ms | $0.013$13.42 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 55.6% | — | Decision | 40.8%38.0%–43.7% | f | 459 / 1,125 | 55 right of 56 kept1 wrong · 1,069 skipped | 0.723 | 11.3%answered 93% | 188 ms | 290 ms | $0.006$6.02 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Sonnet 5.5 (chat): 95.5% at $11.480 per 1,000 questions ($11,480 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 Pareto chat Closed · Workhorse | — | Chat | $11.480$11,480 API list-price equivalent | $11,479.90 | $12.91 | — | Claude subscription |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $11.480$11,480 API list-price equivalent per 1,000 questionsper 1M questions Math 99.7% | — | Chat | 95.5%94.1%–96.5% | a | 1,074 / 1,125 | Not measured | Not measured | 41.3%answered 98% | 6,710 ms | 14,890 ms | $11.480$11,480 API list-price equivalent |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Opus 5.5 (chat): 96.2% at $19.540 per 1,000 questions ($19,540 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 Pareto chat Closed · Flagship | — | Chat | $19.540$19,540 API list-price equivalent | $19,540.15 | $21.98 | — | Claude subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Claude Fable 5.2 (chat) (not running now, 147 of 1,125 stored, last answer 12 Nov 11:32 UTC).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $19.540$19,540 API list-price equivalent per 1,000 questionsper 1M questions Math 99.8% | — | Chat | 96.2%94.9%–97.2% | a | 1,082 / 1,125 | Not measured | Not measured | 38.2%answered 98% | 5,830 ms | 12,240 ms | $19.540$19,540 API list-price equivalent |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.7% | — | Chat | 95.2%93.8%–96.3% | b | 1,071 / 1,125 | Not measured | Not measured | 8.3%answered 100% | 3,820 ms | 13,460 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Liquid AI Alibaba
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
d1.2: 49.2% at $0.006 per 1,000 questions ($6.02 per 1M)
d1.2-ultra: 51.6% at $0.013 per 1,000 questions ($12.64 per 1M)
Qwen3.8 Flash (off · chat): 94.0% at $1.376 per 1,000 questions ($1,376 per 1M)
Qwen4-Max (high · chat): 96.3% at $5.690 per 1,000 questions ($5,690 per 1M)
Liquid AI Alibaba
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| d1.2 Pareto Closed · Undisclosed | — | Decision | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1.2-ultra Pareto Closed · Undisclosed | — | Decision | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
| Qwen3.8 Flash Pareto off · chat Closed · Undisclosed | — | Chat | $1.376$1,376 | $1,376.26 | $1.548 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | Chat | $5.690$5,690 | $5,689.67 | $6.401 | — | provider price × reported usage |
| GPT-6 Astra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Grok 4.7 (chat) (paused for now at 576 of 1,125 (Ümit #8603, zeki #8637, 2026-11-15T11:37Z); cost so far $80.830$80,830 per 1,000 questions API list-price equivalent, over the 576 of 1,125 answered).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 96.7%95.5%–97.6% | a | 1,088 / 1,125 | Not measured | Not measured | 56.4%answered 97% | 6,590 ms | 14,330 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Math 99.6% | — | Chat | 96.7%95.5%–97.6% | a | 1,088 / 1,125 | Not measured | Not measured | 45.5%answered 98% | 44,520 ms | 206,660 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | Chat | 96.5%95.3%–97.5% | a | 1,086 / 1,125 | Not measured | Not measured | 63.2%answered 97% | 7,070 ms | 16,200 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $5.690$5,690 per 1,000 questionsper 1M questions Math 99.8% | — | Chat | 96.3%95.0%–97.2% | a | 1,083 / 1,125 | Not measured | Not measured | 0.0%answered 100% | 11,570 ms | 38,640 ms | $5.690$5,690 |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.2% | — | Chat | 94.0%92.4%–95.2% | b | 1,057 / 1,125 | Not measured | Not measured | 61.1%answered 95% | 5,640 ms | 10,330 ms | $0.000$0.00 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $1.376$1,376 per 1,000 questionsper 1M questions Math 99.5% | — | Chat | 94.0%92.4%–95.2% | b | 1,057 / 1,125 | Not measured | Not measured | 34.6%answered 98% | 17,550 ms | 73,220 ms | $1.376$1,376 |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 69.9% | — | Decision | 51.6%48.6%–54.5% | c | 580 / 1,125 | 13 right of 13 kept0 mistakes · 1,112 skipped | 0.800 | 94.5%answered 28% | 220 ms | 341 ms | $0.013$12.64 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.5% | — | Decision | 49.2%46.2%–52.1% | c | 553 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.742 | 78.8%answered 43% | 222 ms | 390 ms | $0.006$6.02 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino OmniJev Other Celeris StartLux PelaAI Perplexity Liquid AI Inception Apertix AI Microsoft Base Labs Subquadratic Cloudflare Nace.AI Turion.AI TypeSafe scar-ai Strands Agents quincyqiang newsbang victorbennett Patrick-18 Google tattrongvu Blockbrain Labs Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab MachinePulse Poindexter Labs REAX Logits Zefan Cai
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 23.2% at $0.001 per 1,000 questions ($0.95 per 1M) (estimate)
OneJev1.6-0.8B: 27.0% at $0.001 per 1,000 questions ($1.16 per 1M) (estimate)
GLiClass v4 Edge: 23.6% at $0.001 per 1,000 questions ($1.31 per 1M) (estimate)
celeris-1.1-decision: 47.3% at $0.002 per 1,000 questions ($1.54 per 1M)
GLiDE 0.8B: 29.2% at $0.002 per 1,000 questions ($1.58 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 24.7% at $0.002 per 1,000 questions ($1.79 per 1M) (estimate)
KnowLine-0.8B-Gen1: 26.7% at $0.002 per 1,000 questions ($2.00 per 1M) (estimate)
GLiDE 2B: 32.3% at $0.002 per 1,000 questions ($2.26 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 30.8% at $0.003 per 1,000 questions ($2.52 per 1M) (estimate)
Perplexity Decider Flash v1.2: 43.1% at $0.004 per 1,000 questions ($3.76 per 1M)
KnowLine-2B-Gen1: 31.7% at $0.004 per 1,000 questions ($3.78 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 45.9% at $0.005 per 1,000 questions ($4.62 per 1M) (estimate)
KnowLine 4B Gen5.5: 42.3% at $0.006 per 1,000 questions ($5.89 per 1M) (estimate)
d1.2: 49.2% at $0.006 per 1,000 questions ($6.02 per 1M)
d1.1: 46.0% at $0.006 per 1,000 questions ($6.02 per 1M)
d1: 40.8% at $0.006 per 1,000 questions ($6.02 per 1M)
Perplexity Decider 27B v1.2 (API): 53.3% at $0.007 per 1,000 questions ($7.46 per 1M)
Mercury Decide: 45.1% at $0.008 per 1,000 questions ($7.52 per 1M)
StartLux-Decision-v1.1-9B-Instant: 46.6% at $0.010 per 1,000 questions ($10.04 per 1M) (estimate)
d1-ultra: 54.9% at $0.013 per 1,000 questions ($12.64 per 1M)
d1.2-ultra: 51.6% at $0.013 per 1,000 questions ($12.64 per 1M)
GLiDE 9B: 47.8% at $0.013 per 1,000 questions ($13.29 per 1M) (estimate)
Cerno 9B v1.2: 43.3% at $0.013 per 1,000 questions ($13.35 per 1M) (estimate)
Microsoft-Decision-1: 44.4% at $0.013 per 1,000 questions ($13.42 per 1M)
Elephant-1: 46.4% at $0.013 per 1,000 questions ($13.44 per 1M)
GLiDE (v1.2): 55.2% at $0.014 per 1,000 questions ($13.88 per 1M)
d1-2B: 30.7% at $0.015 per 1,000 questions ($14.76 per 1M) (estimate)
SubQ Snap 1.1: 52.4% at $0.015 per 1,000 questions ($14.92 per 1M)
celeris-1-decision: 47.3% at $0.015 per 1,000 questions ($15.40 per 1M)
Clef-Flash: 38.9% at $0.017 per 1,000 questions ($16.83 per 1M)
Drex 2.0: 41.9% at $0.018 per 1,000 questions ($18.41 per 1M)
Drex Max v1.1: 54.8% at $0.022 per 1,000 questions ($22.37 per 1M)
Drex Max: 53.1% at $0.023 per 1,000 questions ($22.55 per 1M)
Turion Jev 31B: 52.3% at $0.025 per 1,000 questions ($24.91 per 1M) (estimate)
Jev 1.14: 48.2% at $0.026 per 1,000 questions ($25.64 per 1M)
Jev 1.13: 46.3% at $0.026 per 1,000 questions ($25.64 per 1M)
vera-medium-omni: 44.1% at $0.026 per 1,000 questions ($25.82 per 1M) (estimate)
Perplexity Decider Flash v1.3: 44.7% at $0.026 per 1,000 questions ($26.44 per 1M) (estimate)
Strands Decider 2B: 22.4% at $0.028 per 1,000 questions ($27.67 per 1M) (estimate)
Chinese Decision Pointer 4B: 40.4% at $0.030 per 1,000 questions ($29.51 per 1M) (estimate)
Adjudex 9B v5: 46.4% at $0.031 per 1,000 questions ($31.36 per 1M) (estimate)
GLiDE 27B: 58.4% at $0.033 per 1,000 questions ($33.16 per 1M) (estimate)
Verdin-4B-v1.1: 39.7% at $0.033 per 1,000 questions ($33.20 per 1M) (estimate)
Dexy-Decision-9B-V2: 45.3% at $0.034 per 1,000 questions ($33.81 per 1M) (estimate)
KnowLine-9B-Gen1: 44.3% at $0.036 per 1,000 questions ($36.27 per 1M) (estimate)
Gemini Decision 1 (20261111): 50.0% at $0.037 per 1,000 questions ($36.77 per 1M)
Mammoth-1: 54.5% at $0.038 per 1,000 questions ($38.39 per 1M)
Jeffan-9B-1211: 48.2% at $0.039 per 1,000 questions ($39.35 per 1M) (estimate)
Cygnet: 39.3% at $0.040 per 1,000 questions ($39.71 per 1M)
KnowLine-9B-Gen2: 43.9% at $0.040 per 1,000 questions ($39.96 per 1M) (estimate)
Valen-Decision-1.5-9B: 44.4% at $0.042 per 1,000 questions ($42.42 per 1M) (estimate)
KnowLine-12B-Gen1: 50.6% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
Winnow-12B: 39.7% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
GLiDE (API): 54.5% at $0.052 per 1,000 questions ($52.03 per 1M)
d3: 54.5% at $0.057 per 1,000 questions ($57.18 per 1M) (estimate)
KnowLine-31B-Gen1.5: 59.0% at $0.061 per 1,000 questions ($61.43 per 1M) (estimate)
Sieve 31B v3: 57.9% at $0.063 per 1,000 questions ($62.95 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 54.8% at $0.081 per 1,000 questions ($81.16 per 1M) (estimate)
OneJev-27B: 40.6% at $0.086 per 1,000 questions ($86.07 per 1M) (estimate)
Metask-Jev-4B: 32.1% at $0.091 per 1,000 questions ($90.99 per 1M) (estimate)
Clef: 40.8% at $0.106 per 1,000 questions ($106 per 1M)
d3-ultra: 54.6% at $0.109 per 1,000 questions ($109 per 1M) (estimate)
KnowLine-31B-Gen2: 58.9% at $0.112 per 1,000 questions ($112 per 1M) (estimate)
Karpo Decision 31B: 60.7% at $0.128 per 1,000 questions ($128 per 1M) (estimate)
Copernicus-31B: 58.7% at $0.136 per 1,000 questions ($136 per 1M) (estimate)
REAX 1.3: 59.3% at $0.160 per 1,000 questions ($160 per 1M) (estimate)
Logits Sigma-1 Duo: 57.8% at $0.211 per 1,000 questions ($211 per 1M) (estimate)
Open-Jev-27B-v1.1: 42.6% at $0.294 per 1,000 questions ($294 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 52.5% at $1.030 per 1,000 questions ($1,030 per 1M)
Fastino OmniJev Other Celeris StartLux PelaAI Perplexity Liquid AI Inception Apertix AI Microsoft Base Labs Subquadratic Cloudflare Nace.AI Turion.AI TypeSafe scar-ai Strands Agents quincyqiang newsbang victorbennett Patrick-18 Google tattrongvu Blockbrain Labs Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab MachinePulse Poindexter Labs REAX Logits Zefan Cai
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | $0.001$0.95estimate · DGX Spark | $0.95 estimate | $0.001064 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | $0.001$1.16estimate · DGX Spark | $1.16 estimate | $0.001301 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Open · 0.4B | — | $0.001$1.31estimate · DGX Spark | $1.31 estimate | $0.001478 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1.1-decision Pareto Closed · Efficient | — | $0.002$1.54 | $1.54 | $0.001733 | — | provider price × reported usage |
| GLiDE 0.8B Open · 0.8B | — | $0.002$1.58estimate · DGX Spark | $1.58 estimate | $0.001773 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | $0.002$1.79estimate · DGX Spark | $1.79 estimate | $0.00201 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | $0.002$2.00estimate · DGX Spark | $2.00 estimate | $0.002246 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Open · 1.9B | — | $0.002$2.26estimate · DGX Spark | $2.26 estimate | $0.002542 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | $0.003$2.52estimate · DGX Spark | $2.52 estimate | $0.002838 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Open · 4.0B | — | $0.004$3.76 list | $3.76 | $0.004229 | $0.034$34.43estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | $0.004$3.78estimate · DGX Spark | $3.78 estimate | $0.004256 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | — | $0.005$4.62estimate · DGX Spark | $4.62 estimate | $0.005202 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | $0.006$5.89estimate · DGX Spark | $5.89 estimate | $0.006621 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1.2 Pareto Closed · Undisclosed | — | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1 Closed · Efficient | — | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | $0.007$7.46 | $7.46 | $0.00839 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | $0.008$7.52 | $7.52 | $0.008459 | — | provider price × reported usage |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | $0.010$10.04estimate · DGX Spark | $10.04 estimate | $0.01129 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-ultra Pareto Closed · Efficient | — | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
| d1.2-ultra Closed · Undisclosed | — | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
| GLiDE 9B Open · 7.9B | — | $0.013$13.29estimate · DGX Spark | $13.29 estimate | $0.01496 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cerno 9B v1.2 Open · 9.0B | — | $0.013$13.35estimate · DGX Spark | $13.35 estimate | $0.01502 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Microsoft-Decision-1 Closed · Efficient | — | $0.013$13.42 | $13.42 | $0.0151 | — | provider price × reported usage |
| Elephant-1 Open · 9.0B | — | $0.013$13.44 list | $13.44 | $0.01512 | $0.058$58.41estimate · RTX PRO 6000 Blackwell | maker's list price |
| GLiDE Pareto v1.2 Open · 27B | — | $0.014$13.88 list | $13.88 | $0.01561 | $0.082$81.77estimate · RTX PRO 6000 Blackwell | maker's list price |
| d1-2B Open · 1.9B | — | $0.015$14.76estimate · RTX PRO 6000 Blackwell | $14.76 estimate | $0.0166 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| SubQ Snap 1.1 Closed · Efficient | — | $0.015$14.92 | $14.92 | $0.01678 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | $0.015$15.40 | $15.40 | $0.01733 | — | provider price × reported usage |
| Clef-Flash Open · 9.0B | — | $0.017$16.83 list | $16.83 | $0.01894 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | $0.018$18.41 | $18.41 | $0.02072 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | $0.022$22.37 | $22.37 | $0.02517 | — | provider price × reported usage |
| Drex Max Open · 27B | — | $0.023$22.55 list | $22.55 | $0.02537 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | $0.025$24.91estimate · DGX Spark | $24.91 estimate | $0.02802 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Jev 1.14 Closed · Efficient | — | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| vera-medium-omni Open · 4.2B | — | $0.026$25.82estimate · RTX PRO 6000 Blackwell | $25.82 estimate | $0.02905 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | $0.026$26.44estimate · RTX PRO 6000 Blackwell | $26.44 estimate | $0.02974 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | $0.028$27.67estimate · RTX PRO 6000 Blackwell | $27.67 estimate | $0.03113 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | $0.030$29.51estimate · RTX PRO 6000 Blackwell | $29.51 estimate | $0.0332 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Adjudex 9B v5 Open · 9.0B | — | $0.031$31.36estimate · RTX PRO 6000 Blackwell | $31.36 estimate | $0.03528 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE 27B Pareto Open · 27B | — | $0.033$33.16estimate · DGX Spark | $33.16 estimate | $0.0373 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Verdin-4B-v1.1 Open · 4.0B | — | $0.033$33.20estimate · RTX PRO 6000 Blackwell | $33.20 estimate | $0.03735 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | $0.034$33.81estimate · RTX PRO 6000 Blackwell | $33.81 estimate | $0.03804 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | $0.036$36.27estimate · RTX PRO 6000 Blackwell | $36.27 estimate | $0.04081 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemini Decision 1 20261111 Closed · Efficient | — | $0.037$36.77 | $36.77 | $0.04137 | — | provider price × reported usage |
| Mammoth-1 Open · 27B | — | $0.038$38.39 list | $38.39 | $0.04319 | $0.151$151estimate · RTX PRO 6000 Blackwell | maker's list price |
| Jeffan-9B-1211 Open · 9.0B | — | $0.039$39.35estimate · RTX PRO 6000 Blackwell | $39.35 estimate | $0.04427 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | $0.040$39.71 list | $39.71 | $0.04468 | $0.031$30.74estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen2 Open · 9.0B | — | $0.040$39.96estimate · RTX PRO 6000 Blackwell | $39.96 estimate | $0.04496 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | $0.042$42.42estimate · RTX PRO 6000 Blackwell | $42.42 estimate | $0.04773 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Open · 12B | — | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE API Closed · Efficient | — | $0.052$52.03 | $52.03 | $0.05854 | — | provider price × reported usage |
| d3 Open · 26B | — | $0.057$57.18estimate · RTX PRO 6000 Blackwell | $57.18 estimate | $0.06433 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | $0.061$61.43estimate · DGX Spark | $61.43 estimate | $0.06911 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Sieve 31B v3 Open · 31B | — | $0.063$62.95estimate · DGX Spark | $62.95 estimate | $0.07082 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | $0.081$81.16estimate · RTX PRO 6000 Blackwell | $81.16 estimate | $0.0913 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| OneJev-27B Open · 27B | — | $0.086$86.07estimate · RTX PRO 6000 Blackwell | $86.07 estimate | $0.09683 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | $0.091$90.99estimate · RTX PRO 6000 Blackwell | $90.99 estimate | $0.1024 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | $0.106$106 list | $106.31 | $0.1196 | $0.141$141estimate · RTX PRO 6000 Blackwell | maker's list price |
| d3-ultra Open · 26B | — | $0.109$109estimate · RTX PRO 6000 Blackwell | $109.44 estimate | $0.1231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen2 Open · 31B | — | $0.112$112estimate · RTX PRO 6000 Blackwell | $111.90 estimate | $0.1259 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | $0.128$128estimate · RTX PRO 6000 Blackwell | $128.50 estimate | $0.1446 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | $0.136$136estimate · RTX PRO 6000 Blackwell | $135.87 estimate | $0.1529 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Open · 57B | — | $0.160$160estimate · RTX PRO 6000 Blackwell | $160.47 estimate | $0.1805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | $0.211$211estimate · RTX PRO 6000 Blackwell | $211.50 estimate | $0.2379 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | $0.294$294estimate · RTX PRO 6000 Blackwell | $294.50 estimate | $0.3313 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | $1.030$1,030 | $1,029.58 | $1.158 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 1,125 of 1,125); shown once.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.128$128estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 81.6% | — | 60.7%57.8%–63.5% | ab | 683 / 1,125 | 169 right of 172 kept3 wrong · 953 skipped | 0.816 | 6.6%answered 97% | 173 ms | 211 ms | $0.128$128estimate · RTX PRO 6000 Blackwell |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.160$160estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 78.0% | — | 59.3%56.4%–62.1% | ac | 667 / 1,125 | 51 right of 51 kept0 mistakes · 1,074 skipped | 0.816 | 5.3%answered 98% | 215 ms | 278 ms | $0.160$160estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.061$61.43estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.4% | — | 59.0%56.1%–61.9% | abcd | 664 / 1,125 | 89 right of 89 kept0 mistakes · 1,036 skipped | 0.804 | 0.4%answered 100% | 983 ms | 1,279 ms | $0.061$61.43estimate · DGX Spark |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.112$112estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 80.3% | — | 58.9%56.0%–61.8% | abcd | 663 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.825 | 0.0%answered 100% | 151 ms | 204 ms | $0.112$112estimate · RTX PRO 6000 Blackwell |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.136$136estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 77.3% | — | 58.7%55.8%–61.5% | abcd | 660 / 1,125 | 66 right of 66 kept0 mistakes · 1,059 skipped | 0.801 | 1.9%answered 99% | 185 ms | 255 ms | $0.136$136estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.033$33.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 72.0% | — | 58.4%55.5%–61.2% | abcde | 657 / 1,125 | 74 right of 75 kept1 wrong · 1,050 skipped | 0.822 | 44.3%answered 79% | 544 ms | 655 ms | $0.033$33.16estimate · DGX Spark |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.063$62.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.1% | — | 57.9%55.0%–60.7% | abcdf | 651 / 1,125 | 79 right of 81 kept2 wrong · 1,044 skipped | 0.797 | 5.5%answered 98% | 1,050 ms | 1,335 ms | $0.063$62.95estimate · DGX Spark |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.211$211estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 75.2% | — | 57.8%54.9%–60.6% | abcdefg | 650 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.796 | 7.1%answered 97% | 285 ms | 399 ms | $0.211$211estimate · RTX PRO 6000 Blackwell |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 69.9% | — | 55.2%52.3%–58.1% | abcdefgh | 621 / 1,125 | 45 right of 45 kept0 mistakes · 1,080 skipped | 0.793 | 50.2%answered 73% | 111 ms | 140 ms | $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 74.1% | — | 54.9%52.0%–57.8% | abcdefgh | 618 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.783 | 91.0%answered 35% | 182 ms | 425 ms | $0.013$12.64 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.022$22.37 per 1,000 questionsper 1M questions Math 70.1% | — | 54.8%51.8%–57.6% | abcdefghi | 616 / 1,125 | 89 right of 90 kept1 wrong · 1,035 skipped | 0.793 | 54.1%answered 69% | 329 ms | 1,540 ms | $0.022$22.37 |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 72.5% | — | 54.8%51.8%–57.6% | abcdefgh | 616 / 1,125 | 62 right of 62 kept0 mistakes · 1,063 skipped | 0.803 | 18.1%answered 91% | 108 ms | 131 ms | $0.081$81.16estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.109$109estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 71.5% | — | 54.6%51.7%–57.5% | bdefghij | 614 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.778 | 10.1%answered 95% | 156 ms | 185 ms | $0.109$109estimate · RTX PRO 6000 Blackwell |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.052$52.03 per 1,000 questionsper 1M questions Math 70.8% | — | 54.5%51.6%–57.4% | cdefghi | 613 / 1,125 | 52 right of 52 kept0 mistakes · 1,073 skipped | 0.814 | 60.0%answered 67% | 271 ms | 291 ms | $0.052$52.03 |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 74.0% | — | 54.5%51.6%–57.4% | cdefghij | 613 / 1,125 | 106 right of 108 kept2 wrong · 1,017 skipped | 0.795 | 72.3%answered 58% | 221 ms | 244 ms | $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.057$57.18estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.4% | — | 54.5%51.6%–57.4% | cdefgh | 613 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.806 | 14.8%answered 93% | 76 ms | 98 ms | $0.057$57.18estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.007$7.46 per 1,000 questionsper 1M questions Math 69.8% | — | 53.3%50.4%–56.2% | defghijk | 600 / 1,125 | 78 right of 80 kept2 wrong · 1,045 skipped | 0.798 | 14.4%answered 93% | 238 ms | 552 ms | $0.007$7.46 |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.3% | — | 53.1%50.1%–56.0% | defghijk | 597 / 1,125 | 147 right of 155 kept8 wrong · 970 skipped | 0.786 | 7.3%answered 97% | 106 ms | 143 ms | $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.030$1,030 per 1,000 questionsper 1M questions Math 69.3% | with web | 52.5%49.6%–55.4% | fghijkl | 591 / 1,125 | 78 right of 79 kept1 wrong · 1,046 skipped | 0.800 | 37.7%answered 80% | 448 ms | 827 ms | $1.030$1,030 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.015$14.92 per 1,000 questionsper 1M questions Math 68.6% | — | 52.4%49.4%–55.3% | fghijklm | 589 / 1,125 | 70 right of 72 kept2 wrong · 1,053 skipped | 0.787 | 65.7%answered 61% | 384 ms | 393 ms | $0.015$14.92 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.025$24.91estimate · DGX Spark per 1,000 questionsper 1M questions Math 69.0% | — | 52.3%49.3%–55.2% | eghijklmn | 588 / 1,125 | 7 right of 7 kept0 mistakes · 1,118 skipped | 0.782 | 1.5%answered 99% | 385 ms | 533 ms | $0.025$24.91estimate · DGX Spark |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 69.9% | — | 51.6%48.6%–54.5% | hijklmno | 580 / 1,125 | 13 right of 13 kept0 mistakes · 1,112 skipped | 0.800 | 94.5%answered 28% | 220 ms | 341 ms | $0.013$12.64 |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 66.3% | — | 50.6%47.7%–53.5% | hijklmnop | 569 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.774 | 8.8%answered 95% | 68 ms | 88 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.037$36.77 per 1,000 questionsper 1M questions Math 71.3% | — | 50.0%47.1%–53.0% | hijklmnopq | 563 / 1,125 | 43 right of 43 kept0 mistakes · 1,082 skipped | 0.768 | 2.0%answered 99% | 228 ms | 344 ms | $0.037$36.77 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.5% | — | 49.2%46.2%–52.1% | hijklmnopqr | 553 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.742 | 78.8%answered 43% | 222 ms | 390 ms | $0.006$6.02 |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.039$39.35estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | 48.2%45.3%–51.1% | jklmnopqrs | 542 / 1,125 | 10 right of 10 kept0 mistakes · 1,115 skipped | 0.753 | 23.4%answered 87% | 52 ms | 66 ms | $0.039$39.35estimate · RTX PRO 6000 Blackwell |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 65.1% | — | 48.2%45.3%–51.1% | ijklmnopqrs | 542 / 1,125 | 61 right of 61 kept0 mistakes · 1,064 skipped | 0.750 | 24.5%answered 86% | 166 ms | 193 ms | $0.026$25.64 |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.013$13.29estimate · DGX Spark per 1,000 questionsper 1M questions Math 57.9% | — | 47.8%44.9%–50.7% | klmnopqrs | 538 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.749 | 30.4%answered 82% | 208 ms | 251 ms | $0.013$13.29estimate · DGX Spark |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.015$15.40 per 1,000 questionsper 1M questions Math 70.0% | — | 47.3%44.4%–50.2% | klmnopqrstu | 532 / 1,125 | No answers at this threshold | 0.735 | 92.3%answered 28% | 390 ms | 394 ms | $0.015$15.40 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.002$1.54 per 1,000 questionsper 1M questions Math 69.9% | — | 47.3%44.4%–50.2% | klmnopqrst | 532 / 1,125 | No answers at this threshold | 0.733 | 91.0%answered 29% | 383 ms | 390 ms | $0.002$1.54 |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.010$10.04estimate · DGX Spark per 1,000 questionsper 1M questions Math 58.4% | — | 46.6%43.7%–49.5% | lmnopqrstuv | 524 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.762 | 17.4%answered 90% | 160 ms | 175 ms | $0.010$10.04estimate · DGX Spark |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.36estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 62.1% | — | 46.4%43.5%–49.3% | lmnopqrstuv | 522 / 1,125 | 54 right of 55 kept1 wrong · 1,070 skipped | 0.771 | 12.2%answered 93% | 41 ms | 46 ms | $0.031$31.36estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 61.3% | — | 46.4%43.5%–49.3% | lmnopqrstuv | 522 / 1,125 | 53 right of 53 kept0 mistakes · 1,072 skipped | 0.768 | 75.6%answered 47% | 79 ms | 98 ms | $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 64.1% | — | 46.3%43.4%–49.2% | lmnopqrstuv | 521 / 1,125 | 65 right of 65 kept0 mistakes · 1,060 skipped | 0.764 | 15.8%answered 91% | 167 ms | 203 ms | $0.026$25.64 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.2% | — | 46.0%43.2%–49.0% | mnopqrstuvw | 518 / 1,125 | 48 right of 48 kept0 mistakes · 1,077 skipped | 0.748 | 90.6%answered 28% | 176 ms | 272 ms | $0.006$6.02 |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.005$4.62estimate · DGX Spark per 1,000 questionsper 1M questions Math 55.8% | — | 45.9%43.0%–48.8% | nopqrstuvx | 516 / 1,125 | 21 right of 21 kept0 mistakes · 1,104 skipped | 0.720 | 16.0%answered 91% | 73 ms | 84 ms | $0.005$4.62estimate · DGX Spark |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.034$33.81estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 58.8% | — | 45.3%42.4%–48.3% | opqrstuvwxy | 510 / 1,125 | 56 right of 56 kept0 mistakes · 1,069 skipped | 0.759 | 12.4%answered 93% | 47 ms | 53 ms | $0.034$33.81estimate · RTX PRO 6000 Blackwell |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.008$7.52 per 1,000 questionsper 1M questions Math 53.5% | — | 45.1%42.2%–48.0% | pqrstuvwxy | 507 / 1,125 | 84 right of 91 kept7 wrong · 1,034 skipped | 0.746 | 20.5%answered 88% | 301 ms | 360 ms | $0.008$7.52 |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.026$26.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.9% | — | 44.7%41.8%–47.6% | pqrstuvwxy | 503 / 1,125 | 17 right of 17 kept0 mistakes · 1,108 skipped | 0.757 | 24.4%answered 85% | 33 ms | 41 ms | $0.026$26.44estimate · RTX PRO 6000 Blackwell |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.013$13.42 per 1,000 questionsper 1M questions Math 59.4% | — | 44.4%41.6%–47.4% | pqrstuvwxy | 500 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.712 | 19.8%answered 88% | 315 ms | 362 ms | $0.013$13.42 |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.042$42.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.3% | — | 44.4%41.5%–47.3% | pqrstuvwxy | 499 / 1,125 | 35 right of 37 kept2 wrong · 1,088 skipped | 0.701 | 13.6%answered 92% | 60 ms | 67 ms | $0.042$42.42estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.036$36.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | 44.3%41.4%–47.2% | pqrstuvwxy | 498 / 1,125 | 30 right of 30 kept0 mistakes · 1,095 skipped | 0.725 | 6.7%answered 96% | 50 ms | 61 ms | $0.036$36.27estimate · RTX PRO 6000 Blackwell |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.026$25.82estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 56.8% | — | 44.1%41.2%–47.0% | qrstuvwxy | 496 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.739 | 13.6%answered 92% | 32 ms | 37 ms | $0.026$25.82estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.040$39.96estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 60.1% | — | 43.9%41.0%–46.8% | qrstuvwxy | 494 / 1,125 | No answers at this threshold | 0.730 | 19.5%answered 88% | 52 ms | 65 ms | $0.040$39.96estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.013$13.35estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | 43.3%40.4%–46.2% | rstuvwxy | 487 / 1,125 | 26 right of 27 kept1 wrong · 1,098 skipped | 0.752 | 6.0%answered 97% | 209 ms | 269 ms | $0.013$13.35estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.2% | — | 43.1%40.2%–46.0% | qrstuvwxy | 485 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 28.9%answered 82% | 45 ms | 55 ms | $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.294$294estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 54.6% | — | 42.6%39.7%–45.5% | rstuvwxy | 479 / 1,125 | No answers at this threshold | 0.733 | 28.2%answered 82% | 413 ms | 576 ms | $0.294$294estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.006$5.89estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | 42.3%39.5%–45.2% | stuvwxy | 476 / 1,125 | 17 right of 18 kept1 wrong · 1,107 skipped | 0.708 | 8.3%answered 95% | 96 ms | 115 ms | $0.006$5.89estimate · DGX Spark |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.018$18.41 per 1,000 questionsper 1M questions Math 59.0% | — | 41.9%39.0%–44.8% | stuvwxy | 471 / 1,125 | 33 right of 33 kept0 mistakes · 1,092 skipped | 0.717 | 37.7%answered 75% | 345 ms | 504 ms | $0.018$18.41 |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.1% | — | 40.8%38.0%–43.7% | uvwxy | 459 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.697 | 3.6%answered 98% | 196 ms | 222 ms | $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 55.6% | — | 40.8%38.0%–43.7% | tuvwxy | 459 / 1,125 | 55 right of 56 kept1 wrong · 1,069 skipped | 0.723 | 11.3%answered 93% | 188 ms | 290 ms | $0.006$6.02 |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.086$86.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 53.8% | — | 40.6%37.8%–43.5% | tuvwxy | 457 / 1,125 | 4 right of 4 kept0 mistakes · 1,121 skipped | 0.726 | 1.2%answered 99% | 108 ms | 136 ms | $0.086$86.07estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.030$29.51estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 50.4% | — | 40.4%37.5%–43.3% | vwxy | 454 / 1,125 | 39 right of 39 kept0 mistakes · 1,086 skipped | 0.724 | 14.6%answered 91% | 38 ms | 48 ms | $0.030$29.51estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.5% | — | 39.7%36.9%–42.6% | wy | 447 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 12.6%answered 92% | 45 ms | 51 ms | $0.033$33.20estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 46.9% | — | 39.7%36.9%–42.6% | xy | 447 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.727 | 29.7%answered 80% | 66 ms | 91 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 44.3% | — | 39.3%36.5%–42.2% | xy | 442 / 1,125 | No answers at this threshold | 0.720 | 11.3%answered 93% | 40 ms | 48 ms | $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 49.8% | — | 38.9%36.1%–41.8% | yz | 438 / 1,125 | No answers at this threshold | 0.659 | 14.4%answered 91% | 110 ms | 137 ms | $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$2.26estimate · DGX Spark per 1,000 questionsper 1M questions Math 36.8% | — | 32.3%29.6%–35.1% | zA | 363 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.632 | 32.9%answered 75% | 34 ms | 42 ms | $0.002$2.26estimate · DGX Spark |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.091$90.99estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 39.2% | — | 32.1%29.4%–34.9% | zA | 361 / 1,125 | No answers at this threshold | 0.660 | 1.5%answered 99% | 123 ms | 152 ms | $0.091$90.99estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.004$3.78estimate · DGX Spark per 1,000 questionsper 1M questions Math 35.3% | — | 31.7%29.1%–34.5% | A | 357 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.601 | 15.7%answered 89% | 59 ms | 70 ms | $0.004$3.78estimate · DGX Spark |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.003$2.52estimate · DGX Spark per 1,000 questionsper 1M questions Math 38.7% | — | 30.8%28.1%–33.5% | AB | 346 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.640 | 31.4%answered 76% | 40 ms | 45 ms | $0.003$2.52estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.015$14.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 37.1% | — | 30.7%28.0%–33.4% | AB | 345 / 1,125 | 3 right of 3 kept0 mistakes · 1,122 skipped | 0.620 | 86.5%answered 22% | 17 ms | 23 ms | $0.015$14.76estimate · RTX PRO 6000 Blackwell |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.002$1.58estimate · DGX Spark per 1,000 questionsper 1M questions Math 32.1% | — | 29.2%26.6%–31.9% | ABC | 328 / 1,125 | 9 right of 9 kept0 mistakes · 1,116 skipped | 0.617 | 26.1%answered 80% | 23 ms | 29 ms | $0.002$1.58estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | 27.0%24.5%–29.7% | ABC | 304 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.545 | 4.6%answered 97% | 16 ms | 19 ms | $0.001$1.16estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$2.00estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | 26.7%24.2%–29.3% | ABC | 300 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.573 | 1.2%answered 99% | 31 ms | 39 ms | $0.002$2.00estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.79estimate · DGX Spark per 1,000 questionsper 1M questions Math 29.7% | — | 24.7%22.3%–27.3% | BC | 278 / 1,125 | No answers at this threshold | 0.641 | 21.4%answered 83% | 27 ms | 31 ms | $0.002$1.79estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.31estimate · DGX Spark per 1,000 questionsper 1M questions Math 22.7% | — | 23.6%21.2%–26.1% | BC | 265 / 1,125 | No answers at this threshold | 0.527 | 12.4%answered 90% | 15 ms | 28 ms | $0.001$1.31estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 20.6% | — | 23.2%20.8%–25.8% | C | 261 / 1,125 | No answers at this threshold | 0.556 | 76.1%answered 30% | 13 ms | 16 ms | $0.001$0.95estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.028$27.67estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 28.7% | — | 22.4%20.1%–24.9% | C | 252 / 1,125 | No answers at this threshold | 0.568 | 8.3%answered 94% | 36 ms | 43 ms | $0.028$27.67estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino OmniJev Other StartLux PelaAI Perplexity Liquid AI scar-ai Strands Agents quincyqiang victorbennett Metask Lab
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 23.2% at $0.001 per 1,000 questions ($0.95 per 1M) (estimate)
OneJev1.6-0.8B: 27.0% at $0.001 per 1,000 questions ($1.16 per 1M) (estimate)
GLiClass v4 Edge: 23.6% at $0.001 per 1,000 questions ($1.31 per 1M) (estimate)
GLiDE 0.8B: 29.2% at $0.002 per 1,000 questions ($1.58 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 24.7% at $0.002 per 1,000 questions ($1.79 per 1M) (estimate)
KnowLine-0.8B-Gen1: 26.7% at $0.002 per 1,000 questions ($2.00 per 1M) (estimate)
GLiDE 2B: 32.3% at $0.002 per 1,000 questions ($2.26 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 30.8% at $0.003 per 1,000 questions ($2.52 per 1M) (estimate)
Perplexity Decider Flash v1.2: 43.1% at $0.004 per 1,000 questions ($3.76 per 1M)
KnowLine-2B-Gen1: 31.7% at $0.004 per 1,000 questions ($3.78 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 45.9% at $0.005 per 1,000 questions ($4.62 per 1M) (estimate)
KnowLine 4B Gen5.5: 42.3% at $0.006 per 1,000 questions ($5.89 per 1M) (estimate)
d1-2B: 30.7% at $0.015 per 1,000 questions ($14.76 per 1M) (estimate)
vera-medium-omni: 44.1% at $0.026 per 1,000 questions ($25.82 per 1M) (estimate)
Perplexity Decider Flash v1.3: 44.7% at $0.026 per 1,000 questions ($26.44 per 1M) (estimate)
Strands Decider 2B: 22.4% at $0.028 per 1,000 questions ($27.67 per 1M) (estimate)
Chinese Decision Pointer 4B: 40.4% at $0.030 per 1,000 questions ($29.51 per 1M) (estimate)
Verdin-4B-v1.1: 39.7% at $0.033 per 1,000 questions ($33.20 per 1M) (estimate)
Metask-Jev-4B: 32.1% at $0.091 per 1,000 questions ($90.99 per 1M) (estimate)
Fastino OmniJev Other StartLux PelaAI Perplexity Liquid AI scar-ai Strands Agents quincyqiang victorbennett Metask Lab
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | $0.001$0.95estimate · DGX Spark | $0.95 estimate | $0.001064 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | $0.001$1.16estimate · DGX Spark | $1.16 estimate | $0.001301 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Open · 0.4B | — | $0.001$1.31estimate · DGX Spark | $1.31 estimate | $0.001478 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Pareto Open · 0.8B | — | $0.002$1.58estimate · DGX Spark | $1.58 estimate | $0.001773 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | $0.002$1.79estimate · DGX Spark | $1.79 estimate | $0.00201 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | $0.002$2.00estimate · DGX Spark | $2.00 estimate | $0.002246 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Pareto Open · 1.9B | — | $0.002$2.26estimate · DGX Spark | $2.26 estimate | $0.002542 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | $0.003$2.52estimate · DGX Spark | $2.52 estimate | $0.002838 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Pareto Open · 4.0B | — | $0.004$3.76 list | $3.76 | $0.004229 | $0.034$34.43estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | $0.004$3.78estimate · DGX Spark | $3.78 estimate | $0.004256 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Pareto Open · 4.0B | — | $0.005$4.62estimate · DGX Spark | $4.62 estimate | $0.005202 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | $0.006$5.89estimate · DGX Spark | $5.89 estimate | $0.006621 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-2B Open · 1.9B | — | $0.015$14.76estimate · RTX PRO 6000 Blackwell | $14.76 estimate | $0.0166 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Open · 4.2B | — | $0.026$25.82estimate · RTX PRO 6000 Blackwell | $25.82 estimate | $0.02905 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | $0.026$26.44estimate · RTX PRO 6000 Blackwell | $26.44 estimate | $0.02974 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | $0.028$27.67estimate · RTX PRO 6000 Blackwell | $27.67 estimate | $0.03113 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | $0.030$29.51estimate · RTX PRO 6000 Blackwell | $29.51 estimate | $0.0332 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | $0.033$33.20estimate · RTX PRO 6000 Blackwell | $33.20 estimate | $0.03735 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | $0.091$90.99estimate · RTX PRO 6000 Blackwell | $90.99 estimate | $0.1024 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.005$4.62estimate · DGX Spark per 1,000 questionsper 1M questions Math 55.8% | — | 45.9%43.0%–48.8% | a | 516 / 1,125 | 21 right of 21 kept0 mistakes · 1,104 skipped | 0.720 | 16.0%answered 91% | 73 ms | 84 ms | $0.005$4.62estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.026$26.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.9% | — | 44.7%41.8%–47.6% | ab | 503 / 1,125 | 17 right of 17 kept0 mistakes · 1,108 skipped | 0.757 | 24.4%answered 85% | 33 ms | 41 ms | $0.026$26.44estimate · RTX PRO 6000 Blackwell |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.026$25.82estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 56.8% | — | 44.1%41.2%–47.0% | abc | 496 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.739 | 13.6%answered 92% | 32 ms | 37 ms | $0.026$25.82estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.2% | — | 43.1%40.2%–46.0% | abc | 485 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 28.9%answered 82% | 45 ms | 55 ms | $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.006$5.89estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | 42.3%39.5%–45.2% | abc | 476 / 1,125 | 17 right of 18 kept1 wrong · 1,107 skipped | 0.708 | 8.3%answered 95% | 96 ms | 115 ms | $0.006$5.89estimate · DGX Spark |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.030$29.51estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 50.4% | — | 40.4%37.5%–43.3% | bc | 454 / 1,125 | 39 right of 39 kept0 mistakes · 1,086 skipped | 0.724 | 14.6%answered 91% | 38 ms | 48 ms | $0.030$29.51estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.5% | — | 39.7%36.9%–42.6% | c | 447 / 1,125 | 35 right of 35 kept0 mistakes · 1,090 skipped | 0.729 | 12.6%answered 92% | 45 ms | 51 ms | $0.033$33.20estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$2.26estimate · DGX Spark per 1,000 questionsper 1M questions Math 36.8% | — | 32.3%29.6%–35.1% | d | 363 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.632 | 32.9%answered 75% | 34 ms | 42 ms | $0.002$2.26estimate · DGX Spark |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.091$90.99estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 39.2% | — | 32.1%29.4%–34.9% | d | 361 / 1,125 | No answers at this threshold | 0.660 | 1.5%answered 99% | 123 ms | 152 ms | $0.091$90.99estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.004$3.78estimate · DGX Spark per 1,000 questionsper 1M questions Math 35.3% | — | 31.7%29.1%–34.5% | d | 357 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.601 | 15.7%answered 89% | 59 ms | 70 ms | $0.004$3.78estimate · DGX Spark |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.003$2.52estimate · DGX Spark per 1,000 questionsper 1M questions Math 38.7% | — | 30.8%28.1%–33.5% | d | 346 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.640 | 31.4%answered 76% | 40 ms | 45 ms | $0.003$2.52estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.015$14.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 37.1% | — | 30.7%28.0%–33.4% | d | 345 / 1,125 | 3 right of 3 kept0 mistakes · 1,122 skipped | 0.620 | 86.5%answered 22% | 17 ms | 23 ms | $0.015$14.76estimate · RTX PRO 6000 Blackwell |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.002$1.58estimate · DGX Spark per 1,000 questionsper 1M questions Math 32.1% | — | 29.2%26.6%–31.9% | de | 328 / 1,125 | 9 right of 9 kept0 mistakes · 1,116 skipped | 0.617 | 26.1%answered 80% | 23 ms | 29 ms | $0.002$1.58estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | 27.0%24.5%–29.7% | def | 304 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.545 | 4.6%answered 97% | 16 ms | 19 ms | $0.001$1.16estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$2.00estimate · DGX Spark per 1,000 questionsper 1M questions Math 28.2% | — | 26.7%24.2%–29.3% | def | 300 / 1,125 | 2 right of 2 kept0 mistakes · 1,123 skipped | 0.573 | 1.2%answered 99% | 31 ms | 39 ms | $0.002$2.00estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.79estimate · DGX Spark per 1,000 questionsper 1M questions Math 29.7% | — | 24.7%22.3%–27.3% | ef | 278 / 1,125 | No answers at this threshold | 0.641 | 21.4%answered 83% | 27 ms | 31 ms | $0.002$1.79estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.31estimate · DGX Spark per 1,000 questionsper 1M questions Math 22.7% | — | 23.6%21.2%–26.1% | ef | 265 / 1,125 | No answers at this threshold | 0.527 | 12.4%answered 90% | 15 ms | 28 ms | $0.001$1.31estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 20.6% | — | 23.2%20.8%–25.8% | f | 261 / 1,125 | No answers at this threshold | 0.556 | 76.1%answered 30% | 13 ms | 16 ms | $0.001$0.95estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.028$27.67estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 28.7% | — | 22.4%20.1%–24.9% | f | 252 / 1,125 | No answers at this threshold | 0.568 | 8.3%answered 94% | 36 ms | 43 ms | $0.028$27.67estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
StartLux Fastino Apertix AI Base Labs Cloudflare newsbang Patrick-18 PelaAI tattrongvu Blockbrain Labs Valen-Team EldanRing
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
StartLux-Decision-v1.1-9B-Instant: 46.6% at $0.010 per 1,000 questions ($10.04 per 1M) (estimate)
GLiDE 9B: 47.8% at $0.013 per 1,000 questions ($13.29 per 1M) (estimate)
Cerno 9B v1.2: 43.3% at $0.013 per 1,000 questions ($13.35 per 1M) (estimate)
Elephant-1: 46.4% at $0.013 per 1,000 questions ($13.44 per 1M)
Clef-Flash: 38.9% at $0.017 per 1,000 questions ($16.83 per 1M)
Adjudex 9B v5: 46.4% at $0.031 per 1,000 questions ($31.36 per 1M) (estimate)
Dexy-Decision-9B-V2: 45.3% at $0.034 per 1,000 questions ($33.81 per 1M) (estimate)
KnowLine-9B-Gen1: 44.3% at $0.036 per 1,000 questions ($36.27 per 1M) (estimate)
Jeffan-9B-1211: 48.2% at $0.039 per 1,000 questions ($39.35 per 1M) (estimate)
Cygnet: 39.3% at $0.040 per 1,000 questions ($39.71 per 1M)
KnowLine-9B-Gen2: 43.9% at $0.040 per 1,000 questions ($39.96 per 1M) (estimate)
Valen-Decision-1.5-9B: 44.4% at $0.042 per 1,000 questions ($42.42 per 1M) (estimate)
KnowLine-12B-Gen1: 50.6% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
Winnow-12B: 39.7% at $0.052 per 1,000 questions ($51.64 per 1M) (estimate)
StartLux Fastino Apertix AI Base Labs Cloudflare newsbang Patrick-18 PelaAI tattrongvu Blockbrain Labs Valen-Team EldanRing
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| StartLux-Decision-v1.1-9B-Instant Pareto Open · 9.0B | — | $0.010$10.04estimate · DGX Spark | $10.04 estimate | $0.01129 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Pareto Open · 7.9B | — | $0.013$13.29estimate · DGX Spark | $13.29 estimate | $0.01496 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cerno 9B v1.2 Open · 9.0B | — | $0.013$13.35estimate · DGX Spark | $13.35 estimate | $0.01502 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Elephant-1 Open · 9.0B | — | $0.013$13.44 list | $13.44 | $0.01512 | $0.058$58.41estimate · RTX PRO 6000 Blackwell | maker's list price |
| Clef-Flash Open · 9.0B | — | $0.017$16.83 list | $16.83 | $0.01894 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Adjudex 9B v5 Open · 9.0B | — | $0.031$31.36estimate · RTX PRO 6000 Blackwell | $31.36 estimate | $0.03528 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | $0.034$33.81estimate · RTX PRO 6000 Blackwell | $33.81 estimate | $0.03804 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | $0.036$36.27estimate · RTX PRO 6000 Blackwell | $36.27 estimate | $0.04081 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Pareto Open · 9.0B | — | $0.039$39.35estimate · RTX PRO 6000 Blackwell | $39.35 estimate | $0.04427 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | $0.040$39.71 list | $39.71 | $0.04468 | $0.031$30.74estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen2 Open · 9.0B | — | $0.040$39.96estimate · RTX PRO 6000 Blackwell | $39.96 estimate | $0.04496 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | $0.042$42.42estimate · RTX PRO 6000 Blackwell | $42.42 estimate | $0.04773 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Pareto Open · 12B | — | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | $0.052$51.64estimate · RTX PRO 6000 Blackwell | $51.64 estimate | $0.0581 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 66.3% | — | 50.6%47.7%–53.5% | a | 569 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.774 | 8.8%answered 95% | 68 ms | 88 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.039$39.35estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | 48.2%45.3%–51.1% | ab | 542 / 1,125 | 10 right of 10 kept0 mistakes · 1,115 skipped | 0.753 | 23.4%answered 87% | 52 ms | 66 ms | $0.039$39.35estimate · RTX PRO 6000 Blackwell |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.013$13.29estimate · DGX Spark per 1,000 questionsper 1M questions Math 57.9% | — | 47.8%44.9%–50.7% | ab | 538 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.749 | 30.4%answered 82% | 208 ms | 251 ms | $0.013$13.29estimate · DGX Spark |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.010$10.04estimate · DGX Spark per 1,000 questionsper 1M questions Math 58.4% | — | 46.6%43.7%–49.5% | ab | 524 / 1,125 | 14 right of 14 kept0 mistakes · 1,111 skipped | 0.762 | 17.4%answered 90% | 160 ms | 175 ms | $0.010$10.04estimate · DGX Spark |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.36estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 62.1% | — | 46.4%43.5%–49.3% | ab | 522 / 1,125 | 54 right of 55 kept1 wrong · 1,070 skipped | 0.771 | 12.2%answered 93% | 41 ms | 46 ms | $0.031$31.36estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 61.3% | — | 46.4%43.5%–49.3% | ab | 522 / 1,125 | 53 right of 53 kept0 mistakes · 1,072 skipped | 0.768 | 75.6%answered 47% | 79 ms | 98 ms | $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.034$33.81estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 58.8% | — | 45.3%42.4%–48.3% | ab | 510 / 1,125 | 56 right of 56 kept0 mistakes · 1,069 skipped | 0.759 | 12.4%answered 93% | 47 ms | 53 ms | $0.034$33.81estimate · RTX PRO 6000 Blackwell |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.042$42.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 55.3% | — | 44.4%41.5%–47.3% | bc | 499 / 1,125 | 35 right of 37 kept2 wrong · 1,088 skipped | 0.701 | 13.6%answered 92% | 60 ms | 67 ms | $0.042$42.42estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.036$36.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 59.8% | — | 44.3%41.4%–47.2% | bc | 498 / 1,125 | 30 right of 30 kept0 mistakes · 1,095 skipped | 0.725 | 6.7%answered 96% | 50 ms | 61 ms | $0.036$36.27estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.040$39.96estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 60.1% | — | 43.9%41.0%–46.8% | bc | 494 / 1,125 | No answers at this threshold | 0.730 | 19.5%answered 88% | 52 ms | 65 ms | $0.040$39.96estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.013$13.35estimate · DGX Spark per 1,000 questionsper 1M questions Math 56.0% | — | 43.3%40.4%–46.2% | bc | 487 / 1,125 | 26 right of 27 kept1 wrong · 1,098 skipped | 0.752 | 6.0%answered 97% | 209 ms | 269 ms | $0.013$13.35estimate · DGX Spark |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 46.9% | — | 39.7%36.9%–42.6% | c | 447 / 1,125 | 32 right of 32 kept0 mistakes · 1,093 skipped | 0.727 | 29.7%answered 80% | 66 ms | 91 ms | $0.052$51.64estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 44.3% | — | 39.3%36.5%–42.2% | c | 442 / 1,125 | No answers at this threshold | 0.720 | 11.3%answered 93% | 40 ms | 48 ms | $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 49.8% | — | 38.9%36.1%–41.8% | c | 438 / 1,125 | No answers at this threshold | 0.659 | 14.4%answered 91% | 110 ms | 137 ms | $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router PelaAI Sthānika AI OmniJev Cloudflare MachinePulse Poindexter Labs Zefan Cai
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Perplexity Decider 27B v1.2 (API): 53.3% at $0.007 per 1,000 questions ($7.46 per 1M)
GLiDE (v1.2): 55.2% at $0.014 per 1,000 questions ($13.88 per 1M)
Drex 2.0: 41.9% at $0.018 per 1,000 questions ($18.41 per 1M)
Drex Max: 53.1% at $0.023 per 1,000 questions ($22.55 per 1M)
Turion Jev 31B: 52.3% at $0.025 per 1,000 questions ($24.91 per 1M) (estimate)
GLiDE 27B: 58.4% at $0.033 per 1,000 questions ($33.16 per 1M) (estimate)
Mammoth-1: 54.5% at $0.038 per 1,000 questions ($38.39 per 1M)
d3: 54.5% at $0.057 per 1,000 questions ($57.18 per 1M) (estimate)
KnowLine-31B-Gen1.5: 59.0% at $0.061 per 1,000 questions ($61.43 per 1M) (estimate)
Sieve 31B v3: 57.9% at $0.063 per 1,000 questions ($62.95 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 54.8% at $0.081 per 1,000 questions ($81.16 per 1M) (estimate)
OneJev-27B: 40.6% at $0.086 per 1,000 questions ($86.07 per 1M) (estimate)
Clef: 40.8% at $0.106 per 1,000 questions ($106 per 1M)
d3-ultra: 54.6% at $0.109 per 1,000 questions ($109 per 1M) (estimate)
KnowLine-31B-Gen2: 58.9% at $0.112 per 1,000 questions ($112 per 1M) (estimate)
Karpo Decision 31B: 60.7% at $0.128 per 1,000 questions ($128 per 1M) (estimate)
Copernicus-31B: 58.7% at $0.136 per 1,000 questions ($136 per 1M) (estimate)
Open-Jev-27B-v1.1: 42.6% at $0.294 per 1,000 questions ($294 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 52.5% at $1.030 per 1,000 questions ($1,030 per 1M)
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router PelaAI Sthānika AI OmniJev Cloudflare MachinePulse Poindexter Labs Zefan Cai
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | $0.007$7.46 | $7.46 | $0.00839 | — | provider price × reported usage |
| GLiDE Pareto v1.2 Open · 27B | — | $0.014$13.88 list | $13.88 | $0.01561 | $0.082$81.77estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | $0.018$18.41 | $18.41 | $0.02072 | — | provider price × reported usage |
| Drex Max Open · 27B | — | $0.023$22.55 list | $22.55 | $0.02537 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | $0.025$24.91estimate · DGX Spark | $24.91 estimate | $0.02802 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 27B Pareto Open · 27B | — | $0.033$33.16estimate · DGX Spark | $33.16 estimate | $0.0373 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Mammoth-1 Open · 27B | — | $0.038$38.39 list | $38.39 | $0.04319 | $0.151$151estimate · RTX PRO 6000 Blackwell | maker's list price |
| d3 Open · 26B | — | $0.057$57.18estimate · RTX PRO 6000 Blackwell | $57.18 estimate | $0.06433 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | $0.061$61.43estimate · DGX Spark | $61.43 estimate | $0.06911 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Sieve 31B v3 Open · 31B | — | $0.063$62.95estimate · DGX Spark | $62.95 estimate | $0.07082 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | $0.081$81.16estimate · RTX PRO 6000 Blackwell | $81.16 estimate | $0.0913 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| OneJev-27B Open · 27B | — | $0.086$86.07estimate · RTX PRO 6000 Blackwell | $86.07 estimate | $0.09683 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | $0.106$106 list | $106.31 | $0.1196 | $0.141$141estimate · RTX PRO 6000 Blackwell | maker's list price |
| d3-ultra Open · 26B | — | $0.109$109estimate · RTX PRO 6000 Blackwell | $109.44 estimate | $0.1231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen2 Open · 31B | — | $0.112$112estimate · RTX PRO 6000 Blackwell | $111.90 estimate | $0.1259 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | $0.128$128estimate · RTX PRO 6000 Blackwell | $128.50 estimate | $0.1446 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | $0.136$136estimate · RTX PRO 6000 Blackwell | $135.87 estimate | $0.1529 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | $0.294$294estimate · RTX PRO 6000 Blackwell | $294.50 estimate | $0.3313 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | $1.030$1,030 | $1,029.58 | $1.158 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.128$128estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 81.6% | — | 60.7%57.8%–63.5% | a | 683 / 1,125 | 169 right of 172 kept3 wrong · 953 skipped | 0.816 | 6.6%answered 97% | 173 ms | 211 ms | $0.128$128estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.061$61.43estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.4% | — | 59.0%56.1%–61.9% | ab | 664 / 1,125 | 89 right of 89 kept0 mistakes · 1,036 skipped | 0.804 | 0.4%answered 100% | 983 ms | 1,279 ms | $0.061$61.43estimate · DGX Spark |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.112$112estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 80.3% | — | 58.9%56.0%–61.8% | ab | 663 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.825 | 0.0%answered 100% | 151 ms | 204 ms | $0.112$112estimate · RTX PRO 6000 Blackwell |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.136$136estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 77.3% | — | 58.7%55.8%–61.5% | ab | 660 / 1,125 | 66 right of 66 kept0 mistakes · 1,059 skipped | 0.801 | 1.9%answered 99% | 185 ms | 255 ms | $0.136$136estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.033$33.16estimate · DGX Spark per 1,000 questionsper 1M questions Math 72.0% | — | 58.4%55.5%–61.2% | ab | 657 / 1,125 | 74 right of 75 kept1 wrong · 1,050 skipped | 0.822 | 44.3%answered 79% | 544 ms | 655 ms | $0.033$33.16estimate · DGX Spark |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.063$62.95estimate · DGX Spark per 1,000 questionsper 1M questions Math 77.1% | — | 57.9%55.0%–60.7% | abc | 651 / 1,125 | 79 right of 81 kept2 wrong · 1,044 skipped | 0.797 | 5.5%answered 98% | 1,050 ms | 1,335 ms | $0.063$62.95estimate · DGX Spark |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 69.9% | — | 55.2%52.3%–58.1% | bcd | 621 / 1,125 | 45 right of 45 kept0 mistakes · 1,080 skipped | 0.793 | 50.2%answered 73% | 111 ms | 140 ms | $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 72.5% | — | 54.8%51.8%–57.6% | bcd | 616 / 1,125 | 62 right of 62 kept0 mistakes · 1,063 skipped | 0.803 | 18.1%answered 91% | 108 ms | 131 ms | $0.081$81.16estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.109$109estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 71.5% | — | 54.6%51.7%–57.5% | bcd | 614 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.778 | 10.1%answered 95% | 156 ms | 185 ms | $0.109$109estimate · RTX PRO 6000 Blackwell |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 74.0% | — | 54.5%51.6%–57.4% | bcd | 613 / 1,125 | 106 right of 108 kept2 wrong · 1,017 skipped | 0.795 | 72.3%answered 58% | 221 ms | 244 ms | $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.057$57.18estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.4% | — | 54.5%51.6%–57.4% | bcd | 613 / 1,125 | 54 right of 54 kept0 mistakes · 1,071 skipped | 0.806 | 14.8%answered 93% | 76 ms | 98 ms | $0.057$57.18estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.007$7.46 per 1,000 questionsper 1M questions Math 69.8% | — | 53.3%50.4%–56.2% | cd | 600 / 1,125 | 78 right of 80 kept2 wrong · 1,045 skipped | 0.798 | 14.4%answered 93% | 238 ms | 552 ms | $0.007$7.46 |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 70.3% | — | 53.1%50.1%–56.0% | cd | 597 / 1,125 | 147 right of 155 kept8 wrong · 970 skipped | 0.786 | 7.3%answered 97% | 106 ms | 143 ms | $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.030$1,030 per 1,000 questionsper 1M questions Math 69.3% | with web | 52.5%49.6%–55.4% | d | 591 / 1,125 | 78 right of 79 kept1 wrong · 1,046 skipped | 0.800 | 37.7%answered 80% | 448 ms | 827 ms | $1.030$1,030 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.025$24.91estimate · DGX Spark per 1,000 questionsper 1M questions Math 69.0% | — | 52.3%49.3%–55.2% | d | 588 / 1,125 | 7 right of 7 kept0 mistakes · 1,118 skipped | 0.782 | 1.5%answered 99% | 385 ms | 533 ms | $0.025$24.91estimate · DGX Spark |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.294$294estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 54.6% | — | 42.6%39.7%–45.5% | e | 479 / 1,125 | No answers at this threshold | 0.733 | 28.2%answered 82% | 413 ms | 576 ms | $0.294$294estimate · RTX PRO 6000 Blackwell |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.018$18.41 per 1,000 questionsper 1M questions Math 59.0% | — | 41.9%39.0%–44.8% | e | 471 / 1,125 | 33 right of 33 kept0 mistakes · 1,092 skipped | 0.717 | 37.7%answered 75% | 345 ms | 504 ms | $0.018$18.41 |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 51.1% | — | 40.8%38.0%–43.7% | e | 459 / 1,125 | 6 right of 6 kept0 mistakes · 1,119 skipped | 0.697 | 3.6%answered 98% | 196 ms | 222 ms | $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.086$86.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 53.8% | — | 40.6%37.8%–43.5% | e | 457 / 1,125 | 4 right of 4 kept0 mistakes · 1,121 skipped | 0.726 | 1.2%answered 99% | 108 ms | 136 ms | $0.086$86.07estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
REAX Logits
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
REAX 1.3: 59.3% at $0.160 per 1,000 questions ($160 per 1M) (estimate)
Logits Sigma-1 Duo: 57.8% at $0.211 per 1,000 questions ($211 per 1M) (estimate)
REAX Logits
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| REAX 1.3 Pareto Open · 57B | — | $0.160$160estimate · RTX PRO 6000 Blackwell | $160.47 estimate | $0.1805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | $0.211$211estimate · RTX PRO 6000 Blackwell | $211.50 estimate | $0.2379 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.160$160estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 78.0% | — | 59.3%56.4%–62.1% | a | 667 / 1,125 | 51 right of 51 kept0 mistakes · 1,074 skipped | 0.816 | 5.3%answered 98% | 215 ms | 278 ms | $0.160$160estimate · RTX PRO 6000 Blackwell |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.211$211estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 75.2% | — | 57.8%54.9%–60.6% | a | 650 / 1,125 | 8 right of 8 kept0 mistakes · 1,117 skipped | 0.796 | 7.1%answered 97% | 285 ms | 399 ms | $0.211$211estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
celeris-1.1-decision: 47.3% at $0.002 per 1,000 questions ($1.54 per 1M)
d1.1: 46.0% at $0.006 per 1,000 questions ($6.02 per 1M)
d1: 40.8% at $0.006 per 1,000 questions ($6.02 per 1M)
Mercury Decide: 45.1% at $0.008 per 1,000 questions ($7.52 per 1M)
d1-ultra: 54.9% at $0.013 per 1,000 questions ($12.64 per 1M)
Microsoft-Decision-1: 44.4% at $0.013 per 1,000 questions ($13.42 per 1M)
SubQ Snap 1.1: 52.4% at $0.015 per 1,000 questions ($14.92 per 1M)
celeris-1-decision: 47.3% at $0.015 per 1,000 questions ($15.40 per 1M)
Drex Max v1.1: 54.8% at $0.022 per 1,000 questions ($22.37 per 1M)
Jev 1.14: 48.2% at $0.026 per 1,000 questions ($25.64 per 1M)
Jev 1.13: 46.3% at $0.026 per 1,000 questions ($25.64 per 1M)
Gemini Decision 1 (20261111): 50.0% at $0.037 per 1,000 questions ($36.77 per 1M)
GLiDE (API): 54.5% at $0.052 per 1,000 questions ($52.03 per 1M)
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| celeris-1.1-decision Pareto Closed · Efficient | — | $0.002$1.54 | $1.54 | $0.001733 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1 Closed · Efficient | — | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | $0.008$7.52 | $7.52 | $0.008459 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | $0.013$13.42 | $13.42 | $0.0151 | — | provider price × reported usage |
| SubQ Snap 1.1 Closed · Efficient | — | $0.015$14.92 | $14.92 | $0.01678 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | $0.015$15.40 | $15.40 | $0.01733 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | $0.022$22.37 | $22.37 | $0.02517 | — | provider price × reported usage |
| Jev 1.14 Closed · Efficient | — | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | $0.026$25.64 | $25.64 | $0.02885 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | $0.037$36.77 | $36.77 | $0.04137 | — | provider price × reported usage |
| GLiDE API Closed · Efficient | — | $0.052$52.03 | $52.03 | $0.05854 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 1,125 of 1,125); shown once.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 74.1% | — | 54.9%52.0%–57.8% | a | 618 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.783 | 91.0%answered 35% | 182 ms | 425 ms | $0.013$12.64 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.022$22.37 per 1,000 questionsper 1M questions Math 70.1% | — | 54.8%51.8%–57.6% | a | 616 / 1,125 | 89 right of 90 kept1 wrong · 1,035 skipped | 0.793 | 54.1%answered 69% | 329 ms | 1,540 ms | $0.022$22.37 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.052$52.03 per 1,000 questionsper 1M questions Math 70.8% | — | 54.5%51.6%–57.4% | a | 613 / 1,125 | 52 right of 52 kept0 mistakes · 1,073 skipped | 0.814 | 60.0%answered 67% | 271 ms | 291 ms | $0.052$52.03 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.015$14.92 per 1,000 questionsper 1M questions Math 68.6% | — | 52.4%49.4%–55.3% | ab | 589 / 1,125 | 70 right of 72 kept2 wrong · 1,053 skipped | 0.787 | 65.7%answered 61% | 384 ms | 393 ms | $0.015$14.92 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.037$36.77 per 1,000 questionsper 1M questions Math 71.3% | — | 50.0%47.1%–53.0% | abc | 563 / 1,125 | 43 right of 43 kept0 mistakes · 1,082 skipped | 0.768 | 2.0%answered 99% | 228 ms | 344 ms | $0.037$36.77 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 65.1% | — | 48.2%45.3%–51.1% | bcd | 542 / 1,125 | 61 right of 61 kept0 mistakes · 1,064 skipped | 0.750 | 24.5%answered 86% | 166 ms | 193 ms | $0.026$25.64 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.015$15.40 per 1,000 questionsper 1M questions Math 70.0% | — | 47.3%44.4%–50.2% | bcd | 532 / 1,125 | No answers at this threshold | 0.735 | 92.3%answered 28% | 390 ms | 394 ms | $0.015$15.40 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.002$1.54 per 1,000 questionsper 1M questions Math 69.9% | — | 47.3%44.4%–50.2% | bcd | 532 / 1,125 | No answers at this threshold | 0.733 | 91.0%answered 29% | 383 ms | 390 ms | $0.002$1.54 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.026$25.64 per 1,000 questionsper 1M questions Math 64.1% | — | 46.3%43.4%–49.2% | cd | 521 / 1,125 | 65 right of 65 kept0 mistakes · 1,060 skipped | 0.764 | 15.8%answered 91% | 167 ms | 203 ms | $0.026$25.64 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.2% | — | 46.0%43.2%–49.0% | cd | 518 / 1,125 | 48 right of 48 kept0 mistakes · 1,077 skipped | 0.748 | 90.6%answered 28% | 176 ms | 272 ms | $0.006$6.02 |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.008$7.52 per 1,000 questionsper 1M questions Math 53.5% | — | 45.1%42.2%–48.0% | cde | 507 / 1,125 | 84 right of 91 kept7 wrong · 1,034 skipped | 0.746 | 20.5%answered 88% | 301 ms | 360 ms | $0.008$7.52 |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.013$13.42 per 1,000 questionsper 1M questions Math 59.4% | — | 44.4%41.6%–47.4% | de | 500 / 1,125 | 20 right of 20 kept0 mistakes · 1,105 skipped | 0.712 | 19.8%answered 88% | 315 ms | 362 ms | $0.013$13.42 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 55.6% | — | 40.8%38.0%–43.7% | e | 459 / 1,125 | 55 right of 56 kept1 wrong · 1,069 skipped | 0.723 | 11.3%answered 93% | 188 ms | 290 ms | $0.006$6.02 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
No workhorse decision-model setup has answers on Logic yet. Decision-model runs appear here as soon as the first one stores answers.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
No flagship decision-model setup has answers on Logic yet. Decision-model runs appear here as soon as the first one stores answers.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
d1.2: 49.2% at $0.006 per 1,000 questions ($6.02 per 1M)
d1.2-ultra: 51.6% at $0.013 per 1,000 questions ($12.64 per 1M)
Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| d1.2 Pareto Closed · Undisclosed | — | $0.006$6.02 | $6.02 | $0.006773 | — | provider price × reported usage |
| d1.2-ultra Pareto Closed · Undisclosed | — | $0.013$12.64 | $12.64 | $0.01422 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.013$12.64 per 1,000 questionsper 1M questions Math 69.9% | — | 51.6%48.6%–54.5% | a | 580 / 1,125 | 13 right of 13 kept0 mistakes · 1,112 skipped | 0.800 | 94.5%answered 28% | 220 ms | 341 ms | $0.013$12.64 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.02 per 1,000 questionsper 1M questions Math 65.5% | — | 49.2%46.2%–52.1% | a | 553 / 1,125 | 28 right of 28 kept0 mistakes · 1,097 skipped | 0.742 | 78.8%answered 43% | 222 ms | 390 ms | $0.006$6.02 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Google eCloud Technology Other MiniMax Z.ai OpenBMB Alibaba Xiaomi DeepSeek Meta Liquid AI Thinking Machines KRAFTON IBM Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Gemma 4 E2B (on · chat): 74.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
Erk-14B (off · chat): 54.2% at $0.390 per 1,000 questions ($390 per 1M) (estimate)
LFM3-1.7B (off · chat): 85.5% at $0.487 per 1,000 questions ($487 per 1M) (estimate)
Gemma 4 E2B (chat): 72.9% at $0.513 per 1,000 questions ($513 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 76.8% at $0.574 per 1,000 questions ($574 per 1M) (estimate)
Gemma 4 E4B (on · chat): 82.6% at $0.650 per 1,000 questions ($650 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 94.0% at $0.803 per 1,000 questions ($803 per 1M)
Gemma 4 E4B (chat): 83.2% at $0.842 per 1,000 questions ($842 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 85.9% at $0.993 per 1,000 questions ($993 per 1M) (estimate)
LFM3-1.7B (on · chat): 84.2% at $1.017 per 1,000 questions ($1,017 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 84.6% at $1.040 per 1,000 questions ($1,040 per 1M) (estimate)
GLM-5.3-Flash (high · chat): 93.2% at $1.057 per 1,000 questions ($1,057 per 1M)
Gemma 4 E2B (on · chat · 32k cap): 75.9% at $1.193 per 1,000 questions ($1,193 per 1M) (estimate)
Gemma 4 31B (chat): 93.3% at $1.244 per 1,000 questions ($1,244 per 1M) (estimate)
Gemma 4 12B Unified (chat): 87.7% at $1.292 per 1,000 questions ($1,292 per 1M) (estimate)
MiniCPM5-2B (off · chat): 83.6% at $1.327 per 1,000 questions ($1,327 per 1M) (estimate)
Qwen3.8 Flash (off · chat): 94.0% at $1.376 per 1,000 questions ($1,376 per 1M)
MiMo-V2.6-Flash (on · chat): 95.1% at $1.390 per 1,000 questions ($1,390 per 1M)
Qwen3.5-4B (off · chat): 90.1% at $1.420 per 1,000 questions ($1,420 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 85.0% at $1.503 per 1,000 questions ($1,503 per 1M) (estimate)
Qwen3.5-2B (off · chat): 64.8% at $1.605 per 1,000 questions ($1,605 per 1M) (estimate)
DeepSeek-V4.1-Flash (high · chat · 1017): 96.2% at $1.647 per 1,000 questions ($1,647 per 1M)
Qwen3-14B (off · chat): 65.3% at $1.772 per 1,000 questions ($1,772 per 1M) (estimate)
Muse Ember (max · chat): 95.5% at $1.820 per 1,000 questions ($1,820 per 1M)
LFM2.5-2.6B (on · chat): 75.4% at $2.480 per 1,000 questions ($2,480 per 1M) (estimate)
MiniCPM5-2B (on · chat): 88.9% at $3.335 per 1,000 questions ($3,335 per 1M) (estimate)
Qwen3.5-4B (on · chat): 92.4% at $3.525 per 1,000 questions ($3,525 per 1M) (estimate)
Inkling 2 Small (high · chat): 95.6% at $4.121 per 1,000 questions ($4,121 per 1M)
Qwen3.5-2B (on · chat): 70.3% at $4.330 per 1,000 questions ($4,330 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 60.2% at $4.664 per 1,000 questions ($4,664 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Qwen3.8-27B (low · chat): 94.0% at $4.884 per 1,000 questions ($4,884 per 1M) (estimate)
Inkling-Small (high · chat · Jul 2026): 95.4% at $5.489 per 1,000 questions ($5,489 per 1M)
Qwen4-Max (high · chat): 96.3% at $5.690 per 1,000 questions ($5,690 per 1M)
LFM2.5-24B-A2B (on · chat): 86.7% at $7.562 per 1,000 questions ($7,562 per 1M) (estimate)
Raon-1 nano (off · chat): 65.1% at $9.270 per 1,000 questions ($9,270 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Granite-4.2-3B (high · chat): 91.6% at $10.893 per 1,000 questions ($10,893 per 1M) (estimate)
Claude Sonnet 5.5 (chat): 95.5% at $11.480 per 1,000 questions ($11,480 per 1M)
Qwen3.5-397B-A17B (on · chat): 95.1% at $13.426 per 1,000 questions ($13,426 per 1M)
Raon-1 nano (on · chat): 70.9% at $15.330 per 1,000 questions ($15,330 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Claude Opus 5.5 (chat): 96.2% at $19.540 per 1,000 questions ($19,540 per 1M)
Google eCloud Technology Other MiniMax Z.ai OpenBMB Alibaba Xiaomi DeepSeek Meta Liquid AI Thinking Machines KRAFTON IBM Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Gemma 4 E2B Pareto on · chat Open · 5.1B | — | $0.363$363estimate · DGX Spark | $363.26 estimate | $0.4087 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | $0.390$390estimate · RTX PRO 6000 Blackwell | $389.79 estimate | $0.4385 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | $0.487$487estimate · DGX Spark | $486.75 estimate | $0.5476 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B chat Open · 5.1B | — | $0.513$513estimate · RTX PRO 6000 Blackwell | $513.37 estimate | $0.5775 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | $0.574$574estimate · RTX PRO 6000 Blackwell | $574.03 estimate | $0.6426 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat Open · 8.0B | — | $0.650$650estimate · DGX Spark | $649.54 estimate | $0.7307 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | $0.803$803 | $803.23 | $0.9036 | — | provider price × reported usage |
| Gemma 4 E4B chat Open · 8.0B | — | $0.842$842estimate · RTX PRO 6000 Blackwell | $842.30 estimate | $0.9476 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B | — | $0.993$993estimate · RTX PRO 6000 Blackwell | $992.52 estimate | $1.091 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B on · chat Open · 1.7B | — | $1.017$1,017estimate · RTX PRO 6000 Blackwell | $1,017.31 estimate | $1.127 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | $1.040$1,040estimate · RTX PRO 6000 Blackwell | $1,039.65 estimate | $1.181 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active | — | $1.057$1,057 | $1,056.60 | $1.189 | — | provider price × reported usage |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | $1.193$1,193estimate · RTX PRO 6000 Blackwell | $1,192.74 estimate | $1.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B chat Open · 31B | — | $1.244$1,244estimate · RTX PRO 6000 Blackwell | $1,243.77 estimate | $1.399 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified chat Open · 12B | — | $1.292$1,292estimate · RTX PRO 6000 Blackwell | $1,291.73 estimate | $1.453 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | $1.327$1,327estimate · RTX PRO 6000 Blackwell | $1,326.98 estimate | $1.594 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8 Flash off · chat Closed · Undisclosed | — | $1.376$1,376 | $1,376.26 | $1.548 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | $1.390$1,390 | $1,390.14 | $1.564 | — | provider price × reported usage |
| Qwen3.5-4B off · chat Open · 4.7B | — | $1.420$1,420estimate · RTX PRO 6000 Blackwell | $1,420.22 estimate | $1.598 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | $1.503$1,503estimate · RTX PRO 6000 Blackwell | $1,502.61 estimate | $1.69 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | $1.605$1,605estimate · RTX PRO 6000 Blackwell | $1,604.67 estimate | $1.805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | $1.647$1,647 | $1,646.92 | $1.853 | — | provider price × reported usage |
| Qwen3-14B off · chat Open · 15B | — | $1.772$1,772estimate · RTX PRO 6000 Blackwell | $1,771.90 estimate | $1.993 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Muse Ember max · chat Closed · Efficient | — | $1.820$1,820 | $1,820.45 | $2.048 | — | provider price × reported usage |
| LFM2.5-2.6B on · chat Open · 2.7B | — | $2.480$2,480estimate · RTX PRO 6000 Blackwell | $2,479.55 estimate | $2.789 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B on · chat Open · 2.5B | — | $3.335$3,335estimate · RTX PRO 6000 Blackwell | $3,334.55 estimate | $3.828 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B on · chat Open · 4.7B | — | $3.525$3,525estimate · RTX PRO 6000 Blackwell | $3,524.73 estimate | $3.965 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | $4.121$4,121 | $4,121.38 | $4.637 | — | provider price × reported usage |
| Qwen3.5-2B on · chat Open · 2.0B | — | $4.330$4,330estimate · RTX PRO 6000 Blackwell | $4,330.14 estimate | $4.871 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | $4.664$4,664estimate · RTX PRO 6000 Blackwell† | $4,663.99 estimate† | $5.247 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B low · chat Open · 27B | — | $4.884$4,884estimate · RTX PRO 6000 Blackwell | $4,884.09 estimate | $5.495 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | $5.489$5,489 | $5,489.38 | $6.176 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | $5.690$5,690 | $5,689.67 | $6.401 | — | provider price × reported usage |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | $7.562$7,562estimate · RTX PRO 6000 Blackwell | $7,561.61 estimate | $8.507 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | $9.270$9,270estimate · RTX PRO 6000 Blackwell† | $9,270.18 estimate† | $10.43 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | $10.893$10,893estimate · RTX PRO 6000 Blackwell | $10,892.67 estimate | $12.25 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Sonnet 5.5 chat Closed · Workhorse | — | $11.480$11,480 API list-price equivalent | $11,479.90 | $12.91 | — | Claude subscription |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active | — | $13.426$13,426 | $13,425.63 | $15.1 | — | provider price × reported usage |
| Raon-1 nano on · chat Open · 3.1B | — | $15.330$15,330estimate · RTX PRO 6000 Blackwell† | $15,330.41 estimate† | $17.25 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Opus 5.5 chat Closed · Flagship | — | $19.540$19,540 API list-price equivalent | $19,540.15 | $21.98 | — | Claude subscription |
| GPT-6 Astra chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Hydrogen chat | — | — | — | — | — | |
| Claude Fable 5.1 chat Closed · Flagship | — | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6 Luna chat Closed · Efficient | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Not ranked (only complete runs are): Spark-X2.6-4B (high · chat) (not running now, 20 of 1,125 stored, last answer 13 Nov 02:18 UTC); Grok 4.7 (chat) (paused for now at 576 of 1,125 (Ümit #8603, zeki #8637, 2026-11-15T11:37Z); cost so far $80.830$80,830 per 1,000 questions API list-price equivalent, over the 576 of 1,125 answered); Claude Fable 5.2 (chat) (not running now, 147 of 1,125 stored, last answer 12 Nov 11:32 UTC); Granite-4.2-3B (off · chat) (still running, 259 of 1,125); GPT-6 Luna (max · chat) (paused for now at 55 of 1,125 (zeki #4129, 2026-11-10T14:13Z)); DeepSeek-V4-Flash (chat) (sample: a random subset by design, 317 answers stored, not the full 1,125); Claude Haiku 5.5 (max · chat) (paused for now at 47 of 1,125 (zeki #4129, 2026-11-10T14:13Z)).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | 96.7%95.5%–97.6% | a | 1,088 / 1,125 | Not measured | Not measured | 56.4%answered 97% | 6,590 ms | 14,330 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Math 99.6% | — | 96.7%95.5%–97.6% | a | 1,088 / 1,125 | Not measured | Not measured | 45.5%answered 98% | 44,520 ms | 206,660 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | 96.5%95.3%–97.5% | a | 1,086 / 1,125 | Not measured | Not measured | 63.2%answered 97% | 7,070 ms | 16,200 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $5.690$5,690 per 1,000 questionsper 1M questions Math 99.8% | — | 96.3%95.0%–97.2% | ab | 1,083 / 1,125 | Not measured | Not measured | 0.0%answered 100% | 11,570 ms | 38,640 ms | $5.690$5,690 |
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $19.540$19,540 API list-price equivalent per 1,000 questionsper 1M questions Math 99.8% | — | 96.2%94.9%–97.2% | abc | 1,082 / 1,125 | Not measured | Not measured | 38.2%answered 98% | 5,830 ms | 12,240 ms | $19.540$19,540 API list-price equivalent |
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $1.647$1,647 per 1,000 questionsper 1M questions Math 99.6% | — | 96.2%94.9%–97.2% | abc | 1,082 / 1,125 | Not measured | Not measured | 3.8%answered 100% | 6,680 ms | 40,040 ms | $1.647$1,647 |
| Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Math 99.6% | — | 96.1%94.8%–97.1% | abcd | 1,081 / 1,125 | Not measured | Not measured | 76.1%answered 95% | 8,210 ms | 24,090 ms | — |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $4.121$4,121 per 1,000 questionsper 1M questions Math 99.5% | — | 95.6%94.3%–96.7% | abcde | 1,076 / 1,125 | Not measured | Not measured | 37.3%answered 98% | 13,180 ms | 144,250 ms | $4.121$4,121 |
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $11.480$11,480 API list-price equivalent per 1,000 questionsper 1M questions Math 99.7% | — | 95.5%94.1%–96.5% | abcdef | 1,074 / 1,125 | Not measured | Not measured | 41.3%answered 98% | 6,710 ms | 14,890 ms | $11.480$11,480 API list-price equivalent |
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $1.820$1,820 per 1,000 questionsper 1M questions Math 99.6% | — | 95.5%94.1%–96.5% | abcdef | 1,074 / 1,125 | Not measured | Not measured | 22.6%answered 99% | 3,110 ms | 14,020 ms | $1.820$1,820 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $5.489$5,489 per 1,000 questionsper 1M questions Math 99.6% | — | 95.4%94.0%–96.5% | abcdef | 1,073 / 1,125 | Not measured | Not measured | 50.0%answered 97% | 13,290 ms | 78,510 ms | $5.489$5,489 |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.7% | — | 95.2%93.8%–96.3% | abcdef | 1,071 / 1,125 | Not measured | Not measured | 8.3%answered 100% | 3,820 ms | 13,460 ms | $0.000$0.00 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $1.390$1,390 per 1,000 questionsper 1M questions Math 99.4% | — | 95.1%93.7%–96.2% | abcdef | 1,070 / 1,125 | Not measured | Not measured | 23.3%answered 98% | 24,740 ms | 179,930 ms | $1.390$1,390 |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active Qwen3.5-397B-A17B on · chat · Alibaba · chat model · reasoning on Parameters: 397B MoE · 17B active Cost $13.426$13,426 per 1,000 questionsper 1M questions Math running | — | 95.1%93.7%–96.2% | abcdef | 1,070 / 1,125 | Not measured | Not measured | 31.7%answered 98% | 84,650 ms | 167,520 ms | $13.426$13,426 |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $4.884$4,884estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | 94.0%92.5%–95.3% | bcdefg | 1,058 / 1,125 | Not measured | Not measured | 39.1%answered 97% | 18,710 ms | 101,030 ms | $4.884$4,884estimate · RTX PRO 6000 Blackwell |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.2% | — | 94.0%92.4%–95.2% | efg | 1,057 / 1,125 | Not measured | Not measured | 61.1%answered 95% | 5,640 ms | 10,330 ms | $0.000$0.00 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.803$803 per 1,000 questionsper 1M questions Math 98.2% | — | 94.0%92.4%–95.2% | cdefg | 1,057 / 1,125 | Not measured | Not measured | 38.3%answered 97% | 5,610 ms | 37,430 ms | $0.803$803 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $1.376$1,376 per 1,000 questionsper 1M questions Math 99.5% | — | 94.0%92.4%–95.2% | defg | 1,057 / 1,125 | Not measured | Not measured | 34.6%answered 98% | 17,550 ms | 73,220 ms | $1.376$1,376 |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $1.244$1,244estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | 93.3%91.7%–94.6% | efgh | 1,050 / 1,125 | Not measured | Not measured | 16.3%answered 99% | 18,290 ms | 39,790 ms | $1.244$1,244estimate · RTX PRO 6000 Blackwell |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $1.057$1,057 per 1,000 questionsper 1M questions Math 98.8% | — | 93.2%91.5%–94.5% | fgh | 1,048 / 1,125 | Not measured | Not measured | 4.2%answered 100% | 12,050 ms | 82,830 ms | $1.057$1,057 |
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $3.525$3,525estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | 92.4%90.7%–93.8% | gh | 1,039 / 1,125 | Not measured | Not measured | 36.1%answered 97% | 142,520 ms | 246,280 ms | $3.525$3,525estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $10.893$10,893estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | 91.6%89.9%–93.1% | ghi | 1,031 / 1,125 | Not measured | Not measured | 53.2%answered 94% | 221,350 ms | 1,229,550 ms | $10.893$10,893estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $1.420$1,420estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | 90.1%88.3%–91.7% | hij | 1,014 / 1,125 | Not measured | Not measured | 33.6%answered 96% | 45,380 ms | 127,510 ms | $1.420$1,420estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $3.335$3,335estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | 88.9%88.3%–89.6% min–max of 3 seeds | ijk | 998 / 1,125seed 0 run | Not measured | Not measured | 51.5%answered 92% | 101,100 ms | 333,427 ms | $3.335$3,335estimate · RTX PRO 6000 Blackwell |
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $1.292$1,292estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.8% | — | 87.7%85.7%–89.5% | jkl | 987 / 1,125 | Not measured | Not measured | 15.5%answered 98% | 42,060 ms | 82,940 ms | $1.292$1,292estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $7.562$7,562estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.0% | — | 86.7%84.6%–88.5% | jklm | 975 / 1,125 | Not measured | Not measured | 37.8%answered 95% | 228,980 ms | 776,930 ms | $7.562$7,562estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.993$993estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | 85.9%85.5%–86.1% min–max of 3 seeds | klm | 968 / 1,125seed 0 run | Not measured | Not measured | 55.9%answered 90% | 31,567 ms | 83,467 ms | $0.993$993estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.487$487estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.7% | — | 85.5%83.3%–87.4% | klm | 962 / 1,125 | Not measured | Not measured | 54.2%answered 90% | 163,870 ms | 656,750 ms | $0.487$487estimate · DGX Spark |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $1.503$1,503estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | 85.0%82.8%–86.9% | klm | 956 / 1,125 | Not measured | Not measured | Not measured | 57,900 ms | 109,440 ms | $1.503$1,503estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $1.040$1,040estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | 84.6%84.4%–85.0% min–max of 3 seeds | klm | 956 / 1,125seed 0 run | Not measured | Not measured | Not measured | 27,950 ms | 107,497 ms | $1.040$1,040estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $1.017$1,017estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | 84.2%83.9%–84.4% min–max of 3 seeds | klm | 950 / 1,125seed 0 run | Not measured | Not measured | 43.6%answered 91% | 29,847 ms | 113,220 ms | $1.017$1,017estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $1.327$1,327estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.2% | — | 83.6%83.1%–83.9% min–max of 3 seeds | lm | 942 / 1,125seed 0 run | Not measured | Not measured | 43.9%answered 88% | 46,453 ms | 111,607 ms | $1.327$1,327estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.842$842estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.1% | — | 83.2%80.9%–85.3% | m | 936 / 1,125 | Not measured | Not measured | 52.4%answered 88% | 32,690 ms | 64,070 ms | $0.842$842estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.650$650estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.2% | — | 82.6%80.3%–84.7% | m | 929 / 1,125 | Not measured | Not measured | 45.1%answered 91% | 284,080 ms | 539,980 ms | $0.650$650estimate · DGX Spark |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.574$574estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.7% | — | 76.8%76.0%–77.6% min–max of 3 seeds | n | 855 / 1,125seed 0 run | Not measured | Not measured | 32.6%answered 91% | 21,223 ms | 45,077 ms | $0.574$574estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $1.193$1,193estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.8% | — | 75.9%73.3%–78.3% | n | 854 / 1,125 | Not measured | Not measured | Not measured | 41,350 ms | 70,680 ms | $1.193$1,193estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $2.480$2,480estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.0% | — | 75.4%72.8%–77.8% | n | 848 / 1,125 | Not measured | Not measured | 44.3%answered 86% | 81,330 ms | 224,000 ms | $2.480$2,480estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.363$363estimate · DGX Spark per 1,000 questionsper 1M questions Math 96.9% | — | 74.6%72.0%–77.0% | n | 839 / 1,125 | Not measured | Not measured | 26.4%answered 93% | 173,220 ms | 297,090 ms | $0.363$363estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.513$513estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 96.8% | — | 72.9%70.2%–75.4% | n | 820 / 1,125 | Not measured | Not measured | 34.0%answered 90% | 21,390 ms | 35,160 ms | $0.513$513estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $15.330$15,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 97.0% | — | 70.9%68.2%–73.5% | no | 798 / 1,125 | Not measured | Not measured | 10.5%answered 95% | 493,230 ms† | 1,396,520 ms† | $15.330$15,330estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $4.330$4,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.7% | — | 70.3%67.6%–72.9% | nop | 791 / 1,125 | Not measured | Not measured | 33.1%answered 91% | 186,330 ms | 272,210 ms | $4.330$4,330estimate · RTX PRO 6000 Blackwell |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $1.772$1,772estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 95.8% | — | 65.3%62.5%–68.1% | opq | 735 / 1,125 | Not measured | Not measured | 16.1%answered 94% | 16,560 ms | 178,680 ms | $1.772$1,772estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $9.270$9,270estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 89.6% | — | 65.1%62.2%–67.8% | opq | 732 / 1,125 | Not measured | Not measured | 35.7%answered 87% | 276,720 ms† | 965,630 ms† | $9.270$9,270estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $1.605$1,605estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.6% | — | 64.8%62.0%–67.5% | pq | 729 / 1,125 | Not measured | Not measured | 35.2%answered 86% | 71,290 ms | 113,470 ms | $1.605$1,605estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $4.664$4,664estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 96.3% | — | 60.2%57.3%–63.0% | qr | 677 / 1,125 | Not measured | Not measured | 58.3%answered 70% | 138,140 ms† | 403,380 ms† | $4.664$4,664estimate · RTX PRO 6000 Blackwell† |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 93.7% | — | 58.0%55.0%–60.8% | r | 652 / 1,125 | Not measured | Not measured | 55.1%answered 72% | 3,860 ms | 5,820 ms | $0.000$0.00 |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.390$390estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 90.8% | — | 54.2%51.3%–57.1% | r | 610 / 1,125 | Not measured | Not measured | 30.7%answered 84% | 10,150 ms | 18,410 ms | $0.390$390estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Google Other OpenBMB Alibaba Liquid AI KRAFTON IBM
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Gemma 4 E2B (on · chat): 74.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
LFM3-1.7B (off · chat): 85.5% at $0.487 per 1,000 questions ($487 per 1M) (estimate)
Gemma 4 E2B (chat): 72.9% at $0.513 per 1,000 questions ($513 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 76.8% at $0.574 per 1,000 questions ($574 per 1M) (estimate)
LFM3-1.7B (on · chat): 84.2% at $1.017 per 1,000 questions ($1,017 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 84.6% at $1.040 per 1,000 questions ($1,040 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 75.9% at $1.193 per 1,000 questions ($1,193 per 1M) (estimate)
MiniCPM5-2B (off · chat): 83.6% at $1.327 per 1,000 questions ($1,327 per 1M) (estimate)
Qwen3.5-4B (off · chat): 90.1% at $1.420 per 1,000 questions ($1,420 per 1M) (estimate)
Qwen3.5-2B (off · chat): 64.8% at $1.605 per 1,000 questions ($1,605 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 75.4% at $2.480 per 1,000 questions ($2,480 per 1M) (estimate)
MiniCPM5-2B (on · chat): 88.9% at $3.335 per 1,000 questions ($3,335 per 1M) (estimate)
Qwen3.5-4B (on · chat): 92.4% at $3.525 per 1,000 questions ($3,525 per 1M) (estimate)
Qwen3.5-2B (on · chat): 70.3% at $4.330 per 1,000 questions ($4,330 per 1M) (estimate)
Raon-1 nano (off · chat): 65.1% at $9.270 per 1,000 questions ($9,270 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Granite-4.2-3B (high · chat): 91.6% at $10.893 per 1,000 questions ($10,893 per 1M) (estimate)
Raon-1 nano (on · chat): 70.9% at $15.330 per 1,000 questions ($15,330 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Google Other OpenBMB Alibaba Liquid AI KRAFTON IBM
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Gemma 4 E2B Pareto on · chat Open · 5.1B | — | $0.363$363estimate · DGX Spark | $363.26 estimate | $0.4087 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | $0.487$487estimate · DGX Spark | $486.75 estimate | $0.5476 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B chat Open · 5.1B | — | $0.513$513estimate · RTX PRO 6000 Blackwell | $513.37 estimate | $0.5775 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | $0.574$574estimate · RTX PRO 6000 Blackwell | $574.03 estimate | $0.6426 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B on · chat Open · 1.7B | — | $1.017$1,017estimate · RTX PRO 6000 Blackwell | $1,017.31 estimate | $1.127 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | $1.040$1,040estimate · RTX PRO 6000 Blackwell | $1,039.65 estimate | $1.181 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | $1.193$1,193estimate · RTX PRO 6000 Blackwell | $1,192.74 estimate | $1.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | $1.327$1,327estimate · RTX PRO 6000 Blackwell | $1,326.98 estimate | $1.594 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto off · chat Open · 4.7B | — | $1.420$1,420estimate · RTX PRO 6000 Blackwell | $1,420.22 estimate | $1.598 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | $1.605$1,605estimate · RTX PRO 6000 Blackwell | $1,604.67 estimate | $1.805 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | $2.480$2,480estimate · RTX PRO 6000 Blackwell | $2,479.55 estimate | $2.789 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B on · chat Open · 2.5B | — | $3.335$3,335estimate · RTX PRO 6000 Blackwell | $3,334.55 estimate | $3.828 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto on · chat Open · 4.7B | — | $3.525$3,525estimate · RTX PRO 6000 Blackwell | $3,524.73 estimate | $3.965 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | $4.330$4,330estimate · RTX PRO 6000 Blackwell | $4,330.14 estimate | $4.871 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | $9.270$9,270estimate · RTX PRO 6000 Blackwell† | $9,270.18 estimate† | $10.43 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | $10.893$10,893estimate · RTX PRO 6000 Blackwell | $10,892.67 estimate | $12.25 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | $15.330$15,330estimate · RTX PRO 6000 Blackwell† | $15,330.41 estimate† | $17.25 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Spark-X2.6-4B (high · chat) (not running now, 20 of 1,125 stored, last answer 13 Nov 02:18 UTC); Granite-4.2-3B (off · chat) (still running, 259 of 1,125).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $3.525$3,525estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | 92.4%90.7%–93.8% | a | 1,039 / 1,125 | Not measured | Not measured | 36.1%answered 97% | 142,520 ms | 246,280 ms | $3.525$3,525estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $10.893$10,893estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | 91.6%89.9%–93.1% | a | 1,031 / 1,125 | Not measured | Not measured | 53.2%answered 94% | 221,350 ms | 1,229,550 ms | $10.893$10,893estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $1.420$1,420estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | 90.1%88.3%–91.7% | ab | 1,014 / 1,125 | Not measured | Not measured | 33.6%answered 96% | 45,380 ms | 127,510 ms | $1.420$1,420estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $3.335$3,335estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | 88.9%88.3%–89.6% min–max of 3 seeds | bc | 998 / 1,125seed 0 run | Not measured | Not measured | 51.5%answered 92% | 101,100 ms | 333,427 ms | $3.335$3,335estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.487$487estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.7% | — | 85.5%83.3%–87.4% | cd | 962 / 1,125 | Not measured | Not measured | 54.2%answered 90% | 163,870 ms | 656,750 ms | $0.487$487estimate · DGX Spark |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $1.040$1,040estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | 84.6%84.4%–85.0% min–max of 3 seeds | d | 956 / 1,125seed 0 run | Not measured | Not measured | Not measured | 27,950 ms | 107,497 ms | $1.040$1,040estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $1.017$1,017estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.9% | — | 84.2%83.9%–84.4% min–max of 3 seeds | d | 950 / 1,125seed 0 run | Not measured | Not measured | 43.6%answered 91% | 29,847 ms | 113,220 ms | $1.017$1,017estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $1.327$1,327estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.2% | — | 83.6%83.1%–83.9% min–max of 3 seeds | d | 942 / 1,125seed 0 run | Not measured | Not measured | 43.9%answered 88% | 46,453 ms | 111,607 ms | $1.327$1,327estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.574$574estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.7% | — | 76.8%76.0%–77.6% min–max of 3 seeds | e | 855 / 1,125seed 0 run | Not measured | Not measured | 32.6%answered 91% | 21,223 ms | 45,077 ms | $0.574$574estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $1.193$1,193estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 97.8% | — | 75.9%73.3%–78.3% | e | 854 / 1,125 | Not measured | Not measured | Not measured | 41,350 ms | 70,680 ms | $1.193$1,193estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $2.480$2,480estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.0% | — | 75.4%72.8%–77.8% | ef | 848 / 1,125 | Not measured | Not measured | 44.3%answered 86% | 81,330 ms | 224,000 ms | $2.480$2,480estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.363$363estimate · DGX Spark per 1,000 questionsper 1M questions Math 96.9% | — | 74.6%72.0%–77.0% | efg | 839 / 1,125 | Not measured | Not measured | 26.4%answered 93% | 173,220 ms | 297,090 ms | $0.363$363estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.513$513estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 96.8% | — | 72.9%70.2%–75.4% | efg | 820 / 1,125 | Not measured | Not measured | 34.0%answered 90% | 21,390 ms | 35,160 ms | $0.513$513estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $15.330$15,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 97.0% | — | 70.9%68.2%–73.5% | fg | 798 / 1,125 | Not measured | Not measured | 10.5%answered 95% | 493,230 ms† | 1,396,520 ms† | $15.330$15,330estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $4.330$4,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.7% | — | 70.3%67.6%–72.9% | g | 791 / 1,125 | Not measured | Not measured | 33.1%answered 91% | 186,330 ms | 272,210 ms | $4.330$4,330estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $9.270$9,270estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 89.6% | — | 65.1%62.2%–67.8% | h | 732 / 1,125 | Not measured | Not measured | 35.7%answered 87% | 276,720 ms† | 965,630 ms† | $9.270$9,270estimate · RTX PRO 6000 Blackwell† |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $1.605$1,605estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 94.6% | — | 64.8%62.0%–67.5% | h | 729 / 1,125 | Not measured | Not measured | 35.2%answered 86% | 71,290 ms | 113,470 ms | $1.605$1,605estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
eCloud Technology Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Erk-14B (off · chat): 54.2% at $0.390 per 1,000 questions ($390 per 1M) (estimate)
Gemma 4 E4B (on · chat): 82.6% at $0.650 per 1,000 questions ($650 per 1M) (estimate)
Gemma 4 E4B (chat): 83.2% at $0.842 per 1,000 questions ($842 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 85.9% at $0.993 per 1,000 questions ($993 per 1M) (estimate)
Gemma 4 12B Unified (chat): 87.7% at $1.292 per 1,000 questions ($1,292 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 85.0% at $1.503 per 1,000 questions ($1,503 per 1M) (estimate)
Qwen3-14B (off · chat): 65.3% at $1.772 per 1,000 questions ($1,772 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 60.2% at $4.664 per 1,000 questions ($4,664 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
eCloud Technology Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Erk-14B Pareto off · chat Open · 15B | — | $0.390$390estimate · RTX PRO 6000 Blackwell | $389.79 estimate | $0.4385 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat Open · 8.0B | — | $0.650$650estimate · DGX Spark | $649.54 estimate | $0.7307 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E4B Pareto chat Open · 8.0B | — | $0.842$842estimate · RTX PRO 6000 Blackwell | $842.30 estimate | $0.9476 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat · 32k cap · sampled Open · 8.0B | — | $0.993$993estimate · RTX PRO 6000 Blackwell | $992.52 estimate | $1.091 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified Pareto chat Open · 12B | — | $1.292$1,292estimate · RTX PRO 6000 Blackwell | $1,291.73 estimate | $1.453 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | $1.503$1,503estimate · RTX PRO 6000 Blackwell | $1,502.61 estimate | $1.69 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | $1.772$1,772estimate · RTX PRO 6000 Blackwell | $1,771.90 estimate | $1.993 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | $4.664$4,664estimate · RTX PRO 6000 Blackwell† | $4,663.99 estimate† | $5.247 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $1.292$1,292estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.8% | — | 87.7%85.7%–89.5% | a | 987 / 1,125 | Not measured | Not measured | 15.5%answered 98% | 42,060 ms | 82,940 ms | $1.292$1,292estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.993$993estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | 85.9%85.5%–86.1% min–max of 3 seeds | ab | 968 / 1,125seed 0 run | Not measured | Not measured | 55.9%answered 90% | 31,567 ms | 83,467 ms | $0.993$993estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $1.503$1,503estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.5% | — | 85.0%82.8%–86.9% | abc | 956 / 1,125 | Not measured | Not measured | Not measured | 57,900 ms | 109,440 ms | $1.503$1,503estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.842$842estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 98.1% | — | 83.2%80.9%–85.3% | bc | 936 / 1,125 | Not measured | Not measured | 52.4%answered 88% | 32,690 ms | 64,070 ms | $0.842$842estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.650$650estimate · DGX Spark per 1,000 questionsper 1M questions Math 98.2% | — | 82.6%80.3%–84.7% | c | 929 / 1,125 | Not measured | Not measured | 45.1%answered 91% | 284,080 ms | 539,980 ms | $0.650$650estimate · DGX Spark |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $1.772$1,772estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 95.8% | — | 65.3%62.5%–68.1% | d | 735 / 1,125 | Not measured | Not measured | 16.1%answered 94% | 16,560 ms | 178,680 ms | $1.772$1,772estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $4.664$4,664estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Math 96.3% | — | 60.2%57.3%–63.0% | e | 677 / 1,125 | Not measured | Not measured | 58.3%answered 70% | 138,140 ms† | 403,380 ms† | $4.664$4,664estimate · RTX PRO 6000 Blackwell† |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.390$390estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 90.8% | — | 54.2%51.3%–57.1% | f | 610 / 1,125 | Not measured | Not measured | 30.7%answered 84% | 10,150 ms | 18,410 ms | $0.390$390estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Gemma 4 31B (chat): 93.3% at $1.244 per 1,000 questions ($1,244 per 1M) (estimate)
Qwen3.8-27B (low · chat): 94.0% at $4.884 per 1,000 questions ($4,884 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 86.7% at $7.562 per 1,000 questions ($7,562 per 1M) (estimate)
Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Gemma 4 31B Pareto chat Open · 31B | — | $1.244$1,244estimate · RTX PRO 6000 Blackwell | $1,243.77 estimate | $1.399 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B Pareto low · chat Open · 27B | — | $4.884$4,884estimate · RTX PRO 6000 Blackwell | $4,884.09 estimate | $5.495 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | $7.562$7,562estimate · RTX PRO 6000 Blackwell | $7,561.61 estimate | $8.507 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $4.884$4,884estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.2% | — | 94.0%92.5%–95.3% | a | 1,058 / 1,125 | Not measured | Not measured | 39.1%answered 97% | 18,710 ms | 101,030 ms | $4.884$4,884estimate · RTX PRO 6000 Blackwell |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $1.244$1,244estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.4% | — | 93.3%91.7%–94.6% | a | 1,050 / 1,125 | Not measured | Not measured | 16.3%answered 99% | 18,290 ms | 39,790 ms | $1.244$1,244estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $7.562$7,562estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Math 99.0% | — | 86.7%84.6%–88.5% | b | 975 / 1,125 | Not measured | Not measured | 37.8%answered 95% | 228,980 ms | 776,930 ms | $7.562$7,562estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
MiniMax Z.ai Xiaomi DeepSeek Thinking Machines Alibaba
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
MiniMax-M3.1-Flash (high · chat): 94.0% at $0.803 per 1,000 questions ($803 per 1M)
GLM-5.3-Flash (high · chat): 93.2% at $1.057 per 1,000 questions ($1,057 per 1M)
MiMo-V2.6-Flash (on · chat): 95.1% at $1.390 per 1,000 questions ($1,390 per 1M)
DeepSeek-V4.1-Flash (high · chat · 1017): 96.2% at $1.647 per 1,000 questions ($1,647 per 1M)
Inkling 2 Small (high · chat): 95.6% at $4.121 per 1,000 questions ($4,121 per 1M)
Inkling-Small (high · chat · Jul 2026): 95.4% at $5.489 per 1,000 questions ($5,489 per 1M)
Qwen3.5-397B-A17B (on · chat): 95.1% at $13.426 per 1,000 questions ($13,426 per 1M)
MiniMax Z.ai Xiaomi DeepSeek Thinking Machines Alibaba
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | $0.803$803 | $803.23 | $0.9036 | — | provider price × reported usage |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active | — | $1.057$1,057 | $1,056.60 | $1.189 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | $1.390$1,390 | $1,390.14 | $1.564 | — | provider price × reported usage |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | $1.647$1,647 | $1,646.92 | $1.853 | — | provider price × reported usage |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | $4.121$4,121 | $4,121.38 | $4.637 | — | provider price × reported usage |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | $5.489$5,489 | $5,489.38 | $6.176 | — | provider price × reported usage |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active | — | $13.426$13,426 | $13,425.63 | $15.1 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): DeepSeek-V4-Flash (chat) (sample: a random subset by design, 317 answers stored, not the full 1,125).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $1.647$1,647 per 1,000 questionsper 1M questions Math 99.6% | — | 96.2%94.9%–97.2% | a | 1,082 / 1,125 | Not measured | Not measured | 3.8%answered 100% | 6,680 ms | 40,040 ms | $1.647$1,647 |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $4.121$4,121 per 1,000 questionsper 1M questions Math 99.5% | — | 95.6%94.3%–96.7% | ab | 1,076 / 1,125 | Not measured | Not measured | 37.3%answered 98% | 13,180 ms | 144,250 ms | $4.121$4,121 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $5.489$5,489 per 1,000 questionsper 1M questions Math 99.6% | — | 95.4%94.0%–96.5% | ab | 1,073 / 1,125 | Not measured | Not measured | 50.0%answered 97% | 13,290 ms | 78,510 ms | $5.489$5,489 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $1.390$1,390 per 1,000 questionsper 1M questions Math 99.4% | — | 95.1%93.7%–96.2% | abc | 1,070 / 1,125 | Not measured | Not measured | 23.3%answered 98% | 24,740 ms | 179,930 ms | $1.390$1,390 |
| Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active Qwen3.5-397B-A17B on · chat · Alibaba · chat model · reasoning on Parameters: 397B MoE · 17B active Cost $13.426$13,426 per 1,000 questionsper 1M questions Math running | — | 95.1%93.7%–96.2% | abc | 1,070 / 1,125 | Not measured | Not measured | 31.7%answered 98% | 84,650 ms | 167,520 ms | $13.426$13,426 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.803$803 per 1,000 questionsper 1M questions Math 98.2% | — | 94.0%92.4%–95.2% | bc | 1,057 / 1,125 | Not measured | Not measured | 38.3%answered 97% | 5,610 ms | 37,430 ms | $0.803$803 |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $1.057$1,057 per 1,000 questionsper 1M questions Math 98.8% | — | 93.2%91.5%–94.5% | c | 1,048 / 1,125 | Not measured | Not measured | 4.2%answered 100% | 12,050 ms | 82,830 ms | $1.057$1,057 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Meta
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Muse Ember (max · chat): 95.5% at $1.820 per 1,000 questions ($1,820 per 1M)
Meta
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Muse Ember Pareto max · chat Closed · Efficient | — | $1.820$1,820 | $1,820.45 | $2.048 | — | provider price × reported usage |
| GPT-6 Luna chat Closed · Efficient | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): GPT-6 Luna (max · chat) (paused for now at 55 of 1,125 (zeki #4129, 2026-11-10T14:13Z)); Claude Haiku 5.5 (max · chat) (paused for now at 47 of 1,125 (zeki #4129, 2026-11-10T14:13Z)).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $1.820$1,820 per 1,000 questionsper 1M questions Math 99.6% | — | 95.5%94.1%–96.5% | a | 1,074 / 1,125 | Not measured | Not measured | 22.6%answered 99% | 3,110 ms | 14,020 ms | $1.820$1,820 |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 93.7% | — | 58.0%55.0%–60.8% | b | 652 / 1,125 | Not measured | Not measured | 55.1%answered 72% | 3,860 ms | 5,820 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Sonnet 5.5 (chat): 95.5% at $11.480 per 1,000 questions ($11,480 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 Pareto chat Closed · Workhorse | — | $11.480$11,480 API list-price equivalent | $11,479.90 | $12.91 | — | Claude subscription |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $11.480$11,480 API list-price equivalent per 1,000 questionsper 1M questions Math 99.7% | — | 95.5%94.1%–96.5% | a | 1,074 / 1,125 | Not measured | Not measured | 41.3%answered 98% | 6,710 ms | 14,890 ms | $11.480$11,480 API list-price equivalent |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Opus 5.5 (chat): 96.2% at $19.540 per 1,000 questions ($19,540 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 Pareto chat Closed · Flagship | — | $19.540$19,540 API list-price equivalent | $19,540.15 | $21.98 | — | Claude subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Claude Fable 5.2 (chat) (not running now, 147 of 1,125 stored, last answer 12 Nov 11:32 UTC).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $19.540$19,540 API list-price equivalent per 1,000 questionsper 1M questions Math 99.8% | — | 96.2%94.9%–97.2% | a | 1,082 / 1,125 | Not measured | Not measured | 38.2%answered 98% | 5,830 ms | 12,240 ms | $19.540$19,540 API list-price equivalent |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.7% | — | 95.2%93.8%–96.3% | b | 1,071 / 1,125 | Not measured | Not measured | 8.3%answered 100% | 3,820 ms | 13,460 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Alibaba
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Qwen3.8 Flash (off · chat): 94.0% at $1.376 per 1,000 questions ($1,376 per 1M)
Qwen4-Max (high · chat): 96.3% at $5.690 per 1,000 questions ($5,690 per 1M)
Alibaba
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Qwen3.8 Flash Pareto off · chat Closed · Undisclosed | — | $1.376$1,376 | $1,376.26 | $1.548 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | $5.690$5,690 | $5,689.67 | $6.401 | — | provider price × reported usage |
| GPT-6 Astra chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Grok 4.7 (chat) (paused for now at 576 of 1,125 (Ümit #8603, zeki #8637, 2026-11-15T11:37Z); cost so far $80.830$80,830 per 1,000 questions API list-price equivalent, over the 576 of 1,125 answered).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 1,125) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | 96.7%95.5%–97.6% | a | 1,088 / 1,125 | Not measured | Not measured | 56.4%answered 97% | 6,590 ms | 14,330 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Math 99.6% | — | 96.7%95.5%–97.6% | a | 1,088 / 1,125 | Not measured | Not measured | 45.5%answered 98% | 44,520 ms | 206,660 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.6% | — | 96.5%95.3%–97.5% | a | 1,086 / 1,125 | Not measured | Not measured | 63.2%answered 97% | 7,070 ms | 16,200 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $5.690$5,690 per 1,000 questionsper 1M questions Math 99.8% | — | 96.3%95.0%–97.2% | a | 1,083 / 1,125 | Not measured | Not measured | 0.0%answered 100% | 11,570 ms | 38,640 ms | $5.690$5,690 |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Math 99.2% | — | 94.0%92.4%–95.2% | b | 1,057 / 1,125 | Not measured | Not measured | 61.1%answered 95% | 5,640 ms | 10,330 ms | $0.000$0.00 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $1.376$1,376 per 1,000 questionsper 1M questions Math 99.5% | — | 94.0%92.4%–95.2% | b | 1,057 / 1,125 | Not measured | Not measured | 34.6%answered 98% | 17,550 ms | 73,220 ms | $1.376$1,376 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.