Math · Cost
Cost on Math
2,163 questions, asked in English. How to read this page · Method
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other Celeris OmniJev StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Nace.AI Apertix AI Cloudflare Turion.AI TypeSafe Google quincyqiang scar-ai Blockbrain Labs Strands Agents newsbang tattrongvu Patrick-18 victorbennett EldanRing Sthānika AI vLLM Semantic Router Valen-Team Metask Lab Poindexter Labs MiniMax MachinePulse Logits REAX Z.ai Zefan Cai Xiaomi OpenBMB DeepSeek eCloud Technology Alibaba Meta Thinking Machines IBM KRAFTON Anthropic xAI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 20.6% at $0.001 per 1,000 questions ($0.77 per 1M) (estimate)
GLiClass v4 Edge: 22.7% at $0.001 per 1,000 questions ($0.85 per 1M) (estimate)
celeris-1.1-decision: 69.9% at $0.001 per 1,000 questions ($0.91 per 1M)
OneJev1.6-0.8B: 28.2% at $0.001 per 1,000 questions ($1.09 per 1M) (estimate)
GLiDE 0.8B: 32.1% at $0.001 per 1,000 questions ($1.28 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 29.7% at $0.002 per 1,000 questions ($1.53 per 1M) (estimate)
KnowLine-0.8B-Gen1: 28.2% at $0.002 per 1,000 questions ($1.69 per 1M) (estimate)
GLiDE 2B: 36.8% at $0.002 per 1,000 questions ($1.78 per 1M) (estimate)
Perplexity Decider Flash v1.2: 55.2% at $0.002 per 1,000 questions ($2.18 per 1M)
StartLux-Decision-v1.1-2B-Instant: 38.7% at $0.002 per 1,000 questions ($2.27 per 1M) (estimate)
d1.2: 65.5% at $0.003 per 1,000 questions ($2.92 per 1M)
d1.1: 65.2% at $0.003 per 1,000 questions ($2.92 per 1M)
d1: 55.6% at $0.003 per 1,000 questions ($2.92 per 1M)
KnowLine-2B-Gen1: 35.3% at $0.003 per 1,000 questions ($3.20 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 55.8% at $0.004 per 1,000 questions ($4.10 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API): 69.8% at $0.004 per 1,000 questions ($4.29 per 1M)
Mercury Decide: 53.5% at $0.004 per 1,000 questions ($4.38 per 1M)
KnowLine 4B Gen5.5: 56.0% at $0.005 per 1,000 questions ($4.56 per 1M) (estimate)
d1-ultra: 74.1% at $0.006 per 1,000 questions ($6.14 per 1M)
d1.2-ultra: 69.9% at $0.006 per 1,000 questions ($6.14 per 1M)
Microsoft-Decision-1: 59.4% at $0.007 per 1,000 questions ($6.81 per 1M)
GLiDE (v1.2): 69.9% at $0.008 per 1,000 questions ($7.55 per 1M)
Elephant-1: 61.3% at $0.008 per 1,000 questions ($7.90 per 1M)
StartLux-Decision-v1.1-9B-Instant: 58.4% at $0.008 per 1,000 questions ($8.34 per 1M) (estimate)
SubQ Snap 1.1: 68.6% at $0.009 per 1,000 questions ($8.59 per 1M)
Drex 2.0: 59.0% at $0.009 per 1,000 questions ($8.92 per 1M)
celeris-1-decision: 70.0% at $0.009 per 1,000 questions ($9.10 per 1M)
Cerno 9B v1.2: 56.0% at $0.010 per 1,000 questions ($9.51 per 1M) (estimate)
GLiDE 9B: 57.9% at $0.010 per 1,000 questions ($10.49 per 1M) (estimate)
Clef-Flash: 49.8% at $0.011 per 1,000 questions ($10.85 per 1M)
Drex Max v1.1: 70.1% at $0.013 per 1,000 questions ($12.88 per 1M)
Drex Max: 70.3% at $0.013 per 1,000 questions ($13.06 per 1M)
d1-2B: 37.1% at $0.014 per 1,000 questions ($14.07 per 1M) (estimate)
Turion Jev 31B: 69.0% at $0.017 per 1,000 questions ($17.46 per 1M) (estimate)
Jev 1.14: 65.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Jev 1.13: 64.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Gemini Decision 1 (20261111): 71.3% at $0.022 per 1,000 questions ($22.02 per 1M)
Perplexity Decider Flash v1.3: 55.9% at $0.022 per 1,000 questions ($22.38 per 1M) (estimate)
Mammoth-1: 74.0% at $0.023 per 1,000 questions ($22.56 per 1M)
Chinese Decision Pointer 4B: 50.4% at $0.023 per 1,000 questions ($22.70 per 1M) (estimate)
vera-medium-omni: 56.8% at $0.023 per 1,000 questions ($23.34 per 1M) (estimate)
Cygnet: 44.3% at $0.024 per 1,000 questions ($23.94 per 1M)
GLiDE 27B: 72.0% at $0.026 per 1,000 questions ($26.16 per 1M) (estimate)
Strands Decider 2B: 28.7% at $0.026 per 1,000 questions ($26.22 per 1M) (estimate)
Adjudex 9B v5: 62.1% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
KnowLine-9B-Gen1: 59.8% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
GLiDE (API): 70.8% at $0.028 per 1,000 questions ($28.30 per 1M)
Jeffan-9B-1211: 59.8% at $0.029 per 1,000 questions ($28.78 per 1M) (estimate)
KnowLine-9B-Gen2: 60.1% at $0.031 per 1,000 questions ($31.02 per 1M) (estimate)
Dexy-Decision-9B-V2: 58.8% at $0.032 per 1,000 questions ($31.98 per 1M) (estimate)
Verdin-4B-v1.1: 51.5% at $0.034 per 1,000 questions ($33.58 per 1M) (estimate)
KnowLine-12B-Gen1: 66.3% at $0.036 per 1,000 questions ($36.13 per 1M) (estimate)
Winnow-12B: 46.9% at $0.038 per 1,000 questions ($38.37 per 1M) (estimate)
Sieve 31B v3: 77.1% at $0.039 per 1,000 questions ($39.14 per 1M) (estimate)
d3: 70.4% at $0.041 per 1,000 questions ($40.61 per 1M) (estimate)
Valen-Decision-1.5-9B: 55.3% at $0.041 per 1,000 questions ($40.93 per 1M) (estimate)
KnowLine-31B-Gen1.5: 77.4% at $0.047 per 1,000 questions ($46.87 per 1M) (estimate)
LFM3-1.7B (off · chat): 98.7% at $0.060 per 1,000 questions ($59.94 per 1M) (estimate)
OneJev-27B: 53.8% at $0.061 per 1,000 questions ($60.76 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 72.5% at $0.066 per 1,000 questions ($65.87 per 1M) (estimate)
Clef: 51.1% at $0.069 per 1,000 questions ($68.54 per 1M)
KnowLine-31B-Gen2: 80.3% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
Metask-Jev-4B: 39.2% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
Copernicus-31B: 77.3% at $0.084 per 1,000 questions ($84.42 per 1M) (estimate)
d3-ultra: 71.5% at $0.089 per 1,000 questions ($88.58 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 98.2% at $0.103 per 1,000 questions ($103 per 1M)
Karpo Decision 31B: 81.6% at $0.113 per 1,000 questions ($113 per 1M) (estimate)
LFM3-1.7B (on · chat): 98.9% at $0.132 per 1,000 questions ($132 per 1M) (estimate)
Gemma 4 E2B (on · chat): 96.9% at $0.138 per 1,000 questions ($138 per 1M) (estimate)
Logits Sigma-1 Duo: 75.2% at $0.141 per 1,000 questions ($141 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 98.9% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
REAX 1.3: 78.0% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
GLM-5.3-Flash (high · chat): 98.8% at $0.166 per 1,000 questions ($166 per 1M)
Open-Jev-27B-v1.1: 54.6% at $0.169 per 1,000 questions ($169 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 97.7% at $0.196 per 1,000 questions ($196 per 1M) (estimate)
Gemma 4 E2B (chat): 96.8% at $0.208 per 1,000 questions ($208 per 1M) (estimate)
MiMo-V2.6-Flash (on · chat): 99.4% at $0.212 per 1,000 questions ($212 per 1M)
MiniCPM5-2B (off · chat): 98.2% at $0.219 per 1,000 questions ($219 per 1M) (estimate)
DeepSeek-V4.1-Flash (high · chat · 1017): 99.6% at $0.227 per 1,000 questions ($227 per 1M)
Gemma 4 E4B (on · chat): 98.2% at $0.233 per 1,000 questions ($233 per 1M) (estimate)
Erk-14B (off · chat): 90.8% at $0.266 per 1,000 questions ($266 per 1M) (estimate)
Gemma 4 E4B (chat): 98.1% at $0.271 per 1,000 questions ($271 per 1M) (estimate)
DeepSeek-V4-Flash (chat): 99.3% at $0.295 per 1,000 questions ($295 per 1M) (estimate)
Qwen3.5-4B (off · chat): 98.5% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 94.0% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 98.5% at $0.318 per 1,000 questions ($318 per 1M) (estimate)
Qwen3-14B (off · chat): 95.8% at $0.355 per 1,000 questions ($355 per 1M) (estimate)
Qwen3.5-2B (off · chat): 94.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
Muse Ember (max · chat): 99.6% at $0.365 per 1,000 questions ($365 per 1M)
Qwen3.8 Flash (off · chat): 99.5% at $0.397 per 1,000 questions ($397 per 1M)
MiniCPM5-2B (on · chat): 99.2% at $0.409 per 1,000 questions ($409 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 97.8% at $0.410 per 1,000 questions ($410 per 1M) (estimate)
Gemma 4 12B Unified (chat): 98.8% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 96.3% at $0.482 per 1,000 questions ($482 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Gemma 4 E4B (on · chat · 32k cap): 98.5% at $0.518 per 1,000 questions ($518 per 1M) (estimate)
Qwen3.8-27B (low · chat): 99.2% at $0.530 per 1,000 questions ($530 per 1M) (estimate)
Gemma 4 31B (chat): 99.4% at $0.531 per 1,000 questions ($531 per 1M) (estimate)
Inkling 2 Small (high · chat): 99.5% at $0.638 per 1,000 questions ($638 per 1M)
Granite-4.2-3B (high · chat): 99.2% at $0.781 per 1,000 questions ($781 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 99.0% at $0.939 per 1,000 questions ($939 per 1M) (estimate)
Inkling-Small (high · chat · Jul 2026): 99.6% at $0.954 per 1,000 questions ($954 per 1M)
Perplexity Decider 27B v1.2 (API · with web): 69.3% at $1.023 per 1,000 questions ($1,023 per 1M)
Qwen3.5-4B (on · chat): 99.4% at $1.271 per 1,000 questions ($1,271 per 1M) (estimate)
Qwen4-Max (high · chat): 99.8% at $1.478 per 1,000 questions ($1,478 per 1M)
Raon-1 nano (on · chat): 97.0% at $1.670 per 1,000 questions ($1,670 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (off · chat): 89.6% at $1.901 per 1,000 questions ($1,901 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Qwen3.5-2B (on · chat): 94.7% at $2.470 per 1,000 questions ($2,470 per 1M) (estimate)
Claude Sonnet 5.5 (chat): 99.7% at $4.734 per 1,000 questions ($4,734 per 1M)
Claude Opus 5.5 (chat): 99.8% at $9.871 per 1,000 questions ($9,871 per 1M)
Grok 4.7 (chat): 99.5% at $12.061 per 1,000 questions ($12,061 per 1M)
Fastino Other Celeris OmniJev StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Nace.AI Apertix AI Cloudflare Turion.AI TypeSafe Google quincyqiang scar-ai Blockbrain Labs Strands Agents newsbang tattrongvu Patrick-18 victorbennett EldanRing Sthānika AI vLLM Semantic Router Valen-Team Metask Lab Poindexter Labs MiniMax MachinePulse Logits REAX Z.ai Zefan Cai Xiaomi OpenBMB DeepSeek eCloud Technology Alibaba Meta Thinking Machines IBM KRAFTON Anthropic xAI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | Decision | $0.001$0.77estimate · DGX Spark | $0.77 estimate | $0.001655 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | Decision | $0.001$0.85estimate · DGX Spark | $0.85 estimate | $0.001833 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1.1-decision Pareto Closed · Efficient | — | Decision | $0.001$0.91 | $0.91 | $0.001968 | — | provider price × reported usage |
| OneJev1.6-0.8B Open · 0.8B | — | Decision | $0.001$1.09estimate · DGX Spark | $1.09 estimate | $0.002365 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Open · 0.8B | — | Decision | $0.001$1.28estimate · DGX Spark | $1.28 estimate | $0.002778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | Decision | $0.002$1.53estimate · DGX Spark | $1.53 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | Decision | $0.002$1.69estimate · DGX Spark | $1.69 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Open · 1.9B | — | Decision | $0.002$1.78estimate · DGX Spark | $1.78 estimate | $0.003843 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Open · 4.0B | — | Decision | $0.002$2.18 list | $2.18 | $0.004707 | $0.030$30.06estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | Decision | $0.002$2.27estimate · DGX Spark | $2.27 estimate | $0.004907 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1.2 Closed · Undisclosed | — | Decision | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | Decision | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1 Closed · Efficient | — | Decision | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| KnowLine-2B-Gen1 Open · 2.0B | — | Decision | $0.003$3.20estimate · DGX Spark | $3.20 estimate | $0.006917 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | — | Decision | $0.004$4.10estimate · DGX Spark | $4.10 estimate | $0.008867 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.2 API Open · 27B | — | Decision | $0.004$4.29 | $4.29 | $0.009285 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | Decision | $0.004$4.38 | $4.38 | $0.009483 | — | provider price × reported usage |
| KnowLine 4B Gen5.5 Open · 4.0B | — | Decision | $0.005$4.56estimate · DGX Spark | $4.56 estimate | $0.009872 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-ultra Pareto Closed · Efficient | — | Decision | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
| d1.2-ultra Closed · Undisclosed | — | Decision | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | Decision | $0.007$6.81 | $6.81 | $0.01472 | — | provider price × reported usage |
| GLiDE v1.2 Open · 27B | — | Decision | $0.008$7.55 list | $7.55 | $0.01632 | $0.058$57.88estimate · RTX PRO 6000 Blackwell | maker's list price |
| Elephant-1 Open · 9.0B | — | Decision | $0.008$7.90 list | $7.90 | $0.01708 | $0.058$58.20estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | Decision | $0.008$8.34estimate · DGX Spark | $8.34 estimate | $0.01803 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| SubQ Snap 1.1 Closed · Efficient | — | Decision | $0.009$8.59 | $8.59 | $0.01857 | — | provider price × reported usage |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | Decision | $0.009$8.92 | $8.92 | $0.01929 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | Decision | $0.009$9.10 | $9.10 | $0.01968 | — | provider price × reported usage |
| Cerno 9B v1.2 Open · 9.0B | — | Decision | $0.010$9.51estimate · DGX Spark | $9.51 estimate | $0.02057 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Open · 7.9B | — | Decision | $0.010$10.49estimate · DGX Spark | $10.49 estimate | $0.0227 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Clef-Flash Open · 9.0B | — | Decision | $0.011$10.85 list | $10.85 | $0.02347 | $0.072$71.95estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex Max v1.1 Closed · Efficient (provisional) | — | Decision | $0.013$12.88 | $12.88 | $0.02786 | — | provider price × reported usage |
| Drex Max Open · 27B | — | Decision | $0.013$13.06 list | $13.06 | $0.02824 | $0.055$54.68estimate · RTX PRO 6000 Blackwell | maker's list price |
| d1-2B Open · 1.9B | — | Decision | $0.014$14.07estimate · RTX PRO 6000 Blackwell | $14.07 estimate | $0.03043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Turion Jev 31B Open · 31B | — | Decision | $0.017$17.46estimate · DGX Spark | $17.46 estimate | $0.03778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Jev 1.14 Closed · Efficient | — | Decision | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | Decision | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | Decision | $0.022$22.02 | $22.02 | $0.04764 | — | provider price × reported usage |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | Decision | $0.022$22.38estimate · RTX PRO 6000 Blackwell | $22.38 estimate | $0.04842 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Mammoth-1 Open · 27B | — | Decision | $0.023$22.56 list | $22.56 | $0.0488 | $0.104$104estimate · RTX PRO 6000 Blackwell | maker's list price |
| Chinese Decision Pointer 4B Open · 4.0B | — | Decision | $0.023$22.70estimate · RTX PRO 6000 Blackwell | $22.70 estimate | $0.04911 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Open · 4.2B | — | Decision | $0.023$23.34estimate · RTX PRO 6000 Blackwell | $23.34 estimate | $0.05049 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | Decision | $0.024$23.94 list | $23.94 | $0.05179 | $0.028$27.50estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| GLiDE 27B Open · 27B | — | Decision | $0.026$26.16estimate · DGX Spark | $26.16 estimate | $0.05657 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Strands Decider 2B Open · 2.0B | — | Decision | $0.026$26.22estimate · RTX PRO 6000 Blackwell | $26.22 estimate | $0.05672 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Adjudex 9B v5 Open · 9.0B | — | Decision | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | Decision | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE API Closed · Efficient | — | Decision | $0.028$28.30 | $28.30 | $0.0612 | — | provider price × reported usage |
| Jeffan-9B-1211 Open · 9.0B | — | Decision | $0.029$28.78estimate · RTX PRO 6000 Blackwell | $28.78 estimate | $0.06225 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | Decision | $0.031$31.02estimate · RTX PRO 6000 Blackwell | $31.02 estimate | $0.06709 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | Decision | $0.032$31.98estimate · RTX PRO 6000 Blackwell | $31.98 estimate | $0.06917 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | Decision | $0.034$33.58estimate · RTX PRO 6000 Blackwell | $33.58 estimate | $0.07263 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Open · 12B | — | Decision | $0.036$36.13estimate · RTX PRO 6000 Blackwell | $36.13 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | Decision | $0.038$38.37estimate · RTX PRO 6000 Blackwell | $38.37 estimate | $0.083 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Sieve 31B v3 Pareto Open · 31B | — | Decision | $0.039$39.14estimate · DGX Spark | $39.14 estimate | $0.08465 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d3 Open · 26B | — | Decision | $0.041$40.61estimate · RTX PRO 6000 Blackwell | $40.61 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | Decision | $0.041$40.93estimate · RTX PRO 6000 Blackwell | $40.93 estimate | $0.08853 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | Decision | $0.047$46.87estimate · DGX Spark | $46.87 estimate | $0.1014 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | Chat | $0.060$59.94estimate · DGX Spark | $59.94 estimate | $0.1296 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | Decision | $0.061$60.76estimate · RTX PRO 6000 Blackwell | $60.76 estimate | $0.1314 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | Decision | $0.066$65.87estimate · RTX PRO 6000 Blackwell | $65.87 estimate | $0.1425 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | Decision | $0.069$68.54 list | $68.54 | $0.1483 | $0.131$131estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-31B-Gen2 Open · 31B | — | Decision | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | Decision | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | Decision | $0.084$84.42estimate · RTX PRO 6000 Blackwell | $84.42 estimate | $0.1826 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | Decision | $0.089$88.58estimate · RTX PRO 6000 Blackwell | $88.58 estimate | $0.1916 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active | — | Chat | $0.103$103 | $102.85 | $0.2225 | — | provider price × reported usage |
| Karpo Decision 31B Open · 31B | — | Decision | $0.113$113estimate · RTX PRO 6000 Blackwell | $112.56 estimate | $0.2435 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B Pareto on · chat Open · 1.7B | — | Chat | $0.132$132estimate · RTX PRO 6000 Blackwell | $132.39 estimate | $0.2808 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | Chat | $0.138$138estimate · DGX Spark | $138.05 estimate | $0.2986 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Logits Sigma-1 Duo Open · 57B | — | Decision | $0.141$141estimate · RTX PRO 6000 Blackwell | $140.70 estimate | $0.3043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | Chat | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.77 estimate | $0.3182 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Open · 57B | — | Decision | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.98 estimate | $0.3071 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active | — | Chat | $0.166$166 | $166.50 | $0.3601 | — | provider price × reported usage |
| Open-Jev-27B-v1.1 Open · 27B | — | Decision | $0.169$169estimate · RTX PRO 6000 Blackwell | $169.16 estimate | $0.3659 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | Chat | $0.196$196estimate · RTX PRO 6000 Blackwell | $195.81 estimate | $0.4219 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B chat Open · 5.1B | — | Chat | $0.208$208estimate · RTX PRO 6000 Blackwell | $208.17 estimate | $0.4503 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | Chat | $0.212$212 | $211.85 | $0.4582 | — | provider price × reported usage |
| MiniCPM5-2B off · chat Open · 2.5B | — | Chat | $0.219$219estimate · RTX PRO 6000 Blackwell | $218.83 estimate | $0.48 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | Chat | $0.227$227 | $227.42 | $0.4919 | — | provider price × reported usage |
| Gemma 4 E4B on · chat Open · 8.0B | — | Chat | $0.233$233estimate · DGX Spark | $232.86 estimate | $0.5037 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | Chat | $0.266$266estimate · RTX PRO 6000 Blackwell | $266.05 estimate | $0.5755 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B chat Open · 8.0B | — | Chat | $0.271$271estimate · RTX PRO 6000 Blackwell | $271.17 estimate | $0.5865 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active | — | Chat | $0.295$295estimate · DGX Spark | $294.82 estimate | $0.6377 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Qwen3.5-4B off · chat Open · 4.7B | — | Chat | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | Chat | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B | — | Chat | $0.318$318estimate · RTX PRO 6000 Blackwell | $317.64 estimate | $0.7034 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | Chat | $0.355$355estimate · RTX PRO 6000 Blackwell | $354.95 estimate | $0.7678 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | Chat | $0.363$363estimate · RTX PRO 6000 Blackwell | $363.26 estimate | $0.7857 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Muse Ember max · chat Closed · Efficient | — | Chat | $0.365$365 | $364.58 | $0.7886 | — | provider price × reported usage |
| Qwen3.8 Flash off · chat Closed · Undisclosed | — | Chat | $0.397$397 | $396.92 | $0.8585 | — | provider price × reported usage |
| MiniCPM5-2B on · chat Open · 2.5B | — | Chat | $0.409$409estimate · RTX PRO 6000 Blackwell | $408.56 estimate | $0.8978 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | Chat | $0.410$410estimate · RTX PRO 6000 Blackwell | $409.63 estimate | $0.886 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified chat Open · 12B | — | Chat | $0.467$467estimate · RTX PRO 6000 Blackwell | $467.19 estimate | $1.011 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | Chat | $0.482$482estimate · RTX PRO 6000 Blackwell† | $482.22 estimate† | $1.043 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | Chat | $0.518$518estimate · RTX PRO 6000 Blackwell | $518.03 estimate | $1.121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B low · chat Open · 27B | — | Chat | $0.530$530estimate · RTX PRO 6000 Blackwell | $529.86 estimate | $1.146 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B chat Open · 31B | — | Chat | $0.531$531estimate · RTX PRO 6000 Blackwell | $530.82 estimate | $1.148 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | Chat | $0.638$638 | $638.48 | $1.381 | — | provider price × reported usage |
| Granite-4.2-3B high · chat Open · 3.0B | — | Chat | $0.781$781estimate · RTX PRO 6000 Blackwell | $780.88 estimate | $1.689 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | Chat | $0.939$939estimate · RTX PRO 6000 Blackwell | $939.17 estimate | $2.031 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | Chat | $0.954$954 | $953.81 | $2.063 | — | provider price × reported usage |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | Decision | $1.023$1,023 | $1,022.79 | $2.212 | — | provider price × reported usage |
| Qwen3.5-4B on · chat Open · 4.7B | — | Chat | $1.271$1,271estimate · RTX PRO 6000 Blackwell | $1,271.09 estimate | $2.749 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | Chat | $1.478$1,478 | $1,478.15 | $3.197 | — | provider price × reported usage |
| Raon-1 nano on · chat Open · 3.1B | — | Chat | $1.670$1,670estimate · RTX PRO 6000 Blackwell† | $1,670.17 estimate† | $3.613 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | Chat | $1.901$1,901estimate · RTX PRO 6000 Blackwell† | $1,900.72 estimate† | $4.111 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | Chat | $2.470$2,470estimate · RTX PRO 6000 Blackwell | $2,469.92 estimate | $5.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Sonnet 5.5 chat Closed · Workhorse | — | Chat | $4.734$4,734 API list-price equivalent | $4,734.30 | $10.24 | — | Claude subscription |
| Claude Opus 5.5 chat Closed · Flagship | — | Chat | $9.871$9,871 API list-price equivalent | $9,870.94 | $21.35 | — | Claude subscription |
| Grok 4.7 chat Closed · Undisclosed | — | Chat | $12.061$12,061 API list-price equivalent | $12,061.26 | $26.09 | — | xAI subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | Chat | — | — | — | — | |
| GPT-6 Astra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Hydrogen chat | — | Chat | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6 Luna chat Closed · Efficient | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 2,151 of 2,163); shown once.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Not ranked (only complete runs are): Claude Fable 5.2 (chat) (not running now, 5 of 2,163 stored, last answer 11 Nov 12:02 UTC); Gemini 4 Carbon (high · chat) (paused for now at 54 of 2,163 (zeki #4681, 2026-11-11T11:53Z)); Granite-4.2-3B (off · chat) (not running now, 25 of 2,163 stored, last answer 16 Nov 20:02 UTC); Qwen3.5-397B-A17B (on · chat) (Logic only; on this test only its 25-question smoke test; cost so far $5.552$5,552 per 1,000 questions, over the 25 of 2,163 answered); GPT-6 Luna (max · chat) (paused for now at 200 of 2,163 (zeki #4129, 2026-11-10T14:13Z); cost so far $0.182$182 per 1,000 questions, over the 200 of 2,163 answered).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $9.871$9,871 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% | — | Chat | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,200 ms | 4,790 ms | $9.871$9,871 API list-price equivalent |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $1.478$1,478 per 1,000 questionsper 1M questions Logic 96.3% | — | Chat | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 4,710 ms | 9,130 ms | $1.478$1,478 |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% | — | Chat | 99.7%99.4%–99.9% | ab | 2,157 / 2,163 | Not measured | Not measured | 60.0%answered 100% | 5,030 ms | 6,490 ms | $0.000$0.00 |
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $4.734$4,734 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% | — | Chat | 99.7%99.3%–99.8% | abcd | 2,156 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,460 ms | 5,970 ms | $4.734$4,734 API list-price equivalent |
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.227$227 per 1,000 questionsper 1M questions Logic 96.2% | — | Chat | 99.6%99.3%–99.8% | abcdefg | 2,155 / 2,163 | Not measured | Not measured | 20.0%answered 100% | 1,910 ms | 3,570 ms | $0.227$227 |
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% | — | Chat | 99.6%99.3%–99.8% | abce | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 5,280 ms | 7,260 ms | $0.000$0.00 |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% | — | Chat | 99.6%99.3%–99.8% | abce | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 4,810 ms | 6,560 ms | $0.000$0.00 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $0.954$954 per 1,000 questionsper 1M questions Logic 95.4% | — | Chat | 99.6%99.3%–99.8% | abce | 2,155 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 2,030 ms | 6,330 ms | $0.954$954 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% | — | Chat | 99.6%99.2%–99.8% | abcdefg | 2,154 / 2,163 | Not measured | Not measured | 75.0%answered 99% | 13,410 ms | 27,800 ms | — |
| Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Logic 96.1% | — | Chat | 99.6%99.2%–99.8% | abcdefg | 2,154 / 2,163 | Not measured | Not measured | 85.7%answered 99% | 4,400 ms | 6,680 ms | — |
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.365$365 per 1,000 questionsper 1M questions Logic 95.5% | — | Chat | 99.6%99.2%–99.8% | abcdefg | 2,154 / 2,163 | Not measured | Not measured | 64.3%answered 100% | 1,260 ms | 2,270 ms | $0.365$365 |
| Grok 4.7 chat Closed · Undisclosed Grok 4.7 chat · xAI · chat model Parameters: not disclosed by maker (closed weights) Cost $12.061$12,061 API list-price equivalent per 1,000 questionsper 1M questions Logic running | — | Chat | 99.5%99.2%–99.7% | abcdefg | 2,153 / 2,163 | Not measured | Not measured | 97.5%answered 98% | 11,100 ms | 66,530 ms | $12.061$12,061 API list-price equivalent |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $0.638$638 per 1,000 questionsper 1M questions Logic 95.6% | — | Chat | 99.5%99.2%–99.7% | abcdef | 2,153 / 2,163 | Not measured | Not measured | 61.5%answered 100% | 2,110 ms | 6,000 ms | $0.638$638 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.397$397 per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 99.5%99.1%–99.7% | abcdefgh | 2,152 / 2,163 | Not measured | Not measured | 16.7%answered 100% | 5,740 ms | 11,800 ms | $0.397$397 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.212$212 per 1,000 questionsper 1M questions Logic 95.1% | — | Chat | 99.4%99.0%–99.7% | abcdefgh | 2,151 / 2,163 | Not measured | Not measured | 50.0%answered 100% | 8,200 ms | 18,090 ms | $0.212$212 |
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $1.271$1,271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% | — | Chat | 99.4%99.0%–99.7% | abcdefgh | 2,151 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 46,630 ms | 96,460 ms | $1.271$1,271estimate · RTX PRO 6000 Blackwell |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.531$531estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% | — | Chat | 99.4%99.0%–99.6% | abcdefghi | 2,150 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 11,110 ms | 16,690 ms | $0.531$531estimate · RTX PRO 6000 Blackwell |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active Cost $0.295$295estimate · DGX Spark per 1,000 questionsper 1M questions Logic running | — | Chat | 99.3%98.9%–99.6% | abcdefghijk | 2,148 / 2,163 | Not measured | Not measured | Not measured | 8,130 ms | 16,920 ms | $0.295$295estimate · DGX Spark |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $0.409$409estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% | — | Chat | 99.2%99.2%–99.3% min–max of 3 seeds | abcdefghij | 2,147 / 2,163seed 0 run | Not measured | Not measured | 53.6%answered 99% | 12,537 ms | 32,720 ms | $0.409$409estimate · RTX PRO 6000 Blackwell |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 99.2%98.7%–99.5% | abcdefghijk | 2,146 / 2,163 | Not measured | Not measured | 77.8%answered 99% | 4,080 ms | 5,870 ms | $0.000$0.00 |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $0.781$781estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% | — | Chat | 99.2%98.7%–99.5% | abcdefghij | 2,146 / 2,163 | Not measured | Not measured | 61.5%answered 99% | 15,340 ms | 53,280 ms | $0.781$781estimate · RTX PRO 6000 Blackwell |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $0.530$530estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 99.2%98.7%–99.5% | abcdefghijk | 2,146 / 2,163 | Not measured | Not measured | 13.0%answered 100% | 10,020 ms | 20,630 ms | $0.530$530estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $0.939$939estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% | — | Chat | 99.0%98.5%–99.4% | bcdefghijk | 2,142 / 2,163 | Not measured | Not measured | 71.4%answered 99% | 27,950 ms | 60,210 ms | $0.939$939estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% | — | Chat | 98.9%98.8%–99.0% min–max of 3 seeds | abcdefghijkl | 2,142 / 2,163seed 0 run | Not measured | Not measured | Not measured | 3,990 ms | 10,603 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.132$132estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% | — | Chat | 98.9%98.8%–99.1% min–max of 3 seeds | bcdefghijkl | 2,139 / 2,163seed 0 run | Not measured | Not measured | 67.6%answered 99% | 3,893 ms | 10,353 ms | $0.132$132estimate · RTX PRO 6000 Blackwell |
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% | — | Chat | 98.8%98.3%–99.2% | cdefghijklm | 2,138 / 2,163 | Not measured | Not measured | 14.3%answered 100% | 19,100 ms | 28,120 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.166$166 per 1,000 questionsper 1M questions Logic 93.2% | — | Chat | 98.8%98.2%–99.2% | dfghijklm | 2,137 / 2,163 | Not measured | Not measured | 17.4%answered 100% | 3,800 ms | 7,970 ms | $0.166$166 |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.060$59.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% | — | Chat | 98.7%98.1%–99.1% | ghijklm | 2,135 / 2,163 | Not measured | Not measured | 67.7%answered 99% | 19,570 ms | 53,410 ms | $0.060$59.94estimate · DGX Spark |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.318$318estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% | — | Chat | 98.5%98.4%–98.8% min–max of 3 seeds | efghijklm | 2,136 / 2,163seed 0 run | Not measured | Not measured | 45.5%answered 99% | 11,190 ms | 22,357 ms | $0.318$318estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.518$518estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% | — | Chat | 98.5%97.9%–99.0% | hijklmn | 2,131 / 2,163 | Not measured | Not measured | Not measured | 20,640 ms | 34,280 ms | $0.518$518estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% | — | Chat | 98.5%97.9%–98.9% | hijklmno | 2,130 / 2,163 | Not measured | Not measured | 36.0%answered 100% | 10,960 ms | 19,970 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.233$233estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% | — | Chat | 98.2%97.6%–98.7% | jklmno | 2,125 / 2,163 | Not measured | Not measured | 37.0%answered 99% | 108,720 ms | 175,470 ms | $0.233$233estimate · DGX Spark |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.103$103 per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 98.2%97.6%–98.7% | ijklmnop | 2,125 / 2,163 | Not measured | Not measured | 40.6%answered 99% | 1,670 ms | 7,470 ms | $0.103$103 |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.219$219estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% | — | Chat | 98.2%98.1%–98.3% min–max of 3 seeds | jklmnop | 2,126 / 2,163seed 0 run | Not measured | Not measured | 31.0%answered 99% | 6,353 ms | 15,697 ms | $0.219$219estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.271$271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% | — | Chat | 98.1%97.4%–98.6% | klmnop | 2,122 / 2,163 | Not measured | Not measured | 44.0%answered 99% | 10,770 ms | 17,700 ms | $0.271$271estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.410$410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% | — | Chat | 97.8%97.1%–98.4% | lmnopq | 2,116 / 2,163 | Not measured | Not measured | Not measured | 14,870 ms | 26,040 ms | $0.410$410estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.196$196estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% | — | Chat | 97.7%97.6%–97.8% min–max of 3 seeds | mnopqr | 2,111 / 2,163seed 0 run | Not measured | Not measured | 23.0%answered 99% | 7,137 ms | 14,787 ms | $0.196$196estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.670$1,670estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% | — | Chat | 97.0%96.2%–97.6% | nopqr | 2,098 / 2,163 | Not measured | Not measured | Not measured | 34,320 ms† | 108,010 ms† | $1.670$1,670estimate · RTX PRO 6000 Blackwell† |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.138$138estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% | — | Chat | 96.9%96.1%–97.6% | opqr | 2,097 / 2,163 | Not measured | Not measured | 16.5%answered 99% | 64,440 ms | 106,260 ms | $0.138$138estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.208$208estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% | — | Chat | 96.8%95.9%–97.4% | pqrs | 2,093 / 2,163 | Not measured | Not measured | 25.3%answered 99% | 8,120 ms | 13,650 ms | $0.208$208estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $0.482$482estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% | — | Chat | 96.3%95.4%–97.0% | qrst | 2,082 / 2,163 | Not measured | Not measured | 45.2%answered 97% | 14,530 ms† | 33,190 ms† | $0.482$482estimate · RTX PRO 6000 Blackwell† |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.355$355estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% | — | Chat | 95.8%94.9%–96.6% | rstu | 2,072 / 2,163 | Not measured | Not measured | 39.3%answered 98% | 8,040 ms | 18,980 ms | $0.355$355estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $2.470$2,470estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% | — | Chat | 94.7%93.7%–95.6% | stu | 2,049 / 2,163 | Not measured | Not measured | 20.3%answered 99% | 98,390 ms | 194,980 ms | $2.470$2,470estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.363$363estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% | — | Chat | 94.6%93.6%–95.5% | stu | 2,047 / 2,163 | Not measured | Not measured | 31.3%answered 98% | 11,200 ms | 32,140 ms | $0.363$363estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% | — | Chat | 94.0%92.9%–94.9% | tu | 2,033 / 2,163 | Not measured | Not measured | 26.4%answered 99% | 9,860 ms | 21,240 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% | — | Chat | 93.7%92.6%–94.6% | u | 2,026 / 2,163 | Not measured | Not measured | 68.9%answered 95% | 3,830 ms | 5,000 ms | $0.000$0.00 |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.266$266estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% | — | Chat | 90.8%89.5%–91.9% | v | 1,963 / 2,163 | Not measured | Not measured | 35.9%answered 96% | 7,890 ms | 14,890 ms | $0.266$266estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.901$1,901estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% | — | Chat | 89.6%88.2%–90.8% | v | 1,937 / 2,163 | Not measured | Not measured | 22.1%answered 98% | 25,550 ms† | 134,840 ms† | $1.901$1,901estimate · RTX PRO 6000 Blackwell† |
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.113$113estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% | — | Decision | 81.6%79.9%–83.1% | w | 1,764 / 2,163 | 837 right of 837 kept0 mistakes · 1,326 skipped | 0.903 | 4.8%answered 99% | 155 ms | 186 ms | $0.113$113estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% | — | Decision | 80.3%78.5%–81.9% | wx | 1,736 / 2,163 | 469 right of 470 kept1 wrong · 1,693 skipped | 0.898 | 0.2%answered 100% | 114 ms | 120 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% | — | Decision | 78.0%76.2%–79.7% | xy | 1,687 / 2,163 | 178 right of 178 kept0 mistakes · 1,985 skipped | 0.903 | 2.3%answered 99% | 189 ms | 232 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.047$46.87estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% | — | Decision | 77.4%75.6%–79.1% | yz | 1,674 / 2,163 | 464 right of 465 kept1 wrong · 1,698 skipped | 0.895 | 1.1%answered 100% | 764 ms | 867 ms | $0.047$46.87estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.084$84.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% | — | Decision | 77.3%75.5%–79.1% | yz | 1,673 / 2,163 | 318 right of 318 kept0 mistakes · 1,845 skipped | 0.899 | 1.7%answered 100% | 118 ms | 142 ms | $0.084$84.42estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.039$39.14estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% | — | Decision | 77.1%75.3%–78.8% | yz | 1,668 / 2,163 | 215 right of 215 kept0 mistakes · 1,948 skipped | 0.891 | 2.1%answered 100% | 637 ms | 735 ms | $0.039$39.14estimate · DGX Spark |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% | — | Decision | 75.2%73.4%–77.0% | zA | 1,627 / 2,163 | 15 right of 15 kept0 mistakes · 2,148 skipped | 0.899 | 2.1%answered 99% | 196 ms | 221 ms | $0.141$141estimate · RTX PRO 6000 Blackwell |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 54.9% | — | Decision | 74.1%72.2%–75.9% | yzAB | 1,603 / 2,163 | 213 right of 213 kept0 mistakes · 1,950 skipped | 0.876 | 87.4%answered 61% | 179 ms | 426 ms | $0.006$6.14 |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | Decision | 74.0%72.1%–75.8% | zAB | 1,600 / 2,163 | 412 right of 412 kept0 mistakes · 1,751 skipped | 0.895 | 70.6%answered 75% | 140 ms | 173 ms | $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.066$65.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% | — | Decision | 72.5%70.6%–74.3% | ABC | 1,568 / 2,163 | 323 right of 323 kept0 mistakes · 1,840 skipped | 0.856 | 9.4%answered 97% | 91 ms | 103 ms | $0.066$65.87estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.026$26.16estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% | — | Decision | 72.0%70.1%–73.8% | ABCD | 1,557 / 2,163 | 198 right of 199 kept1 wrong · 1,964 skipped | 0.864 | 21.6%answered 94% | 425 ms | 461 ms | $0.026$26.16estimate · DGX Spark |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.089$88.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% | — | Decision | 71.5%69.6%–73.4% | BCD | 1,547 / 2,163 | 11 right of 11 kept0 mistakes · 2,152 skipped | 0.857 | 5.3%answered 98% | 118 ms | 148 ms | $0.089$88.58estimate · RTX PRO 6000 Blackwell |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.022$22.02 per 1,000 questionsper 1M questions Logic 50.0% | — | Decision | 71.3%69.4%–73.2% | ABCD | 1,543 / 2,163 | 214 right of 214 kept0 mistakes · 1,949 skipped | 0.859 | 3.9%answered 99% | 226 ms | 351 ms | $0.022$22.02 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.028$28.30 per 1,000 questionsper 1M questions Logic 54.5% | — | Decision | 70.8%68.8%–72.7% | BCD | 1,531 / 2,163 | 161 right of 161 kept0 mistakes · 2,002 skipped | 0.865 | 40.6%answered 86% | 264 ms | 279 ms | $0.028$28.30 |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.041$40.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | Decision | 70.4%68.5%–72.3% | BCDE | 1,523 / 2,163 | 162 right of 163 kept1 wrong · 2,000 skipped | 0.878 | 4.7%answered 99% | 55 ms | 62 ms | $0.041$40.61estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% | — | Decision | 70.3%68.4%–72.2% | BCDE | 1,521 / 2,163 | 719 right of 729 kept10 wrong · 1,434 skipped | 0.861 | 4.5%answered 99% | 76 ms | 89 ms | $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.013$12.88 per 1,000 questionsper 1M questions Logic 54.8% | — | Decision | 70.1%68.1%–72.0% | BCDE | 1,516 / 2,163 | 315 right of 315 kept0 mistakes · 1,848 skipped | 0.867 | 43.5%answered 84% | 326 ms | 473 ms | $0.013$12.88 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.009$9.10 per 1,000 questionsper 1M questions Logic 47.3% | — | Decision | 70.0%68.0%–71.9% | BCDEF | 1,514 / 2,163 | No answers at this threshold | 0.858 | 82.7%answered 62% | 387 ms | 391 ms | $0.009$9.10 |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% | — | Decision | 69.9%67.9%–71.8% | CDEF | 1,511 / 2,163 | 131 right of 131 kept0 mistakes · 2,032 skipped | 0.862 | 35.8%answered 88% | 79 ms | 84 ms | $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$0.91 per 1,000 questionsper 1M questions Logic 47.3% | — | Decision | 69.9%67.9%–71.8% | BCDEF | 1,511 / 2,163 | No answers at this threshold | 0.859 | 82.9%answered 62% | 381 ms | 386 ms | $0.001$0.91 |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 51.6% | — | Decision | 69.9%67.9%–71.8% | CDE | 1,511 / 2,163 | 115 right of 115 kept0 mistakes · 2,048 skipped | 0.876 | 86.7%answered 58% | 215 ms | 347 ms | $0.006$6.14 |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.004$4.29 per 1,000 questionsper 1M questions Logic 53.3% | — | Decision | 69.8%67.8%–71.7% | CDEF | 1,509 / 2,163 | 331 right of 331 kept0 mistakes · 1,832 skipped | 0.869 | 7.0%answered 98% | 226 ms | 368 ms | $0.004$4.29 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.023$1,023 per 1,000 questionsper 1M questions Logic 52.5% | with web | Decision | 69.3%67.3%–71.2% | CDEFG | 1,499 / 2,163 | 326 right of 326 kept0 mistakes · 1,837 skipped | 0.873 | 15.4%answered 95% | 422 ms | 641 ms | $1.023$1,023 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.017$17.46estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% | — | Decision | 69.0%67.0%–70.9% | CDEFG | 1,493 / 2,163 | 43 right of 43 kept0 mistakes · 2,120 skipped | 0.864 | 2.2%answered 99% | 278 ms | 329 ms | $0.017$17.46estimate · DGX Spark |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.009$8.59 per 1,000 questionsper 1M questions Logic 52.4% | — | Decision | 68.6%66.6%–70.5% | DEFG | 1,483 / 2,163 | 360 right of 361 kept1 wrong · 1,802 skipped | 0.839 | 52.5%answered 80% | 381 ms | 390 ms | $0.009$8.59 |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.036$36.13estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% | — | Decision | 66.3%64.2%–68.2% | EFGH | 1,433 / 2,163 | 7 right of 7 kept0 mistakes · 2,156 skipped | 0.819 | 1.1%answered 100% | 49 ms | 55 ms | $0.036$36.13estimate · RTX PRO 6000 Blackwell |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 49.2% | — | Decision | 65.5%63.5%–67.5% | FGHI | 1,417 / 2,163 | 141 right of 141 kept0 mistakes · 2,022 skipped | 0.838 | 56.3%answered 76% | 204 ms | 386 ms | $0.003$2.92 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 46.0% | — | Decision | 65.2%63.2%–67.2% | GHI | 1,410 / 2,163 | 183 right of 183 kept0 mistakes · 1,980 skipped | 0.829 | 82.5%answered 56% | 172 ms | 290 ms | $0.003$2.92 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 48.2% | — | Decision | 65.1%63.1%–67.1% | GHI | 1,409 / 2,163 | 202 right of 202 kept0 mistakes · 1,961 skipped | 0.829 | 7.7%answered 97% | 163 ms | 185 ms | $0.019$18.99 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 46.3% | — | Decision | 64.1%62.0%–66.1% | HIJ | 1,386 / 2,163 | 248 right of 248 kept0 mistakes · 1,915 skipped | 0.834 | 5.7%answered 98% | 166 ms | 206 ms | $0.019$18.99 |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | Decision | 62.1%60.0%–64.1% | HIJK | 1,343 / 2,163 | 197 right of 198 kept1 wrong · 1,965 skipped | 0.806 | 3.1%answered 99% | 36 ms | 44 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | Decision | 61.3%59.3%–63.4% | IJK | 1,327 / 2,163 | 182 right of 183 kept1 wrong · 1,980 skipped | 0.803 | 65.0%answered 67% | 80 ms | 92 ms | $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.02estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% | — | Decision | 60.1%58.0%–62.1% | JKL | 1,300 / 2,163 | No answers at this threshold | 0.803 | 3.2%answered 99% | 41 ms | 46 ms | $0.031$31.02estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.029$28.78estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% | — | Decision | 59.8%57.7%–61.9% | JKL | 1,294 / 2,163 | 64 right of 64 kept0 mistakes · 2,099 skipped | 0.814 | 6.6%answered 97% | 37 ms | 51 ms | $0.029$28.78estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% | — | Decision | 59.8%57.7%–61.8% | JKL | 1,293 / 2,163 | 123 right of 123 kept0 mistakes · 2,040 skipped | 0.799 | 0.9%answered 100% | 37 ms | 43 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.007$6.81 per 1,000 questionsper 1M questions Logic 44.4% | — | Decision | 59.4%57.3%–61.4% | KLM | 1,284 / 2,163 | 27 right of 27 kept0 mistakes · 2,136 skipped | 0.787 | 5.1%answered 98% | 310 ms | 359 ms | $0.007$6.81 |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.009$8.92 per 1,000 questionsper 1M questions Logic 41.9% | — | Decision | 59.0%56.9%–61.0% | KLM | 1,276 / 2,163 | 174 right of 174 kept0 mistakes · 1,989 skipped | 0.813 | 20.7%answered 91% | 334 ms | 432 ms | $0.009$8.92 |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.032$31.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% | — | Decision | 58.8%56.7%–60.9% | KLM | 1,272 / 2,163 | 207 right of 210 kept3 wrong · 1,953 skipped | 0.804 | 1.7%answered 99% | 45 ms | 50 ms | $0.032$31.98estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.008$8.34estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% | — | Decision | 58.4%56.3%–60.5% | KLMN | 1,264 / 2,163 | 49 right of 49 kept0 mistakes · 2,114 skipped | 0.781 | 4.2%answered 98% | 140 ms | 146 ms | $0.008$8.34estimate · DGX Spark |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.010$10.49estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% | — | Decision | 57.9%55.8%–60.0% | KLMN | 1,253 / 2,163 | 73 right of 73 kept0 mistakes · 2,090 skipped | 0.798 | 24.6%answered 89% | 174 ms | 182 ms | $0.010$10.49estimate · DGX Spark |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.023$23.34estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% | — | Decision | 56.8%54.7%–58.9% | LMN | 1,229 / 2,163 | 33 right of 33 kept0 mistakes · 2,130 skipped | 0.789 | 3.8%answered 98% | 31 ms | 35 ms | $0.023$23.34estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.010$9.51estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% | — | Decision | 56.0%53.9%–58.1% | LMNOP | 1,212 / 2,163 | 244 right of 254 kept10 wrong · 1,909 skipped | 0.780 | 1.2%answered 99% | 153 ms | 175 ms | $0.010$9.51estimate · DGX Spark |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$4.56estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% | — | Decision | 56.0%53.9%–58.1% | LMNO | 1,211 / 2,163 | 54 right of 54 kept0 mistakes · 2,109 skipped | 0.765 | 1.6%answered 99% | 74 ms | 84 ms | $0.005$4.56estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.022$22.38estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% | — | Decision | 55.9%53.8%–58.0% | LMNO | 1,210 / 2,163 | 113 right of 114 kept1 wrong · 2,049 skipped | 0.787 | 8.4%answered 96% | 28 ms | 31 ms | $0.022$22.38estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.10estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% | — | Decision | 55.8%53.7%–57.9% | LMNO | 1,208 / 2,163 | 96 right of 96 kept0 mistakes · 2,067 skipped | 0.779 | 13.4%answered 94% | 67 ms | 69 ms | $0.004$4.10estimate · DGX Spark |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 40.8% | — | Decision | 55.6%53.5%–57.7% | LMNOP | 1,203 / 2,163 | 152 right of 152 kept0 mistakes · 2,011 skipped | 0.767 | 2.9%answered 99% | 184 ms | 297 ms | $0.003$2.92 |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$40.93estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% | — | Decision | 55.3%53.2%–57.4% | MNOP | 1,197 / 2,163 | 130 right of 130 kept0 mistakes · 2,033 skipped | 0.790 | 2.6%answered 99% | 58 ms | 64 ms | $0.041$40.93estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% | — | Decision | 55.2%53.1%–57.3% | MNOP | 1,195 / 2,163 | 150 right of 155 kept5 wrong · 2,008 skipped | 0.781 | 12.0%answered 94% | 40 ms | 47 ms | $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.169$169estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% | — | Decision | 54.6%52.4%–56.6% | MNOPQ | 1,180 / 2,163 | No answers at this threshold | 0.764 | 5.0%answered 98% | 233 ms | 272 ms | $0.169$169estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.061$60.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% | — | Decision | 53.8%51.7%–55.9% | NOPQ | 1,163 / 2,163 | 26 right of 26 kept0 mistakes · 2,137 skipped | 0.783 | 1.8%answered 99% | 79 ms | 106 ms | $0.061$60.76estimate · RTX PRO 6000 Blackwell |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.004$4.38 per 1,000 questionsper 1M questions Logic 45.1% | — | Decision | 53.5%51.4%–55.6% | NOPQ | 1,157 / 2,163 | 272 right of 281 kept9 wrong · 1,882 skipped | 0.789 | 3.2%answered 99% | 288 ms | 342 ms | $0.004$4.38 |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.034$33.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | Decision | 51.5%49.4%–53.6% | PQR | 1,114 / 2,163 | 126 right of 130 kept4 wrong · 2,033 skipped | 0.752 | 6.4%answered 97% | 47 ms | 52 ms | $0.034$33.58estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% | — | Decision | 51.1%49.0%–53.2% | OPQR | 1,105 / 2,163 | 12 right of 12 kept0 mistakes · 2,151 skipped | 0.767 | 1.6%answered 99% | 181 ms | 210 ms | $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.023$22.70estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% | — | Decision | 50.4%48.3%–52.5% | QR | 1,091 / 2,163 | 149 right of 155 kept6 wrong · 2,008 skipped | 0.746 | 3.4%answered 98% | 28 ms | 32 ms | $0.023$22.70estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% | — | Decision | 49.8%47.7%–51.9% | QR | 1,077 / 2,163 | No answers at this threshold | 0.730 | 2.0%answered 99% | 99 ms | 127 ms | $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.038$38.37estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | Decision | 46.9%44.8%–49.0% | RS | 1,014 / 2,163 | 20 right of 21 kept1 wrong · 2,142 skipped | 0.743 | 9.2%answered 95% | 52 ms | 61 ms | $0.038$38.37estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% | — | Decision | 44.3%42.3%–46.4% | S | 959 / 2,163 | No answers at this threshold | 0.735 | 4.9%answered 97% | 37 ms | 41 ms | $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% | — | Decision | 39.2%37.1%–41.2% | T | 847 / 2,163 | No answers at this threshold | 0.651 | 2.2%answered 99% | 113 ms | 144 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.27estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% | — | Decision | 38.7%36.6%–40.7% | T | 836 / 2,163 | 2 right of 2 kept0 mistakes · 2,161 skipped | 0.685 | 7.2%answered 95% | 36 ms | 37 ms | $0.002$2.27estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% | — | Decision | 37.1%35.1%–39.1% | TU | 802 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.656 | 80.2%answered 34% | 16 ms | 21 ms | $0.014$14.07estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.78estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% | — | Decision | 36.8%34.7%–38.8% | T | 795 / 2,163 | 15 right of 16 kept1 wrong · 2,147 skipped | 0.635 | 17.3%answered 89% | 28 ms | 30 ms | $0.002$1.78estimate · DGX Spark |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.20estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% | — | Decision | 35.3%33.3%–37.3% | TU | 763 / 2,163 | 8 right of 8 kept0 mistakes · 2,155 skipped | 0.636 | 1.7%answered 99% | 50 ms | 54 ms | $0.003$3.20estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% | — | Decision | 32.1%30.2%–34.1% | UV | 694 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.589 | 20.8%answered 85% | 20 ms | 21 ms | $0.001$1.28estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.53estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% | — | Decision | 29.7%27.8%–31.6% | V | 642 / 2,163 | No answers at this threshold | 0.583 | 11.3%answered 92% | 24 ms | 25 ms | $0.002$1.53estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.026$26.22estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% | — | Decision | 28.7%26.8%–30.6% | V | 620 / 2,163 | No answers at this threshold | 0.569 | 3.2%answered 98% | 35 ms | 41 ms | $0.026$26.22estimate · RTX PRO 6000 Blackwell |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.09estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% | — | Decision | 28.2%26.4%–30.2% | V | 611 / 2,163 | No answers at this threshold | 0.592 | 0.6%answered 100% | 15 ms | 17 ms | $0.001$1.09estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.69estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% | — | Decision | 28.2%26.3%–30.1% | V | 610 / 2,163 | No answers at this threshold | 0.596 | 0.3%answered 100% | 25 ms | 28 ms | $0.002$1.69estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$0.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% | — | Decision | 22.7%20.9%–24.5% | W | 490 / 2,163 | No answers at this threshold | 0.501 | 17.2%answered 86% | 12 ms | 13 ms | $0.001$0.85estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.77estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% | — | Decision | 20.6%18.9%–22.3% | W | 445 / 2,163 | No answers at this threshold | 0.500 | 72.8%answered 33% | 11 ms | 12 ms | $0.001$0.77estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI quincyqiang scar-ai Strands Agents victorbennett Metask Lab Google OpenBMB Alibaba IBM KRAFTON
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 20.6% at $0.001 per 1,000 questions ($0.77 per 1M) (estimate)
GLiClass v4 Edge: 22.7% at $0.001 per 1,000 questions ($0.85 per 1M) (estimate)
OneJev1.6-0.8B: 28.2% at $0.001 per 1,000 questions ($1.09 per 1M) (estimate)
GLiDE 0.8B: 32.1% at $0.001 per 1,000 questions ($1.28 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 29.7% at $0.002 per 1,000 questions ($1.53 per 1M) (estimate)
KnowLine-0.8B-Gen1: 28.2% at $0.002 per 1,000 questions ($1.69 per 1M) (estimate)
GLiDE 2B: 36.8% at $0.002 per 1,000 questions ($1.78 per 1M) (estimate)
Perplexity Decider Flash v1.2: 55.2% at $0.002 per 1,000 questions ($2.18 per 1M)
StartLux-Decision-v1.1-2B-Instant: 38.7% at $0.002 per 1,000 questions ($2.27 per 1M) (estimate)
KnowLine-2B-Gen1: 35.3% at $0.003 per 1,000 questions ($3.20 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 55.8% at $0.004 per 1,000 questions ($4.10 per 1M) (estimate)
KnowLine 4B Gen5.5: 56.0% at $0.005 per 1,000 questions ($4.56 per 1M) (estimate)
d1-2B: 37.1% at $0.014 per 1,000 questions ($14.07 per 1M) (estimate)
Perplexity Decider Flash v1.3: 55.9% at $0.022 per 1,000 questions ($22.38 per 1M) (estimate)
Chinese Decision Pointer 4B: 50.4% at $0.023 per 1,000 questions ($22.70 per 1M) (estimate)
vera-medium-omni: 56.8% at $0.023 per 1,000 questions ($23.34 per 1M) (estimate)
Strands Decider 2B: 28.7% at $0.026 per 1,000 questions ($26.22 per 1M) (estimate)
Verdin-4B-v1.1: 51.5% at $0.034 per 1,000 questions ($33.58 per 1M) (estimate)
LFM3-1.7B (off · chat): 98.7% at $0.060 per 1,000 questions ($59.94 per 1M) (estimate)
Metask-Jev-4B: 39.2% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
LFM3-1.7B (on · chat): 98.9% at $0.132 per 1,000 questions ($132 per 1M) (estimate)
Gemma 4 E2B (on · chat): 96.9% at $0.138 per 1,000 questions ($138 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 98.9% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 97.7% at $0.196 per 1,000 questions ($196 per 1M) (estimate)
Gemma 4 E2B (chat): 96.8% at $0.208 per 1,000 questions ($208 per 1M) (estimate)
MiniCPM5-2B (off · chat): 98.2% at $0.219 per 1,000 questions ($219 per 1M) (estimate)
Qwen3.5-4B (off · chat): 98.5% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 94.0% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
Qwen3.5-2B (off · chat): 94.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
MiniCPM5-2B (on · chat): 99.2% at $0.409 per 1,000 questions ($409 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 97.8% at $0.410 per 1,000 questions ($410 per 1M) (estimate)
Granite-4.2-3B (high · chat): 99.2% at $0.781 per 1,000 questions ($781 per 1M) (estimate)
Qwen3.5-4B (on · chat): 99.4% at $1.271 per 1,000 questions ($1,271 per 1M) (estimate)
Raon-1 nano (on · chat): 97.0% at $1.670 per 1,000 questions ($1,670 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (off · chat): 89.6% at $1.901 per 1,000 questions ($1,901 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Qwen3.5-2B (on · chat): 94.7% at $2.470 per 1,000 questions ($2,470 per 1M) (estimate)
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI quincyqiang scar-ai Strands Agents victorbennett Metask Lab Google OpenBMB Alibaba IBM KRAFTON
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | Decision | $0.001$0.77estimate · DGX Spark | $0.77 estimate | $0.001655 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | Decision | $0.001$0.85estimate · DGX Spark | $0.85 estimate | $0.001833 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | Decision | $0.001$1.09estimate · DGX Spark | $1.09 estimate | $0.002365 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Pareto Open · 0.8B | — | Decision | $0.001$1.28estimate · DGX Spark | $1.28 estimate | $0.002778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | Decision | $0.002$1.53estimate · DGX Spark | $1.53 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | Decision | $0.002$1.69estimate · DGX Spark | $1.69 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Pareto Open · 1.9B | — | Decision | $0.002$1.78estimate · DGX Spark | $1.78 estimate | $0.003843 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Pareto Open · 4.0B | — | Decision | $0.002$2.18 list | $2.18 | $0.004707 | $0.030$30.06estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | Decision | $0.002$2.27estimate · DGX Spark | $2.27 estimate | $0.004907 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-2B-Gen1 Open · 2.0B | — | Decision | $0.003$3.20estimate · DGX Spark | $3.20 estimate | $0.006917 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Pareto Open · 4.0B | — | Decision | $0.004$4.10estimate · DGX Spark | $4.10 estimate | $0.008867 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Pareto Open · 4.0B | — | Decision | $0.005$4.56estimate · DGX Spark | $4.56 estimate | $0.009872 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-2B Open · 1.9B | — | Decision | $0.014$14.07estimate · RTX PRO 6000 Blackwell | $14.07 estimate | $0.03043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | Decision | $0.022$22.38estimate · RTX PRO 6000 Blackwell | $22.38 estimate | $0.04842 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | Decision | $0.023$22.70estimate · RTX PRO 6000 Blackwell | $22.70 estimate | $0.04911 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Pareto Open · 4.2B | — | Decision | $0.023$23.34estimate · RTX PRO 6000 Blackwell | $23.34 estimate | $0.05049 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | Decision | $0.026$26.22estimate · RTX PRO 6000 Blackwell | $26.22 estimate | $0.05672 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | Decision | $0.034$33.58estimate · RTX PRO 6000 Blackwell | $33.58 estimate | $0.07263 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | Chat | $0.060$59.94estimate · DGX Spark | $59.94 estimate | $0.1296 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Metask-Jev-4B Open · 4.5B | — | Decision | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B Pareto on · chat Open · 1.7B | — | Chat | $0.132$132estimate · RTX PRO 6000 Blackwell | $132.39 estimate | $0.2808 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | Chat | $0.138$138estimate · DGX Spark | $138.05 estimate | $0.2986 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | Chat | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.77 estimate | $0.3182 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | Chat | $0.196$196estimate · RTX PRO 6000 Blackwell | $195.81 estimate | $0.4219 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B chat Open · 5.1B | — | Chat | $0.208$208estimate · RTX PRO 6000 Blackwell | $208.17 estimate | $0.4503 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | Chat | $0.219$219estimate · RTX PRO 6000 Blackwell | $218.83 estimate | $0.48 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B off · chat Open · 4.7B | — | Chat | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | Chat | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | Chat | $0.363$363estimate · RTX PRO 6000 Blackwell | $363.26 estimate | $0.7857 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B Pareto on · chat Open · 2.5B | — | Chat | $0.409$409estimate · RTX PRO 6000 Blackwell | $408.56 estimate | $0.8978 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | Chat | $0.410$410estimate · RTX PRO 6000 Blackwell | $409.63 estimate | $0.886 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | Chat | $0.781$781estimate · RTX PRO 6000 Blackwell | $780.88 estimate | $1.689 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto on · chat Open · 4.7B | — | Chat | $1.271$1,271estimate · RTX PRO 6000 Blackwell | $1,271.09 estimate | $2.749 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | Chat | $1.670$1,670estimate · RTX PRO 6000 Blackwell† | $1,670.17 estimate† | $3.613 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | Chat | $1.901$1,901estimate · RTX PRO 6000 Blackwell† | $1,900.72 estimate† | $4.111 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | Chat | $2.470$2,470estimate · RTX PRO 6000 Blackwell | $2,469.92 estimate | $5.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Granite-4.2-3B (off · chat) (not running now, 25 of 2,163 stored, last answer 16 Nov 20:02 UTC).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $1.271$1,271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% | — | Chat | 99.4%99.0%–99.7% | a | 2,151 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 46,630 ms | 96,460 ms | $1.271$1,271estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $0.409$409estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% | — | Chat | 99.2%99.2%–99.3% min–max of 3 seeds | ab | 2,147 / 2,163seed 0 run | Not measured | Not measured | 53.6%answered 99% | 12,537 ms | 32,720 ms | $0.409$409estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $0.781$781estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% | — | Chat | 99.2%98.7%–99.5% | ab | 2,146 / 2,163 | Not measured | Not measured | 61.5%answered 99% | 15,340 ms | 53,280 ms | $0.781$781estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% | — | Chat | 98.9%98.8%–99.0% min–max of 3 seeds | abc | 2,142 / 2,163seed 0 run | Not measured | Not measured | Not measured | 3,990 ms | 10,603 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.132$132estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% | — | Chat | 98.9%98.8%–99.1% min–max of 3 seeds | abcd | 2,139 / 2,163seed 0 run | Not measured | Not measured | 67.6%answered 99% | 3,893 ms | 10,353 ms | $0.132$132estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.060$59.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% | — | Chat | 98.7%98.1%–99.1% | abcde | 2,135 / 2,163 | Not measured | Not measured | 67.7%answered 99% | 19,570 ms | 53,410 ms | $0.060$59.94estimate · DGX Spark |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% | — | Chat | 98.5%97.9%–98.9% | bcde | 2,130 / 2,163 | Not measured | Not measured | 36.0%answered 100% | 10,960 ms | 19,970 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.219$219estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% | — | Chat | 98.2%98.1%–98.3% min–max of 3 seeds | cdef | 2,126 / 2,163seed 0 run | Not measured | Not measured | 31.0%answered 99% | 6,353 ms | 15,697 ms | $0.219$219estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.410$410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% | — | Chat | 97.8%97.1%–98.4% | defg | 2,116 / 2,163 | Not measured | Not measured | Not measured | 14,870 ms | 26,040 ms | $0.410$410estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.196$196estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% | — | Chat | 97.7%97.6%–97.8% min–max of 3 seeds | efg | 2,111 / 2,163seed 0 run | Not measured | Not measured | 23.0%answered 99% | 7,137 ms | 14,787 ms | $0.196$196estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.670$1,670estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% | — | Chat | 97.0%96.2%–97.6% | fg | 2,098 / 2,163 | Not measured | Not measured | Not measured | 34,320 ms† | 108,010 ms† | $1.670$1,670estimate · RTX PRO 6000 Blackwell† |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.138$138estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% | — | Chat | 96.9%96.1%–97.6% | fg | 2,097 / 2,163 | Not measured | Not measured | 16.5%answered 99% | 64,440 ms | 106,260 ms | $0.138$138estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.208$208estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% | — | Chat | 96.8%95.9%–97.4% | g | 2,093 / 2,163 | Not measured | Not measured | 25.3%answered 99% | 8,120 ms | 13,650 ms | $0.208$208estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $2.470$2,470estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% | — | Chat | 94.7%93.7%–95.6% | h | 2,049 / 2,163 | Not measured | Not measured | 20.3%answered 99% | 98,390 ms | 194,980 ms | $2.470$2,470estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.363$363estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% | — | Chat | 94.6%93.6%–95.5% | h | 2,047 / 2,163 | Not measured | Not measured | 31.3%answered 98% | 11,200 ms | 32,140 ms | $0.363$363estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% | — | Chat | 94.0%92.9%–94.9% | h | 2,033 / 2,163 | Not measured | Not measured | 26.4%answered 99% | 9,860 ms | 21,240 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.901$1,901estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% | — | Chat | 89.6%88.2%–90.8% | i | 1,937 / 2,163 | Not measured | Not measured | 22.1%answered 98% | 25,550 ms† | 134,840 ms† | $1.901$1,901estimate · RTX PRO 6000 Blackwell† |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.023$23.34estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% | — | Decision | 56.8%54.7%–58.9% | j | 1,229 / 2,163 | 33 right of 33 kept0 mistakes · 2,130 skipped | 0.789 | 3.8%answered 98% | 31 ms | 35 ms | $0.023$23.34estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$4.56estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% | — | Decision | 56.0%53.9%–58.1% | j | 1,211 / 2,163 | 54 right of 54 kept0 mistakes · 2,109 skipped | 0.765 | 1.6%answered 99% | 74 ms | 84 ms | $0.005$4.56estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.022$22.38estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% | — | Decision | 55.9%53.8%–58.0% | j | 1,210 / 2,163 | 113 right of 114 kept1 wrong · 2,049 skipped | 0.787 | 8.4%answered 96% | 28 ms | 31 ms | $0.022$22.38estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.10estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% | — | Decision | 55.8%53.7%–57.9% | j | 1,208 / 2,163 | 96 right of 96 kept0 mistakes · 2,067 skipped | 0.779 | 13.4%answered 94% | 67 ms | 69 ms | $0.004$4.10estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% | — | Decision | 55.2%53.1%–57.3% | j | 1,195 / 2,163 | 150 right of 155 kept5 wrong · 2,008 skipped | 0.781 | 12.0%answered 94% | 40 ms | 47 ms | $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.034$33.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | Decision | 51.5%49.4%–53.6% | k | 1,114 / 2,163 | 126 right of 130 kept4 wrong · 2,033 skipped | 0.752 | 6.4%answered 97% | 47 ms | 52 ms | $0.034$33.58estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.023$22.70estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% | — | Decision | 50.4%48.3%–52.5% | k | 1,091 / 2,163 | 149 right of 155 kept6 wrong · 2,008 skipped | 0.746 | 3.4%answered 98% | 28 ms | 32 ms | $0.023$22.70estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% | — | Decision | 39.2%37.1%–41.2% | l | 847 / 2,163 | No answers at this threshold | 0.651 | 2.2%answered 99% | 113 ms | 144 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.27estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% | — | Decision | 38.7%36.6%–40.7% | l | 836 / 2,163 | 2 right of 2 kept0 mistakes · 2,161 skipped | 0.685 | 7.2%answered 95% | 36 ms | 37 ms | $0.002$2.27estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% | — | Decision | 37.1%35.1%–39.1% | l | 802 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.656 | 80.2%answered 34% | 16 ms | 21 ms | $0.014$14.07estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.78estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% | — | Decision | 36.8%34.7%–38.8% | l | 795 / 2,163 | 15 right of 16 kept1 wrong · 2,147 skipped | 0.635 | 17.3%answered 89% | 28 ms | 30 ms | $0.002$1.78estimate · DGX Spark |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.20estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% | — | Decision | 35.3%33.3%–37.3% | lm | 763 / 2,163 | 8 right of 8 kept0 mistakes · 2,155 skipped | 0.636 | 1.7%answered 99% | 50 ms | 54 ms | $0.003$3.20estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% | — | Decision | 32.1%30.2%–34.1% | mn | 694 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.589 | 20.8%answered 85% | 20 ms | 21 ms | $0.001$1.28estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.53estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% | — | Decision | 29.7%27.8%–31.6% | n | 642 / 2,163 | No answers at this threshold | 0.583 | 11.3%answered 92% | 24 ms | 25 ms | $0.002$1.53estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.026$26.22estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% | — | Decision | 28.7%26.8%–30.6% | n | 620 / 2,163 | No answers at this threshold | 0.569 | 3.2%answered 98% | 35 ms | 41 ms | $0.026$26.22estimate · RTX PRO 6000 Blackwell |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.09estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% | — | Decision | 28.2%26.4%–30.2% | n | 611 / 2,163 | No answers at this threshold | 0.592 | 0.6%answered 100% | 15 ms | 17 ms | $0.001$1.09estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.69estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% | — | Decision | 28.2%26.3%–30.1% | n | 610 / 2,163 | No answers at this threshold | 0.596 | 0.3%answered 100% | 25 ms | 28 ms | $0.002$1.69estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$0.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% | — | Decision | 22.7%20.9%–24.5% | o | 490 / 2,163 | No answers at this threshold | 0.501 | 17.2%answered 86% | 12 ms | 13 ms | $0.001$0.85estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.77estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% | — | Decision | 20.6%18.9%–22.3% | o | 445 / 2,163 | No answers at this threshold | 0.500 | 72.8%answered 33% | 11 ms | 12 ms | $0.001$0.77estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Base Labs StartLux Apertix AI Fastino Cloudflare Blockbrain Labs newsbang PelaAI tattrongvu Patrick-18 EldanRing Valen-Team Google eCloud Technology Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Elephant-1: 61.3% at $0.008 per 1,000 questions ($7.90 per 1M)
StartLux-Decision-v1.1-9B-Instant: 58.4% at $0.008 per 1,000 questions ($8.34 per 1M) (estimate)
Cerno 9B v1.2: 56.0% at $0.010 per 1,000 questions ($9.51 per 1M) (estimate)
GLiDE 9B: 57.9% at $0.010 per 1,000 questions ($10.49 per 1M) (estimate)
Clef-Flash: 49.8% at $0.011 per 1,000 questions ($10.85 per 1M)
Cygnet: 44.3% at $0.024 per 1,000 questions ($23.94 per 1M)
Adjudex 9B v5: 62.1% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
KnowLine-9B-Gen1: 59.8% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
Jeffan-9B-1211: 59.8% at $0.029 per 1,000 questions ($28.78 per 1M) (estimate)
KnowLine-9B-Gen2: 60.1% at $0.031 per 1,000 questions ($31.02 per 1M) (estimate)
Dexy-Decision-9B-V2: 58.8% at $0.032 per 1,000 questions ($31.98 per 1M) (estimate)
KnowLine-12B-Gen1: 66.3% at $0.036 per 1,000 questions ($36.13 per 1M) (estimate)
Winnow-12B: 46.9% at $0.038 per 1,000 questions ($38.37 per 1M) (estimate)
Valen-Decision-1.5-9B: 55.3% at $0.041 per 1,000 questions ($40.93 per 1M) (estimate)
Gemma 4 E4B (on · chat): 98.2% at $0.233 per 1,000 questions ($233 per 1M) (estimate)
Erk-14B (off · chat): 90.8% at $0.266 per 1,000 questions ($266 per 1M) (estimate)
Gemma 4 E4B (chat): 98.1% at $0.271 per 1,000 questions ($271 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 98.5% at $0.318 per 1,000 questions ($318 per 1M) (estimate)
Qwen3-14B (off · chat): 95.8% at $0.355 per 1,000 questions ($355 per 1M) (estimate)
Gemma 4 12B Unified (chat): 98.8% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 96.3% at $0.482 per 1,000 questions ($482 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Gemma 4 E4B (on · chat · 32k cap): 98.5% at $0.518 per 1,000 questions ($518 per 1M) (estimate)
Base Labs StartLux Apertix AI Fastino Cloudflare Blockbrain Labs newsbang PelaAI tattrongvu Patrick-18 EldanRing Valen-Team Google eCloud Technology Alibaba Liquid AI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Elephant-1 Pareto Open · 9.0B | — | Decision | $0.008$7.90 list | $7.90 | $0.01708 | $0.058$58.20estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | Decision | $0.008$8.34estimate · DGX Spark | $8.34 estimate | $0.01803 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cerno 9B v1.2 Open · 9.0B | — | Decision | $0.010$9.51estimate · DGX Spark | $9.51 estimate | $0.02057 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Open · 7.9B | — | Decision | $0.010$10.49estimate · DGX Spark | $10.49 estimate | $0.0227 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Clef-Flash Open · 9.0B | — | Decision | $0.011$10.85 list | $10.85 | $0.02347 | $0.072$71.95estimate · RTX PRO 6000 Blackwell | maker's list price |
| Cygnet Open · 12B | — | Decision | $0.024$23.94 list | $23.94 | $0.05179 | $0.028$27.50estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| Adjudex 9B v5 Pareto Open · 9.0B | — | Decision | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | Decision | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Open · 9.0B | — | Decision | $0.029$28.78estimate · RTX PRO 6000 Blackwell | $28.78 estimate | $0.06225 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | Decision | $0.031$31.02estimate · RTX PRO 6000 Blackwell | $31.02 estimate | $0.06709 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | Decision | $0.032$31.98estimate · RTX PRO 6000 Blackwell | $31.98 estimate | $0.06917 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Pareto Open · 12B | — | Decision | $0.036$36.13estimate · RTX PRO 6000 Blackwell | $36.13 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | Decision | $0.038$38.37estimate · RTX PRO 6000 Blackwell | $38.37 estimate | $0.083 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | Decision | $0.041$40.93estimate · RTX PRO 6000 Blackwell | $40.93 estimate | $0.08853 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat Open · 8.0B | — | Chat | $0.233$233estimate · DGX Spark | $232.86 estimate | $0.5037 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | Chat | $0.266$266estimate · RTX PRO 6000 Blackwell | $266.05 estimate | $0.5755 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B chat Open · 8.0B | — | Chat | $0.271$271estimate · RTX PRO 6000 Blackwell | $271.17 estimate | $0.5865 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat · 32k cap · sampled Open · 8.0B | — | Chat | $0.318$318estimate · RTX PRO 6000 Blackwell | $317.64 estimate | $0.7034 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | Chat | $0.355$355estimate · RTX PRO 6000 Blackwell | $354.95 estimate | $0.7678 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified Pareto chat Open · 12B | — | Chat | $0.467$467estimate · RTX PRO 6000 Blackwell | $467.19 estimate | $1.011 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | Chat | $0.482$482estimate · RTX PRO 6000 Blackwell† | $482.22 estimate† | $1.043 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | Chat | $0.518$518estimate · RTX PRO 6000 Blackwell | $518.03 estimate | $1.121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% | — | Chat | 98.8%98.3%–99.2% | a | 2,138 / 2,163 | Not measured | Not measured | 14.3%answered 100% | 19,100 ms | 28,120 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.318$318estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% | — | Chat | 98.5%98.4%–98.8% min–max of 3 seeds | a | 2,136 / 2,163seed 0 run | Not measured | Not measured | 45.5%answered 99% | 11,190 ms | 22,357 ms | $0.318$318estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.518$518estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% | — | Chat | 98.5%97.9%–99.0% | a | 2,131 / 2,163 | Not measured | Not measured | Not measured | 20,640 ms | 34,280 ms | $0.518$518estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.233$233estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% | — | Chat | 98.2%97.6%–98.7% | a | 2,125 / 2,163 | Not measured | Not measured | 37.0%answered 99% | 108,720 ms | 175,470 ms | $0.233$233estimate · DGX Spark |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.271$271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% | — | Chat | 98.1%97.4%–98.6% | a | 2,122 / 2,163 | Not measured | Not measured | 44.0%answered 99% | 10,770 ms | 17,700 ms | $0.271$271estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $0.482$482estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% | — | Chat | 96.3%95.4%–97.0% | b | 2,082 / 2,163 | Not measured | Not measured | 45.2%answered 97% | 14,530 ms† | 33,190 ms† | $0.482$482estimate · RTX PRO 6000 Blackwell† |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.355$355estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% | — | Chat | 95.8%94.9%–96.6% | b | 2,072 / 2,163 | Not measured | Not measured | 39.3%answered 98% | 8,040 ms | 18,980 ms | $0.355$355estimate · RTX PRO 6000 Blackwell |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.266$266estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% | — | Chat | 90.8%89.5%–91.9% | c | 1,963 / 2,163 | Not measured | Not measured | 35.9%answered 96% | 7,890 ms | 14,890 ms | $0.266$266estimate · RTX PRO 6000 Blackwell |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.036$36.13estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% | — | Decision | 66.3%64.2%–68.2% | d | 1,433 / 2,163 | 7 right of 7 kept0 mistakes · 2,156 skipped | 0.819 | 1.1%answered 100% | 49 ms | 55 ms | $0.036$36.13estimate · RTX PRO 6000 Blackwell |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | Decision | 62.1%60.0%–64.1% | e | 1,343 / 2,163 | 197 right of 198 kept1 wrong · 1,965 skipped | 0.806 | 3.1%answered 99% | 36 ms | 44 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | Decision | 61.3%59.3%–63.4% | ef | 1,327 / 2,163 | 182 right of 183 kept1 wrong · 1,980 skipped | 0.803 | 65.0%answered 67% | 80 ms | 92 ms | $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.02estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% | — | Decision | 60.1%58.0%–62.1% | efg | 1,300 / 2,163 | No answers at this threshold | 0.803 | 3.2%answered 99% | 41 ms | 46 ms | $0.031$31.02estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.029$28.78estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% | — | Decision | 59.8%57.7%–61.9% | efg | 1,294 / 2,163 | 64 right of 64 kept0 mistakes · 2,099 skipped | 0.814 | 6.6%answered 97% | 37 ms | 51 ms | $0.029$28.78estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% | — | Decision | 59.8%57.7%–61.8% | efg | 1,293 / 2,163 | 123 right of 123 kept0 mistakes · 2,040 skipped | 0.799 | 0.9%answered 100% | 37 ms | 43 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.032$31.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% | — | Decision | 58.8%56.7%–60.9% | fgh | 1,272 / 2,163 | 207 right of 210 kept3 wrong · 1,953 skipped | 0.804 | 1.7%answered 99% | 45 ms | 50 ms | $0.032$31.98estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.008$8.34estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% | — | Decision | 58.4%56.3%–60.5% | fgh | 1,264 / 2,163 | 49 right of 49 kept0 mistakes · 2,114 skipped | 0.781 | 4.2%answered 98% | 140 ms | 146 ms | $0.008$8.34estimate · DGX Spark |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.010$10.49estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% | — | Decision | 57.9%55.8%–60.0% | gh | 1,253 / 2,163 | 73 right of 73 kept0 mistakes · 2,090 skipped | 0.798 | 24.6%answered 89% | 174 ms | 182 ms | $0.010$10.49estimate · DGX Spark |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.010$9.51estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% | — | Decision | 56.0%53.9%–58.1% | h | 1,212 / 2,163 | 244 right of 254 kept10 wrong · 1,909 skipped | 0.780 | 1.2%answered 99% | 153 ms | 175 ms | $0.010$9.51estimate · DGX Spark |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$40.93estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% | — | Decision | 55.3%53.2%–57.4% | h | 1,197 / 2,163 | 130 right of 130 kept0 mistakes · 2,033 skipped | 0.790 | 2.6%answered 99% | 58 ms | 64 ms | $0.041$40.93estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% | — | Decision | 49.8%47.7%–51.9% | i | 1,077 / 2,163 | No answers at this threshold | 0.730 | 2.0%answered 99% | 99 ms | 127 ms | $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.038$38.37estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | Decision | 46.9%44.8%–49.0% | ij | 1,014 / 2,163 | 20 right of 21 kept1 wrong · 2,142 skipped | 0.743 | 9.2%answered 95% | 52 ms | 61 ms | $0.038$38.37estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% | — | Decision | 44.3%42.3%–46.4% | j | 959 / 2,163 | No answers at this threshold | 0.735 | 4.9%answered 97% | 37 ms | 41 ms | $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Perplexity Fastino Nace.AI Turion.AI Base Labs Sthānika AI vLLM Semantic Router PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai Alibaba Google Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Perplexity Decider 27B v1.2 (API): 69.8% at $0.004 per 1,000 questions ($4.29 per 1M)
GLiDE (v1.2): 69.9% at $0.008 per 1,000 questions ($7.55 per 1M)
Drex 2.0: 59.0% at $0.009 per 1,000 questions ($8.92 per 1M)
Drex Max: 70.3% at $0.013 per 1,000 questions ($13.06 per 1M)
Turion Jev 31B: 69.0% at $0.017 per 1,000 questions ($17.46 per 1M) (estimate)
Mammoth-1: 74.0% at $0.023 per 1,000 questions ($22.56 per 1M)
GLiDE 27B: 72.0% at $0.026 per 1,000 questions ($26.16 per 1M) (estimate)
Sieve 31B v3: 77.1% at $0.039 per 1,000 questions ($39.14 per 1M) (estimate)
d3: 70.4% at $0.041 per 1,000 questions ($40.61 per 1M) (estimate)
KnowLine-31B-Gen1.5: 77.4% at $0.047 per 1,000 questions ($46.87 per 1M) (estimate)
OneJev-27B: 53.8% at $0.061 per 1,000 questions ($60.76 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 72.5% at $0.066 per 1,000 questions ($65.87 per 1M) (estimate)
Clef: 51.1% at $0.069 per 1,000 questions ($68.54 per 1M)
KnowLine-31B-Gen2: 80.3% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
Copernicus-31B: 77.3% at $0.084 per 1,000 questions ($84.42 per 1M) (estimate)
d3-ultra: 71.5% at $0.089 per 1,000 questions ($88.58 per 1M) (estimate)
Karpo Decision 31B: 81.6% at $0.113 per 1,000 questions ($113 per 1M) (estimate)
Open-Jev-27B-v1.1: 54.6% at $0.169 per 1,000 questions ($169 per 1M) (estimate)
Qwen3.8-27B (low · chat): 99.2% at $0.530 per 1,000 questions ($530 per 1M) (estimate)
Gemma 4 31B (chat): 99.4% at $0.531 per 1,000 questions ($531 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 99.0% at $0.939 per 1,000 questions ($939 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 69.3% at $1.023 per 1,000 questions ($1,023 per 1M)
Perplexity Fastino Nace.AI Turion.AI Base Labs Sthānika AI vLLM Semantic Router PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai Alibaba Google Liquid AI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | Decision | $0.004$4.29 | $4.29 | $0.009285 | — | provider price × reported usage |
| GLiDE Pareto v1.2 Open · 27B | — | Decision | $0.008$7.55 list | $7.55 | $0.01632 | $0.058$57.88estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | Decision | $0.009$8.92 | $8.92 | $0.01929 | — | provider price × reported usage |
| Drex Max Pareto Open · 27B | — | Decision | $0.013$13.06 list | $13.06 | $0.02824 | $0.055$54.68estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | Decision | $0.017$17.46estimate · DGX Spark | $17.46 estimate | $0.03778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Mammoth-1 Pareto Open · 27B | — | Decision | $0.023$22.56 list | $22.56 | $0.0488 | $0.104$104estimate · RTX PRO 6000 Blackwell | maker's list price |
| GLiDE 27B Open · 27B | — | Decision | $0.026$26.16estimate · DGX Spark | $26.16 estimate | $0.05657 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Sieve 31B v3 Pareto Open · 31B | — | Decision | $0.039$39.14estimate · DGX Spark | $39.14 estimate | $0.08465 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d3 Open · 26B | — | Decision | $0.041$40.61estimate · RTX PRO 6000 Blackwell | $40.61 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | Decision | $0.047$46.87estimate · DGX Spark | $46.87 estimate | $0.1014 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | Decision | $0.061$60.76estimate · RTX PRO 6000 Blackwell | $60.76 estimate | $0.1314 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | Decision | $0.066$65.87estimate · RTX PRO 6000 Blackwell | $65.87 estimate | $0.1425 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | Decision | $0.069$68.54 list | $68.54 | $0.1483 | $0.131$131estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-31B-Gen2 Pareto Open · 31B | — | Decision | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | Decision | $0.084$84.42estimate · RTX PRO 6000 Blackwell | $84.42 estimate | $0.1826 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | Decision | $0.089$88.58estimate · RTX PRO 6000 Blackwell | $88.58 estimate | $0.1916 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | Decision | $0.113$113estimate · RTX PRO 6000 Blackwell | $112.56 estimate | $0.2435 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | Decision | $0.169$169estimate · RTX PRO 6000 Blackwell | $169.16 estimate | $0.3659 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B Pareto low · chat Open · 27B | — | Chat | $0.530$530estimate · RTX PRO 6000 Blackwell | $529.86 estimate | $1.146 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B Pareto chat Open · 31B | — | Chat | $0.531$531estimate · RTX PRO 6000 Blackwell | $530.82 estimate | $1.148 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | Chat | $0.939$939estimate · RTX PRO 6000 Blackwell | $939.17 estimate | $2.031 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | Decision | $1.023$1,023 | $1,022.79 | $2.212 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.531$531estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% | — | Chat | 99.4%99.0%–99.6% | a | 2,150 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 11,110 ms | 16,690 ms | $0.531$531estimate · RTX PRO 6000 Blackwell |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $0.530$530estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 99.2%98.7%–99.5% | a | 2,146 / 2,163 | Not measured | Not measured | 13.0%answered 100% | 10,020 ms | 20,630 ms | $0.530$530estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $0.939$939estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% | — | Chat | 99.0%98.5%–99.4% | a | 2,142 / 2,163 | Not measured | Not measured | 71.4%answered 99% | 27,950 ms | 60,210 ms | $0.939$939estimate · RTX PRO 6000 Blackwell |
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.113$113estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% | — | Decision | 81.6%79.9%–83.1% | b | 1,764 / 2,163 | 837 right of 837 kept0 mistakes · 1,326 skipped | 0.903 | 4.8%answered 99% | 155 ms | 186 ms | $0.113$113estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% | — | Decision | 80.3%78.5%–81.9% | b | 1,736 / 2,163 | 469 right of 470 kept1 wrong · 1,693 skipped | 0.898 | 0.2%answered 100% | 114 ms | 120 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.047$46.87estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% | — | Decision | 77.4%75.6%–79.1% | c | 1,674 / 2,163 | 464 right of 465 kept1 wrong · 1,698 skipped | 0.895 | 1.1%answered 100% | 764 ms | 867 ms | $0.047$46.87estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.084$84.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% | — | Decision | 77.3%75.5%–79.1% | c | 1,673 / 2,163 | 318 right of 318 kept0 mistakes · 1,845 skipped | 0.899 | 1.7%answered 100% | 118 ms | 142 ms | $0.084$84.42estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.039$39.14estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% | — | Decision | 77.1%75.3%–78.8% | cd | 1,668 / 2,163 | 215 right of 215 kept0 mistakes · 1,948 skipped | 0.891 | 2.1%answered 100% | 637 ms | 735 ms | $0.039$39.14estimate · DGX Spark |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | Decision | 74.0%72.1%–75.8% | de | 1,600 / 2,163 | 412 right of 412 kept0 mistakes · 1,751 skipped | 0.895 | 70.6%answered 75% | 140 ms | 173 ms | $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.066$65.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% | — | Decision | 72.5%70.6%–74.3% | ef | 1,568 / 2,163 | 323 right of 323 kept0 mistakes · 1,840 skipped | 0.856 | 9.4%answered 97% | 91 ms | 103 ms | $0.066$65.87estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.026$26.16estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% | — | Decision | 72.0%70.1%–73.8% | efg | 1,557 / 2,163 | 198 right of 199 kept1 wrong · 1,964 skipped | 0.864 | 21.6%answered 94% | 425 ms | 461 ms | $0.026$26.16estimate · DGX Spark |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.089$88.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% | — | Decision | 71.5%69.6%–73.4% | efg | 1,547 / 2,163 | 11 right of 11 kept0 mistakes · 2,152 skipped | 0.857 | 5.3%answered 98% | 118 ms | 148 ms | $0.089$88.58estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.041$40.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | Decision | 70.4%68.5%–72.3% | fg | 1,523 / 2,163 | 162 right of 163 kept1 wrong · 2,000 skipped | 0.878 | 4.7%answered 99% | 55 ms | 62 ms | $0.041$40.61estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% | — | Decision | 70.3%68.4%–72.2% | fg | 1,521 / 2,163 | 719 right of 729 kept10 wrong · 1,434 skipped | 0.861 | 4.5%answered 99% | 76 ms | 89 ms | $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% | — | Decision | 69.9%67.9%–71.8% | fg | 1,511 / 2,163 | 131 right of 131 kept0 mistakes · 2,032 skipped | 0.862 | 35.8%answered 88% | 79 ms | 84 ms | $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.004$4.29 per 1,000 questionsper 1M questions Logic 53.3% | — | Decision | 69.8%67.8%–71.7% | g | 1,509 / 2,163 | 331 right of 331 kept0 mistakes · 1,832 skipped | 0.869 | 7.0%answered 98% | 226 ms | 368 ms | $0.004$4.29 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.023$1,023 per 1,000 questionsper 1M questions Logic 52.5% | with web | Decision | 69.3%67.3%–71.2% | g | 1,499 / 2,163 | 326 right of 326 kept0 mistakes · 1,837 skipped | 0.873 | 15.4%answered 95% | 422 ms | 641 ms | $1.023$1,023 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.017$17.46estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% | — | Decision | 69.0%67.0%–70.9% | g | 1,493 / 2,163 | 43 right of 43 kept0 mistakes · 2,120 skipped | 0.864 | 2.2%answered 99% | 278 ms | 329 ms | $0.017$17.46estimate · DGX Spark |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.009$8.92 per 1,000 questionsper 1M questions Logic 41.9% | — | Decision | 59.0%56.9%–61.0% | h | 1,276 / 2,163 | 174 right of 174 kept0 mistakes · 1,989 skipped | 0.813 | 20.7%answered 91% | 334 ms | 432 ms | $0.009$8.92 |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.169$169estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% | — | Decision | 54.6%52.4%–56.6% | i | 1,180 / 2,163 | No answers at this threshold | 0.764 | 5.0%answered 98% | 233 ms | 272 ms | $0.169$169estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.061$60.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% | — | Decision | 53.8%51.7%–55.9% | i | 1,163 / 2,163 | 26 right of 26 kept0 mistakes · 2,137 skipped | 0.783 | 1.8%answered 99% | 79 ms | 106 ms | $0.061$60.76estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% | — | Decision | 51.1%49.0%–53.2% | i | 1,105 / 2,163 | 12 right of 12 kept0 mistakes · 2,151 skipped | 0.767 | 1.6%answered 99% | 181 ms | 210 ms | $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
MiniMax Logits REAX Z.ai Xiaomi DeepSeek Thinking Machines
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
MiniMax-M3.1-Flash (high · chat): 98.2% at $0.103 per 1,000 questions ($103 per 1M)
Logits Sigma-1 Duo: 75.2% at $0.141 per 1,000 questions ($141 per 1M) (estimate)
REAX 1.3: 78.0% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
GLM-5.3-Flash (high · chat): 98.8% at $0.166 per 1,000 questions ($166 per 1M)
MiMo-V2.6-Flash (on · chat): 99.4% at $0.212 per 1,000 questions ($212 per 1M)
DeepSeek-V4.1-Flash (high · chat · 1017): 99.6% at $0.227 per 1,000 questions ($227 per 1M)
DeepSeek-V4-Flash (chat): 99.3% at $0.295 per 1,000 questions ($295 per 1M) (estimate)
Inkling 2 Small (high · chat): 99.5% at $0.638 per 1,000 questions ($638 per 1M)
Inkling-Small (high · chat · Jul 2026): 99.6% at $0.954 per 1,000 questions ($954 per 1M)
MiniMax Logits REAX Z.ai Xiaomi DeepSeek Thinking Machines
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | Chat | $0.103$103 | $102.85 | $0.2225 | — | provider price × reported usage |
| Logits Sigma-1 Duo Open · 57B | — | Decision | $0.141$141estimate · RTX PRO 6000 Blackwell | $140.70 estimate | $0.3043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Open · 57B | — | Decision | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.98 estimate | $0.3071 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLM-5.3-Flash Pareto high · chat Open · 320B MoE · 18B active | — | Chat | $0.166$166 | $166.50 | $0.3601 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | Chat | $0.212$212 | $211.85 | $0.4582 | — | provider price × reported usage |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | Chat | $0.227$227 | $227.42 | $0.4919 | — | provider price × reported usage |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active | — | Chat | $0.295$295estimate · DGX Spark | $294.82 estimate | $0.6377 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | Chat | $0.638$638 | $638.48 | $1.381 | — | provider price × reported usage |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | Chat | $0.954$954 | $953.81 | $2.063 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Qwen3.5-397B-A17B (on · chat) (Logic only; on this test only its 25-question smoke test; cost so far $5.552$5,552 per 1,000 questions, over the 25 of 2,163 answered).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.227$227 per 1,000 questionsper 1M questions Logic 96.2% | — | Chat | 99.6%99.3%–99.8% | a | 2,155 / 2,163 | Not measured | Not measured | 20.0%answered 100% | 1,910 ms | 3,570 ms | $0.227$227 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $0.954$954 per 1,000 questionsper 1M questions Logic 95.4% | — | Chat | 99.6%99.3%–99.8% | a | 2,155 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 2,030 ms | 6,330 ms | $0.954$954 |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $0.638$638 per 1,000 questionsper 1M questions Logic 95.6% | — | Chat | 99.5%99.2%–99.7% | a | 2,153 / 2,163 | Not measured | Not measured | 61.5%answered 100% | 2,110 ms | 6,000 ms | $0.638$638 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.212$212 per 1,000 questionsper 1M questions Logic 95.1% | — | Chat | 99.4%99.0%–99.7% | ab | 2,151 / 2,163 | Not measured | Not measured | 50.0%answered 100% | 8,200 ms | 18,090 ms | $0.212$212 |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active Cost $0.295$295estimate · DGX Spark per 1,000 questionsper 1M questions Logic running | — | Chat | 99.3%98.9%–99.6% | ab | 2,148 / 2,163 | Not measured | Not measured | Not measured | 8,130 ms | 16,920 ms | $0.295$295estimate · DGX Spark |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.166$166 per 1,000 questionsper 1M questions Logic 93.2% | — | Chat | 98.8%98.2%–99.2% | bc | 2,137 / 2,163 | Not measured | Not measured | 17.4%answered 100% | 3,800 ms | 7,970 ms | $0.166$166 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.103$103 per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 98.2%97.6%–98.7% | c | 2,125 / 2,163 | Not measured | Not measured | 40.6%answered 99% | 1,670 ms | 7,470 ms | $0.103$103 |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% | — | Decision | 78.0%76.2%–79.7% | d | 1,687 / 2,163 | 178 right of 178 kept0 mistakes · 1,985 skipped | 0.903 | 2.3%answered 99% | 189 ms | 232 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% | — | Decision | 75.2%73.4%–77.0% | e | 1,627 / 2,163 | 15 right of 15 kept0 mistakes · 2,148 skipped | 0.899 | 2.1%answered 99% | 196 ms | 221 ms | $0.141$141estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino Meta
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
celeris-1.1-decision: 69.9% at $0.001 per 1,000 questions ($0.91 per 1M)
d1.1: 65.2% at $0.003 per 1,000 questions ($2.92 per 1M)
d1: 55.6% at $0.003 per 1,000 questions ($2.92 per 1M)
Mercury Decide: 53.5% at $0.004 per 1,000 questions ($4.38 per 1M)
d1-ultra: 74.1% at $0.006 per 1,000 questions ($6.14 per 1M)
Microsoft-Decision-1: 59.4% at $0.007 per 1,000 questions ($6.81 per 1M)
SubQ Snap 1.1: 68.6% at $0.009 per 1,000 questions ($8.59 per 1M)
celeris-1-decision: 70.0% at $0.009 per 1,000 questions ($9.10 per 1M)
Drex Max v1.1: 70.1% at $0.013 per 1,000 questions ($12.88 per 1M)
Jev 1.14: 65.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Jev 1.13: 64.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Gemini Decision 1 (20261111): 71.3% at $0.022 per 1,000 questions ($22.02 per 1M)
GLiDE (API): 70.8% at $0.028 per 1,000 questions ($28.30 per 1M)
Muse Ember (max · chat): 99.6% at $0.365 per 1,000 questions ($365 per 1M)
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino Meta
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| celeris-1.1-decision Pareto Closed · Efficient | — | Decision | $0.001$0.91 | $0.91 | $0.001968 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | Decision | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1 Closed · Efficient | — | Decision | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | Decision | $0.004$4.38 | $4.38 | $0.009483 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | Decision | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | Decision | $0.007$6.81 | $6.81 | $0.01472 | — | provider price × reported usage |
| SubQ Snap 1.1 Closed · Efficient | — | Decision | $0.009$8.59 | $8.59 | $0.01857 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | Decision | $0.009$9.10 | $9.10 | $0.01968 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | Decision | $0.013$12.88 | $12.88 | $0.02786 | — | provider price × reported usage |
| Jev 1.14 Closed · Efficient | — | Decision | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | Decision | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | Decision | $0.022$22.02 | $22.02 | $0.04764 | — | provider price × reported usage |
| GLiDE API Closed · Efficient | — | Decision | $0.028$28.30 | $28.30 | $0.0612 | — | provider price × reported usage |
| Muse Ember Pareto max · chat Closed · Efficient | — | Chat | $0.365$365 | $364.58 | $0.7886 | — | provider price × reported usage |
| GPT-6 Luna chat Closed · Efficient | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 2,151 of 2,163); shown once.
Not ranked (only complete runs are): GPT-6 Luna (max · chat) (paused for now at 200 of 2,163 (zeki #4129, 2026-11-10T14:13Z); cost so far $0.182$182 per 1,000 questions, over the 200 of 2,163 answered).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.365$365 per 1,000 questionsper 1M questions Logic 95.5% | — | Chat | 99.6%99.2%–99.8% | a | 2,154 / 2,163 | Not measured | Not measured | 64.3%answered 100% | 1,260 ms | 2,270 ms | $0.365$365 |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% | — | Chat | 93.7%92.6%–94.6% | b | 2,026 / 2,163 | Not measured | Not measured | 68.9%answered 95% | 3,830 ms | 5,000 ms | $0.000$0.00 |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 54.9% | — | Decision | 74.1%72.2%–75.9% | c | 1,603 / 2,163 | 213 right of 213 kept0 mistakes · 1,950 skipped | 0.876 | 87.4%answered 61% | 179 ms | 426 ms | $0.006$6.14 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.022$22.02 per 1,000 questionsper 1M questions Logic 50.0% | — | Decision | 71.3%69.4%–73.2% | cd | 1,543 / 2,163 | 214 right of 214 kept0 mistakes · 1,949 skipped | 0.859 | 3.9%answered 99% | 226 ms | 351 ms | $0.022$22.02 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.028$28.30 per 1,000 questionsper 1M questions Logic 54.5% | — | Decision | 70.8%68.8%–72.7% | d | 1,531 / 2,163 | 161 right of 161 kept0 mistakes · 2,002 skipped | 0.865 | 40.6%answered 86% | 264 ms | 279 ms | $0.028$28.30 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.013$12.88 per 1,000 questionsper 1M questions Logic 54.8% | — | Decision | 70.1%68.1%–72.0% | d | 1,516 / 2,163 | 315 right of 315 kept0 mistakes · 1,848 skipped | 0.867 | 43.5%answered 84% | 326 ms | 473 ms | $0.013$12.88 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.009$9.10 per 1,000 questionsper 1M questions Logic 47.3% | — | Decision | 70.0%68.0%–71.9% | d | 1,514 / 2,163 | No answers at this threshold | 0.858 | 82.7%answered 62% | 387 ms | 391 ms | $0.009$9.10 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$0.91 per 1,000 questionsper 1M questions Logic 47.3% | — | Decision | 69.9%67.9%–71.8% | d | 1,511 / 2,163 | No answers at this threshold | 0.859 | 82.9%answered 62% | 381 ms | 386 ms | $0.001$0.91 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.009$8.59 per 1,000 questionsper 1M questions Logic 52.4% | — | Decision | 68.6%66.6%–70.5% | d | 1,483 / 2,163 | 360 right of 361 kept1 wrong · 1,802 skipped | 0.839 | 52.5%answered 80% | 381 ms | 390 ms | $0.009$8.59 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 46.0% | — | Decision | 65.2%63.2%–67.2% | e | 1,410 / 2,163 | 183 right of 183 kept0 mistakes · 1,980 skipped | 0.829 | 82.5%answered 56% | 172 ms | 290 ms | $0.003$2.92 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 48.2% | — | Decision | 65.1%63.1%–67.1% | e | 1,409 / 2,163 | 202 right of 202 kept0 mistakes · 1,961 skipped | 0.829 | 7.7%answered 97% | 163 ms | 185 ms | $0.019$18.99 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 46.3% | — | Decision | 64.1%62.0%–66.1% | e | 1,386 / 2,163 | 248 right of 248 kept0 mistakes · 1,915 skipped | 0.834 | 5.7%answered 98% | 166 ms | 206 ms | $0.019$18.99 |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.007$6.81 per 1,000 questionsper 1M questions Logic 44.4% | — | Decision | 59.4%57.3%–61.4% | f | 1,284 / 2,163 | 27 right of 27 kept0 mistakes · 2,136 skipped | 0.787 | 5.1%answered 98% | 310 ms | 359 ms | $0.007$6.81 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 40.8% | — | Decision | 55.6%53.5%–57.7% | g | 1,203 / 2,163 | 152 right of 152 kept0 mistakes · 2,011 skipped | 0.767 | 2.9%answered 99% | 184 ms | 297 ms | $0.003$2.92 |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.004$4.38 per 1,000 questionsper 1M questions Logic 45.1% | — | Decision | 53.5%51.4%–55.6% | g | 1,157 / 2,163 | 272 right of 281 kept9 wrong · 1,882 skipped | 0.789 | 3.2%answered 99% | 288 ms | 342 ms | $0.004$4.38 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Sonnet 5.5 (chat): 99.7% at $4.734 per 1,000 questions ($4,734 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 Pareto chat Closed · Workhorse | — | Chat | $4.734$4,734 API list-price equivalent | $4,734.30 | $10.24 | — | Claude subscription |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $4.734$4,734 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% | — | Chat | 99.7%99.3%–99.8% | a | 2,156 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,460 ms | 5,970 ms | $4.734$4,734 API list-price equivalent |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Opus 5.5 (chat): 99.8% at $9.871 per 1,000 questions ($9,871 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 Pareto chat Closed · Flagship | — | Chat | $9.871$9,871 API list-price equivalent | $9,870.94 | $21.35 | — | Claude subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Claude Fable 5.2 (chat) (not running now, 5 of 2,163 stored, last answer 11 Nov 12:02 UTC).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $9.871$9,871 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% | — | Chat | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,200 ms | 4,790 ms | $9.871$9,871 API list-price equivalent |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% | — | Chat | 99.7%99.4%–99.9% | a | 2,157 / 2,163 | Not measured | Not measured | 60.0%answered 100% | 5,030 ms | 6,490 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Liquid AI Alibaba xAI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
d1.2: 65.5% at $0.003 per 1,000 questions ($2.92 per 1M)
d1.2-ultra: 69.9% at $0.006 per 1,000 questions ($6.14 per 1M)
Qwen3.8 Flash (off · chat): 99.5% at $0.397 per 1,000 questions ($397 per 1M)
Qwen4-Max (high · chat): 99.8% at $1.478 per 1,000 questions ($1,478 per 1M)
Grok 4.7 (chat): 99.5% at $12.061 per 1,000 questions ($12,061 per 1M)
Liquid AI Alibaba xAI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| d1.2 Pareto Closed · Undisclosed | — | Decision | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1.2-ultra Pareto Closed · Undisclosed | — | Decision | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
| Qwen3.8 Flash Pareto off · chat Closed · Undisclosed | — | Chat | $0.397$397 | $396.92 | $0.8585 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | Chat | $1.478$1,478 | $1,478.15 | $3.197 | — | provider price × reported usage |
| Grok 4.7 chat Closed · Undisclosed | — | Chat | $12.061$12,061 API list-price equivalent | $12,061.26 | $26.09 | — | xAI subscription |
| GPT-6 Astra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Gemini 4 Carbon (high · chat) (paused for now at 54 of 2,163 (zeki #4681, 2026-11-11T11:53Z)).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $1.478$1,478 per 1,000 questionsper 1M questions Logic 96.3% | — | Chat | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 4,710 ms | 9,130 ms | $1.478$1,478 |
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% | — | Chat | 99.6%99.3%–99.8% | ab | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 5,280 ms | 7,260 ms | $0.000$0.00 |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% | — | Chat | 99.6%99.3%–99.8% | ab | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 4,810 ms | 6,560 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% | — | Chat | 99.6%99.2%–99.8% | ab | 2,154 / 2,163 | Not measured | Not measured | 75.0%answered 99% | 13,410 ms | 27,800 ms | — |
| Grok 4.7 chat Closed · Undisclosed Grok 4.7 chat · xAI · chat model Parameters: not disclosed by maker (closed weights) Cost $12.061$12,061 API list-price equivalent per 1,000 questionsper 1M questions Logic running | — | Chat | 99.5%99.2%–99.7% | ab | 2,153 / 2,163 | Not measured | Not measured | 97.5%answered 98% | 11,100 ms | 66,530 ms | $12.061$12,061 API list-price equivalent |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.397$397 per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 99.5%99.1%–99.7% | ab | 2,152 / 2,163 | Not measured | Not measured | 16.7%answered 100% | 5,740 ms | 11,800 ms | $0.397$397 |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% | — | Chat | 99.2%98.7%–99.5% | b | 2,146 / 2,163 | Not measured | Not measured | 77.8%answered 99% | 4,080 ms | 5,870 ms | $0.000$0.00 |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 51.6% | — | Decision | 69.9%67.9%–71.8% | c | 1,511 / 2,163 | 115 right of 115 kept0 mistakes · 2,048 skipped | 0.876 | 86.7%answered 58% | 215 ms | 347 ms | $0.006$6.14 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 49.2% | — | Decision | 65.5%63.5%–67.5% | d | 1,417 / 2,163 | 141 right of 141 kept0 mistakes · 2,022 skipped | 0.838 | 56.3%answered 76% | 204 ms | 386 ms | $0.003$2.92 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other Celeris OmniJev StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Nace.AI Apertix AI Cloudflare Turion.AI TypeSafe Google quincyqiang scar-ai Blockbrain Labs Strands Agents newsbang tattrongvu Patrick-18 victorbennett EldanRing Sthānika AI vLLM Semantic Router Valen-Team Metask Lab Poindexter Labs MachinePulse Logits REAX Zefan Cai
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 20.6% at $0.001 per 1,000 questions ($0.77 per 1M) (estimate)
GLiClass v4 Edge: 22.7% at $0.001 per 1,000 questions ($0.85 per 1M) (estimate)
celeris-1.1-decision: 69.9% at $0.001 per 1,000 questions ($0.91 per 1M)
OneJev1.6-0.8B: 28.2% at $0.001 per 1,000 questions ($1.09 per 1M) (estimate)
GLiDE 0.8B: 32.1% at $0.001 per 1,000 questions ($1.28 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 29.7% at $0.002 per 1,000 questions ($1.53 per 1M) (estimate)
KnowLine-0.8B-Gen1: 28.2% at $0.002 per 1,000 questions ($1.69 per 1M) (estimate)
GLiDE 2B: 36.8% at $0.002 per 1,000 questions ($1.78 per 1M) (estimate)
Perplexity Decider Flash v1.2: 55.2% at $0.002 per 1,000 questions ($2.18 per 1M)
StartLux-Decision-v1.1-2B-Instant: 38.7% at $0.002 per 1,000 questions ($2.27 per 1M) (estimate)
d1.2: 65.5% at $0.003 per 1,000 questions ($2.92 per 1M)
d1.1: 65.2% at $0.003 per 1,000 questions ($2.92 per 1M)
d1: 55.6% at $0.003 per 1,000 questions ($2.92 per 1M)
KnowLine-2B-Gen1: 35.3% at $0.003 per 1,000 questions ($3.20 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 55.8% at $0.004 per 1,000 questions ($4.10 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API): 69.8% at $0.004 per 1,000 questions ($4.29 per 1M)
Mercury Decide: 53.5% at $0.004 per 1,000 questions ($4.38 per 1M)
KnowLine 4B Gen5.5: 56.0% at $0.005 per 1,000 questions ($4.56 per 1M) (estimate)
d1-ultra: 74.1% at $0.006 per 1,000 questions ($6.14 per 1M)
d1.2-ultra: 69.9% at $0.006 per 1,000 questions ($6.14 per 1M)
Microsoft-Decision-1: 59.4% at $0.007 per 1,000 questions ($6.81 per 1M)
GLiDE (v1.2): 69.9% at $0.008 per 1,000 questions ($7.55 per 1M)
Elephant-1: 61.3% at $0.008 per 1,000 questions ($7.90 per 1M)
StartLux-Decision-v1.1-9B-Instant: 58.4% at $0.008 per 1,000 questions ($8.34 per 1M) (estimate)
SubQ Snap 1.1: 68.6% at $0.009 per 1,000 questions ($8.59 per 1M)
Drex 2.0: 59.0% at $0.009 per 1,000 questions ($8.92 per 1M)
celeris-1-decision: 70.0% at $0.009 per 1,000 questions ($9.10 per 1M)
Cerno 9B v1.2: 56.0% at $0.010 per 1,000 questions ($9.51 per 1M) (estimate)
GLiDE 9B: 57.9% at $0.010 per 1,000 questions ($10.49 per 1M) (estimate)
Clef-Flash: 49.8% at $0.011 per 1,000 questions ($10.85 per 1M)
Drex Max v1.1: 70.1% at $0.013 per 1,000 questions ($12.88 per 1M)
Drex Max: 70.3% at $0.013 per 1,000 questions ($13.06 per 1M)
d1-2B: 37.1% at $0.014 per 1,000 questions ($14.07 per 1M) (estimate)
Turion Jev 31B: 69.0% at $0.017 per 1,000 questions ($17.46 per 1M) (estimate)
Jev 1.14: 65.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Jev 1.13: 64.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Gemini Decision 1 (20261111): 71.3% at $0.022 per 1,000 questions ($22.02 per 1M)
Perplexity Decider Flash v1.3: 55.9% at $0.022 per 1,000 questions ($22.38 per 1M) (estimate)
Mammoth-1: 74.0% at $0.023 per 1,000 questions ($22.56 per 1M)
Chinese Decision Pointer 4B: 50.4% at $0.023 per 1,000 questions ($22.70 per 1M) (estimate)
vera-medium-omni: 56.8% at $0.023 per 1,000 questions ($23.34 per 1M) (estimate)
Cygnet: 44.3% at $0.024 per 1,000 questions ($23.94 per 1M)
GLiDE 27B: 72.0% at $0.026 per 1,000 questions ($26.16 per 1M) (estimate)
Strands Decider 2B: 28.7% at $0.026 per 1,000 questions ($26.22 per 1M) (estimate)
Adjudex 9B v5: 62.1% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
KnowLine-9B-Gen1: 59.8% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
GLiDE (API): 70.8% at $0.028 per 1,000 questions ($28.30 per 1M)
Jeffan-9B-1211: 59.8% at $0.029 per 1,000 questions ($28.78 per 1M) (estimate)
KnowLine-9B-Gen2: 60.1% at $0.031 per 1,000 questions ($31.02 per 1M) (estimate)
Dexy-Decision-9B-V2: 58.8% at $0.032 per 1,000 questions ($31.98 per 1M) (estimate)
Verdin-4B-v1.1: 51.5% at $0.034 per 1,000 questions ($33.58 per 1M) (estimate)
KnowLine-12B-Gen1: 66.3% at $0.036 per 1,000 questions ($36.13 per 1M) (estimate)
Winnow-12B: 46.9% at $0.038 per 1,000 questions ($38.37 per 1M) (estimate)
Sieve 31B v3: 77.1% at $0.039 per 1,000 questions ($39.14 per 1M) (estimate)
d3: 70.4% at $0.041 per 1,000 questions ($40.61 per 1M) (estimate)
Valen-Decision-1.5-9B: 55.3% at $0.041 per 1,000 questions ($40.93 per 1M) (estimate)
KnowLine-31B-Gen1.5: 77.4% at $0.047 per 1,000 questions ($46.87 per 1M) (estimate)
OneJev-27B: 53.8% at $0.061 per 1,000 questions ($60.76 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 72.5% at $0.066 per 1,000 questions ($65.87 per 1M) (estimate)
Clef: 51.1% at $0.069 per 1,000 questions ($68.54 per 1M)
KnowLine-31B-Gen2: 80.3% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
Metask-Jev-4B: 39.2% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
Copernicus-31B: 77.3% at $0.084 per 1,000 questions ($84.42 per 1M) (estimate)
d3-ultra: 71.5% at $0.089 per 1,000 questions ($88.58 per 1M) (estimate)
Karpo Decision 31B: 81.6% at $0.113 per 1,000 questions ($113 per 1M) (estimate)
Logits Sigma-1 Duo: 75.2% at $0.141 per 1,000 questions ($141 per 1M) (estimate)
REAX 1.3: 78.0% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
Open-Jev-27B-v1.1: 54.6% at $0.169 per 1,000 questions ($169 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 69.3% at $1.023 per 1,000 questions ($1,023 per 1M)
Fastino Other Celeris OmniJev StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Nace.AI Apertix AI Cloudflare Turion.AI TypeSafe Google quincyqiang scar-ai Blockbrain Labs Strands Agents newsbang tattrongvu Patrick-18 victorbennett EldanRing Sthānika AI vLLM Semantic Router Valen-Team Metask Lab Poindexter Labs MachinePulse Logits REAX Zefan Cai
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | $0.001$0.77estimate · DGX Spark | $0.77 estimate | $0.001655 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | $0.001$0.85estimate · DGX Spark | $0.85 estimate | $0.001833 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1.1-decision Pareto Closed · Efficient | — | $0.001$0.91 | $0.91 | $0.001968 | — | provider price × reported usage |
| OneJev1.6-0.8B Open · 0.8B | — | $0.001$1.09estimate · DGX Spark | $1.09 estimate | $0.002365 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Open · 0.8B | — | $0.001$1.28estimate · DGX Spark | $1.28 estimate | $0.002778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | $0.002$1.53estimate · DGX Spark | $1.53 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | $0.002$1.69estimate · DGX Spark | $1.69 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Open · 1.9B | — | $0.002$1.78estimate · DGX Spark | $1.78 estimate | $0.003843 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Open · 4.0B | — | $0.002$2.18 list | $2.18 | $0.004707 | $0.030$30.06estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | $0.002$2.27estimate · DGX Spark | $2.27 estimate | $0.004907 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1.2 Closed · Undisclosed | — | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1 Closed · Efficient | — | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| KnowLine-2B-Gen1 Open · 2.0B | — | $0.003$3.20estimate · DGX Spark | $3.20 estimate | $0.006917 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | — | $0.004$4.10estimate · DGX Spark | $4.10 estimate | $0.008867 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.2 API Open · 27B | — | $0.004$4.29 | $4.29 | $0.009285 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | $0.004$4.38 | $4.38 | $0.009483 | — | provider price × reported usage |
| KnowLine 4B Gen5.5 Open · 4.0B | — | $0.005$4.56estimate · DGX Spark | $4.56 estimate | $0.009872 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-ultra Pareto Closed · Efficient | — | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
| d1.2-ultra Closed · Undisclosed | — | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | $0.007$6.81 | $6.81 | $0.01472 | — | provider price × reported usage |
| GLiDE v1.2 Open · 27B | — | $0.008$7.55 list | $7.55 | $0.01632 | $0.058$57.88estimate · RTX PRO 6000 Blackwell | maker's list price |
| Elephant-1 Open · 9.0B | — | $0.008$7.90 list | $7.90 | $0.01708 | $0.058$58.20estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | $0.008$8.34estimate · DGX Spark | $8.34 estimate | $0.01803 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| SubQ Snap 1.1 Closed · Efficient | — | $0.009$8.59 | $8.59 | $0.01857 | — | provider price × reported usage |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | $0.009$8.92 | $8.92 | $0.01929 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | $0.009$9.10 | $9.10 | $0.01968 | — | provider price × reported usage |
| Cerno 9B v1.2 Open · 9.0B | — | $0.010$9.51estimate · DGX Spark | $9.51 estimate | $0.02057 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Open · 7.9B | — | $0.010$10.49estimate · DGX Spark | $10.49 estimate | $0.0227 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Clef-Flash Open · 9.0B | — | $0.011$10.85 list | $10.85 | $0.02347 | $0.072$71.95estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex Max v1.1 Closed · Efficient (provisional) | — | $0.013$12.88 | $12.88 | $0.02786 | — | provider price × reported usage |
| Drex Max Open · 27B | — | $0.013$13.06 list | $13.06 | $0.02824 | $0.055$54.68estimate · RTX PRO 6000 Blackwell | maker's list price |
| d1-2B Open · 1.9B | — | $0.014$14.07estimate · RTX PRO 6000 Blackwell | $14.07 estimate | $0.03043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Turion Jev 31B Open · 31B | — | $0.017$17.46estimate · DGX Spark | $17.46 estimate | $0.03778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Jev 1.14 Closed · Efficient | — | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | $0.022$22.02 | $22.02 | $0.04764 | — | provider price × reported usage |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | $0.022$22.38estimate · RTX PRO 6000 Blackwell | $22.38 estimate | $0.04842 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Mammoth-1 Open · 27B | — | $0.023$22.56 list | $22.56 | $0.0488 | $0.104$104estimate · RTX PRO 6000 Blackwell | maker's list price |
| Chinese Decision Pointer 4B Open · 4.0B | — | $0.023$22.70estimate · RTX PRO 6000 Blackwell | $22.70 estimate | $0.04911 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Open · 4.2B | — | $0.023$23.34estimate · RTX PRO 6000 Blackwell | $23.34 estimate | $0.05049 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | $0.024$23.94 list | $23.94 | $0.05179 | $0.028$27.50estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| GLiDE 27B Open · 27B | — | $0.026$26.16estimate · DGX Spark | $26.16 estimate | $0.05657 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Strands Decider 2B Open · 2.0B | — | $0.026$26.22estimate · RTX PRO 6000 Blackwell | $26.22 estimate | $0.05672 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Adjudex 9B v5 Open · 9.0B | — | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE API Closed · Efficient | — | $0.028$28.30 | $28.30 | $0.0612 | — | provider price × reported usage |
| Jeffan-9B-1211 Open · 9.0B | — | $0.029$28.78estimate · RTX PRO 6000 Blackwell | $28.78 estimate | $0.06225 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | $0.031$31.02estimate · RTX PRO 6000 Blackwell | $31.02 estimate | $0.06709 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | $0.032$31.98estimate · RTX PRO 6000 Blackwell | $31.98 estimate | $0.06917 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | $0.034$33.58estimate · RTX PRO 6000 Blackwell | $33.58 estimate | $0.07263 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Open · 12B | — | $0.036$36.13estimate · RTX PRO 6000 Blackwell | $36.13 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | $0.038$38.37estimate · RTX PRO 6000 Blackwell | $38.37 estimate | $0.083 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Sieve 31B v3 Pareto Open · 31B | — | $0.039$39.14estimate · DGX Spark | $39.14 estimate | $0.08465 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d3 Open · 26B | — | $0.041$40.61estimate · RTX PRO 6000 Blackwell | $40.61 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | $0.041$40.93estimate · RTX PRO 6000 Blackwell | $40.93 estimate | $0.08853 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | $0.047$46.87estimate · DGX Spark | $46.87 estimate | $0.1014 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | $0.061$60.76estimate · RTX PRO 6000 Blackwell | $60.76 estimate | $0.1314 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | $0.066$65.87estimate · RTX PRO 6000 Blackwell | $65.87 estimate | $0.1425 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | $0.069$68.54 list | $68.54 | $0.1483 | $0.131$131estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-31B-Gen2 Pareto Open · 31B | — | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | $0.084$84.42estimate · RTX PRO 6000 Blackwell | $84.42 estimate | $0.1826 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | $0.089$88.58estimate · RTX PRO 6000 Blackwell | $88.58 estimate | $0.1916 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | $0.113$113estimate · RTX PRO 6000 Blackwell | $112.56 estimate | $0.2435 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | $0.141$141estimate · RTX PRO 6000 Blackwell | $140.70 estimate | $0.3043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Open · 57B | — | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.98 estimate | $0.3071 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | $0.169$169estimate · RTX PRO 6000 Blackwell | $169.16 estimate | $0.3659 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | $1.023$1,023 | $1,022.79 | $2.212 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 2,151 of 2,163); shown once.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.113$113estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% | — | 81.6%79.9%–83.1% | a | 1,764 / 2,163 | 837 right of 837 kept0 mistakes · 1,326 skipped | 0.903 | 4.8%answered 99% | 155 ms | 186 ms | $0.113$113estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% | — | 80.3%78.5%–81.9% | ab | 1,736 / 2,163 | 469 right of 470 kept1 wrong · 1,693 skipped | 0.898 | 0.2%answered 100% | 114 ms | 120 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% | — | 78.0%76.2%–79.7% | bc | 1,687 / 2,163 | 178 right of 178 kept0 mistakes · 1,985 skipped | 0.903 | 2.3%answered 99% | 189 ms | 232 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.047$46.87estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% | — | 77.4%75.6%–79.1% | cd | 1,674 / 2,163 | 464 right of 465 kept1 wrong · 1,698 skipped | 0.895 | 1.1%answered 100% | 764 ms | 867 ms | $0.047$46.87estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.084$84.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% | — | 77.3%75.5%–79.1% | cd | 1,673 / 2,163 | 318 right of 318 kept0 mistakes · 1,845 skipped | 0.899 | 1.7%answered 100% | 118 ms | 142 ms | $0.084$84.42estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.039$39.14estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% | — | 77.1%75.3%–78.8% | cd | 1,668 / 2,163 | 215 right of 215 kept0 mistakes · 1,948 skipped | 0.891 | 2.1%answered 100% | 637 ms | 735 ms | $0.039$39.14estimate · DGX Spark |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% | — | 75.2%73.4%–77.0% | de | 1,627 / 2,163 | 15 right of 15 kept0 mistakes · 2,148 skipped | 0.899 | 2.1%answered 99% | 196 ms | 221 ms | $0.141$141estimate · RTX PRO 6000 Blackwell |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 54.9% | — | 74.1%72.2%–75.9% | defg | 1,603 / 2,163 | 213 right of 213 kept0 mistakes · 1,950 skipped | 0.876 | 87.4%answered 61% | 179 ms | 426 ms | $0.006$6.14 |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | 74.0%72.1%–75.8% | defh | 1,600 / 2,163 | 412 right of 412 kept0 mistakes · 1,751 skipped | 0.895 | 70.6%answered 75% | 140 ms | 173 ms | $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.066$65.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% | — | 72.5%70.6%–74.3% | efghi | 1,568 / 2,163 | 323 right of 323 kept0 mistakes · 1,840 skipped | 0.856 | 9.4%answered 97% | 91 ms | 103 ms | $0.066$65.87estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.026$26.16estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% | — | 72.0%70.1%–73.8% | efghij | 1,557 / 2,163 | 198 right of 199 kept1 wrong · 1,964 skipped | 0.864 | 21.6%answered 94% | 425 ms | 461 ms | $0.026$26.16estimate · DGX Spark |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.089$88.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% | — | 71.5%69.6%–73.4% | fghij | 1,547 / 2,163 | 11 right of 11 kept0 mistakes · 2,152 skipped | 0.857 | 5.3%answered 98% | 118 ms | 148 ms | $0.089$88.58estimate · RTX PRO 6000 Blackwell |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.022$22.02 per 1,000 questionsper 1M questions Logic 50.0% | — | 71.3%69.4%–73.2% | efghij | 1,543 / 2,163 | 214 right of 214 kept0 mistakes · 1,949 skipped | 0.859 | 3.9%answered 99% | 226 ms | 351 ms | $0.022$22.02 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.028$28.30 per 1,000 questionsper 1M questions Logic 54.5% | — | 70.8%68.8%–72.7% | fghij | 1,531 / 2,163 | 161 right of 161 kept0 mistakes · 2,002 skipped | 0.865 | 40.6%answered 86% | 264 ms | 279 ms | $0.028$28.30 |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.041$40.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | 70.4%68.5%–72.3% | gij | 1,523 / 2,163 | 162 right of 163 kept1 wrong · 2,000 skipped | 0.878 | 4.7%answered 99% | 55 ms | 62 ms | $0.041$40.61estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% | — | 70.3%68.4%–72.2% | gijk | 1,521 / 2,163 | 719 right of 729 kept10 wrong · 1,434 skipped | 0.861 | 4.5%answered 99% | 76 ms | 89 ms | $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.013$12.88 per 1,000 questionsper 1M questions Logic 54.8% | — | 70.1%68.1%–72.0% | ijk | 1,516 / 2,163 | 315 right of 315 kept0 mistakes · 1,848 skipped | 0.867 | 43.5%answered 84% | 326 ms | 473 ms | $0.013$12.88 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.009$9.10 per 1,000 questionsper 1M questions Logic 47.3% | — | 70.0%68.0%–71.9% | hijk | 1,514 / 2,163 | No answers at this threshold | 0.858 | 82.7%answered 62% | 387 ms | 391 ms | $0.009$9.10 |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% | — | 69.9%67.9%–71.8% | ijk | 1,511 / 2,163 | 131 right of 131 kept0 mistakes · 2,032 skipped | 0.862 | 35.8%answered 88% | 79 ms | 84 ms | $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$0.91 per 1,000 questionsper 1M questions Logic 47.3% | — | 69.9%67.9%–71.8% | ijkl | 1,511 / 2,163 | No answers at this threshold | 0.859 | 82.9%answered 62% | 381 ms | 386 ms | $0.001$0.91 |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 51.6% | — | 69.9%67.9%–71.8% | ijk | 1,511 / 2,163 | 115 right of 115 kept0 mistakes · 2,048 skipped | 0.876 | 86.7%answered 58% | 215 ms | 347 ms | $0.006$6.14 |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.004$4.29 per 1,000 questionsper 1M questions Logic 53.3% | — | 69.8%67.8%–71.7% | ijkl | 1,509 / 2,163 | 331 right of 331 kept0 mistakes · 1,832 skipped | 0.869 | 7.0%answered 98% | 226 ms | 368 ms | $0.004$4.29 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.023$1,023 per 1,000 questionsper 1M questions Logic 52.5% | with web | 69.3%67.3%–71.2% | jklm | 1,499 / 2,163 | 326 right of 326 kept0 mistakes · 1,837 skipped | 0.873 | 15.4%answered 95% | 422 ms | 641 ms | $1.023$1,023 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.017$17.46estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% | — | 69.0%67.0%–70.9% | ijklmn | 1,493 / 2,163 | 43 right of 43 kept0 mistakes · 2,120 skipped | 0.864 | 2.2%answered 99% | 278 ms | 329 ms | $0.017$17.46estimate · DGX Spark |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.009$8.59 per 1,000 questionsper 1M questions Logic 52.4% | — | 68.6%66.6%–70.5% | jklmn | 1,483 / 2,163 | 360 right of 361 kept1 wrong · 1,802 skipped | 0.839 | 52.5%answered 80% | 381 ms | 390 ms | $0.009$8.59 |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.036$36.13estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% | — | 66.3%64.2%–68.2% | klmno | 1,433 / 2,163 | 7 right of 7 kept0 mistakes · 2,156 skipped | 0.819 | 1.1%answered 100% | 49 ms | 55 ms | $0.036$36.13estimate · RTX PRO 6000 Blackwell |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 49.2% | — | 65.5%63.5%–67.5% | lmnop | 1,417 / 2,163 | 141 right of 141 kept0 mistakes · 2,022 skipped | 0.838 | 56.3%answered 76% | 204 ms | 386 ms | $0.003$2.92 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 46.0% | — | 65.2%63.2%–67.2% | mnop | 1,410 / 2,163 | 183 right of 183 kept0 mistakes · 1,980 skipped | 0.829 | 82.5%answered 56% | 172 ms | 290 ms | $0.003$2.92 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 48.2% | — | 65.1%63.1%–67.1% | nop | 1,409 / 2,163 | 202 right of 202 kept0 mistakes · 1,961 skipped | 0.829 | 7.7%answered 97% | 163 ms | 185 ms | $0.019$18.99 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 46.3% | — | 64.1%62.0%–66.1% | opq | 1,386 / 2,163 | 248 right of 248 kept0 mistakes · 1,915 skipped | 0.834 | 5.7%answered 98% | 166 ms | 206 ms | $0.019$18.99 |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | 62.1%60.0%–64.1% | opqr | 1,343 / 2,163 | 197 right of 198 kept1 wrong · 1,965 skipped | 0.806 | 3.1%answered 99% | 36 ms | 44 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | 61.3%59.3%–63.4% | pqr | 1,327 / 2,163 | 182 right of 183 kept1 wrong · 1,980 skipped | 0.803 | 65.0%answered 67% | 80 ms | 92 ms | $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.02estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% | — | 60.1%58.0%–62.1% | qrs | 1,300 / 2,163 | No answers at this threshold | 0.803 | 3.2%answered 99% | 41 ms | 46 ms | $0.031$31.02estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.029$28.78estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% | — | 59.8%57.7%–61.9% | qrs | 1,294 / 2,163 | 64 right of 64 kept0 mistakes · 2,099 skipped | 0.814 | 6.6%answered 97% | 37 ms | 51 ms | $0.029$28.78estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% | — | 59.8%57.7%–61.8% | qrs | 1,293 / 2,163 | 123 right of 123 kept0 mistakes · 2,040 skipped | 0.799 | 0.9%answered 100% | 37 ms | 43 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.007$6.81 per 1,000 questionsper 1M questions Logic 44.4% | — | 59.4%57.3%–61.4% | rst | 1,284 / 2,163 | 27 right of 27 kept0 mistakes · 2,136 skipped | 0.787 | 5.1%answered 98% | 310 ms | 359 ms | $0.007$6.81 |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.009$8.92 per 1,000 questionsper 1M questions Logic 41.9% | — | 59.0%56.9%–61.0% | rstu | 1,276 / 2,163 | 174 right of 174 kept0 mistakes · 1,989 skipped | 0.813 | 20.7%answered 91% | 334 ms | 432 ms | $0.009$8.92 |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.032$31.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% | — | 58.8%56.7%–60.9% | rstu | 1,272 / 2,163 | 207 right of 210 kept3 wrong · 1,953 skipped | 0.804 | 1.7%answered 99% | 45 ms | 50 ms | $0.032$31.98estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.008$8.34estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% | — | 58.4%56.3%–60.5% | rstuv | 1,264 / 2,163 | 49 right of 49 kept0 mistakes · 2,114 skipped | 0.781 | 4.2%answered 98% | 140 ms | 146 ms | $0.008$8.34estimate · DGX Spark |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.010$10.49estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% | — | 57.9%55.8%–60.0% | rstuvw | 1,253 / 2,163 | 73 right of 73 kept0 mistakes · 2,090 skipped | 0.798 | 24.6%answered 89% | 174 ms | 182 ms | $0.010$10.49estimate · DGX Spark |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.023$23.34estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% | — | 56.8%54.7%–58.9% | stuvw | 1,229 / 2,163 | 33 right of 33 kept0 mistakes · 2,130 skipped | 0.789 | 3.8%answered 98% | 31 ms | 35 ms | $0.023$23.34estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.010$9.51estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% | — | 56.0%53.9%–58.1% | stuvw | 1,212 / 2,163 | 244 right of 254 kept10 wrong · 1,909 skipped | 0.780 | 1.2%answered 99% | 153 ms | 175 ms | $0.010$9.51estimate · DGX Spark |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$4.56estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% | — | 56.0%53.9%–58.1% | stuvw | 1,211 / 2,163 | 54 right of 54 kept0 mistakes · 2,109 skipped | 0.765 | 1.6%answered 99% | 74 ms | 84 ms | $0.005$4.56estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.022$22.38estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% | — | 55.9%53.8%–58.0% | stuvw | 1,210 / 2,163 | 113 right of 114 kept1 wrong · 2,049 skipped | 0.787 | 8.4%answered 96% | 28 ms | 31 ms | $0.022$22.38estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.10estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% | — | 55.8%53.7%–57.9% | stuvwx | 1,208 / 2,163 | 96 right of 96 kept0 mistakes · 2,067 skipped | 0.779 | 13.4%answered 94% | 67 ms | 69 ms | $0.004$4.10estimate · DGX Spark |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 40.8% | — | 55.6%53.5%–57.7% | stuvwxy | 1,203 / 2,163 | 152 right of 152 kept0 mistakes · 2,011 skipped | 0.767 | 2.9%answered 99% | 184 ms | 297 ms | $0.003$2.92 |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$40.93estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% | — | 55.3%53.2%–57.4% | tuvwxy | 1,197 / 2,163 | 130 right of 130 kept0 mistakes · 2,033 skipped | 0.790 | 2.6%answered 99% | 58 ms | 64 ms | $0.041$40.93estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% | — | 55.2%53.1%–57.3% | tuvwxy | 1,195 / 2,163 | 150 right of 155 kept5 wrong · 2,008 skipped | 0.781 | 12.0%answered 94% | 40 ms | 47 ms | $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.169$169estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% | — | 54.6%52.4%–56.6% | uvwxyz | 1,180 / 2,163 | No answers at this threshold | 0.764 | 5.0%answered 98% | 233 ms | 272 ms | $0.169$169estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.061$60.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% | — | 53.8%51.7%–55.9% | vwxyzA | 1,163 / 2,163 | 26 right of 26 kept0 mistakes · 2,137 skipped | 0.783 | 1.8%answered 99% | 79 ms | 106 ms | $0.061$60.76estimate · RTX PRO 6000 Blackwell |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.004$4.38 per 1,000 questionsper 1M questions Logic 45.1% | — | 53.5%51.4%–55.6% | wxyzA | 1,157 / 2,163 | 272 right of 281 kept9 wrong · 1,882 skipped | 0.789 | 3.2%answered 99% | 288 ms | 342 ms | $0.004$4.38 |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.034$33.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | 51.5%49.4%–53.6% | yzAB | 1,114 / 2,163 | 126 right of 130 kept4 wrong · 2,033 skipped | 0.752 | 6.4%answered 97% | 47 ms | 52 ms | $0.034$33.58estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% | — | 51.1%49.0%–53.2% | xyzAB | 1,105 / 2,163 | 12 right of 12 kept0 mistakes · 2,151 skipped | 0.767 | 1.6%answered 99% | 181 ms | 210 ms | $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.023$22.70estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% | — | 50.4%48.3%–52.5% | zAB | 1,091 / 2,163 | 149 right of 155 kept6 wrong · 2,008 skipped | 0.746 | 3.4%answered 98% | 28 ms | 32 ms | $0.023$22.70estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% | — | 49.8%47.7%–51.9% | AB | 1,077 / 2,163 | No answers at this threshold | 0.730 | 2.0%answered 99% | 99 ms | 127 ms | $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.038$38.37estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | 46.9%44.8%–49.0% | BC | 1,014 / 2,163 | 20 right of 21 kept1 wrong · 2,142 skipped | 0.743 | 9.2%answered 95% | 52 ms | 61 ms | $0.038$38.37estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% | — | 44.3%42.3%–46.4% | C | 959 / 2,163 | No answers at this threshold | 0.735 | 4.9%answered 97% | 37 ms | 41 ms | $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% | — | 39.2%37.1%–41.2% | D | 847 / 2,163 | No answers at this threshold | 0.651 | 2.2%answered 99% | 113 ms | 144 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.27estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% | — | 38.7%36.6%–40.7% | D | 836 / 2,163 | 2 right of 2 kept0 mistakes · 2,161 skipped | 0.685 | 7.2%answered 95% | 36 ms | 37 ms | $0.002$2.27estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% | — | 37.1%35.1%–39.1% | D | 802 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.656 | 80.2%answered 34% | 16 ms | 21 ms | $0.014$14.07estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.78estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% | — | 36.8%34.7%–38.8% | D | 795 / 2,163 | 15 right of 16 kept1 wrong · 2,147 skipped | 0.635 | 17.3%answered 89% | 28 ms | 30 ms | $0.002$1.78estimate · DGX Spark |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.20estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% | — | 35.3%33.3%–37.3% | DE | 763 / 2,163 | 8 right of 8 kept0 mistakes · 2,155 skipped | 0.636 | 1.7%answered 99% | 50 ms | 54 ms | $0.003$3.20estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% | — | 32.1%30.2%–34.1% | EF | 694 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.589 | 20.8%answered 85% | 20 ms | 21 ms | $0.001$1.28estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.53estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% | — | 29.7%27.8%–31.6% | F | 642 / 2,163 | No answers at this threshold | 0.583 | 11.3%answered 92% | 24 ms | 25 ms | $0.002$1.53estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.026$26.22estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% | — | 28.7%26.8%–30.6% | F | 620 / 2,163 | No answers at this threshold | 0.569 | 3.2%answered 98% | 35 ms | 41 ms | $0.026$26.22estimate · RTX PRO 6000 Blackwell |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.09estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% | — | 28.2%26.4%–30.2% | F | 611 / 2,163 | No answers at this threshold | 0.592 | 0.6%answered 100% | 15 ms | 17 ms | $0.001$1.09estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.69estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% | — | 28.2%26.3%–30.1% | F | 610 / 2,163 | No answers at this threshold | 0.596 | 0.3%answered 100% | 25 ms | 28 ms | $0.002$1.69estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$0.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% | — | 22.7%20.9%–24.5% | G | 490 / 2,163 | No answers at this threshold | 0.501 | 17.2%answered 86% | 12 ms | 13 ms | $0.001$0.85estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.77estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% | — | 20.6%18.9%–22.3% | G | 445 / 2,163 | No answers at this threshold | 0.500 | 72.8%answered 33% | 11 ms | 12 ms | $0.001$0.77estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI quincyqiang scar-ai Strands Agents victorbennett Metask Lab
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 20.6% at $0.001 per 1,000 questions ($0.77 per 1M) (estimate)
GLiClass v4 Edge: 22.7% at $0.001 per 1,000 questions ($0.85 per 1M) (estimate)
OneJev1.6-0.8B: 28.2% at $0.001 per 1,000 questions ($1.09 per 1M) (estimate)
GLiDE 0.8B: 32.1% at $0.001 per 1,000 questions ($1.28 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 29.7% at $0.002 per 1,000 questions ($1.53 per 1M) (estimate)
KnowLine-0.8B-Gen1: 28.2% at $0.002 per 1,000 questions ($1.69 per 1M) (estimate)
GLiDE 2B: 36.8% at $0.002 per 1,000 questions ($1.78 per 1M) (estimate)
Perplexity Decider Flash v1.2: 55.2% at $0.002 per 1,000 questions ($2.18 per 1M)
StartLux-Decision-v1.1-2B-Instant: 38.7% at $0.002 per 1,000 questions ($2.27 per 1M) (estimate)
KnowLine-2B-Gen1: 35.3% at $0.003 per 1,000 questions ($3.20 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 55.8% at $0.004 per 1,000 questions ($4.10 per 1M) (estimate)
KnowLine 4B Gen5.5: 56.0% at $0.005 per 1,000 questions ($4.56 per 1M) (estimate)
d1-2B: 37.1% at $0.014 per 1,000 questions ($14.07 per 1M) (estimate)
Perplexity Decider Flash v1.3: 55.9% at $0.022 per 1,000 questions ($22.38 per 1M) (estimate)
Chinese Decision Pointer 4B: 50.4% at $0.023 per 1,000 questions ($22.70 per 1M) (estimate)
vera-medium-omni: 56.8% at $0.023 per 1,000 questions ($23.34 per 1M) (estimate)
Strands Decider 2B: 28.7% at $0.026 per 1,000 questions ($26.22 per 1M) (estimate)
Verdin-4B-v1.1: 51.5% at $0.034 per 1,000 questions ($33.58 per 1M) (estimate)
Metask-Jev-4B: 39.2% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI quincyqiang scar-ai Strands Agents victorbennett Metask Lab
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | $0.001$0.77estimate · DGX Spark | $0.77 estimate | $0.001655 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | $0.001$0.85estimate · DGX Spark | $0.85 estimate | $0.001833 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | $0.001$1.09estimate · DGX Spark | $1.09 estimate | $0.002365 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Pareto Open · 0.8B | — | $0.001$1.28estimate · DGX Spark | $1.28 estimate | $0.002778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | $0.002$1.53estimate · DGX Spark | $1.53 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | $0.002$1.69estimate · DGX Spark | $1.69 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Pareto Open · 1.9B | — | $0.002$1.78estimate · DGX Spark | $1.78 estimate | $0.003843 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Pareto Open · 4.0B | — | $0.002$2.18 list | $2.18 | $0.004707 | $0.030$30.06estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | $0.002$2.27estimate · DGX Spark | $2.27 estimate | $0.004907 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-2B-Gen1 Open · 2.0B | — | $0.003$3.20estimate · DGX Spark | $3.20 estimate | $0.006917 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Pareto Open · 4.0B | — | $0.004$4.10estimate · DGX Spark | $4.10 estimate | $0.008867 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Pareto Open · 4.0B | — | $0.005$4.56estimate · DGX Spark | $4.56 estimate | $0.009872 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-2B Open · 1.9B | — | $0.014$14.07estimate · RTX PRO 6000 Blackwell | $14.07 estimate | $0.03043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | $0.022$22.38estimate · RTX PRO 6000 Blackwell | $22.38 estimate | $0.04842 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | $0.023$22.70estimate · RTX PRO 6000 Blackwell | $22.70 estimate | $0.04911 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Pareto Open · 4.2B | — | $0.023$23.34estimate · RTX PRO 6000 Blackwell | $23.34 estimate | $0.05049 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | $0.026$26.22estimate · RTX PRO 6000 Blackwell | $26.22 estimate | $0.05672 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | $0.034$33.58estimate · RTX PRO 6000 Blackwell | $33.58 estimate | $0.07263 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.023$23.34estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% | — | 56.8%54.7%–58.9% | a | 1,229 / 2,163 | 33 right of 33 kept0 mistakes · 2,130 skipped | 0.789 | 3.8%answered 98% | 31 ms | 35 ms | $0.023$23.34estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$4.56estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% | — | 56.0%53.9%–58.1% | a | 1,211 / 2,163 | 54 right of 54 kept0 mistakes · 2,109 skipped | 0.765 | 1.6%answered 99% | 74 ms | 84 ms | $0.005$4.56estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.022$22.38estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% | — | 55.9%53.8%–58.0% | a | 1,210 / 2,163 | 113 right of 114 kept1 wrong · 2,049 skipped | 0.787 | 8.4%answered 96% | 28 ms | 31 ms | $0.022$22.38estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.10estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% | — | 55.8%53.7%–57.9% | a | 1,208 / 2,163 | 96 right of 96 kept0 mistakes · 2,067 skipped | 0.779 | 13.4%answered 94% | 67 ms | 69 ms | $0.004$4.10estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% | — | 55.2%53.1%–57.3% | a | 1,195 / 2,163 | 150 right of 155 kept5 wrong · 2,008 skipped | 0.781 | 12.0%answered 94% | 40 ms | 47 ms | $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.034$33.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | 51.5%49.4%–53.6% | b | 1,114 / 2,163 | 126 right of 130 kept4 wrong · 2,033 skipped | 0.752 | 6.4%answered 97% | 47 ms | 52 ms | $0.034$33.58estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.023$22.70estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% | — | 50.4%48.3%–52.5% | b | 1,091 / 2,163 | 149 right of 155 kept6 wrong · 2,008 skipped | 0.746 | 3.4%answered 98% | 28 ms | 32 ms | $0.023$22.70estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% | — | 39.2%37.1%–41.2% | c | 847 / 2,163 | No answers at this threshold | 0.651 | 2.2%answered 99% | 113 ms | 144 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.27estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% | — | 38.7%36.6%–40.7% | c | 836 / 2,163 | 2 right of 2 kept0 mistakes · 2,161 skipped | 0.685 | 7.2%answered 95% | 36 ms | 37 ms | $0.002$2.27estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% | — | 37.1%35.1%–39.1% | c | 802 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.656 | 80.2%answered 34% | 16 ms | 21 ms | $0.014$14.07estimate · RTX PRO 6000 Blackwell |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.78estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% | — | 36.8%34.7%–38.8% | c | 795 / 2,163 | 15 right of 16 kept1 wrong · 2,147 skipped | 0.635 | 17.3%answered 89% | 28 ms | 30 ms | $0.002$1.78estimate · DGX Spark |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.20estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% | — | 35.3%33.3%–37.3% | cd | 763 / 2,163 | 8 right of 8 kept0 mistakes · 2,155 skipped | 0.636 | 1.7%answered 99% | 50 ms | 54 ms | $0.003$3.20estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% | — | 32.1%30.2%–34.1% | de | 694 / 2,163 | 4 right of 4 kept0 mistakes · 2,159 skipped | 0.589 | 20.8%answered 85% | 20 ms | 21 ms | $0.001$1.28estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.53estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% | — | 29.7%27.8%–31.6% | ef | 642 / 2,163 | No answers at this threshold | 0.583 | 11.3%answered 92% | 24 ms | 25 ms | $0.002$1.53estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.026$26.22estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% | — | 28.7%26.8%–30.6% | ef | 620 / 2,163 | No answers at this threshold | 0.569 | 3.2%answered 98% | 35 ms | 41 ms | $0.026$26.22estimate · RTX PRO 6000 Blackwell |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.09estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% | — | 28.2%26.4%–30.2% | f | 611 / 2,163 | No answers at this threshold | 0.592 | 0.6%answered 100% | 15 ms | 17 ms | $0.001$1.09estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.69estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% | — | 28.2%26.3%–30.1% | f | 610 / 2,163 | No answers at this threshold | 0.596 | 0.3%answered 100% | 25 ms | 28 ms | $0.002$1.69estimate · DGX Spark |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$0.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% | — | 22.7%20.9%–24.5% | g | 490 / 2,163 | No answers at this threshold | 0.501 | 17.2%answered 86% | 12 ms | 13 ms | $0.001$0.85estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.77estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% | — | 20.6%18.9%–22.3% | g | 445 / 2,163 | No answers at this threshold | 0.500 | 72.8%answered 33% | 11 ms | 12 ms | $0.001$0.77estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Base Labs StartLux Apertix AI Fastino Cloudflare Blockbrain Labs newsbang PelaAI tattrongvu Patrick-18 EldanRing Valen-Team
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Elephant-1: 61.3% at $0.008 per 1,000 questions ($7.90 per 1M)
StartLux-Decision-v1.1-9B-Instant: 58.4% at $0.008 per 1,000 questions ($8.34 per 1M) (estimate)
Cerno 9B v1.2: 56.0% at $0.010 per 1,000 questions ($9.51 per 1M) (estimate)
GLiDE 9B: 57.9% at $0.010 per 1,000 questions ($10.49 per 1M) (estimate)
Clef-Flash: 49.8% at $0.011 per 1,000 questions ($10.85 per 1M)
Cygnet: 44.3% at $0.024 per 1,000 questions ($23.94 per 1M)
Adjudex 9B v5: 62.1% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
KnowLine-9B-Gen1: 59.8% at $0.027 per 1,000 questions ($26.54 per 1M) (estimate)
Jeffan-9B-1211: 59.8% at $0.029 per 1,000 questions ($28.78 per 1M) (estimate)
KnowLine-9B-Gen2: 60.1% at $0.031 per 1,000 questions ($31.02 per 1M) (estimate)
Dexy-Decision-9B-V2: 58.8% at $0.032 per 1,000 questions ($31.98 per 1M) (estimate)
KnowLine-12B-Gen1: 66.3% at $0.036 per 1,000 questions ($36.13 per 1M) (estimate)
Winnow-12B: 46.9% at $0.038 per 1,000 questions ($38.37 per 1M) (estimate)
Valen-Decision-1.5-9B: 55.3% at $0.041 per 1,000 questions ($40.93 per 1M) (estimate)
Base Labs StartLux Apertix AI Fastino Cloudflare Blockbrain Labs newsbang PelaAI tattrongvu Patrick-18 EldanRing Valen-Team
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Elephant-1 Pareto Open · 9.0B | — | $0.008$7.90 list | $7.90 | $0.01708 | $0.058$58.20estimate · RTX PRO 6000 Blackwell | maker's list price |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | $0.008$8.34estimate · DGX Spark | $8.34 estimate | $0.01803 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cerno 9B v1.2 Open · 9.0B | — | $0.010$9.51estimate · DGX Spark | $9.51 estimate | $0.02057 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Open · 7.9B | — | $0.010$10.49estimate · DGX Spark | $10.49 estimate | $0.0227 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Clef-Flash Open · 9.0B | — | $0.011$10.85 list | $10.85 | $0.02347 | $0.072$71.95estimate · RTX PRO 6000 Blackwell | maker's list price |
| Cygnet Open · 12B | — | $0.024$23.94 list | $23.94 | $0.05179 | $0.028$27.50estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| Adjudex 9B v5 Pareto Open · 9.0B | — | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen1 Open · 9.0B | — | $0.027$26.54estimate · RTX PRO 6000 Blackwell | $26.54 estimate | $0.05741 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Open · 9.0B | — | $0.029$28.78estimate · RTX PRO 6000 Blackwell | $28.78 estimate | $0.06225 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | $0.031$31.02estimate · RTX PRO 6000 Blackwell | $31.02 estimate | $0.06709 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | $0.032$31.98estimate · RTX PRO 6000 Blackwell | $31.98 estimate | $0.06917 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Pareto Open · 12B | — | $0.036$36.13estimate · RTX PRO 6000 Blackwell | $36.13 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | $0.038$38.37estimate · RTX PRO 6000 Blackwell | $38.37 estimate | $0.083 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | $0.041$40.93estimate · RTX PRO 6000 Blackwell | $40.93 estimate | $0.08853 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.036$36.13estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% | — | 66.3%64.2%–68.2% | a | 1,433 / 2,163 | 7 right of 7 kept0 mistakes · 2,156 skipped | 0.819 | 1.1%answered 100% | 49 ms | 55 ms | $0.036$36.13estimate · RTX PRO 6000 Blackwell |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | 62.1%60.0%–64.1% | b | 1,343 / 2,163 | 197 right of 198 kept1 wrong · 1,965 skipped | 0.806 | 3.1%answered 99% | 36 ms | 44 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% | — | 61.3%59.3%–63.4% | bc | 1,327 / 2,163 | 182 right of 183 kept1 wrong · 1,980 skipped | 0.803 | 65.0%answered 67% | 80 ms | 92 ms | $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.031$31.02estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% | — | 60.1%58.0%–62.1% | bcd | 1,300 / 2,163 | No answers at this threshold | 0.803 | 3.2%answered 99% | 41 ms | 46 ms | $0.031$31.02estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.029$28.78estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% | — | 59.8%57.7%–61.9% | bcd | 1,294 / 2,163 | 64 right of 64 kept0 mistakes · 2,099 skipped | 0.814 | 6.6%answered 97% | 37 ms | 51 ms | $0.029$28.78estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% | — | 59.8%57.7%–61.8% | bcd | 1,293 / 2,163 | 123 right of 123 kept0 mistakes · 2,040 skipped | 0.799 | 0.9%answered 100% | 37 ms | 43 ms | $0.027$26.54estimate · RTX PRO 6000 Blackwell |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.032$31.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% | — | 58.8%56.7%–60.9% | cde | 1,272 / 2,163 | 207 right of 210 kept3 wrong · 1,953 skipped | 0.804 | 1.7%answered 99% | 45 ms | 50 ms | $0.032$31.98estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.008$8.34estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% | — | 58.4%56.3%–60.5% | cde | 1,264 / 2,163 | 49 right of 49 kept0 mistakes · 2,114 skipped | 0.781 | 4.2%answered 98% | 140 ms | 146 ms | $0.008$8.34estimate · DGX Spark |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.010$10.49estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% | — | 57.9%55.8%–60.0% | de | 1,253 / 2,163 | 73 right of 73 kept0 mistakes · 2,090 skipped | 0.798 | 24.6%answered 89% | 174 ms | 182 ms | $0.010$10.49estimate · DGX Spark |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.010$9.51estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% | — | 56.0%53.9%–58.1% | e | 1,212 / 2,163 | 244 right of 254 kept10 wrong · 1,909 skipped | 0.780 | 1.2%answered 99% | 153 ms | 175 ms | $0.010$9.51estimate · DGX Spark |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$40.93estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% | — | 55.3%53.2%–57.4% | e | 1,197 / 2,163 | 130 right of 130 kept0 mistakes · 2,033 skipped | 0.790 | 2.6%answered 99% | 58 ms | 64 ms | $0.041$40.93estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% | — | 49.8%47.7%–51.9% | f | 1,077 / 2,163 | No answers at this threshold | 0.730 | 2.0%answered 99% | 99 ms | 127 ms | $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.038$38.37estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% | — | 46.9%44.8%–49.0% | fg | 1,014 / 2,163 | 20 right of 21 kept1 wrong · 2,142 skipped | 0.743 | 9.2%answered 95% | 52 ms | 61 ms | $0.038$38.37estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% | — | 44.3%42.3%–46.4% | g | 959 / 2,163 | No answers at this threshold | 0.735 | 4.9%answered 97% | 37 ms | 41 ms | $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Perplexity Fastino Nace.AI Turion.AI Base Labs Sthānika AI vLLM Semantic Router PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Perplexity Decider 27B v1.2 (API): 69.8% at $0.004 per 1,000 questions ($4.29 per 1M)
GLiDE (v1.2): 69.9% at $0.008 per 1,000 questions ($7.55 per 1M)
Drex 2.0: 59.0% at $0.009 per 1,000 questions ($8.92 per 1M)
Drex Max: 70.3% at $0.013 per 1,000 questions ($13.06 per 1M)
Turion Jev 31B: 69.0% at $0.017 per 1,000 questions ($17.46 per 1M) (estimate)
Mammoth-1: 74.0% at $0.023 per 1,000 questions ($22.56 per 1M)
GLiDE 27B: 72.0% at $0.026 per 1,000 questions ($26.16 per 1M) (estimate)
Sieve 31B v3: 77.1% at $0.039 per 1,000 questions ($39.14 per 1M) (estimate)
d3: 70.4% at $0.041 per 1,000 questions ($40.61 per 1M) (estimate)
KnowLine-31B-Gen1.5: 77.4% at $0.047 per 1,000 questions ($46.87 per 1M) (estimate)
OneJev-27B: 53.8% at $0.061 per 1,000 questions ($60.76 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 72.5% at $0.066 per 1,000 questions ($65.87 per 1M) (estimate)
Clef: 51.1% at $0.069 per 1,000 questions ($68.54 per 1M)
KnowLine-31B-Gen2: 80.3% at $0.083 per 1,000 questions ($83.14 per 1M) (estimate)
Copernicus-31B: 77.3% at $0.084 per 1,000 questions ($84.42 per 1M) (estimate)
d3-ultra: 71.5% at $0.089 per 1,000 questions ($88.58 per 1M) (estimate)
Karpo Decision 31B: 81.6% at $0.113 per 1,000 questions ($113 per 1M) (estimate)
Open-Jev-27B-v1.1: 54.6% at $0.169 per 1,000 questions ($169 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 69.3% at $1.023 per 1,000 questions ($1,023 per 1M)
Perplexity Fastino Nace.AI Turion.AI Base Labs Sthānika AI vLLM Semantic Router PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | $0.004$4.29 | $4.29 | $0.009285 | — | provider price × reported usage |
| GLiDE Pareto v1.2 Open · 27B | — | $0.008$7.55 list | $7.55 | $0.01632 | $0.058$57.88estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | $0.009$8.92 | $8.92 | $0.01929 | — | provider price × reported usage |
| Drex Max Pareto Open · 27B | — | $0.013$13.06 list | $13.06 | $0.02824 | $0.055$54.68estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | $0.017$17.46estimate · DGX Spark | $17.46 estimate | $0.03778 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Mammoth-1 Pareto Open · 27B | — | $0.023$22.56 list | $22.56 | $0.0488 | $0.104$104estimate · RTX PRO 6000 Blackwell | maker's list price |
| GLiDE 27B Open · 27B | — | $0.026$26.16estimate · DGX Spark | $26.16 estimate | $0.05657 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Sieve 31B v3 Pareto Open · 31B | — | $0.039$39.14estimate · DGX Spark | $39.14 estimate | $0.08465 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d3 Open · 26B | — | $0.041$40.61estimate · RTX PRO 6000 Blackwell | $40.61 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | $0.047$46.87estimate · DGX Spark | $46.87 estimate | $0.1014 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | $0.061$60.76estimate · RTX PRO 6000 Blackwell | $60.76 estimate | $0.1314 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | $0.066$65.87estimate · RTX PRO 6000 Blackwell | $65.87 estimate | $0.1425 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | $0.069$68.54 list | $68.54 | $0.1483 | $0.131$131estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-31B-Gen2 Pareto Open · 31B | — | $0.083$83.14estimate · RTX PRO 6000 Blackwell | $83.14 estimate | $0.1798 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | $0.084$84.42estimate · RTX PRO 6000 Blackwell | $84.42 estimate | $0.1826 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | $0.089$88.58estimate · RTX PRO 6000 Blackwell | $88.58 estimate | $0.1916 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | $0.113$113estimate · RTX PRO 6000 Blackwell | $112.56 estimate | $0.2435 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | $0.169$169estimate · RTX PRO 6000 Blackwell | $169.16 estimate | $0.3659 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | $1.023$1,023 | $1,022.79 | $2.212 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.113$113estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% | — | 81.6%79.9%–83.1% | a | 1,764 / 2,163 | 837 right of 837 kept0 mistakes · 1,326 skipped | 0.903 | 4.8%answered 99% | 155 ms | 186 ms | $0.113$113estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% | — | 80.3%78.5%–81.9% | a | 1,736 / 2,163 | 469 right of 470 kept1 wrong · 1,693 skipped | 0.898 | 0.2%answered 100% | 114 ms | 120 ms | $0.083$83.14estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.047$46.87estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% | — | 77.4%75.6%–79.1% | b | 1,674 / 2,163 | 464 right of 465 kept1 wrong · 1,698 skipped | 0.895 | 1.1%answered 100% | 764 ms | 867 ms | $0.047$46.87estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.084$84.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% | — | 77.3%75.5%–79.1% | b | 1,673 / 2,163 | 318 right of 318 kept0 mistakes · 1,845 skipped | 0.899 | 1.7%answered 100% | 118 ms | 142 ms | $0.084$84.42estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.039$39.14estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% | — | 77.1%75.3%–78.8% | bc | 1,668 / 2,163 | 215 right of 215 kept0 mistakes · 1,948 skipped | 0.891 | 2.1%answered 100% | 637 ms | 735 ms | $0.039$39.14estimate · DGX Spark |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | 74.0%72.1%–75.8% | cd | 1,600 / 2,163 | 412 right of 412 kept0 mistakes · 1,751 skipped | 0.895 | 70.6%answered 75% | 140 ms | 173 ms | $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.066$65.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% | — | 72.5%70.6%–74.3% | de | 1,568 / 2,163 | 323 right of 323 kept0 mistakes · 1,840 skipped | 0.856 | 9.4%answered 97% | 91 ms | 103 ms | $0.066$65.87estimate · RTX PRO 6000 Blackwell |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.026$26.16estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% | — | 72.0%70.1%–73.8% | def | 1,557 / 2,163 | 198 right of 199 kept1 wrong · 1,964 skipped | 0.864 | 21.6%answered 94% | 425 ms | 461 ms | $0.026$26.16estimate · DGX Spark |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.089$88.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% | — | 71.5%69.6%–73.4% | def | 1,547 / 2,163 | 11 right of 11 kept0 mistakes · 2,152 skipped | 0.857 | 5.3%answered 98% | 118 ms | 148 ms | $0.089$88.58estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.041$40.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% | — | 70.4%68.5%–72.3% | ef | 1,523 / 2,163 | 162 right of 163 kept1 wrong · 2,000 skipped | 0.878 | 4.7%answered 99% | 55 ms | 62 ms | $0.041$40.61estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% | — | 70.3%68.4%–72.2% | ef | 1,521 / 2,163 | 719 right of 729 kept10 wrong · 1,434 skipped | 0.861 | 4.5%answered 99% | 76 ms | 89 ms | $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% | — | 69.9%67.9%–71.8% | ef | 1,511 / 2,163 | 131 right of 131 kept0 mistakes · 2,032 skipped | 0.862 | 35.8%answered 88% | 79 ms | 84 ms | $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.004$4.29 per 1,000 questionsper 1M questions Logic 53.3% | — | 69.8%67.8%–71.7% | f | 1,509 / 2,163 | 331 right of 331 kept0 mistakes · 1,832 skipped | 0.869 | 7.0%answered 98% | 226 ms | 368 ms | $0.004$4.29 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.023$1,023 per 1,000 questionsper 1M questions Logic 52.5% | with web | 69.3%67.3%–71.2% | f | 1,499 / 2,163 | 326 right of 326 kept0 mistakes · 1,837 skipped | 0.873 | 15.4%answered 95% | 422 ms | 641 ms | $1.023$1,023 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.017$17.46estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% | — | 69.0%67.0%–70.9% | f | 1,493 / 2,163 | 43 right of 43 kept0 mistakes · 2,120 skipped | 0.864 | 2.2%answered 99% | 278 ms | 329 ms | $0.017$17.46estimate · DGX Spark |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.009$8.92 per 1,000 questionsper 1M questions Logic 41.9% | — | 59.0%56.9%–61.0% | g | 1,276 / 2,163 | 174 right of 174 kept0 mistakes · 1,989 skipped | 0.813 | 20.7%answered 91% | 334 ms | 432 ms | $0.009$8.92 |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.169$169estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% | — | 54.6%52.4%–56.6% | h | 1,180 / 2,163 | No answers at this threshold | 0.764 | 5.0%answered 98% | 233 ms | 272 ms | $0.169$169estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.061$60.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% | — | 53.8%51.7%–55.9% | h | 1,163 / 2,163 | 26 right of 26 kept0 mistakes · 2,137 skipped | 0.783 | 1.8%answered 99% | 79 ms | 106 ms | $0.061$60.76estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% | — | 51.1%49.0%–53.2% | h | 1,105 / 2,163 | 12 right of 12 kept0 mistakes · 2,151 skipped | 0.767 | 1.6%answered 99% | 181 ms | 210 ms | $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Logits REAX
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Logits Sigma-1 Duo: 75.2% at $0.141 per 1,000 questions ($141 per 1M) (estimate)
REAX 1.3: 78.0% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
Logits REAX
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Logits Sigma-1 Duo Pareto Open · 57B | — | $0.141$141estimate · RTX PRO 6000 Blackwell | $140.70 estimate | $0.3043 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Pareto Open · 57B | — | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.98 estimate | $0.3071 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% | — | 78.0%76.2%–79.7% | a | 1,687 / 2,163 | 178 right of 178 kept0 mistakes · 1,985 skipped | 0.903 | 2.3%answered 99% | 189 ms | 232 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% | — | 75.2%73.4%–77.0% | b | 1,627 / 2,163 | 15 right of 15 kept0 mistakes · 2,148 skipped | 0.899 | 2.1%answered 99% | 196 ms | 221 ms | $0.141$141estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
celeris-1.1-decision: 69.9% at $0.001 per 1,000 questions ($0.91 per 1M)
d1.1: 65.2% at $0.003 per 1,000 questions ($2.92 per 1M)
d1: 55.6% at $0.003 per 1,000 questions ($2.92 per 1M)
Mercury Decide: 53.5% at $0.004 per 1,000 questions ($4.38 per 1M)
d1-ultra: 74.1% at $0.006 per 1,000 questions ($6.14 per 1M)
Microsoft-Decision-1: 59.4% at $0.007 per 1,000 questions ($6.81 per 1M)
SubQ Snap 1.1: 68.6% at $0.009 per 1,000 questions ($8.59 per 1M)
celeris-1-decision: 70.0% at $0.009 per 1,000 questions ($9.10 per 1M)
Drex Max v1.1: 70.1% at $0.013 per 1,000 questions ($12.88 per 1M)
Jev 1.14: 65.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Jev 1.13: 64.1% at $0.019 per 1,000 questions ($18.99 per 1M)
Gemini Decision 1 (20261111): 71.3% at $0.022 per 1,000 questions ($22.02 per 1M)
GLiDE (API): 70.8% at $0.028 per 1,000 questions ($28.30 per 1M)
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| celeris-1.1-decision Pareto Closed · Efficient | — | $0.001$0.91 | $0.91 | $0.001968 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1 Closed · Efficient | — | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | $0.004$4.38 | $4.38 | $0.009483 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | $0.007$6.81 | $6.81 | $0.01472 | — | provider price × reported usage |
| SubQ Snap 1.1 Closed · Efficient | — | $0.009$8.59 | $8.59 | $0.01857 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | $0.009$9.10 | $9.10 | $0.01968 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | $0.013$12.88 | $12.88 | $0.02786 | — | provider price × reported usage |
| Jev 1.14 Closed · Efficient | — | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | $0.019$18.99 | $18.99 | $0.04108 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | $0.022$22.02 | $22.02 | $0.04764 | — | provider price × reported usage |
| GLiDE API Closed · Efficient | — | $0.028$28.30 | $28.30 | $0.0612 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 2,151 of 2,163); shown once.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 54.9% | — | 74.1%72.2%–75.9% | a | 1,603 / 2,163 | 213 right of 213 kept0 mistakes · 1,950 skipped | 0.876 | 87.4%answered 61% | 179 ms | 426 ms | $0.006$6.14 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.022$22.02 per 1,000 questionsper 1M questions Logic 50.0% | — | 71.3%69.4%–73.2% | ab | 1,543 / 2,163 | 214 right of 214 kept0 mistakes · 1,949 skipped | 0.859 | 3.9%answered 99% | 226 ms | 351 ms | $0.022$22.02 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.028$28.30 per 1,000 questionsper 1M questions Logic 54.5% | — | 70.8%68.8%–72.7% | b | 1,531 / 2,163 | 161 right of 161 kept0 mistakes · 2,002 skipped | 0.865 | 40.6%answered 86% | 264 ms | 279 ms | $0.028$28.30 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.013$12.88 per 1,000 questionsper 1M questions Logic 54.8% | — | 70.1%68.1%–72.0% | b | 1,516 / 2,163 | 315 right of 315 kept0 mistakes · 1,848 skipped | 0.867 | 43.5%answered 84% | 326 ms | 473 ms | $0.013$12.88 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.009$9.10 per 1,000 questionsper 1M questions Logic 47.3% | — | 70.0%68.0%–71.9% | b | 1,514 / 2,163 | No answers at this threshold | 0.858 | 82.7%answered 62% | 387 ms | 391 ms | $0.009$9.10 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$0.91 per 1,000 questionsper 1M questions Logic 47.3% | — | 69.9%67.9%–71.8% | b | 1,511 / 2,163 | No answers at this threshold | 0.859 | 82.9%answered 62% | 381 ms | 386 ms | $0.001$0.91 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.009$8.59 per 1,000 questionsper 1M questions Logic 52.4% | — | 68.6%66.6%–70.5% | b | 1,483 / 2,163 | 360 right of 361 kept1 wrong · 1,802 skipped | 0.839 | 52.5%answered 80% | 381 ms | 390 ms | $0.009$8.59 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 46.0% | — | 65.2%63.2%–67.2% | c | 1,410 / 2,163 | 183 right of 183 kept0 mistakes · 1,980 skipped | 0.829 | 82.5%answered 56% | 172 ms | 290 ms | $0.003$2.92 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 48.2% | — | 65.1%63.1%–67.1% | c | 1,409 / 2,163 | 202 right of 202 kept0 mistakes · 1,961 skipped | 0.829 | 7.7%answered 97% | 163 ms | 185 ms | $0.019$18.99 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.019$18.99 per 1,000 questionsper 1M questions Logic 46.3% | — | 64.1%62.0%–66.1% | c | 1,386 / 2,163 | 248 right of 248 kept0 mistakes · 1,915 skipped | 0.834 | 5.7%answered 98% | 166 ms | 206 ms | $0.019$18.99 |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.007$6.81 per 1,000 questionsper 1M questions Logic 44.4% | — | 59.4%57.3%–61.4% | d | 1,284 / 2,163 | 27 right of 27 kept0 mistakes · 2,136 skipped | 0.787 | 5.1%answered 98% | 310 ms | 359 ms | $0.007$6.81 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 40.8% | — | 55.6%53.5%–57.7% | e | 1,203 / 2,163 | 152 right of 152 kept0 mistakes · 2,011 skipped | 0.767 | 2.9%answered 99% | 184 ms | 297 ms | $0.003$2.92 |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.004$4.38 per 1,000 questionsper 1M questions Logic 45.1% | — | 53.5%51.4%–55.6% | e | 1,157 / 2,163 | 272 right of 281 kept9 wrong · 1,882 skipped | 0.789 | 3.2%answered 99% | 288 ms | 342 ms | $0.004$4.38 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
No workhorse decision-model setup has answers on Math yet. Decision-model runs appear here as soon as the first one stores answers.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
No flagship decision-model setup has answers on Math yet. Decision-model runs appear here as soon as the first one stores answers.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
d1.2: 65.5% at $0.003 per 1,000 questions ($2.92 per 1M)
d1.2-ultra: 69.9% at $0.006 per 1,000 questions ($6.14 per 1M)
Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| d1.2 Pareto Closed · Undisclosed | — | $0.003$2.92 | $2.92 | $0.006325 | — | provider price × reported usage |
| d1.2-ultra Pareto Closed · Undisclosed | — | $0.006$6.14 | $6.14 | $0.01328 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.006$6.14 per 1,000 questionsper 1M questions Logic 51.6% | — | 69.9%67.9%–71.8% | a | 1,511 / 2,163 | 115 right of 115 kept0 mistakes · 2,048 skipped | 0.876 | 86.7%answered 58% | 215 ms | 347 ms | $0.006$6.14 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.003$2.92 per 1,000 questionsper 1M questions Logic 49.2% | — | 65.5%63.5%–67.5% | b | 1,417 / 2,163 | 141 right of 141 kept0 mistakes · 2,022 skipped | 0.838 | 56.3%answered 76% | 204 ms | 386 ms | $0.003$2.92 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Other MiniMax Google Z.ai Xiaomi OpenBMB DeepSeek eCloud Technology Alibaba Liquid AI Meta Thinking Machines IBM KRAFTON Anthropic xAI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
LFM3-1.7B (off · chat): 98.7% at $0.060 per 1,000 questions ($59.94 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 98.2% at $0.103 per 1,000 questions ($103 per 1M)
LFM3-1.7B (on · chat): 98.9% at $0.132 per 1,000 questions ($132 per 1M) (estimate)
Gemma 4 E2B (on · chat): 96.9% at $0.138 per 1,000 questions ($138 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 98.9% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
GLM-5.3-Flash (high · chat): 98.8% at $0.166 per 1,000 questions ($166 per 1M)
Gemma 4 E2B (on · chat · 32k cap · sampled): 97.7% at $0.196 per 1,000 questions ($196 per 1M) (estimate)
Gemma 4 E2B (chat): 96.8% at $0.208 per 1,000 questions ($208 per 1M) (estimate)
MiMo-V2.6-Flash (on · chat): 99.4% at $0.212 per 1,000 questions ($212 per 1M)
MiniCPM5-2B (off · chat): 98.2% at $0.219 per 1,000 questions ($219 per 1M) (estimate)
DeepSeek-V4.1-Flash (high · chat · 1017): 99.6% at $0.227 per 1,000 questions ($227 per 1M)
Gemma 4 E4B (on · chat): 98.2% at $0.233 per 1,000 questions ($233 per 1M) (estimate)
Erk-14B (off · chat): 90.8% at $0.266 per 1,000 questions ($266 per 1M) (estimate)
Gemma 4 E4B (chat): 98.1% at $0.271 per 1,000 questions ($271 per 1M) (estimate)
DeepSeek-V4-Flash (chat): 99.3% at $0.295 per 1,000 questions ($295 per 1M) (estimate)
Qwen3.5-4B (off · chat): 98.5% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 94.0% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 98.5% at $0.318 per 1,000 questions ($318 per 1M) (estimate)
Qwen3-14B (off · chat): 95.8% at $0.355 per 1,000 questions ($355 per 1M) (estimate)
Qwen3.5-2B (off · chat): 94.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
Muse Ember (max · chat): 99.6% at $0.365 per 1,000 questions ($365 per 1M)
Qwen3.8 Flash (off · chat): 99.5% at $0.397 per 1,000 questions ($397 per 1M)
MiniCPM5-2B (on · chat): 99.2% at $0.409 per 1,000 questions ($409 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 97.8% at $0.410 per 1,000 questions ($410 per 1M) (estimate)
Gemma 4 12B Unified (chat): 98.8% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 96.3% at $0.482 per 1,000 questions ($482 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Gemma 4 E4B (on · chat · 32k cap): 98.5% at $0.518 per 1,000 questions ($518 per 1M) (estimate)
Qwen3.8-27B (low · chat): 99.2% at $0.530 per 1,000 questions ($530 per 1M) (estimate)
Gemma 4 31B (chat): 99.4% at $0.531 per 1,000 questions ($531 per 1M) (estimate)
Inkling 2 Small (high · chat): 99.5% at $0.638 per 1,000 questions ($638 per 1M)
Granite-4.2-3B (high · chat): 99.2% at $0.781 per 1,000 questions ($781 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 99.0% at $0.939 per 1,000 questions ($939 per 1M) (estimate)
Inkling-Small (high · chat · Jul 2026): 99.6% at $0.954 per 1,000 questions ($954 per 1M)
Qwen3.5-4B (on · chat): 99.4% at $1.271 per 1,000 questions ($1,271 per 1M) (estimate)
Qwen4-Max (high · chat): 99.8% at $1.478 per 1,000 questions ($1,478 per 1M)
Raon-1 nano (on · chat): 97.0% at $1.670 per 1,000 questions ($1,670 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (off · chat): 89.6% at $1.901 per 1,000 questions ($1,901 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Qwen3.5-2B (on · chat): 94.7% at $2.470 per 1,000 questions ($2,470 per 1M) (estimate)
Claude Sonnet 5.5 (chat): 99.7% at $4.734 per 1,000 questions ($4,734 per 1M)
Claude Opus 5.5 (chat): 99.8% at $9.871 per 1,000 questions ($9,871 per 1M)
Grok 4.7 (chat): 99.5% at $12.061 per 1,000 questions ($12,061 per 1M)
Other MiniMax Google Z.ai Xiaomi OpenBMB DeepSeek eCloud Technology Alibaba Liquid AI Meta Thinking Machines IBM KRAFTON Anthropic xAI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | $0.060$59.94estimate · DGX Spark | $59.94 estimate | $0.1296 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active | — | $0.103$103 | $102.85 | $0.2225 | — | provider price × reported usage |
| LFM3-1.7B Pareto on · chat Open · 1.7B | — | $0.132$132estimate · RTX PRO 6000 Blackwell | $132.39 estimate | $0.2808 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | $0.138$138estimate · DGX Spark | $138.05 estimate | $0.2986 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.77 estimate | $0.3182 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active | — | $0.166$166 | $166.50 | $0.3601 | — | provider price × reported usage |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | $0.196$196estimate · RTX PRO 6000 Blackwell | $195.81 estimate | $0.4219 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B chat Open · 5.1B | — | $0.208$208estimate · RTX PRO 6000 Blackwell | $208.17 estimate | $0.4503 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | $0.212$212 | $211.85 | $0.4582 | — | provider price × reported usage |
| MiniCPM5-2B off · chat Open · 2.5B | — | $0.219$219estimate · RTX PRO 6000 Blackwell | $218.83 estimate | $0.48 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | $0.227$227 | $227.42 | $0.4919 | — | provider price × reported usage |
| Gemma 4 E4B on · chat Open · 8.0B | — | $0.233$233estimate · DGX Spark | $232.86 estimate | $0.5037 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | $0.266$266estimate · RTX PRO 6000 Blackwell | $266.05 estimate | $0.5755 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B chat Open · 8.0B | — | $0.271$271estimate · RTX PRO 6000 Blackwell | $271.17 estimate | $0.5865 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active | — | $0.295$295estimate · DGX Spark | $294.82 estimate | $0.6377 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Qwen3.5-4B off · chat Open · 4.7B | — | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B | — | $0.318$318estimate · RTX PRO 6000 Blackwell | $317.64 estimate | $0.7034 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | $0.355$355estimate · RTX PRO 6000 Blackwell | $354.95 estimate | $0.7678 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | $0.363$363estimate · RTX PRO 6000 Blackwell | $363.26 estimate | $0.7857 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Muse Ember max · chat Closed · Efficient | — | $0.365$365 | $364.58 | $0.7886 | — | provider price × reported usage |
| Qwen3.8 Flash off · chat Closed · Undisclosed | — | $0.397$397 | $396.92 | $0.8585 | — | provider price × reported usage |
| MiniCPM5-2B on · chat Open · 2.5B | — | $0.409$409estimate · RTX PRO 6000 Blackwell | $408.56 estimate | $0.8978 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | $0.410$410estimate · RTX PRO 6000 Blackwell | $409.63 estimate | $0.886 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified chat Open · 12B | — | $0.467$467estimate · RTX PRO 6000 Blackwell | $467.19 estimate | $1.011 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | $0.482$482estimate · RTX PRO 6000 Blackwell† | $482.22 estimate† | $1.043 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | $0.518$518estimate · RTX PRO 6000 Blackwell | $518.03 estimate | $1.121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B low · chat Open · 27B | — | $0.530$530estimate · RTX PRO 6000 Blackwell | $529.86 estimate | $1.146 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B chat Open · 31B | — | $0.531$531estimate · RTX PRO 6000 Blackwell | $530.82 estimate | $1.148 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | $0.638$638 | $638.48 | $1.381 | — | provider price × reported usage |
| Granite-4.2-3B high · chat Open · 3.0B | — | $0.781$781estimate · RTX PRO 6000 Blackwell | $780.88 estimate | $1.689 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | $0.939$939estimate · RTX PRO 6000 Blackwell | $939.17 estimate | $2.031 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | $0.954$954 | $953.81 | $2.063 | — | provider price × reported usage |
| Qwen3.5-4B on · chat Open · 4.7B | — | $1.271$1,271estimate · RTX PRO 6000 Blackwell | $1,271.09 estimate | $2.749 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | $1.478$1,478 | $1,478.15 | $3.197 | — | provider price × reported usage |
| Raon-1 nano on · chat Open · 3.1B | — | $1.670$1,670estimate · RTX PRO 6000 Blackwell† | $1,670.17 estimate† | $3.613 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | $1.901$1,901estimate · RTX PRO 6000 Blackwell† | $1,900.72 estimate† | $4.111 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | $2.470$2,470estimate · RTX PRO 6000 Blackwell | $2,469.92 estimate | $5.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Sonnet 5.5 chat Closed · Workhorse | — | $4.734$4,734 API list-price equivalent | $4,734.30 | $10.24 | — | Claude subscription |
| Claude Opus 5.5 chat Closed · Flagship | — | $9.871$9,871 API list-price equivalent | $9,870.94 | $21.35 | — | Claude subscription |
| Grok 4.7 chat Closed · Undisclosed | — | $12.061$12,061 API list-price equivalent | $12,061.26 | $26.09 | — | xAI subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | — | — | — | — | |
| GPT-6 Astra chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Hydrogen chat | — | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6 Luna chat Closed · Efficient | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Not ranked (only complete runs are): Claude Fable 5.2 (chat) (not running now, 5 of 2,163 stored, last answer 11 Nov 12:02 UTC); Gemini 4 Carbon (high · chat) (paused for now at 54 of 2,163 (zeki #4681, 2026-11-11T11:53Z)); Granite-4.2-3B (off · chat) (not running now, 25 of 2,163 stored, last answer 16 Nov 20:02 UTC); Qwen3.5-397B-A17B (on · chat) (Logic only; on this test only its 25-question smoke test; cost so far $5.552$5,552 per 1,000 questions, over the 25 of 2,163 answered); GPT-6 Luna (max · chat) (paused for now at 200 of 2,163 (zeki #4129, 2026-11-10T14:13Z); cost so far $0.182$182 per 1,000 questions, over the 200 of 2,163 answered).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $9.871$9,871 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% | — | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,200 ms | 4,790 ms | $9.871$9,871 API list-price equivalent |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $1.478$1,478 per 1,000 questionsper 1M questions Logic 96.3% | — | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 4,710 ms | 9,130 ms | $1.478$1,478 |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% | — | 99.7%99.4%–99.9% | abc | 2,157 / 2,163 | Not measured | Not measured | 60.0%answered 100% | 5,030 ms | 6,490 ms | $0.000$0.00 |
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $4.734$4,734 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% | — | 99.7%99.3%–99.8% | abd | 2,156 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,460 ms | 5,970 ms | $4.734$4,734 API list-price equivalent |
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.227$227 per 1,000 questionsper 1M questions Logic 96.2% | — | 99.6%99.3%–99.8% | abcdef | 2,155 / 2,163 | Not measured | Not measured | 20.0%answered 100% | 1,910 ms | 3,570 ms | $0.227$227 |
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% | — | 99.6%99.3%–99.8% | abce | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 5,280 ms | 7,260 ms | $0.000$0.00 |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% | — | 99.6%99.3%–99.8% | abce | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 4,810 ms | 6,560 ms | $0.000$0.00 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $0.954$954 per 1,000 questionsper 1M questions Logic 95.4% | — | 99.6%99.3%–99.8% | abce | 2,155 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 2,030 ms | 6,330 ms | $0.954$954 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% | — | 99.6%99.2%–99.8% | abcdefg | 2,154 / 2,163 | Not measured | Not measured | 75.0%answered 99% | 13,410 ms | 27,800 ms | — |
| Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Logic 96.1% | — | 99.6%99.2%–99.8% | abcdef | 2,154 / 2,163 | Not measured | Not measured | 85.7%answered 99% | 4,400 ms | 6,680 ms | — |
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.365$365 per 1,000 questionsper 1M questions Logic 95.5% | — | 99.6%99.2%–99.8% | abcdefg | 2,154 / 2,163 | Not measured | Not measured | 64.3%answered 100% | 1,260 ms | 2,270 ms | $0.365$365 |
| Grok 4.7 chat Closed · Undisclosed Grok 4.7 chat · xAI · chat model Parameters: not disclosed by maker (closed weights) Cost $12.061$12,061 API list-price equivalent per 1,000 questionsper 1M questions Logic running | — | 99.5%99.2%–99.7% | abcdefg | 2,153 / 2,163 | Not measured | Not measured | 97.5%answered 98% | 11,100 ms | 66,530 ms | $12.061$12,061 API list-price equivalent |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $0.638$638 per 1,000 questionsper 1M questions Logic 95.6% | — | 99.5%99.2%–99.7% | abcdef | 2,153 / 2,163 | Not measured | Not measured | 61.5%answered 100% | 2,110 ms | 6,000 ms | $0.638$638 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.397$397 per 1,000 questionsper 1M questions Logic 94.0% | — | 99.5%99.1%–99.7% | abcdefgh | 2,152 / 2,163 | Not measured | Not measured | 16.7%answered 100% | 5,740 ms | 11,800 ms | $0.397$397 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.212$212 per 1,000 questionsper 1M questions Logic 95.1% | — | 99.4%99.0%–99.7% | abcdefgh | 2,151 / 2,163 | Not measured | Not measured | 50.0%answered 100% | 8,200 ms | 18,090 ms | $0.212$212 |
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $1.271$1,271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% | — | 99.4%99.0%–99.7% | abcdefgh | 2,151 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 46,630 ms | 96,460 ms | $1.271$1,271estimate · RTX PRO 6000 Blackwell |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.531$531estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% | — | 99.4%99.0%–99.6% | abcdefgh | 2,150 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 11,110 ms | 16,690 ms | $0.531$531estimate · RTX PRO 6000 Blackwell |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active Cost $0.295$295estimate · DGX Spark per 1,000 questionsper 1M questions Logic running | — | 99.3%98.9%–99.6% | abcdefghij | 2,148 / 2,163 | Not measured | Not measured | Not measured | 8,130 ms | 16,920 ms | $0.295$295estimate · DGX Spark |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $0.409$409estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% | — | 99.2%99.2%–99.3% min–max of 3 seeds | abcdefghi | 2,147 / 2,163seed 0 run | Not measured | Not measured | 53.6%answered 99% | 12,537 ms | 32,720 ms | $0.409$409estimate · RTX PRO 6000 Blackwell |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% | — | 99.2%98.7%–99.5% | abcdefghij | 2,146 / 2,163 | Not measured | Not measured | 77.8%answered 99% | 4,080 ms | 5,870 ms | $0.000$0.00 |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $0.781$781estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% | — | 99.2%98.7%–99.5% | abcdefghi | 2,146 / 2,163 | Not measured | Not measured | 61.5%answered 99% | 15,340 ms | 53,280 ms | $0.781$781estimate · RTX PRO 6000 Blackwell |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $0.530$530estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% | — | 99.2%98.7%–99.5% | abcdefghij | 2,146 / 2,163 | Not measured | Not measured | 13.0%answered 100% | 10,020 ms | 20,630 ms | $0.530$530estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $0.939$939estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% | — | 99.0%98.5%–99.4% | bcdefghij | 2,142 / 2,163 | Not measured | Not measured | 71.4%answered 99% | 27,950 ms | 60,210 ms | $0.939$939estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% | — | 98.9%98.8%–99.0% min–max of 3 seeds | abcdefghij | 2,142 / 2,163seed 0 run | Not measured | Not measured | Not measured | 3,990 ms | 10,603 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.132$132estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% | — | 98.9%98.8%–99.1% min–max of 3 seeds | cefghijk | 2,139 / 2,163seed 0 run | Not measured | Not measured | 67.6%answered 99% | 3,893 ms | 10,353 ms | $0.132$132estimate · RTX PRO 6000 Blackwell |
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% | — | 98.8%98.3%–99.2% | dfghijkl | 2,138 / 2,163 | Not measured | Not measured | 14.3%answered 100% | 19,100 ms | 28,120 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.166$166 per 1,000 questionsper 1M questions Logic 93.2% | — | 98.8%98.2%–99.2% | fghijkl | 2,137 / 2,163 | Not measured | Not measured | 17.4%answered 100% | 3,800 ms | 7,970 ms | $0.166$166 |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.060$59.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% | — | 98.7%98.1%–99.1% | ghijkl | 2,135 / 2,163 | Not measured | Not measured | 67.7%answered 99% | 19,570 ms | 53,410 ms | $0.060$59.94estimate · DGX Spark |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.318$318estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% | — | 98.5%98.4%–98.8% min–max of 3 seeds | efghijkl | 2,136 / 2,163seed 0 run | Not measured | Not measured | 45.5%answered 99% | 11,190 ms | 22,357 ms | $0.318$318estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.518$518estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% | — | 98.5%97.9%–99.0% | hijkl | 2,131 / 2,163 | Not measured | Not measured | Not measured | 20,640 ms | 34,280 ms | $0.518$518estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% | — | 98.5%97.9%–98.9% | hijklm | 2,130 / 2,163 | Not measured | Not measured | 36.0%answered 100% | 10,960 ms | 19,970 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.233$233estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% | — | 98.2%97.6%–98.7% | ijklm | 2,125 / 2,163 | Not measured | Not measured | 37.0%answered 99% | 108,720 ms | 175,470 ms | $0.233$233estimate · DGX Spark |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.103$103 per 1,000 questionsper 1M questions Logic 94.0% | — | 98.2%97.6%–98.7% | ijklmn | 2,125 / 2,163 | Not measured | Not measured | 40.6%answered 99% | 1,670 ms | 7,470 ms | $0.103$103 |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.219$219estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% | — | 98.2%98.1%–98.3% min–max of 3 seeds | ijklmn | 2,126 / 2,163seed 0 run | Not measured | Not measured | 31.0%answered 99% | 6,353 ms | 15,697 ms | $0.219$219estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.271$271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% | — | 98.1%97.4%–98.6% | jklmn | 2,122 / 2,163 | Not measured | Not measured | 44.0%answered 99% | 10,770 ms | 17,700 ms | $0.271$271estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.410$410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% | — | 97.8%97.1%–98.4% | klmno | 2,116 / 2,163 | Not measured | Not measured | Not measured | 14,870 ms | 26,040 ms | $0.410$410estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.196$196estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% | — | 97.7%97.6%–97.8% min–max of 3 seeds | lmnop | 2,111 / 2,163seed 0 run | Not measured | Not measured | 23.0%answered 99% | 7,137 ms | 14,787 ms | $0.196$196estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.670$1,670estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% | — | 97.0%96.2%–97.6% | mnop | 2,098 / 2,163 | Not measured | Not measured | Not measured | 34,320 ms† | 108,010 ms† | $1.670$1,670estimate · RTX PRO 6000 Blackwell† |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.138$138estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% | — | 96.9%96.1%–97.6% | mnop | 2,097 / 2,163 | Not measured | Not measured | 16.5%answered 99% | 64,440 ms | 106,260 ms | $0.138$138estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.208$208estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% | — | 96.8%95.9%–97.4% | nopq | 2,093 / 2,163 | Not measured | Not measured | 25.3%answered 99% | 8,120 ms | 13,650 ms | $0.208$208estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $0.482$482estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% | — | 96.3%95.4%–97.0% | opqr | 2,082 / 2,163 | Not measured | Not measured | 45.2%answered 97% | 14,530 ms† | 33,190 ms† | $0.482$482estimate · RTX PRO 6000 Blackwell† |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.355$355estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% | — | 95.8%94.9%–96.6% | pqrs | 2,072 / 2,163 | Not measured | Not measured | 39.3%answered 98% | 8,040 ms | 18,980 ms | $0.355$355estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $2.470$2,470estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% | — | 94.7%93.7%–95.6% | qrs | 2,049 / 2,163 | Not measured | Not measured | 20.3%answered 99% | 98,390 ms | 194,980 ms | $2.470$2,470estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.363$363estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% | — | 94.6%93.6%–95.5% | rs | 2,047 / 2,163 | Not measured | Not measured | 31.3%answered 98% | 11,200 ms | 32,140 ms | $0.363$363estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% | — | 94.0%92.9%–94.9% | rs | 2,033 / 2,163 | Not measured | Not measured | 26.4%answered 99% | 9,860 ms | 21,240 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% | — | 93.7%92.6%–94.6% | s | 2,026 / 2,163 | Not measured | Not measured | 68.9%answered 95% | 3,830 ms | 5,000 ms | $0.000$0.00 |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.266$266estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% | — | 90.8%89.5%–91.9% | t | 1,963 / 2,163 | Not measured | Not measured | 35.9%answered 96% | 7,890 ms | 14,890 ms | $0.266$266estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.901$1,901estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% | — | 89.6%88.2%–90.8% | t | 1,937 / 2,163 | Not measured | Not measured | 22.1%answered 98% | 25,550 ms† | 134,840 ms† | $1.901$1,901estimate · RTX PRO 6000 Blackwell† |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Other Google OpenBMB Alibaba Liquid AI IBM KRAFTON
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
LFM3-1.7B (off · chat): 98.7% at $0.060 per 1,000 questions ($59.94 per 1M) (estimate)
LFM3-1.7B (on · chat): 98.9% at $0.132 per 1,000 questions ($132 per 1M) (estimate)
Gemma 4 E2B (on · chat): 96.9% at $0.138 per 1,000 questions ($138 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 98.9% at $0.142 per 1,000 questions ($142 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 97.7% at $0.196 per 1,000 questions ($196 per 1M) (estimate)
Gemma 4 E2B (chat): 96.8% at $0.208 per 1,000 questions ($208 per 1M) (estimate)
MiniCPM5-2B (off · chat): 98.2% at $0.219 per 1,000 questions ($219 per 1M) (estimate)
Qwen3.5-4B (off · chat): 98.5% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 94.0% at $0.300 per 1,000 questions ($300 per 1M) (estimate)
Qwen3.5-2B (off · chat): 94.6% at $0.363 per 1,000 questions ($363 per 1M) (estimate)
MiniCPM5-2B (on · chat): 99.2% at $0.409 per 1,000 questions ($409 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 97.8% at $0.410 per 1,000 questions ($410 per 1M) (estimate)
Granite-4.2-3B (high · chat): 99.2% at $0.781 per 1,000 questions ($781 per 1M) (estimate)
Qwen3.5-4B (on · chat): 99.4% at $1.271 per 1,000 questions ($1,271 per 1M) (estimate)
Raon-1 nano (on · chat): 97.0% at $1.670 per 1,000 questions ($1,670 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (off · chat): 89.6% at $1.901 per 1,000 questions ($1,901 per 1M) (estimate) (other engine: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Qwen3.5-2B (on · chat): 94.7% at $2.470 per 1,000 questions ($2,470 per 1M) (estimate)
Other Google OpenBMB Alibaba Liquid AI IBM KRAFTON
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | $0.060$59.94estimate · DGX Spark | $59.94 estimate | $0.1296 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B Pareto on · chat Open · 1.7B | — | $0.132$132estimate · RTX PRO 6000 Blackwell | $132.39 estimate | $0.2808 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | $0.138$138estimate · DGX Spark | $138.05 estimate | $0.2986 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | $0.142$142estimate · RTX PRO 6000 Blackwell | $141.77 estimate | $0.3182 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | $0.196$196estimate · RTX PRO 6000 Blackwell | $195.81 estimate | $0.4219 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B chat Open · 5.1B | — | $0.208$208estimate · RTX PRO 6000 Blackwell | $208.17 estimate | $0.4503 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | $0.219$219estimate · RTX PRO 6000 Blackwell | $218.83 estimate | $0.48 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B off · chat Open · 4.7B | — | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | $0.300$300estimate · RTX PRO 6000 Blackwell | $299.95 estimate | $0.6488 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | $0.363$363estimate · RTX PRO 6000 Blackwell | $363.26 estimate | $0.7857 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B Pareto on · chat Open · 2.5B | — | $0.409$409estimate · RTX PRO 6000 Blackwell | $408.56 estimate | $0.8978 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | $0.410$410estimate · RTX PRO 6000 Blackwell | $409.63 estimate | $0.886 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | $0.781$781estimate · RTX PRO 6000 Blackwell | $780.88 estimate | $1.689 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto on · chat Open · 4.7B | — | $1.271$1,271estimate · RTX PRO 6000 Blackwell | $1,271.09 estimate | $2.749 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | $1.670$1,670estimate · RTX PRO 6000 Blackwell† | $1,670.17 estimate† | $3.613 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | $1.901$1,901estimate · RTX PRO 6000 Blackwell† | $1,900.72 estimate† | $4.111 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | $2.470$2,470estimate · RTX PRO 6000 Blackwell | $2,469.92 estimate | $5.342 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Granite-4.2-3B (off · chat) (not running now, 25 of 2,163 stored, last answer 16 Nov 20:02 UTC).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $1.271$1,271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% | — | 99.4%99.0%–99.7% | a | 2,151 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 46,630 ms | 96,460 ms | $1.271$1,271estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $0.409$409estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% | — | 99.2%99.2%–99.3% min–max of 3 seeds | ab | 2,147 / 2,163seed 0 run | Not measured | Not measured | 53.6%answered 99% | 12,537 ms | 32,720 ms | $0.409$409estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $0.781$781estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% | — | 99.2%98.7%–99.5% | ab | 2,146 / 2,163 | Not measured | Not measured | 61.5%answered 99% | 15,340 ms | 53,280 ms | $0.781$781estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% | — | 98.9%98.8%–99.0% min–max of 3 seeds | abc | 2,142 / 2,163seed 0 run | Not measured | Not measured | Not measured | 3,990 ms | 10,603 ms | $0.142$142estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.132$132estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% | — | 98.9%98.8%–99.1% min–max of 3 seeds | abc | 2,139 / 2,163seed 0 run | Not measured | Not measured | 67.6%answered 99% | 3,893 ms | 10,353 ms | $0.132$132estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.060$59.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% | — | 98.7%98.1%–99.1% | bcd | 2,135 / 2,163 | Not measured | Not measured | 67.7%answered 99% | 19,570 ms | 53,410 ms | $0.060$59.94estimate · DGX Spark |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% | — | 98.5%97.9%–98.9% | bcde | 2,130 / 2,163 | Not measured | Not measured | 36.0%answered 100% | 10,960 ms | 19,970 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.219$219estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% | — | 98.2%98.1%–98.3% min–max of 3 seeds | cdef | 2,126 / 2,163seed 0 run | Not measured | Not measured | 31.0%answered 99% | 6,353 ms | 15,697 ms | $0.219$219estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.410$410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% | — | 97.8%97.1%–98.4% | defg | 2,116 / 2,163 | Not measured | Not measured | Not measured | 14,870 ms | 26,040 ms | $0.410$410estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.196$196estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% | — | 97.7%97.6%–97.8% min–max of 3 seeds | efg | 2,111 / 2,163seed 0 run | Not measured | Not measured | 23.0%answered 99% | 7,137 ms | 14,787 ms | $0.196$196estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.670$1,670estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% | — | 97.0%96.2%–97.6% | fg | 2,098 / 2,163 | Not measured | Not measured | Not measured | 34,320 ms† | 108,010 ms† | $1.670$1,670estimate · RTX PRO 6000 Blackwell† |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.138$138estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% | — | 96.9%96.1%–97.6% | g | 2,097 / 2,163 | Not measured | Not measured | 16.5%answered 99% | 64,440 ms | 106,260 ms | $0.138$138estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.208$208estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% | — | 96.8%95.9%–97.4% | g | 2,093 / 2,163 | Not measured | Not measured | 25.3%answered 99% | 8,120 ms | 13,650 ms | $0.208$208estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $2.470$2,470estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% | — | 94.7%93.7%–95.6% | h | 2,049 / 2,163 | Not measured | Not measured | 20.3%answered 99% | 98,390 ms | 194,980 ms | $2.470$2,470estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.363$363estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% | — | 94.6%93.6%–95.5% | h | 2,047 / 2,163 | Not measured | Not measured | 31.3%answered 98% | 11,200 ms | 32,140 ms | $0.363$363estimate · RTX PRO 6000 Blackwell |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% | — | 94.0%92.9%–94.9% | h | 2,033 / 2,163 | Not measured | Not measured | 26.4%answered 99% | 9,860 ms | 21,240 ms | $0.300$300estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $1.901$1,901estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% | — | 89.6%88.2%–90.8% | i | 1,937 / 2,163 | Not measured | Not measured | 22.1%answered 98% | 25,550 ms† | 134,840 ms† | $1.901$1,901estimate · RTX PRO 6000 Blackwell† |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Google eCloud Technology Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Gemma 4 E4B (on · chat): 98.2% at $0.233 per 1,000 questions ($233 per 1M) (estimate)
Erk-14B (off · chat): 90.8% at $0.266 per 1,000 questions ($266 per 1M) (estimate)
Gemma 4 E4B (chat): 98.1% at $0.271 per 1,000 questions ($271 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 98.5% at $0.318 per 1,000 questions ($318 per 1M) (estimate)
Qwen3-14B (off · chat): 95.8% at $0.355 per 1,000 questions ($355 per 1M) (estimate)
Gemma 4 12B Unified (chat): 98.8% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 96.3% at $0.482 per 1,000 questions ($482 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Gemma 4 E4B (on · chat · 32k cap): 98.5% at $0.518 per 1,000 questions ($518 per 1M) (estimate)
Google eCloud Technology Alibaba Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Gemma 4 E4B Pareto on · chat Open · 8.0B | — | $0.233$233estimate · DGX Spark | $232.86 estimate | $0.5037 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | $0.266$266estimate · RTX PRO 6000 Blackwell | $266.05 estimate | $0.5755 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B chat Open · 8.0B | — | $0.271$271estimate · RTX PRO 6000 Blackwell | $271.17 estimate | $0.5865 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat · 32k cap · sampled Open · 8.0B | — | $0.318$318estimate · RTX PRO 6000 Blackwell | $317.64 estimate | $0.7034 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | $0.355$355estimate · RTX PRO 6000 Blackwell | $354.95 estimate | $0.7678 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified Pareto chat Open · 12B | — | $0.467$467estimate · RTX PRO 6000 Blackwell | $467.19 estimate | $1.011 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | $0.482$482estimate · RTX PRO 6000 Blackwell† | $482.22 estimate† | $1.043 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | $0.518$518estimate · RTX PRO 6000 Blackwell | $518.03 estimate | $1.121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% | — | 98.8%98.3%–99.2% | a | 2,138 / 2,163 | Not measured | Not measured | 14.3%answered 100% | 19,100 ms | 28,120 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.318$318estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% | — | 98.5%98.4%–98.8% min–max of 3 seeds | a | 2,136 / 2,163seed 0 run | Not measured | Not measured | 45.5%answered 99% | 11,190 ms | 22,357 ms | $0.318$318estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.518$518estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% | — | 98.5%97.9%–99.0% | a | 2,131 / 2,163 | Not measured | Not measured | Not measured | 20,640 ms | 34,280 ms | $0.518$518estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.233$233estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% | — | 98.2%97.6%–98.7% | a | 2,125 / 2,163 | Not measured | Not measured | 37.0%answered 99% | 108,720 ms | 175,470 ms | $0.233$233estimate · DGX Spark |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.271$271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% | — | 98.1%97.4%–98.6% | a | 2,122 / 2,163 | Not measured | Not measured | 44.0%answered 99% | 10,770 ms | 17,700 ms | $0.271$271estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $0.482$482estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% | — | 96.3%95.4%–97.0% | b | 2,082 / 2,163 | Not measured | Not measured | 45.2%answered 97% | 14,530 ms† | 33,190 ms† | $0.482$482estimate · RTX PRO 6000 Blackwell† |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.355$355estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% | — | 95.8%94.9%–96.6% | b | 2,072 / 2,163 | Not measured | Not measured | 39.3%answered 98% | 8,040 ms | 18,980 ms | $0.355$355estimate · RTX PRO 6000 Blackwell |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.266$266estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% | — | 90.8%89.5%–91.9% | c | 1,963 / 2,163 | Not measured | Not measured | 35.9%answered 96% | 7,890 ms | 14,890 ms | $0.266$266estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Alibaba Google Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Qwen3.8-27B (low · chat): 99.2% at $0.530 per 1,000 questions ($530 per 1M) (estimate)
Gemma 4 31B (chat): 99.4% at $0.531 per 1,000 questions ($531 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 99.0% at $0.939 per 1,000 questions ($939 per 1M) (estimate)
Alibaba Google Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Qwen3.8-27B Pareto low · chat Open · 27B | — | $0.530$530estimate · RTX PRO 6000 Blackwell | $529.86 estimate | $1.146 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B Pareto chat Open · 31B | — | $0.531$531estimate · RTX PRO 6000 Blackwell | $530.82 estimate | $1.148 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | $0.939$939estimate · RTX PRO 6000 Blackwell | $939.17 estimate | $2.031 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.531$531estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% | — | 99.4%99.0%–99.6% | a | 2,150 / 2,163 | Not measured | Not measured | 21.4%answered 100% | 11,110 ms | 16,690 ms | $0.531$531estimate · RTX PRO 6000 Blackwell |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $0.530$530estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% | — | 99.2%98.7%–99.5% | a | 2,146 / 2,163 | Not measured | Not measured | 13.0%answered 100% | 10,020 ms | 20,630 ms | $0.530$530estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $0.939$939estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% | — | 99.0%98.5%–99.4% | a | 2,142 / 2,163 | Not measured | Not measured | 71.4%answered 99% | 27,950 ms | 60,210 ms | $0.939$939estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
MiniMax Z.ai Xiaomi DeepSeek Thinking Machines
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
MiniMax-M3.1-Flash (high · chat): 98.2% at $0.103 per 1,000 questions ($103 per 1M)
GLM-5.3-Flash (high · chat): 98.8% at $0.166 per 1,000 questions ($166 per 1M)
MiMo-V2.6-Flash (on · chat): 99.4% at $0.212 per 1,000 questions ($212 per 1M)
DeepSeek-V4.1-Flash (high · chat · 1017): 99.6% at $0.227 per 1,000 questions ($227 per 1M)
DeepSeek-V4-Flash (chat): 99.3% at $0.295 per 1,000 questions ($295 per 1M) (estimate)
Inkling 2 Small (high · chat): 99.5% at $0.638 per 1,000 questions ($638 per 1M)
Inkling-Small (high · chat · Jul 2026): 99.6% at $0.954 per 1,000 questions ($954 per 1M)
MiniMax Z.ai Xiaomi DeepSeek Thinking Machines
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | $0.103$103 | $102.85 | $0.2225 | — | provider price × reported usage |
| GLM-5.3-Flash Pareto high · chat Open · 320B MoE · 18B active | — | $0.166$166 | $166.50 | $0.3601 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | $0.212$212 | $211.85 | $0.4582 | — | provider price × reported usage |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | $0.227$227 | $227.42 | $0.4919 | — | provider price × reported usage |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active | — | $0.295$295estimate · DGX Spark | $294.82 estimate | $0.6377 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | $0.638$638 | $638.48 | $1.381 | — | provider price × reported usage |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | $0.954$954 | $953.81 | $2.063 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Qwen3.5-397B-A17B (on · chat) (Logic only; on this test only its 25-question smoke test; cost so far $5.552$5,552 per 1,000 questions, over the 25 of 2,163 answered).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.227$227 per 1,000 questionsper 1M questions Logic 96.2% | — | 99.6%99.3%–99.8% | a | 2,155 / 2,163 | Not measured | Not measured | 20.0%answered 100% | 1,910 ms | 3,570 ms | $0.227$227 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $0.954$954 per 1,000 questionsper 1M questions Logic 95.4% | — | 99.6%99.3%–99.8% | a | 2,155 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 2,030 ms | 6,330 ms | $0.954$954 |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $0.638$638 per 1,000 questionsper 1M questions Logic 95.6% | — | 99.5%99.2%–99.7% | a | 2,153 / 2,163 | Not measured | Not measured | 61.5%answered 100% | 2,110 ms | 6,000 ms | $0.638$638 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.212$212 per 1,000 questionsper 1M questions Logic 95.1% | — | 99.4%99.0%–99.7% | ab | 2,151 / 2,163 | Not measured | Not measured | 50.0%answered 100% | 8,200 ms | 18,090 ms | $0.212$212 |
| DeepSeek-V4-Flash chat Open · 285B MoE · 13B active DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active Cost $0.295$295estimate · DGX Spark per 1,000 questionsper 1M questions Logic running | — | 99.3%98.9%–99.6% | ab | 2,148 / 2,163 | Not measured | Not measured | Not measured | 8,130 ms | 16,920 ms | $0.295$295estimate · DGX Spark |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.166$166 per 1,000 questionsper 1M questions Logic 93.2% | — | 98.8%98.2%–99.2% | bc | 2,137 / 2,163 | Not measured | Not measured | 17.4%answered 100% | 3,800 ms | 7,970 ms | $0.166$166 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.103$103 per 1,000 questionsper 1M questions Logic 94.0% | — | 98.2%97.6%–98.7% | c | 2,125 / 2,163 | Not measured | Not measured | 40.6%answered 99% | 1,670 ms | 7,470 ms | $0.103$103 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Meta
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Muse Ember (max · chat): 99.6% at $0.365 per 1,000 questions ($365 per 1M)
Meta
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Muse Ember Pareto max · chat Closed · Efficient | — | $0.365$365 | $364.58 | $0.7886 | — | provider price × reported usage |
| GPT-6 Luna chat Closed · Efficient | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): GPT-6 Luna (max · chat) (paused for now at 200 of 2,163 (zeki #4129, 2026-11-10T14:13Z); cost so far $0.182$182 per 1,000 questions, over the 200 of 2,163 answered).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.365$365 per 1,000 questionsper 1M questions Logic 95.5% | — | 99.6%99.2%–99.8% | a | 2,154 / 2,163 | Not measured | Not measured | 64.3%answered 100% | 1,260 ms | 2,270 ms | $0.365$365 |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% | — | 93.7%92.6%–94.6% | b | 2,026 / 2,163 | Not measured | Not measured | 68.9%answered 95% | 3,830 ms | 5,000 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Sonnet 5.5 (chat): 99.7% at $4.734 per 1,000 questions ($4,734 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 Pareto chat Closed · Workhorse | — | $4.734$4,734 API list-price equivalent | $4,734.30 | $10.24 | — | Claude subscription |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $4.734$4,734 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% | — | 99.7%99.3%–99.8% | a | 2,156 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,460 ms | 5,970 ms | $4.734$4,734 API list-price equivalent |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Opus 5.5 (chat): 99.8% at $9.871 per 1,000 questions ($9,871 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 Pareto chat Closed · Flagship | — | $9.871$9,871 API list-price equivalent | $9,870.94 | $21.35 | — | Claude subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Claude Fable 5.2 (chat) (not running now, 5 of 2,163 stored, last answer 11 Nov 12:02 UTC).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $9.871$9,871 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% | — | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 62.5%answered 100% | 3,200 ms | 4,790 ms | $9.871$9,871 API list-price equivalent |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% | — | 99.7%99.4%–99.9% | a | 2,157 / 2,163 | Not measured | Not measured | 60.0%answered 100% | 5,030 ms | 6,490 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Alibaba xAI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Qwen3.8 Flash (off · chat): 99.5% at $0.397 per 1,000 questions ($397 per 1M)
Qwen4-Max (high · chat): 99.8% at $1.478 per 1,000 questions ($1,478 per 1M)
Grok 4.7 (chat): 99.5% at $12.061 per 1,000 questions ($12,061 per 1M)
Alibaba xAI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Qwen3.8 Flash Pareto off · chat Closed · Undisclosed | — | $0.397$397 | $396.92 | $0.8585 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | $1.478$1,478 | $1,478.15 | $3.197 | — | provider price × reported usage |
| Grok 4.7 chat Closed · Undisclosed | — | $12.061$12,061 API list-price equivalent | $12,061.26 | $26.09 | — | xAI subscription |
| GPT-6 Astra chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not ranked (only complete runs are): Gemini 4 Carbon (high · chat) (paused for now at 54 of 2,163 (zeki #4681, 2026-11-11T11:53Z)).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 2,163) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $1.478$1,478 per 1,000 questionsper 1M questions Logic 96.3% | — | 99.8%99.5%–99.9% | a | 2,158 / 2,163 | Not measured | Not measured | 54.5%answered 100% | 4,710 ms | 9,130 ms | $1.478$1,478 |
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% | — | 99.6%99.3%–99.8% | ab | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 5,280 ms | 7,260 ms | $0.000$0.00 |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% | — | 99.6%99.3%–99.8% | ab | 2,155 / 2,163 | Not measured | Not measured | 78.6%answered 99% | 4,810 ms | 6,560 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% | — | 99.6%99.2%–99.8% | ab | 2,154 / 2,163 | Not measured | Not measured | 75.0%answered 99% | 13,410 ms | 27,800 ms | — |
| Grok 4.7 chat Closed · Undisclosed Grok 4.7 chat · xAI · chat model Parameters: not disclosed by maker (closed weights) Cost $12.061$12,061 API list-price equivalent per 1,000 questionsper 1M questions Logic running | — | 99.5%99.2%–99.7% | ab | 2,153 / 2,163 | Not measured | Not measured | 97.5%answered 98% | 11,100 ms | 66,530 ms | $12.061$12,061 API list-price equivalent |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.397$397 per 1,000 questionsper 1M questions Logic 94.0% | — | 99.5%99.1%–99.7% | ab | 2,152 / 2,163 | Not measured | Not measured | 16.7%answered 100% | 5,740 ms | 11,800 ms | $0.397$397 |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% | — | 99.2%98.7%–99.5% | b | 2,146 / 2,163 | Not measured | Not measured | 77.8%answered 99% | 4,080 ms | 5,870 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.