Logic and Math · Cost
Cost on Logic and Math
Logic and Math pooled: 3,288 questions; only setups that finished both. How to read this page · Method
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other OmniJev Celeris StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Apertix AI Nace.AI Cloudflare Turion.AI TypeSafe scar-ai quincyqiang Strands Agents Google newsbang Blockbrain Labs tattrongvu Patrick-18 victorbennett Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab Poindexter Labs MachinePulse REAX Logits Zefan Cai eCloud Technology MiniMax Z.ai OpenBMB Xiaomi Alibaba DeepSeek Meta Thinking Machines IBM KRAFTON Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 21.5% at $0.001 per 1,000 questions ($0.83 per 1M) (estimate)
GLiClass v4 Edge: 23.0% at $0.001 per 1,000 questions ($1.01 per 1M) (estimate)
OneJev1.6-0.8B: 27.8% at $0.001 per 1,000 questions ($1.11 per 1M) (estimate)
celeris-1.1-decision: 62.1% at $0.001 per 1,000 questions ($1.13 per 1M)
GLiDE 0.8B: 31.1% at $0.001 per 1,000 questions ($1.38 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 28.0% at $0.002 per 1,000 questions ($1.62 per 1M) (estimate)
KnowLine-0.8B-Gen1: 27.7% at $0.002 per 1,000 questions ($1.80 per 1M) (estimate)
GLiDE 2B: 35.2% at $0.002 per 1,000 questions ($1.94 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 35.9% at $0.002 per 1,000 questions ($2.36 per 1M) (estimate)
Perplexity Decider Flash v1.2: 51.1% at $0.003 per 1,000 questions ($2.72 per 1M)
KnowLine-2B-Gen1: 34.1% at $0.003 per 1,000 questions ($3.40 per 1M) (estimate)
d1.2: 59.9% at $0.004 per 1,000 questions ($3.98 per 1M)
d1.1: 58.6% at $0.004 per 1,000 questions ($3.98 per 1M)
d1: 50.5% at $0.004 per 1,000 questions ($3.98 per 1M)
StartLux-Decision-v1.1-4B-Instant: 52.4% at $0.004 per 1,000 questions ($4.28 per 1M) (estimate)
KnowLine 4B Gen5.5: 51.3% at $0.005 per 1,000 questions ($5.02 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API): 64.1% at $0.005 per 1,000 questions ($5.38 per 1M)
Mercury Decide: 50.6% at $0.005 per 1,000 questions ($5.46 per 1M)
d1-ultra: 67.5% at $0.008 per 1,000 questions ($8.37 per 1M)
d1.2-ultra: 63.6% at $0.008 per 1,000 questions ($8.37 per 1M)
StartLux-Decision-v1.1-9B-Instant: 54.4% at $0.009 per 1,000 questions ($8.92 per 1M) (estimate)
Microsoft-Decision-1: 54.3% at $0.009 per 1,000 questions ($9.07 per 1M)
GLiDE (v1.2): 64.8% at $0.010 per 1,000 questions ($9.71 per 1M)
Elephant-1: 56.2% at $0.010 per 1,000 questions ($9.79 per 1M)
SubQ Snap 1.1: 63.0% at $0.011 per 1,000 questions ($10.75 per 1M)
Cerno 9B v1.2: 51.7% at $0.011 per 1,000 questions ($10.82 per 1M) (estimate)
celeris-1-decision: 62.2% at $0.011 per 1,000 questions ($11.25 per 1M)
GLiDE 9B: 54.5% at $0.011 per 1,000 questions ($11.45 per 1M) (estimate)
Drex 2.0: 53.1% at $0.012 per 1,000 questions ($12.17 per 1M)
Clef-Flash: 46.1% at $0.013 per 1,000 questions ($12.90 per 1M)
d1-2B: 34.9% at $0.014 per 1,000 questions ($14.30 per 1M) (estimate)
Drex Max v1.1: 64.8% at $0.016 per 1,000 questions ($16.13 per 1M)
Drex Max: 64.4% at $0.016 per 1,000 questions ($16.31 per 1M)
Turion Jev 31B: 63.3% at $0.020 per 1,000 questions ($20.01 per 1M) (estimate)
Jev 1.14: 59.3% at $0.021 per 1,000 questions ($21.27 per 1M)
Jev 1.13: 58.0% at $0.021 per 1,000 questions ($21.27 per 1M)
Perplexity Decider Flash v1.3: 52.1% at $0.024 per 1,000 questions ($23.77 per 1M) (estimate)
vera-medium-omni: 52.5% at $0.024 per 1,000 questions ($24.19 per 1M) (estimate)
Chinese Decision Pointer 4B: 47.0% at $0.025 per 1,000 questions ($25.03 per 1M) (estimate)
Strands Decider 2B: 26.5% at $0.027 per 1,000 questions ($26.72 per 1M) (estimate)
Gemini Decision 1 (20261111): 64.1% at $0.027 per 1,000 questions ($27.07 per 1M)
Mammoth-1: 67.3% at $0.028 per 1,000 questions ($27.98 per 1M)
Adjudex 9B v5: 56.7% at $0.028 per 1,000 questions ($28.19 per 1M) (estimate)
GLiDE 27B: 67.3% at $0.029 per 1,000 questions ($28.55 per 1M) (estimate)
Cygnet: 42.6% at $0.029 per 1,000 questions ($29.34 per 1M)
KnowLine-9B-Gen1: 54.5% at $0.030 per 1,000 questions ($29.87 per 1M) (estimate)
Jeffan-9B-1211: 55.8% at $0.032 per 1,000 questions ($32.40 per 1M) (estimate)
Dexy-Decision-9B-V2: 54.2% at $0.033 per 1,000 questions ($32.61 per 1M) (estimate)
Verdin-4B-v1.1: 47.5% at $0.033 per 1,000 questions ($33.45 per 1M) (estimate)
KnowLine-9B-Gen2: 54.6% at $0.034 per 1,000 questions ($34.08 per 1M) (estimate)
GLiDE (API): 65.2% at $0.036 per 1,000 questions ($36.42 per 1M)
KnowLine-12B-Gen1: 60.9% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Valen-Decision-1.5-9B: 51.6% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Winnow-12B: 44.4% at $0.043 per 1,000 questions ($42.91 per 1M) (estimate)
d3: 65.0% at $0.046 per 1,000 questions ($46.28 per 1M) (estimate)
Sieve 31B v3: 70.5% at $0.047 per 1,000 questions ($47.29 per 1M) (estimate)
KnowLine-31B-Gen1.5: 71.1% at $0.052 per 1,000 questions ($51.85 per 1M) (estimate)
OneJev-27B: 49.3% at $0.069 per 1,000 questions ($69.42 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 66.4% at $0.071 per 1,000 questions ($71.10 per 1M) (estimate)
Clef: 47.6% at $0.081 per 1,000 questions ($81.46 per 1M)
Metask-Jev-4B: 36.7% at $0.086 per 1,000 questions ($85.83 per 1M) (estimate)
KnowLine-31B-Gen2: 73.0% at $0.093 per 1,000 questions ($92.98 per 1M) (estimate)
d3-ultra: 65.7% at $0.096 per 1,000 questions ($95.71 per 1M) (estimate)
Copernicus-31B: 71.0% at $0.102 per 1,000 questions ($102 per 1M) (estimate)
Karpo Decision 31B: 74.4% at $0.118 per 1,000 questions ($118 per 1M) (estimate)
REAX 1.3: 71.6% at $0.148 per 1,000 questions ($148 per 1M) (estimate)
Logits Sigma-1 Duo: 69.3% at $0.165 per 1,000 questions ($165 per 1M) (estimate)
LFM3-1.7B (off · chat): 94.2% at $0.206 per 1,000 questions ($206 per 1M) (estimate)
Open-Jev-27B-v1.1: 50.5% at $0.212 per 1,000 questions ($212 per 1M) (estimate)
Gemma 4 E2B (on · chat): 89.3% at $0.215 per 1,000 questions ($215 per 1M) (estimate)
Erk-14B (off · chat): 78.3% at $0.308 per 1,000 questions ($308 per 1M) (estimate)
Gemma 4 E2B (chat): 88.6% at $0.313 per 1,000 questions ($313 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 90.5% at $0.325 per 1,000 questions ($325 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 96.8% at $0.342 per 1,000 questions ($342 per 1M)
Gemma 4 E4B (on · chat): 92.9% at $0.375 per 1,000 questions ($375 per 1M) (estimate)
LFM3-1.7B (on · chat): 93.9% at $0.435 per 1,000 questions ($435 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 94.0% at $0.449 per 1,000 questions ($449 per 1M) (estimate)
Gemma 4 E4B (chat): 93.0% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
GLM-5.3-Flash (high · chat): 96.9% at $0.471 per 1,000 questions ($471 per 1M)
Gemma 4 E4B (on · chat · 32k cap · sampled): 94.2% at $0.549 per 1,000 questions ($549 per 1M) (estimate)
MiniCPM5-2B (off · chat): 93.2% at $0.598 per 1,000 questions ($598 per 1M) (estimate)
MiMo-V2.6-Flash (on · chat): 98.0% at $0.615 per 1,000 questions ($615 per 1M)
Gemma 4 E2B (on · chat · 32k cap): 90.3% at $0.678 per 1,000 questions ($678 per 1M) (estimate)
Qwen3.5-4B (off · chat): 95.6% at $0.683 per 1,000 questions ($683 per 1M) (estimate)
DeepSeek-V4.1-Flash (high · chat · 1017): 98.4% at $0.713 per 1,000 questions ($713 per 1M)
Qwen3.8 Flash (off · chat): 97.6% at $0.732 per 1,000 questions ($732 per 1M)
Gemma 4 12B Unified (chat): 95.0% at $0.749 per 1,000 questions ($749 per 1M) (estimate)
Gemma 4 31B (chat): 97.3% at $0.775 per 1,000 questions ($775 per 1M) (estimate)
Qwen3.5-2B (off · chat): 84.4% at $0.788 per 1,000 questions ($788 per 1M) (estimate)
Qwen3-14B (off · chat): 85.4% at $0.840 per 1,000 questions ($840 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 93.9% at $0.855 per 1,000 questions ($855 per 1M) (estimate)
Muse Ember (max · chat): 98.2% at $0.863 per 1,000 questions ($863 per 1M)
Perplexity Decider 27B v1.2 (API · with web): 63.6% at $1.025 per 1,000 questions ($1,025 per 1M)
LFM2.5-2.6B (on · chat): 87.6% at $1.046 per 1,000 questions ($1,046 per 1M) (estimate)
MiniCPM5-2B (on · chat): 95.7% at $1.410 per 1,000 questions ($1,410 per 1M) (estimate)
Inkling 2 Small (high · chat): 98.2% at $1.830 per 1,000 questions ($1,830 per 1M)
LFM2.5-8B-A1B (on · chat): 83.9% at $1.913 per 1,000 questions ($1,913 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Qwen3.8-27B (low · chat): 97.4% at $2.020 per 1,000 questions ($2,020 per 1M) (estimate)
Qwen3.5-4B (on · chat): 97.0% at $2.042 per 1,000 questions ($2,042 per 1M) (estimate)
Inkling-Small (high · chat · Jul 2026): 98.2% at $2.506 per 1,000 questions ($2,506 per 1M)
Qwen4-Max (high · chat): 98.6% at $2.919 per 1,000 questions ($2,919 per 1M)
Qwen3.5-2B (on · chat): 86.4% at $3.106 per 1,000 questions ($3,106 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 94.8% at $3.205 per 1,000 questions ($3,205 per 1M) (estimate)
Granite-4.2-3B (high · chat): 96.6% at $4.241 per 1,000 questions ($4,241 per 1M) (estimate)
Raon-1 nano (off · chat): 81.2% at $4.422 per 1,000 questions ($4,422 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (on · chat): 88.1% at $6.344 per 1,000 questions ($6,344 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Claude Sonnet 5.5 (chat): 98.2% at $7.042 per 1,000 questions ($7,042 per 1M)
Claude Opus 5.5 (chat): 98.5% at $13.179 per 1,000 questions ($13,179 per 1M)
Fastino Other OmniJev Celeris StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Apertix AI Nace.AI Cloudflare Turion.AI TypeSafe scar-ai quincyqiang Strands Agents Google newsbang Blockbrain Labs tattrongvu Patrick-18 victorbennett Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab Poindexter Labs MachinePulse REAX Logits Zefan Cai eCloud Technology MiniMax Z.ai OpenBMB Xiaomi Alibaba DeepSeek Meta Thinking Machines IBM KRAFTON Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | Decision | $0.001$0.83estimate · DGX Spark | $0.83 estimate | $0.002719 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | Decision | $0.001$1.01estimate · DGX Spark | $1.01 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | Decision | $0.001$1.11estimate · DGX Spark | $1.11 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1.1-decision Pareto Closed · Efficient | — | Decision | $0.001$1.13 | $1.13 | $0.003701 | — | provider price × reported usage |
| GLiDE 0.8B Open · 0.8B | — | Decision | $0.001$1.38estimate · DGX Spark | $1.38 estimate | $0.004552 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | Decision | $0.002$1.62estimate · DGX Spark | $1.62 estimate | $0.00532 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | Decision | $0.002$1.80estimate · DGX Spark | $1.80 estimate | $0.005912 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Open · 1.9B | — | Decision | $0.002$1.94estimate · DGX Spark | $1.94 estimate | $0.006385 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | Decision | $0.002$2.36estimate · DGX Spark | $2.36 estimate | $0.007744 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Open · 4.0B | — | Decision | $0.003$2.72 list | $2.72 | $0.008936 | $0.032$31.55estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | Decision | $0.003$3.40estimate · DGX Spark | $3.40 estimate | $0.01117 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1.2 Closed · Undisclosed | — | Decision | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | Decision | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1 Closed · Efficient | — | Decision | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | — | Decision | $0.004$4.28estimate · DGX Spark | $4.28 estimate | $0.01407 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | Decision | $0.005$5.02estimate · DGX Spark | $5.02 estimate | $0.01649 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | Decision | $0.005$5.38 | $5.38 | $0.01768 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | Decision | $0.005$5.46 | $5.46 | $0.01794 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | Decision | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
| d1.2-ultra Closed · Undisclosed | — | Decision | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | Decision | $0.009$8.92estimate · DGX Spark | $8.92 estimate | $0.02932 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Microsoft-Decision-1 Closed · Efficient | — | Decision | $0.009$9.07 | $9.07 | $0.02983 | — | provider price × reported usage |
| GLiDE v1.2 Open · 27B | — | Decision | $0.010$9.71 list | $9.71 | $0.03193 | $0.066$66.05estimate · RTX PRO 6000 Blackwell | maker's list price |
| Elephant-1 Open · 9.0B | — | Decision | $0.010$9.79 list | $9.79 | $0.0322 | $0.058$58.27estimate · RTX PRO 6000 Blackwell | maker's list price |
| SubQ Snap 1.1 Closed · Efficient | — | Decision | $0.011$10.75 | $10.75 | $0.03535 | — | provider price × reported usage |
| Cerno 9B v1.2 Open · 9.0B | — | Decision | $0.011$10.82estimate · DGX Spark | $10.82 estimate | $0.03559 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1-decision Closed · Efficient | — | Decision | $0.011$11.25 | $11.25 | $0.03701 | — | provider price × reported usage |
| GLiDE 9B Open · 7.9B | — | Decision | $0.011$11.45estimate · DGX Spark | $11.45 estimate | $0.03766 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | Decision | $0.012$12.17 | $12.17 | $0.04001 | — | provider price × reported usage |
| Clef-Flash Open · 9.0B | — | Decision | $0.013$12.90 list | $12.90 | $0.04241 | $0.075$75.10estimate · RTX PRO 6000 Blackwell | maker's list price |
| d1-2B Open · 1.9B | — | Decision | $0.014$14.30estimate · RTX PRO 6000 Blackwell | $14.30 estimate | $0.04703 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Drex Max v1.1 Closed · Efficient (provisional) | — | Decision | $0.016$16.13 | $16.13 | $0.05303 | — | provider price × reported usage |
| Drex Max Open · 27B | — | Decision | $0.016$16.31 list | $16.31 | $0.05362 | $0.064$63.74estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | Decision | $0.020$20.01estimate · DGX Spark | $20.01 estimate | $0.0658 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Jev 1.14 Closed · Efficient | — | Decision | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | Decision | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | Decision | $0.024$23.77estimate · RTX PRO 6000 Blackwell | $23.77 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Open · 4.2B | — | Decision | $0.024$24.19estimate · RTX PRO 6000 Blackwell | $24.19 estimate | $0.07954 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | Decision | $0.025$25.03estimate · RTX PRO 6000 Blackwell | $25.03 estimate | $0.08231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | Decision | $0.027$26.72estimate · RTX PRO 6000 Blackwell | $26.72 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemini Decision 1 20261111 Closed · Efficient | — | Decision | $0.027$27.07 | $27.07 | $0.08901 | — | provider price × reported usage |
| Mammoth-1 Open · 27B | — | Decision | $0.028$27.98 list | $27.98 | $0.09199 | $0.120$120estimate · RTX PRO 6000 Blackwell | maker's list price |
| Adjudex 9B v5 Open · 9.0B | — | Decision | $0.028$28.19estimate · RTX PRO 6000 Blackwell | $28.19 estimate | $0.09268 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE 27B Open · 27B | — | Decision | $0.029$28.55estimate · DGX Spark | $28.55 estimate | $0.09388 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cygnet Open · 12B | — | Decision | $0.029$29.34 list | $29.34 | $0.09647 | $0.029$28.61estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen1 Open · 9.0B | — | Decision | $0.030$29.87estimate · RTX PRO 6000 Blackwell | $29.87 estimate | $0.09822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Open · 9.0B | — | Decision | $0.032$32.40estimate · RTX PRO 6000 Blackwell | $32.40 estimate | $0.1065 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | Decision | $0.033$32.61estimate · RTX PRO 6000 Blackwell | $32.61 estimate | $0.1072 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | Decision | $0.033$33.45estimate · RTX PRO 6000 Blackwell | $33.45 estimate | $0.11 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | Decision | $0.034$34.08estimate · RTX PRO 6000 Blackwell | $34.08 estimate | $0.1121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE API Closed · Efficient | — | Decision | $0.036$36.42 | $36.42 | $0.1197 | — | provider price × reported usage |
| KnowLine-12B-Gen1 Open · 12B | — | Decision | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | Decision | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | Decision | $0.043$42.91estimate · RTX PRO 6000 Blackwell | $42.91 estimate | $0.1411 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3 Open · 26B | — | Decision | $0.046$46.28estimate · RTX PRO 6000 Blackwell | $46.28 estimate | $0.1522 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Sieve 31B v3 Pareto Open · 31B | — | Decision | $0.047$47.29estimate · DGX Spark | $47.29 estimate | $0.1555 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | Decision | $0.052$51.85estimate · DGX Spark | $51.85 estimate | $0.1705 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | Decision | $0.069$69.42estimate · RTX PRO 6000 Blackwell | $69.42 estimate | $0.2283 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | Decision | $0.071$71.10estimate · RTX PRO 6000 Blackwell | $71.10 estimate | $0.2338 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | Decision | $0.081$81.46 list | $81.46 | $0.2679 | $0.135$135estimate · RTX PRO 6000 Blackwell | maker's list price |
| Metask-Jev-4B Open · 4.5B | — | Decision | $0.086$85.83estimate · RTX PRO 6000 Blackwell | $85.83 estimate | $0.2822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen2 Pareto Open · 31B | — | Decision | $0.093$92.98estimate · RTX PRO 6000 Blackwell | $92.98 estimate | $0.3057 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | Decision | $0.096$95.71estimate · RTX PRO 6000 Blackwell | $95.71 estimate | $0.3147 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | Decision | $0.102$102estimate · RTX PRO 6000 Blackwell | $102.03 estimate | $0.3355 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | Decision | $0.118$118estimate · RTX PRO 6000 Blackwell | $118.01 estimate | $0.388 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Open · 57B | — | Decision | $0.148$148estimate · RTX PRO 6000 Blackwell | $148.30 estimate | $0.4876 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | Decision | $0.165$165estimate · RTX PRO 6000 Blackwell | $164.92 estimate | $0.5423 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | Chat | $0.206$206estimate · DGX Spark | $205.97 estimate | $0.6772 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Open-Jev-27B-v1.1 Open · 27B | — | Decision | $0.212$212estimate · RTX PRO 6000 Blackwell | $212.04 estimate | $0.6972 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | Chat | $0.215$215estimate · DGX Spark | $215.10 estimate | $0.7073 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | Chat | $0.308$308estimate · RTX PRO 6000 Blackwell | $308.39 estimate | $1.014 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B chat Open · 5.1B | — | Chat | $0.313$313estimate · RTX PRO 6000 Blackwell | $312.60 estimate | $1.028 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | Chat | $0.325$325estimate · RTX PRO 6000 Blackwell | $325.22 estimate | $1.064 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | Chat | $0.342$342 | $342.49 | $1.126 | — | provider price × reported usage |
| Gemma 4 E4B on · chat Open · 8.0B | — | Chat | $0.375$375estimate · DGX Spark | $375.43 estimate | $1.234 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B on · chat Open · 1.7B | — | Chat | $0.435$435estimate · RTX PRO 6000 Blackwell | $435.17 estimate | $1.408 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | Chat | $0.449$449estimate · RTX PRO 6000 Blackwell | $448.98 estimate | $1.499 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B chat Open · 8.0B | — | Chat | $0.467$467estimate · RTX PRO 6000 Blackwell | $466.58 estimate | $1.534 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLM-5.3-Flash Pareto high · chat Open · 320B MoE · 18B active | — | Chat | $0.471$471 | $471.05 | $1.549 | — | provider price × reported usage |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B | — | Chat | $0.549$549estimate · RTX PRO 6000 Blackwell | $548.55 estimate | $1.795 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | Chat | $0.598$598estimate · RTX PRO 6000 Blackwell | $597.99 estimate | $2.074 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | Chat | $0.615$615 | $615.00 | $2.022 | — | provider price × reported usage |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | Chat | $0.678$678estimate · RTX PRO 6000 Blackwell | $677.57 estimate | $2.228 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B off · chat Open · 4.7B | — | Chat | $0.683$683estimate · RTX PRO 6000 Blackwell | $683.25 estimate | $2.247 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | Chat | $0.713$713 | $713.10 | $2.345 | — | provider price × reported usage |
| Qwen3.8 Flash off · chat Closed · Undisclosed | — | Chat | $0.732$732 | $732.01 | $2.407 | — | provider price × reported usage |
| Gemma 4 12B Unified chat Open · 12B | — | Chat | $0.749$749estimate · RTX PRO 6000 Blackwell | $749.31 estimate | $2.464 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B chat Open · 31B | — | Chat | $0.775$775estimate · RTX PRO 6000 Blackwell | $774.76 estimate | $2.547 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | Chat | $0.788$788estimate · RTX PRO 6000 Blackwell | $788.01 estimate | $2.591 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | Chat | $0.840$840estimate · RTX PRO 6000 Blackwell | $839.76 estimate | $2.761 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | Chat | $0.855$855estimate · RTX PRO 6000 Blackwell | $854.91 estimate | $2.811 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Muse Ember max · chat Closed · Efficient | — | Chat | $0.863$863 | $862.71 | $2.837 | — | provider price × reported usage |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | Decision | $1.025$1,025 | $1,025.11 | $3.371 | — | provider price × reported usage |
| LFM2.5-2.6B on · chat Open · 2.7B | — | Chat | $1.046$1,046estimate · RTX PRO 6000 Blackwell | $1,045.70 estimate | $3.438 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B on · chat Open · 2.5B | — | Chat | $1.410$1,410estimate · RTX PRO 6000 Blackwell | $1,409.70 estimate | $4.726 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | Chat | $1.830$1,830 | $1,830.16 | $6.018 | — | provider price × reported usage |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | Chat | $1.913$1,913estimate · RTX PRO 6000 Blackwell† | $1,913.02 estimate† | $6.29 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B low · chat Open · 27B | — | Chat | $2.020$2,020estimate · RTX PRO 6000 Blackwell | $2,019.68 estimate | $6.641 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B on · chat Open · 4.7B | — | Chat | $2.042$2,042estimate · RTX PRO 6000 Blackwell | $2,042.18 estimate | $6.715 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | Chat | $2.506$2,506 | $2,505.67 | $8.239 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | Chat | $2.919$2,919 | $2,919.14 | $9.598 | — | provider price × reported usage |
| Qwen3.5-2B on · chat Open · 2.0B | — | Chat | $3.106$3,106estimate · RTX PRO 6000 Blackwell | $3,106.40 estimate | $10.21 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | Chat | $3.205$3,205estimate · RTX PRO 6000 Blackwell | $3,205.06 estimate | $10.54 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | Chat | $4.241$4,241estimate · RTX PRO 6000 Blackwell | $4,240.67 estimate | $13.94 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | Chat | $4.422$4,422estimate · RTX PRO 6000 Blackwell† | $4,422.21 estimate† | $14.54 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | Chat | $6.344$6,344estimate · RTX PRO 6000 Blackwell† | $6,344.06 estimate† | $20.86 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Sonnet 5.5 chat Closed · Workhorse | — | Chat | $7.042$7,042 API list-price equivalent | $7,042.33 | $23.16 | — | Claude subscription |
| Claude Opus 5.5 chat Closed · Flagship | — | Chat | $13.179$13,179 API list-price equivalent | $13,179.29 | $43.33 | — | Claude subscription |
| GPT-6 Astra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Hydrogen chat | — | Chat | — | — | — | — | |
| Claude Fable 5.1 chat Closed · Flagship | — | Chat | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6 Luna chat Closed · Efficient | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 3,276 of 3,288); shown once.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 9 setups not yet: DeepSeek-V4-Flash (chat) (missing Logic); Qwen3.5-397B-A17B (on · chat) (missing Math); GPT-6 Luna (max · chat) (missing Logic and Math); Claude Haiku 5.5 (max · chat) (missing Logic and Math); Claude Fable 5.2 (chat) (missing Logic and Math); Grok 4.7 (chat) (missing Logic); Spark-X2.6-4B (high · chat) (missing Logic and Math); Granite-4.2-3B (off · chat) (missing Logic and Math); Gemini 4 Carbon (high · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% · Math 99.6% | — | Chat | 98.6%98.2%–99.0% | a | 3,243 / 3,288 | Not measured | Not measured | 60.9%answered 99% | 5,580 ms | 9,580 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% · Math 99.6% | — | Chat | 98.6%98.1%–98.9% | a | 3,242 / 3,288 | Not measured | Not measured | 53.3%answered 99% | 16,350 ms | 90,630 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% · Math 99.6% | — | Chat | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 66.2%answered 99% | 5,240 ms | 10,410 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $2.919$2,919 per 1,000 questionsper 1M questions Logic 96.3% · Math 99.8% | — | Chat | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 10.7%answered 100% | 5,700 ms | 19,750 ms | $2.919$2,919 |
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $13.179$13,179 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% · Math 99.8% | — | Chat | 98.5%98.1%–98.9% | a | 3,240 / 3,288 | Not measured | Not measured | 41.3%answered 99% | 3,600 ms | 8,380 ms | $13.179$13,179 API list-price equivalent |
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.713$713 per 1,000 questionsper 1M questions Logic 96.2% · Math 99.6% | — | Chat | 98.4%98.0%–98.8% | ab | 3,237 / 3,288 | Not measured | Not measured | 6.3%answered 100% | 2,270 ms | 14,870 ms | $0.713$713 |
| Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Logic 96.1% · Math 99.6% | — | Chat | 98.4%97.9%–98.8% | abc | 3,235 / 3,288 | Not measured | Not measured | 78.3%answered 98% | 4,920 ms | 12,150 ms | — |
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $7.042$7,042 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% · Math 99.7% | — | Chat | 98.2%97.7%–98.6% | abcd | 3,230 / 3,288 | Not measured | Not measured | 43.7%answered 99% | 4,030 ms | 10,080 ms | $7.042$7,042 API list-price equivalent |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $1.830$1,830 per 1,000 questionsper 1M questions Logic 95.6% · Math 99.5% | — | Chat | 98.2%97.7%–98.6% | abcd | 3,229 / 3,288 | Not measured | Not measured | 41.7%answered 99% | 2,870 ms | 35,530 ms | $1.830$1,830 |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% · Math 99.7% | — | Chat | 98.2%97.7%–98.6% | abcd | 3,228 / 3,288 | Not measured | Not measured | 15.7%answered 100% | 4,940 ms | 8,620 ms | $0.000$0.00 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $2.506$2,506 per 1,000 questionsper 1M questions Logic 95.4% · Math 99.6% | — | Chat | 98.2%97.7%–98.6% | abcd | 3,228 / 3,288 | Not measured | Not measured | 50.6%answered 99% | 2,750 ms | 34,100 ms | $2.506$2,506 |
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.863$863 per 1,000 questionsper 1M questions Logic 95.5% · Math 99.6% | — | Chat | 98.2%97.7%–98.6% | abcd | 3,228 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 1,460 ms | 6,300 ms | $0.863$863 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.615$615 per 1,000 questionsper 1M questions Logic 95.1% · Math 99.4% | — | Chat | 98.0%97.4%–98.4% | abcde | 3,221 / 3,288 | Not measured | Not measured | 27.6%answered 99% | 10,590 ms | 53,970 ms | $0.615$615 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.732$732 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.5% | — | Chat | 97.6%97.0%–98.1% | bcdef | 3,209 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 6,780 ms | 36,290 ms | $0.732$732 |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $2.020$2,020estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | Chat | 97.4%96.8%–97.9% | cdef | 3,204 / 3,288 | Not measured | Not measured | 33.9%answered 99% | 11,200 ms | 39,560 ms | $2.020$2,020estimate · RTX PRO 6000 Blackwell |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | Chat | 97.4%96.8%–97.9% | def | 3,203 / 3,288 | Not measured | Not measured | 65.0%answered 98% | 4,440 ms | 8,290 ms | $0.000$0.00 |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.775$775estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% · Math 99.4% | — | Chat | 97.3%96.7%–97.8% | def | 3,200 / 3,288 | Not measured | Not measured | 17.0%answered 99% | 12,575 ms | 25,280 ms | $0.775$775estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $2.042$2,042estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% · Math 99.4% | — | Chat | 97.0%96.4%–97.5% | ef | 3,190 / 3,288 | Not measured | Not measured | 34.2%answered 99% | 63,270 ms | 193,950 ms | $2.042$2,042estimate · RTX PRO 6000 Blackwell |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.471$471 per 1,000 questionsper 1M questions Logic 93.2% · Math 98.8% | — | Chat | 96.9%96.2%–97.4% | efg | 3,185 / 3,288 | Not measured | Not measured | 7.4%answered 100% | 4,640 ms | 25,900 ms | $0.471$471 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.342$342 per 1,000 questionsper 1M questions Logic 94.0% · Math 98.2% | — | Chat | 96.8%96.1%–97.3% | fg | 3,182 / 3,288 | Not measured | Not measured | 38.9%answered 99% | 2,170 ms | 15,760 ms | $0.342$342 |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $4.241$4,241estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% · Math 99.2% | — | Chat | 96.6%96.0%–97.2% | fg | 3,177 / 3,288 | Not measured | Not measured | 54.7%answered 98% | 25,615 ms | 523,100 ms | $4.241$4,241estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $1.410$1,410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% · Math 99.2% | — | Chat | 95.7%95.4%–96.0% min–max of 3 seeds | gh | 3,145 / 3,288seed 0 run | Not measured | Not measured | 51.8%answered 97% | 19,260 ms | 194,917 ms | $1.410$1,410estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.683$683estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% · Math 98.5% | — | Chat | 95.6%94.9%–96.3% | ghi | 3,144 / 3,288 | Not measured | Not measured | 34.0%answered 98% | 14,295 ms | 78,230 ms | $0.683$683estimate · RTX PRO 6000 Blackwell |
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.749$749estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% · Math 98.8% | — | Chat | 95.0%94.2%–95.7% | hij | 3,125 / 3,288 | Not measured | Not measured | 15.3%answered 99% | 22,345 ms | 53,390 ms | $0.749$749estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $3.205$3,205estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% · Math 99.0% | — | Chat | 94.8%94.0%–95.5% | hijk | 3,117 / 3,288 | Not measured | Not measured | 44.0%answered 97% | 42,425 ms | 432,510 ms | $3.205$3,205estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.549$549estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% · Math 98.5% | — | Chat | 94.2%94.0%–94.4% min–max of 3 seeds | hijkl | 3,104 / 3,288seed 0 run | Not measured | Not measured | 54.1%answered 96% | 13,945 ms | 51,987 ms | $0.549$549estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.206$206estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% · Math 98.7% | — | Chat | 94.2%93.3%–94.9% | hijkl | 3,097 / 3,288 | Not measured | Not measured | 55.9%answered 96% | 29,040 ms | 343,820 ms | $0.206$206estimate · DGX Spark |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.449$449estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% · Math 98.9% | — | Chat | 94.0%93.8%–94.2% min–max of 3 seeds | hijkl | 3,098 / 3,288seed 0 run | Not measured | Not measured | Not measured | 5,667 ms | 58,313 ms | $0.449$449estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.435$435estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% · Math 98.9% | — | Chat | 93.9%93.89%–93.95% min–max of 3 seeds | ijkl | 3,089 / 3,288seed 0 run | Not measured | Not measured | 46.7%answered 96% | 5,630 ms | 63,067 ms | $0.435$435estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.855$855estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% · Math 98.5% | — | Chat | 93.9%93.0%–94.7% | jkl | 3,087 / 3,288 | Not measured | Not measured | Not measured | 25,530 ms | 77,380 ms | $0.855$855estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.598$598estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% · Math 98.2% | — | Chat | 93.2%92.9%–93.3% min–max of 3 seeds | jkl | 3,068 / 3,288seed 0 run | Not measured | Not measured | 41.0%answered 95% | 9,065 ms | 75,703 ms | $0.598$598estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% · Math 98.1% | — | Chat | 93.0%92.1%–93.8% | kl | 3,058 / 3,288 | Not measured | Not measured | 51.0%answered 95% | 13,150 ms | 44,990 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.375$375estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% · Math 98.2% | — | Chat | 92.9%92.0%–93.7% | l | 3,054 / 3,288 | Not measured | Not measured | 43.7%answered 96% | 131,535 ms | 392,620 ms | $0.375$375estimate · DGX Spark |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.325$325estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% · Math 97.7% | — | Chat | 90.5%90.2%–90.9% min–max of 3 seeds | mn | 2,966 / 3,288seed 0 run | Not measured | Not measured | 31.0%answered 97% | 8,873 ms | 32,503 ms | $0.325$325estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.678$678estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% · Math 97.8% | — | Chat | 90.3%89.3%–91.3% | m | 2,970 / 3,288 | Not measured | Not measured | Not measured | 19,100 ms | 54,630 ms | $0.678$678estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.215$215estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% · Math 96.9% | — | Chat | 89.3%88.2%–90.3% | mno | 2,936 / 3,288 | Not measured | Not measured | 24.3%answered 97% | 77,885 ms | 232,800 ms | $0.215$215estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.313$313estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% · Math 96.8% | — | Chat | 88.6%87.5%–89.6% | mnop | 2,913 / 3,288 | Not measured | Not measured | 32.3%answered 96% | 9,935 ms | 27,690 ms | $0.313$313estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $6.344$6,344estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% · Math 97.0% | — | Chat | 88.1%86.9%–89.1% | nop | 2,896 / 3,288 | Not measured | Not measured | Not measured | 54,985 ms† | 987,660 ms† | $6.344$6,344estimate · RTX PRO 6000 Blackwell† |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $1.046$1,046estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% · Math 94.0% | — | Chat | 87.6%86.5%–88.7% | opq | 2,881 / 3,288 | Not measured | Not measured | 40.6%answered 95% | 14,035 ms | 142,970 ms | $1.046$1,046estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $3.106$3,106estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% · Math 94.7% | — | Chat | 86.4%85.2%–87.5% | pqr | 2,840 / 3,288 | Not measured | Not measured | 30.9%answered 97% | 121,840 ms | 241,590 ms | $3.106$3,106estimate · RTX PRO 6000 Blackwell |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.840$840estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% · Math 95.8% | — | Chat | 85.4%84.1%–86.5% | qr | 2,807 / 3,288 | Not measured | Not measured | 20.9%answered 96% | 9,380 ms | 35,110 ms | $0.840$840estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.788$788estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% · Math 94.6% | — | Chat | 84.4%83.1%–85.6% | r | 2,776 / 3,288 | Not measured | Not measured | 34.3%answered 94% | 17,805 ms | 92,100 ms | $0.788$788estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $1.913$1,913estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% · Math 96.3% | — | Chat | 83.9%82.6%–85.1% | rs | 2,759 / 3,288 | Not measured | Not measured | 55.9%answered 88% | 20,095 ms† | 230,570 ms† | $1.913$1,913estimate · RTX PRO 6000 Blackwell† |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% · Math 93.7% | — | Chat | 81.4%80.1%–82.7% | s | 2,678 / 3,288 | Not measured | Not measured | Not measured | 3,830 ms | 5,300 ms | $0.000$0.00 |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $4.422$4,422estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% · Math 89.6% | — | Chat | 81.2%79.8%–82.5% | st | 2,669 / 3,288 | Not measured | Not measured | 31.1%answered 94% | 42,585 ms† | 593,950 ms† | $4.422$4,422estimate · RTX PRO 6000 Blackwell† |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.308$308estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% · Math 90.8% | — | Chat | 78.3%76.8%–79.6% | t | 2,573 / 3,288 | Not measured | Not measured | 32.3%answered 92% | 8,740 ms | 16,420 ms | $0.308$308estimate · RTX PRO 6000 Blackwell |
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.118$118estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% · Math 81.6% | — | Decision | 74.4%72.9%–75.9% | u | 2,447 / 3,288 | 1,006 right of 1,009 kept3 wrong · 2,279 skipped | 0.882 | 5.7%answered 98% | 162 ms | 198 ms | $0.118$118estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.093$92.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% · Math 80.3% | — | Decision | 73.0%71.4%–74.5% | uv | 2,399 / 3,288 | 523 right of 524 kept1 wrong · 2,764 skipped | 0.882 | 0.1%answered 100% | 116 ms | 177 ms | $0.093$92.98estimate · RTX PRO 6000 Blackwell |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.148$148estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% · Math 78.0% | — | Decision | 71.6%70.0%–73.1% | v | 2,354 / 3,288 | 229 right of 229 kept0 mistakes · 3,059 skipped | 0.878 | 3.8%answered 99% | 200 ms | 260 ms | $0.148$148estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.052$51.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% · Math 77.4% | — | Decision | 71.1%69.5%–72.6% | vw | 2,338 / 3,288 | 553 right of 554 kept1 wrong · 2,734 skipped | 0.870 | 0.7%answered 100% | 811 ms | 1,111 ms | $0.052$51.85estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.102$102estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% · Math 77.3% | — | Decision | 71.0%69.4%–72.5% | vw | 2,333 / 3,288 | 384 right of 384 kept0 mistakes · 2,904 skipped | 0.873 | 1.8%answered 99% | 128 ms | 218 ms | $0.102$102estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.047$47.29estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% · Math 77.1% | — | Decision | 70.5%68.9%–72.1% | vwx | 2,319 / 3,288 | 294 right of 296 kept2 wrong · 2,992 skipped | 0.865 | 3.8%answered 99% | 676 ms | 1,173 ms | $0.047$47.29estimate · DGX Spark |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.165$165estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% · Math 75.2% | — | Decision | 69.3%67.7%–70.8% | wxy | 2,277 / 3,288 | 23 right of 23 kept0 mistakes · 3,265 skipped | 0.868 | 4.4%answered 99% | 207 ms | 324 ms | $0.165$165estimate · RTX PRO 6000 Blackwell |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 54.9% · Math 74.1% | — | Decision | 67.5%65.9%–69.1% | xyzA | 2,221 / 3,288 | 233 right of 233 kept0 mistakes · 3,055 skipped | 0.852 | 89.1%answered 52% | 180 ms | 426 ms | $0.008$8.37 |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.029$28.55estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% · Math 72.0% | — | Decision | 67.3%65.7%–68.9% | xyz | 2,214 / 3,288 | 272 right of 274 kept2 wrong · 3,014 skipped | 0.852 | 31.9%answered 89% | 450 ms | 587 ms | $0.029$28.55estimate · DGX Spark |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 74.0% | — | Decision | 67.3%65.7%–68.9% | yzA | 2,213 / 3,288 | 518 right of 520 kept2 wrong · 2,768 skipped | 0.868 | 71.4%answered 69% | 150 ms | 232 ms | $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.071$71.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% · Math 72.5% | — | Decision | 66.4%64.8%–68.0% | zAB | 2,184 / 3,288 | 385 right of 385 kept0 mistakes · 2,903 skipped | 0.844 | 13.5%answered 95% | 97 ms | 125 ms | $0.071$71.10estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.096$95.71estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% · Math 71.5% | — | Decision | 65.7%64.1%–67.3% | zABC | 2,161 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.835 | 7.5%answered 97% | 129 ms | 168 ms | $0.096$95.71estimate · RTX PRO 6000 Blackwell |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.036$36.42 per 1,000 questionsper 1M questions Logic 54.5% · Math 70.8% | — | Decision | 65.2%63.6%–66.8% | zABC | 2,144 / 3,288 | 213 right of 213 kept0 mistakes · 3,075 skipped | 0.852 | 49.2%answered 80% | 266 ms | 284 ms | $0.036$36.42 |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.046$46.28estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 70.4% | — | Decision | 65.0%63.3%–66.6% | zABC | 2,136 / 3,288 | 216 right of 217 kept1 wrong · 3,071 skipped | 0.858 | 9.3%answered 97% | 57 ms | 86 ms | $0.046$46.28estimate · RTX PRO 6000 Blackwell |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.016$16.13 per 1,000 questionsper 1M questions Logic 54.8% · Math 70.1% | — | Decision | 64.8%63.2%–66.5% | zABC | 2,132 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.846 | 48.3%answered 79% | 327 ms | 518 ms | $0.016$16.13 |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% · Math 69.9% | — | Decision | 64.8%63.2%–66.5% | ABC | 2,132 / 3,288 | 176 right of 176 kept0 mistakes · 3,112 skipped | 0.843 | 42.3%answered 83% | 81 ms | 136 ms | $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% · Math 70.3% | — | Decision | 64.4%62.8%–66.0% | zABC | 2,118 / 3,288 | 866 right of 884 kept18 wrong · 2,404 skipped | 0.841 | 5.7%answered 98% | 81 ms | 130 ms | $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.005$5.38 per 1,000 questionsper 1M questions Logic 53.3% · Math 69.8% | — | Decision | 64.1%62.5%–65.8% | BCD | 2,109 / 3,288 | 409 right of 411 kept2 wrong · 2,877 skipped | 0.849 | 10.3%answered 96% | 230 ms | 439 ms | $0.005$5.38 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.027$27.07 per 1,000 questionsper 1M questions Logic 50.0% · Math 71.3% | — | Decision | 64.1%62.4%–65.7% | zABCD | 2,106 / 3,288 | 257 right of 257 kept0 mistakes · 3,031 skipped | 0.838 | 3.0%answered 99% | 227 ms | 347 ms | $0.027$27.07 |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 51.6% · Math 69.9% | — | Decision | 63.6%61.9%–65.2% | BCD | 2,091 / 3,288 | 128 right of 128 kept0 mistakes · 3,160 skipped | 0.856 | 90.2%answered 48% | 217 ms | 347 ms | $0.008$8.37 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.025$1,025 per 1,000 questionsper 1M questions Logic 52.5% · Math 69.3% | with web | Decision | 63.6%61.9%–65.2% | CD | 2,090 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.854 | 25.8%answered 90% | 430 ms | 707 ms | $1.025$1,025 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.020$20.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% · Math 69.0% | — | Decision | 63.3%61.6%–64.9% | BCDE | 2,081 / 3,288 | 50 right of 50 kept0 mistakes · 3,238 skipped | 0.844 | 1.9%answered 99% | 301 ms | 448 ms | $0.020$20.01estimate · DGX Spark |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.011$10.75 per 1,000 questionsper 1M questions Logic 52.4% · Math 68.6% | — | Decision | 63.0%61.4%–64.7% | CDE | 2,072 / 3,288 | 430 right of 433 kept3 wrong · 2,855 skipped | 0.827 | 58.3%answered 73% | 382 ms | 391 ms | $0.011$10.75 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.011$11.25 per 1,000 questionsper 1M questions Logic 47.3% · Math 70.0% | — | Decision | 62.2%60.6%–63.9% | CDEF | 2,046 / 3,288 | No answers at this threshold | 0.829 | 87.2%answered 50% | 388 ms | 392 ms | $0.011$11.25 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$1.13 per 1,000 questionsper 1M questions Logic 47.3% · Math 69.9% | — | Decision | 62.1%60.5%–63.8% | CDEF | 2,043 / 3,288 | No answers at this threshold | 0.830 | 86.7%answered 51% | 381 ms | 388 ms | $0.001$1.13 |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% · Math 66.3% | — | Decision | 60.9%59.2%–62.5% | DEFG | 2,002 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.810 | 4.5%answered 98% | 51 ms | 76 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 49.2% · Math 65.5% | — | Decision | 59.9%58.2%–61.6% | EFGH | 1,970 / 3,288 | 169 right of 169 kept0 mistakes · 3,119 skipped | 0.812 | 66.9%answered 65% | 208 ms | 390 ms | $0.004$3.98 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 48.2% · Math 65.1% | — | Decision | 59.3%57.6%–61.0% | FGHI | 1,951 / 3,288 | 263 right of 263 kept0 mistakes · 3,025 skipped | 0.810 | 15.4%answered 93% | 164 ms | 188 ms | $0.021$21.27 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 46.0% · Math 65.2% | — | Decision | 58.6%56.9%–60.3% | FGHI | 1,928 / 3,288 | 231 right of 231 kept0 mistakes · 3,057 skipped | 0.811 | 86.0%answered 46% | 174 ms | 285 ms | $0.004$3.98 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 46.3% · Math 64.1% | — | Decision | 58.0%56.3%–59.7% | GHIJ | 1,907 / 3,288 | 313 right of 313 kept0 mistakes · 2,975 skipped | 0.817 | 10.2%answered 96% | 166 ms | 205 ms | $0.021$21.27 |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.028$28.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 62.1% | — | Decision | 56.7%55.0%–58.4% | HIJK | 1,865 / 3,288 | 251 right of 253 kept2 wrong · 3,035 skipped | 0.802 | 7.0%answered 97% | 38 ms | 45 ms | $0.028$28.19estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 61.3% | — | Decision | 56.2%54.5%–57.9% | IJK | 1,849 / 3,288 | 235 right of 236 kept1 wrong · 3,052 skipped | 0.797 | 69.4%answered 60% | 80 ms | 94 ms | $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.032$32.40estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% · Math 59.8% | — | Decision | 55.8%54.1%–57.5% | IJKL | 1,836 / 3,288 | 74 right of 74 kept0 mistakes · 3,214 skipped | 0.798 | 13.5%answered 94% | 39 ms | 53 ms | $0.032$32.40estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.034$34.08estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% · Math 60.1% | — | Decision | 54.6%52.9%–56.3% | JKLM | 1,794 / 3,288 | No answers at this threshold | 0.786 | 10.4%answered 95% | 42 ms | 57 ms | $0.034$34.08estimate · RTX PRO 6000 Blackwell |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.011$11.45estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% · Math 57.9% | — | Decision | 54.5%52.8%–56.2% | JKLM | 1,791 / 3,288 | 105 right of 105 kept0 mistakes · 3,183 skipped | 0.786 | 26.9%answered 86% | 177 ms | 222 ms | $0.011$11.45estimate · DGX Spark |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.030$29.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% · Math 59.8% | — | Decision | 54.5%52.8%–56.2% | JKLMN | 1,791 / 3,288 | 153 right of 153 kept0 mistakes · 3,135 skipped | 0.782 | 3.4%answered 98% | 38 ms | 55 ms | $0.030$29.87estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.009$8.92estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% · Math 58.4% | — | Decision | 54.4%52.7%–56.1% | KLMNO | 1,788 / 3,288 | 63 right of 63 kept0 mistakes · 3,225 skipped | 0.779 | 9.6%answered 95% | 142 ms | 165 ms | $0.009$8.92estimate · DGX Spark |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.009$9.07 per 1,000 questionsper 1M questions Logic 44.4% · Math 59.4% | — | Decision | 54.3%52.6%–56.0% | KLMNOP | 1,784 / 3,288 | 47 right of 47 kept0 mistakes · 3,241 skipped | 0.769 | 11.4%answered 94% | 312 ms | 360 ms | $0.009$9.07 |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.033$32.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% · Math 58.8% | — | Decision | 54.2%52.5%–55.9% | KLMNOP | 1,782 / 3,288 | 263 right of 266 kept3 wrong · 3,022 skipped | 0.795 | 6.1%answered 97% | 46 ms | 51 ms | $0.033$32.61estimate · RTX PRO 6000 Blackwell |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.012$12.17 per 1,000 questionsper 1M questions Logic 41.9% · Math 59.0% | — | Decision | 53.1%51.4%–54.8% | KLMNOP | 1,747 / 3,288 | 207 right of 207 kept0 mistakes · 3,081 skipped | 0.789 | 28.1%answered 85% | 339 ms | 472 ms | $0.012$12.17 |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.024$24.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% · Math 56.8% | — | Decision | 52.5%50.8%–54.2% | LMNOPQ | 1,725 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.777 | 7.9%answered 96% | 31 ms | 36 ms | $0.024$24.19estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% · Math 55.8% | — | Decision | 52.4%50.7%–54.1% | LMNOPQ | 1,724 / 3,288 | 117 right of 117 kept0 mistakes · 3,171 skipped | 0.763 | 14.4%answered 93% | 68 ms | 80 ms | $0.004$4.28estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.024$23.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% · Math 55.9% | — | Decision | 52.1%50.4%–53.8% | MNOPQ | 1,713 / 3,288 | 130 right of 131 kept1 wrong · 3,157 skipped | 0.782 | 15.0%answered 92% | 29 ms | 35 ms | $0.024$23.77estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.011$10.82estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% · Math 56.0% | — | Decision | 51.7%50.0%–53.4% | MNOPQ | 1,699 / 3,288 | 270 right of 281 kept11 wrong · 3,007 skipped | 0.776 | 3.1%answered 99% | 163 ms | 240 ms | $0.011$10.82estimate · DGX Spark |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% · Math 55.3% | — | Decision | 51.6%49.9%–53.3% | MNOPQ | 1,696 / 3,288 | 165 right of 167 kept2 wrong · 3,121 skipped | 0.765 | 7.1%answered 96% | 59 ms | 65 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$5.02estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% · Math 56.0% | — | Decision | 51.3%49.6%–53.0% | MNOPQR | 1,687 / 3,288 | 71 right of 72 kept1 wrong · 3,216 skipped | 0.752 | 4.4%answered 98% | 76 ms | 104 ms | $0.005$5.02estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% · Math 55.2% | — | Decision | 51.1%49.4%–52.8% | MNOPQR | 1,680 / 3,288 | 185 right of 190 kept5 wrong · 3,098 skipped | 0.769 | 18.9%answered 90% | 41 ms | 51 ms | $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.005$5.46 per 1,000 questionsper 1M questions Logic 45.1% · Math 53.5% | — | Decision | 50.6%48.9%–52.3% | NOPQRS | 1,664 / 3,288 | 356 right of 372 kept16 wrong · 2,916 skipped | 0.777 | 10.1%answered 95% | 293 ms | 349 ms | $0.005$5.46 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 40.8% · Math 55.6% | — | Decision | 50.5%48.8%–52.3% | OPQRS | 1,662 / 3,288 | 207 right of 208 kept1 wrong · 3,080 skipped | 0.757 | 6.4%answered 97% | 185 ms | 292 ms | $0.004$3.98 |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.212$212estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% · Math 54.6% | — | Decision | 50.5%48.7%–52.2% | PQRS | 1,659 / 3,288 | No answers at this threshold | 0.759 | 14.5%answered 92% | 250 ms | 482 ms | $0.212$212estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.069$69.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% · Math 53.8% | — | Decision | 49.3%47.6%–51.0% | QRST | 1,620 / 3,288 | 30 right of 30 kept0 mistakes · 3,258 skipped | 0.770 | 1.6%answered 99% | 80 ms | 134 ms | $0.069$69.42estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% · Math 51.1% | — | Decision | 47.6%45.9%–49.3% | RSTU | 1,564 / 3,288 | 18 right of 18 kept0 mistakes · 3,270 skipped | 0.747 | 2.4%answered 99% | 185 ms | 216 ms | $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.45estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 51.5% | — | Decision | 47.5%45.8%–49.2% | STU | 1,561 / 3,288 | 161 right of 165 kept4 wrong · 3,123 skipped | 0.747 | 8.9%answered 95% | 46 ms | 52 ms | $0.033$33.45estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.025$25.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% · Math 50.4% | — | Decision | 47.0%45.3%–48.7% | STU | 1,545 / 3,288 | 188 right of 194 kept6 wrong · 3,094 skipped | 0.743 | 7.8%answered 96% | 30 ms | 42 ms | $0.025$25.03estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% · Math 49.8% | — | Decision | 46.1%44.4%–47.8% | TUV | 1,515 / 3,288 | No answers at this threshold | 0.707 | 6.9%answered 96% | 103 ms | 131 ms | $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.043$42.91estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 46.9% | — | Decision | 44.4%42.7%–46.1% | UV | 1,461 / 3,288 | 52 right of 53 kept1 wrong · 3,235 skipped | 0.737 | 17.1%answered 90% | 55 ms | 72 ms | $0.043$42.91estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% · Math 44.3% | — | Decision | 42.6%40.9%–44.3% | V | 1,401 / 3,288 | No answers at this threshold | 0.729 | 7.3%answered 96% | 38 ms | 44 ms | $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.086$85.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% · Math 39.2% | — | Decision | 36.7%35.1%–38.4% | W | 1,208 / 3,288 | No answers at this threshold | 0.659 | 2.0%answered 99% | 117 ms | 146 ms | $0.086$85.83estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.36estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% · Math 38.7% | — | Decision | 35.9%34.3%–37.6% | W | 1,182 / 3,288 | 4 right of 4 kept0 mistakes · 3,284 skipped | 0.675 | 16.4%answered 89% | 36 ms | 42 ms | $0.002$2.36estimate · DGX Spark |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% · Math 36.8% | — | Decision | 35.2%33.6%–36.9% | W | 1,158 / 3,288 | 29 right of 30 kept1 wrong · 3,258 skipped | 0.637 | 23.1%answered 84% | 28 ms | 37 ms | $0.002$1.94estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.30estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% · Math 37.1% | — | Decision | 34.9%33.3%–36.5% | WX | 1,147 / 3,288 | 7 right of 7 kept0 mistakes · 3,281 skipped | 0.648 | 82.5%answered 30% | 17 ms | 21 ms | $0.014$14.30estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.40estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% · Math 35.3% | — | Decision | 34.1%32.5%–35.7% | WX | 1,120 / 3,288 | 16 right of 16 kept0 mistakes · 3,272 skipped | 0.626 | 6.8%answered 95% | 51 ms | 63 ms | $0.003$3.40estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.38estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% · Math 32.1% | — | Decision | 31.1%29.5%–32.7% | XY | 1,022 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.602 | 22.7%answered 83% | 20 ms | 25 ms | $0.001$1.38estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.62estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% · Math 29.7% | — | Decision | 28.0%26.5%–29.5% | YZ | 920 / 3,288 | No answers at this threshold | 0.605 | 14.9%answered 89% | 24 ms | 29 ms | $0.002$1.62estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.11estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% · Math 28.2% | — | Decision | 27.8%26.3%–29.4% | YZ | 915 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.576 | 2.0%answered 99% | 15 ms | 17 ms | $0.001$1.11estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.80estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% · Math 28.2% | — | Decision | 27.7%26.2%–29.2% | YZ | 910 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.590 | 0.6%answered 100% | 26 ms | 35 ms | $0.002$1.80estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.027$26.72estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% · Math 28.7% | — | Decision | 26.5%25.0%–28.1% | Zα | 872 / 3,288 | No answers at this threshold | 0.573 | 5.0%answered 96% | 35 ms | 42 ms | $0.027$26.72estimate · RTX PRO 6000 Blackwell |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% · Math 22.7% | — | Decision | 23.0%21.6%–24.4% | αβ | 755 / 3,288 | No answers at this threshold | 0.511 | 15.6%answered 87% | 12 ms | 23 ms | $0.001$1.01estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.83estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% · Math 20.6% | — | Decision | 21.5%20.1%–22.9% | β | 706 / 3,288 | No answers at this threshold | 0.520 | 73.9%answered 32% | 11 ms | 14 ms | $0.001$0.83estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI scar-ai quincyqiang Strands Agents victorbennett Metask Lab Google OpenBMB Alibaba IBM KRAFTON
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 21.5% at $0.001 per 1,000 questions ($0.83 per 1M) (estimate)
GLiClass v4 Edge: 23.0% at $0.001 per 1,000 questions ($1.01 per 1M) (estimate)
OneJev1.6-0.8B: 27.8% at $0.001 per 1,000 questions ($1.11 per 1M) (estimate)
GLiDE 0.8B: 31.1% at $0.001 per 1,000 questions ($1.38 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 28.0% at $0.002 per 1,000 questions ($1.62 per 1M) (estimate)
KnowLine-0.8B-Gen1: 27.7% at $0.002 per 1,000 questions ($1.80 per 1M) (estimate)
GLiDE 2B: 35.2% at $0.002 per 1,000 questions ($1.94 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 35.9% at $0.002 per 1,000 questions ($2.36 per 1M) (estimate)
Perplexity Decider Flash v1.2: 51.1% at $0.003 per 1,000 questions ($2.72 per 1M)
KnowLine-2B-Gen1: 34.1% at $0.003 per 1,000 questions ($3.40 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 52.4% at $0.004 per 1,000 questions ($4.28 per 1M) (estimate)
KnowLine 4B Gen5.5: 51.3% at $0.005 per 1,000 questions ($5.02 per 1M) (estimate)
d1-2B: 34.9% at $0.014 per 1,000 questions ($14.30 per 1M) (estimate)
Perplexity Decider Flash v1.3: 52.1% at $0.024 per 1,000 questions ($23.77 per 1M) (estimate)
vera-medium-omni: 52.5% at $0.024 per 1,000 questions ($24.19 per 1M) (estimate)
Chinese Decision Pointer 4B: 47.0% at $0.025 per 1,000 questions ($25.03 per 1M) (estimate)
Strands Decider 2B: 26.5% at $0.027 per 1,000 questions ($26.72 per 1M) (estimate)
Verdin-4B-v1.1: 47.5% at $0.033 per 1,000 questions ($33.45 per 1M) (estimate)
Metask-Jev-4B: 36.7% at $0.086 per 1,000 questions ($85.83 per 1M) (estimate)
LFM3-1.7B (off · chat): 94.2% at $0.206 per 1,000 questions ($206 per 1M) (estimate)
Gemma 4 E2B (on · chat): 89.3% at $0.215 per 1,000 questions ($215 per 1M) (estimate)
Gemma 4 E2B (chat): 88.6% at $0.313 per 1,000 questions ($313 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 90.5% at $0.325 per 1,000 questions ($325 per 1M) (estimate)
LFM3-1.7B (on · chat): 93.9% at $0.435 per 1,000 questions ($435 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 94.0% at $0.449 per 1,000 questions ($449 per 1M) (estimate)
MiniCPM5-2B (off · chat): 93.2% at $0.598 per 1,000 questions ($598 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 90.3% at $0.678 per 1,000 questions ($678 per 1M) (estimate)
Qwen3.5-4B (off · chat): 95.6% at $0.683 per 1,000 questions ($683 per 1M) (estimate)
Qwen3.5-2B (off · chat): 84.4% at $0.788 per 1,000 questions ($788 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 87.6% at $1.046 per 1,000 questions ($1,046 per 1M) (estimate)
MiniCPM5-2B (on · chat): 95.7% at $1.410 per 1,000 questions ($1,410 per 1M) (estimate)
Qwen3.5-4B (on · chat): 97.0% at $2.042 per 1,000 questions ($2,042 per 1M) (estimate)
Qwen3.5-2B (on · chat): 86.4% at $3.106 per 1,000 questions ($3,106 per 1M) (estimate)
Granite-4.2-3B (high · chat): 96.6% at $4.241 per 1,000 questions ($4,241 per 1M) (estimate)
Raon-1 nano (off · chat): 81.2% at $4.422 per 1,000 questions ($4,422 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (on · chat): 88.1% at $6.344 per 1,000 questions ($6,344 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI scar-ai quincyqiang Strands Agents victorbennett Metask Lab Google OpenBMB Alibaba IBM KRAFTON
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | Decision | $0.001$0.83estimate · DGX Spark | $0.83 estimate | $0.002719 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | Decision | $0.001$1.01estimate · DGX Spark | $1.01 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | Decision | $0.001$1.11estimate · DGX Spark | $1.11 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Pareto Open · 0.8B | — | Decision | $0.001$1.38estimate · DGX Spark | $1.38 estimate | $0.004552 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | Decision | $0.002$1.62estimate · DGX Spark | $1.62 estimate | $0.00532 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | Decision | $0.002$1.80estimate · DGX Spark | $1.80 estimate | $0.005912 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Pareto Open · 1.9B | — | Decision | $0.002$1.94estimate · DGX Spark | $1.94 estimate | $0.006385 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Pareto Open · 2.0B | — | Decision | $0.002$2.36estimate · DGX Spark | $2.36 estimate | $0.007744 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Pareto Open · 4.0B | — | Decision | $0.003$2.72 list | $2.72 | $0.008936 | $0.032$31.55estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | Decision | $0.003$3.40estimate · DGX Spark | $3.40 estimate | $0.01117 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Pareto Open · 4.0B | — | Decision | $0.004$4.28estimate · DGX Spark | $4.28 estimate | $0.01407 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | Decision | $0.005$5.02estimate · DGX Spark | $5.02 estimate | $0.01649 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-2B Open · 1.9B | — | Decision | $0.014$14.30estimate · RTX PRO 6000 Blackwell | $14.30 estimate | $0.04703 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | Decision | $0.024$23.77estimate · RTX PRO 6000 Blackwell | $23.77 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Pareto Open · 4.2B | — | Decision | $0.024$24.19estimate · RTX PRO 6000 Blackwell | $24.19 estimate | $0.07954 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | Decision | $0.025$25.03estimate · RTX PRO 6000 Blackwell | $25.03 estimate | $0.08231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | Decision | $0.027$26.72estimate · RTX PRO 6000 Blackwell | $26.72 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | Decision | $0.033$33.45estimate · RTX PRO 6000 Blackwell | $33.45 estimate | $0.11 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | Decision | $0.086$85.83estimate · RTX PRO 6000 Blackwell | $85.83 estimate | $0.2822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | Chat | $0.206$206estimate · DGX Spark | $205.97 estimate | $0.6772 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | Chat | $0.215$215estimate · DGX Spark | $215.10 estimate | $0.7073 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B chat Open · 5.1B | — | Chat | $0.313$313estimate · RTX PRO 6000 Blackwell | $312.60 estimate | $1.028 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | Chat | $0.325$325estimate · RTX PRO 6000 Blackwell | $325.22 estimate | $1.064 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B on · chat Open · 1.7B | — | Chat | $0.435$435estimate · RTX PRO 6000 Blackwell | $435.17 estimate | $1.408 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | Chat | $0.449$449estimate · RTX PRO 6000 Blackwell | $448.98 estimate | $1.499 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | Chat | $0.598$598estimate · RTX PRO 6000 Blackwell | $597.99 estimate | $2.074 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | Chat | $0.678$678estimate · RTX PRO 6000 Blackwell | $677.57 estimate | $2.228 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto off · chat Open · 4.7B | — | Chat | $0.683$683estimate · RTX PRO 6000 Blackwell | $683.25 estimate | $2.247 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | Chat | $0.788$788estimate · RTX PRO 6000 Blackwell | $788.01 estimate | $2.591 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | Chat | $1.046$1,046estimate · RTX PRO 6000 Blackwell | $1,045.70 estimate | $3.438 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B Pareto on · chat Open · 2.5B | — | Chat | $1.410$1,410estimate · RTX PRO 6000 Blackwell | $1,409.70 estimate | $4.726 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto on · chat Open · 4.7B | — | Chat | $2.042$2,042estimate · RTX PRO 6000 Blackwell | $2,042.18 estimate | $6.715 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | Chat | $3.106$3,106estimate · RTX PRO 6000 Blackwell | $3,106.40 estimate | $10.21 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | Chat | $4.241$4,241estimate · RTX PRO 6000 Blackwell | $4,240.67 estimate | $13.94 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | Chat | $4.422$4,422estimate · RTX PRO 6000 Blackwell† | $4,422.21 estimate† | $14.54 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | Chat | $6.344$6,344estimate · RTX PRO 6000 Blackwell† | $6,344.06 estimate† | $20.86 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: Spark-X2.6-4B (high · chat) (missing Logic and Math); Granite-4.2-3B (off · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $2.042$2,042estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% · Math 99.4% | — | Chat | 97.0%96.4%–97.5% | a | 3,190 / 3,288 | Not measured | Not measured | 34.2%answered 99% | 63,270 ms | 193,950 ms | $2.042$2,042estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $4.241$4,241estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% · Math 99.2% | — | Chat | 96.6%96.0%–97.2% | ab | 3,177 / 3,288 | Not measured | Not measured | 54.7%answered 98% | 25,615 ms | 523,100 ms | $4.241$4,241estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $1.410$1,410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% · Math 99.2% | — | Chat | 95.7%95.4%–96.0% min–max of 3 seeds | b | 3,145 / 3,288seed 0 run | Not measured | Not measured | 51.8%answered 97% | 19,260 ms | 194,917 ms | $1.410$1,410estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.683$683estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% · Math 98.5% | — | Chat | 95.6%94.9%–96.3% | bc | 3,144 / 3,288 | Not measured | Not measured | 34.0%answered 98% | 14,295 ms | 78,230 ms | $0.683$683estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.206$206estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% · Math 98.7% | — | Chat | 94.2%93.3%–94.9% | cd | 3,097 / 3,288 | Not measured | Not measured | 55.9%answered 96% | 29,040 ms | 343,820 ms | $0.206$206estimate · DGX Spark |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.449$449estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% · Math 98.9% | — | Chat | 94.0%93.8%–94.2% min–max of 3 seeds | cd | 3,098 / 3,288seed 0 run | Not measured | Not measured | Not measured | 5,667 ms | 58,313 ms | $0.449$449estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.435$435estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% · Math 98.9% | — | Chat | 93.9%93.89%–93.95% min–max of 3 seeds | d | 3,089 / 3,288seed 0 run | Not measured | Not measured | 46.7%answered 96% | 5,630 ms | 63,067 ms | $0.435$435estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.598$598estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% · Math 98.2% | — | Chat | 93.2%92.9%–93.3% min–max of 3 seeds | d | 3,068 / 3,288seed 0 run | Not measured | Not measured | 41.0%answered 95% | 9,065 ms | 75,703 ms | $0.598$598estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.325$325estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% · Math 97.7% | — | Chat | 90.5%90.2%–90.9% min–max of 3 seeds | e | 2,966 / 3,288seed 0 run | Not measured | Not measured | 31.0%answered 97% | 8,873 ms | 32,503 ms | $0.325$325estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.678$678estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% · Math 97.8% | — | Chat | 90.3%89.3%–91.3% | e | 2,970 / 3,288 | Not measured | Not measured | Not measured | 19,100 ms | 54,630 ms | $0.678$678estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.215$215estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% · Math 96.9% | — | Chat | 89.3%88.2%–90.3% | ef | 2,936 / 3,288 | Not measured | Not measured | 24.3%answered 97% | 77,885 ms | 232,800 ms | $0.215$215estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.313$313estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% · Math 96.8% | — | Chat | 88.6%87.5%–89.6% | ef | 2,913 / 3,288 | Not measured | Not measured | 32.3%answered 96% | 9,935 ms | 27,690 ms | $0.313$313estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $6.344$6,344estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% · Math 97.0% | — | Chat | 88.1%86.9%–89.1% | fg | 2,896 / 3,288 | Not measured | Not measured | Not measured | 54,985 ms† | 987,660 ms† | $6.344$6,344estimate · RTX PRO 6000 Blackwell† |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $1.046$1,046estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% · Math 94.0% | — | Chat | 87.6%86.5%–88.7% | fg | 2,881 / 3,288 | Not measured | Not measured | 40.6%answered 95% | 14,035 ms | 142,970 ms | $1.046$1,046estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $3.106$3,106estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% · Math 94.7% | — | Chat | 86.4%85.2%–87.5% | gh | 2,840 / 3,288 | Not measured | Not measured | 30.9%answered 97% | 121,840 ms | 241,590 ms | $3.106$3,106estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.788$788estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% · Math 94.6% | — | Chat | 84.4%83.1%–85.6% | h | 2,776 / 3,288 | Not measured | Not measured | 34.3%answered 94% | 17,805 ms | 92,100 ms | $0.788$788estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $4.422$4,422estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% · Math 89.6% | — | Chat | 81.2%79.8%–82.5% | i | 2,669 / 3,288 | Not measured | Not measured | 31.1%answered 94% | 42,585 ms† | 593,950 ms† | $4.422$4,422estimate · RTX PRO 6000 Blackwell† |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.024$24.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% · Math 56.8% | — | Decision | 52.5%50.8%–54.2% | j | 1,725 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.777 | 7.9%answered 96% | 31 ms | 36 ms | $0.024$24.19estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% · Math 55.8% | — | Decision | 52.4%50.7%–54.1% | j | 1,724 / 3,288 | 117 right of 117 kept0 mistakes · 3,171 skipped | 0.763 | 14.4%answered 93% | 68 ms | 80 ms | $0.004$4.28estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.024$23.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% · Math 55.9% | — | Decision | 52.1%50.4%–53.8% | j | 1,713 / 3,288 | 130 right of 131 kept1 wrong · 3,157 skipped | 0.782 | 15.0%answered 92% | 29 ms | 35 ms | $0.024$23.77estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$5.02estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% · Math 56.0% | — | Decision | 51.3%49.6%–53.0% | j | 1,687 / 3,288 | 71 right of 72 kept1 wrong · 3,216 skipped | 0.752 | 4.4%answered 98% | 76 ms | 104 ms | $0.005$5.02estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% · Math 55.2% | — | Decision | 51.1%49.4%–52.8% | j | 1,680 / 3,288 | 185 right of 190 kept5 wrong · 3,098 skipped | 0.769 | 18.9%answered 90% | 41 ms | 51 ms | $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.45estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 51.5% | — | Decision | 47.5%45.8%–49.2% | k | 1,561 / 3,288 | 161 right of 165 kept4 wrong · 3,123 skipped | 0.747 | 8.9%answered 95% | 46 ms | 52 ms | $0.033$33.45estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.025$25.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% · Math 50.4% | — | Decision | 47.0%45.3%–48.7% | k | 1,545 / 3,288 | 188 right of 194 kept6 wrong · 3,094 skipped | 0.743 | 7.8%answered 96% | 30 ms | 42 ms | $0.025$25.03estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.086$85.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% · Math 39.2% | — | Decision | 36.7%35.1%–38.4% | l | 1,208 / 3,288 | No answers at this threshold | 0.659 | 2.0%answered 99% | 117 ms | 146 ms | $0.086$85.83estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.36estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% · Math 38.7% | — | Decision | 35.9%34.3%–37.6% | l | 1,182 / 3,288 | 4 right of 4 kept0 mistakes · 3,284 skipped | 0.675 | 16.4%answered 89% | 36 ms | 42 ms | $0.002$2.36estimate · DGX Spark |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% · Math 36.8% | — | Decision | 35.2%33.6%–36.9% | l | 1,158 / 3,288 | 29 right of 30 kept1 wrong · 3,258 skipped | 0.637 | 23.1%answered 84% | 28 ms | 37 ms | $0.002$1.94estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.30estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% · Math 37.1% | — | Decision | 34.9%33.3%–36.5% | l | 1,147 / 3,288 | 7 right of 7 kept0 mistakes · 3,281 skipped | 0.648 | 82.5%answered 30% | 17 ms | 21 ms | $0.014$14.30estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.40estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% · Math 35.3% | — | Decision | 34.1%32.5%–35.7% | lm | 1,120 / 3,288 | 16 right of 16 kept0 mistakes · 3,272 skipped | 0.626 | 6.8%answered 95% | 51 ms | 63 ms | $0.003$3.40estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.38estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% · Math 32.1% | — | Decision | 31.1%29.5%–32.7% | mn | 1,022 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.602 | 22.7%answered 83% | 20 ms | 25 ms | $0.001$1.38estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.62estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% · Math 29.7% | — | Decision | 28.0%26.5%–29.5% | no | 920 / 3,288 | No answers at this threshold | 0.605 | 14.9%answered 89% | 24 ms | 29 ms | $0.002$1.62estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.11estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% · Math 28.2% | — | Decision | 27.8%26.3%–29.4% | o | 915 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.576 | 2.0%answered 99% | 15 ms | 17 ms | $0.001$1.11estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.80estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% · Math 28.2% | — | Decision | 27.7%26.2%–29.2% | o | 910 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.590 | 0.6%answered 100% | 26 ms | 35 ms | $0.002$1.80estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.027$26.72estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% · Math 28.7% | — | Decision | 26.5%25.0%–28.1% | o | 872 / 3,288 | No answers at this threshold | 0.573 | 5.0%answered 96% | 35 ms | 42 ms | $0.027$26.72estimate · RTX PRO 6000 Blackwell |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% · Math 22.7% | — | Decision | 23.0%21.6%–24.4% | p | 755 / 3,288 | No answers at this threshold | 0.511 | 15.6%answered 87% | 12 ms | 23 ms | $0.001$1.01estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.83estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% · Math 20.6% | — | Decision | 21.5%20.1%–22.9% | p | 706 / 3,288 | No answers at this threshold | 0.520 | 73.9%answered 32% | 11 ms | 14 ms | $0.001$0.83estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
StartLux Base Labs Apertix AI Fastino Cloudflare newsbang Blockbrain Labs PelaAI tattrongvu Patrick-18 Valen-Team EldanRing eCloud Technology Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
StartLux-Decision-v1.1-9B-Instant: 54.4% at $0.009 per 1,000 questions ($8.92 per 1M) (estimate)
Elephant-1: 56.2% at $0.010 per 1,000 questions ($9.79 per 1M)
Cerno 9B v1.2: 51.7% at $0.011 per 1,000 questions ($10.82 per 1M) (estimate)
GLiDE 9B: 54.5% at $0.011 per 1,000 questions ($11.45 per 1M) (estimate)
Clef-Flash: 46.1% at $0.013 per 1,000 questions ($12.90 per 1M)
Adjudex 9B v5: 56.7% at $0.028 per 1,000 questions ($28.19 per 1M) (estimate)
Cygnet: 42.6% at $0.029 per 1,000 questions ($29.34 per 1M)
KnowLine-9B-Gen1: 54.5% at $0.030 per 1,000 questions ($29.87 per 1M) (estimate)
Jeffan-9B-1211: 55.8% at $0.032 per 1,000 questions ($32.40 per 1M) (estimate)
Dexy-Decision-9B-V2: 54.2% at $0.033 per 1,000 questions ($32.61 per 1M) (estimate)
KnowLine-9B-Gen2: 54.6% at $0.034 per 1,000 questions ($34.08 per 1M) (estimate)
KnowLine-12B-Gen1: 60.9% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Valen-Decision-1.5-9B: 51.6% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Winnow-12B: 44.4% at $0.043 per 1,000 questions ($42.91 per 1M) (estimate)
Erk-14B (off · chat): 78.3% at $0.308 per 1,000 questions ($308 per 1M) (estimate)
Gemma 4 E4B (on · chat): 92.9% at $0.375 per 1,000 questions ($375 per 1M) (estimate)
Gemma 4 E4B (chat): 93.0% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 94.2% at $0.549 per 1,000 questions ($549 per 1M) (estimate)
Gemma 4 12B Unified (chat): 95.0% at $0.749 per 1,000 questions ($749 per 1M) (estimate)
Qwen3-14B (off · chat): 85.4% at $0.840 per 1,000 questions ($840 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 93.9% at $0.855 per 1,000 questions ($855 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 83.9% at $1.913 per 1,000 questions ($1,913 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
StartLux Base Labs Apertix AI Fastino Cloudflare newsbang Blockbrain Labs PelaAI tattrongvu Patrick-18 Valen-Team EldanRing eCloud Technology Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| StartLux-Decision-v1.1-9B-Instant Pareto Open · 9.0B | — | Decision | $0.009$8.92estimate · DGX Spark | $8.92 estimate | $0.02932 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Elephant-1 Pareto Open · 9.0B | — | Decision | $0.010$9.79 list | $9.79 | $0.0322 | $0.058$58.27estimate · RTX PRO 6000 Blackwell | maker's list price |
| Cerno 9B v1.2 Open · 9.0B | — | Decision | $0.011$10.82estimate · DGX Spark | $10.82 estimate | $0.03559 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Open · 7.9B | — | Decision | $0.011$11.45estimate · DGX Spark | $11.45 estimate | $0.03766 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Clef-Flash Open · 9.0B | — | Decision | $0.013$12.90 list | $12.90 | $0.04241 | $0.075$75.10estimate · RTX PRO 6000 Blackwell | maker's list price |
| Adjudex 9B v5 Pareto Open · 9.0B | — | Decision | $0.028$28.19estimate · RTX PRO 6000 Blackwell | $28.19 estimate | $0.09268 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | Decision | $0.029$29.34 list | $29.34 | $0.09647 | $0.029$28.61estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen1 Open · 9.0B | — | Decision | $0.030$29.87estimate · RTX PRO 6000 Blackwell | $29.87 estimate | $0.09822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Open · 9.0B | — | Decision | $0.032$32.40estimate · RTX PRO 6000 Blackwell | $32.40 estimate | $0.1065 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | Decision | $0.033$32.61estimate · RTX PRO 6000 Blackwell | $32.61 estimate | $0.1072 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | Decision | $0.034$34.08estimate · RTX PRO 6000 Blackwell | $34.08 estimate | $0.1121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Pareto Open · 12B | — | Decision | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | Decision | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | Decision | $0.043$42.91estimate · RTX PRO 6000 Blackwell | $42.91 estimate | $0.1411 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Erk-14B Pareto off · chat Open · 15B | — | Chat | $0.308$308estimate · RTX PRO 6000 Blackwell | $308.39 estimate | $1.014 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat Open · 8.0B | — | Chat | $0.375$375estimate · DGX Spark | $375.43 estimate | $1.234 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E4B Pareto chat Open · 8.0B | — | Chat | $0.467$467estimate · RTX PRO 6000 Blackwell | $466.58 estimate | $1.534 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat · 32k cap · sampled Open · 8.0B | — | Chat | $0.549$549estimate · RTX PRO 6000 Blackwell | $548.55 estimate | $1.795 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified Pareto chat Open · 12B | — | Chat | $0.749$749estimate · RTX PRO 6000 Blackwell | $749.31 estimate | $2.464 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | Chat | $0.840$840estimate · RTX PRO 6000 Blackwell | $839.76 estimate | $2.761 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | Chat | $0.855$855estimate · RTX PRO 6000 Blackwell | $854.91 estimate | $2.811 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | Chat | $1.913$1,913estimate · RTX PRO 6000 Blackwell† | $1,913.02 estimate† | $6.29 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.749$749estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% · Math 98.8% | — | Chat | 95.0%94.2%–95.7% | a | 3,125 / 3,288 | Not measured | Not measured | 15.3%answered 99% | 22,345 ms | 53,390 ms | $0.749$749estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.549$549estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% · Math 98.5% | — | Chat | 94.2%94.0%–94.4% min–max of 3 seeds | ab | 3,104 / 3,288seed 0 run | Not measured | Not measured | 54.1%answered 96% | 13,945 ms | 51,987 ms | $0.549$549estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.855$855estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% · Math 98.5% | — | Chat | 93.9%93.0%–94.7% | abc | 3,087 / 3,288 | Not measured | Not measured | Not measured | 25,530 ms | 77,380 ms | $0.855$855estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% · Math 98.1% | — | Chat | 93.0%92.1%–93.8% | bc | 3,058 / 3,288 | Not measured | Not measured | 51.0%answered 95% | 13,150 ms | 44,990 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.375$375estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% · Math 98.2% | — | Chat | 92.9%92.0%–93.7% | c | 3,054 / 3,288 | Not measured | Not measured | 43.7%answered 96% | 131,535 ms | 392,620 ms | $0.375$375estimate · DGX Spark |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.840$840estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% · Math 95.8% | — | Chat | 85.4%84.1%–86.5% | d | 2,807 / 3,288 | Not measured | Not measured | 20.9%answered 96% | 9,380 ms | 35,110 ms | $0.840$840estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $1.913$1,913estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% · Math 96.3% | — | Chat | 83.9%82.6%–85.1% | d | 2,759 / 3,288 | Not measured | Not measured | 55.9%answered 88% | 20,095 ms† | 230,570 ms† | $1.913$1,913estimate · RTX PRO 6000 Blackwell† |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.308$308estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% · Math 90.8% | — | Chat | 78.3%76.8%–79.6% | e | 2,573 / 3,288 | Not measured | Not measured | 32.3%answered 92% | 8,740 ms | 16,420 ms | $0.308$308estimate · RTX PRO 6000 Blackwell |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% · Math 66.3% | — | Decision | 60.9%59.2%–62.5% | f | 2,002 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.810 | 4.5%answered 98% | 51 ms | 76 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.028$28.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 62.1% | — | Decision | 56.7%55.0%–58.4% | g | 1,865 / 3,288 | 251 right of 253 kept2 wrong · 3,035 skipped | 0.802 | 7.0%answered 97% | 38 ms | 45 ms | $0.028$28.19estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 61.3% | — | Decision | 56.2%54.5%–57.9% | g | 1,849 / 3,288 | 235 right of 236 kept1 wrong · 3,052 skipped | 0.797 | 69.4%answered 60% | 80 ms | 94 ms | $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.032$32.40estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% · Math 59.8% | — | Decision | 55.8%54.1%–57.5% | g | 1,836 / 3,288 | 74 right of 74 kept0 mistakes · 3,214 skipped | 0.798 | 13.5%answered 94% | 39 ms | 53 ms | $0.032$32.40estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.034$34.08estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% · Math 60.1% | — | Decision | 54.6%52.9%–56.3% | g | 1,794 / 3,288 | No answers at this threshold | 0.786 | 10.4%answered 95% | 42 ms | 57 ms | $0.034$34.08estimate · RTX PRO 6000 Blackwell |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.011$11.45estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% · Math 57.9% | — | Decision | 54.5%52.8%–56.2% | ghi | 1,791 / 3,288 | 105 right of 105 kept0 mistakes · 3,183 skipped | 0.786 | 26.9%answered 86% | 177 ms | 222 ms | $0.011$11.45estimate · DGX Spark |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.030$29.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% · Math 59.8% | — | Decision | 54.5%52.8%–56.2% | gh | 1,791 / 3,288 | 153 right of 153 kept0 mistakes · 3,135 skipped | 0.782 | 3.4%answered 98% | 38 ms | 55 ms | $0.030$29.87estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.009$8.92estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% · Math 58.4% | — | Decision | 54.4%52.7%–56.1% | ghi | 1,788 / 3,288 | 63 right of 63 kept0 mistakes · 3,225 skipped | 0.779 | 9.6%answered 95% | 142 ms | 165 ms | $0.009$8.92estimate · DGX Spark |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.033$32.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% · Math 58.8% | — | Decision | 54.2%52.5%–55.9% | ghi | 1,782 / 3,288 | 263 right of 266 kept3 wrong · 3,022 skipped | 0.795 | 6.1%answered 97% | 46 ms | 51 ms | $0.033$32.61estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.011$10.82estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% · Math 56.0% | — | Decision | 51.7%50.0%–53.4% | hi | 1,699 / 3,288 | 270 right of 281 kept11 wrong · 3,007 skipped | 0.776 | 3.1%answered 99% | 163 ms | 240 ms | $0.011$10.82estimate · DGX Spark |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% · Math 55.3% | — | Decision | 51.6%49.9%–53.3% | i | 1,696 / 3,288 | 165 right of 167 kept2 wrong · 3,121 skipped | 0.765 | 7.1%answered 96% | 59 ms | 65 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% · Math 49.8% | — | Decision | 46.1%44.4%–47.8% | j | 1,515 / 3,288 | No answers at this threshold | 0.707 | 6.9%answered 96% | 103 ms | 131 ms | $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.043$42.91estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 46.9% | — | Decision | 44.4%42.7%–46.1% | jk | 1,461 / 3,288 | 52 right of 53 kept1 wrong · 3,235 skipped | 0.737 | 17.1%answered 90% | 55 ms | 72 ms | $0.043$42.91estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% · Math 44.3% | — | Decision | 42.6%40.9%–44.3% | k | 1,401 / 3,288 | No answers at this threshold | 0.729 | 7.3%answered 96% | 38 ms | 44 ms | $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router Sthānika AI PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Perplexity Decider 27B v1.2 (API): 64.1% at $0.005 per 1,000 questions ($5.38 per 1M)
GLiDE (v1.2): 64.8% at $0.010 per 1,000 questions ($9.71 per 1M)
Drex 2.0: 53.1% at $0.012 per 1,000 questions ($12.17 per 1M)
Drex Max: 64.4% at $0.016 per 1,000 questions ($16.31 per 1M)
Turion Jev 31B: 63.3% at $0.020 per 1,000 questions ($20.01 per 1M) (estimate)
Mammoth-1: 67.3% at $0.028 per 1,000 questions ($27.98 per 1M)
GLiDE 27B: 67.3% at $0.029 per 1,000 questions ($28.55 per 1M) (estimate)
d3: 65.0% at $0.046 per 1,000 questions ($46.28 per 1M) (estimate)
Sieve 31B v3: 70.5% at $0.047 per 1,000 questions ($47.29 per 1M) (estimate)
KnowLine-31B-Gen1.5: 71.1% at $0.052 per 1,000 questions ($51.85 per 1M) (estimate)
OneJev-27B: 49.3% at $0.069 per 1,000 questions ($69.42 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 66.4% at $0.071 per 1,000 questions ($71.10 per 1M) (estimate)
Clef: 47.6% at $0.081 per 1,000 questions ($81.46 per 1M)
KnowLine-31B-Gen2: 73.0% at $0.093 per 1,000 questions ($92.98 per 1M) (estimate)
d3-ultra: 65.7% at $0.096 per 1,000 questions ($95.71 per 1M) (estimate)
Copernicus-31B: 71.0% at $0.102 per 1,000 questions ($102 per 1M) (estimate)
Karpo Decision 31B: 74.4% at $0.118 per 1,000 questions ($118 per 1M) (estimate)
Open-Jev-27B-v1.1: 50.5% at $0.212 per 1,000 questions ($212 per 1M) (estimate)
Gemma 4 31B (chat): 97.3% at $0.775 per 1,000 questions ($775 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 63.6% at $1.025 per 1,000 questions ($1,025 per 1M)
Qwen3.8-27B (low · chat): 97.4% at $2.020 per 1,000 questions ($2,020 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 94.8% at $3.205 per 1,000 questions ($3,205 per 1M) (estimate)
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router Sthānika AI PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | Decision | $0.005$5.38 | $5.38 | $0.01768 | — | provider price × reported usage |
| GLiDE Pareto v1.2 Open · 27B | — | Decision | $0.010$9.71 list | $9.71 | $0.03193 | $0.066$66.05estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | Decision | $0.012$12.17 | $12.17 | $0.04001 | — | provider price × reported usage |
| Drex Max Open · 27B | — | Decision | $0.016$16.31 list | $16.31 | $0.05362 | $0.064$63.74estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | Decision | $0.020$20.01estimate · DGX Spark | $20.01 estimate | $0.0658 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Mammoth-1 Pareto Open · 27B | — | Decision | $0.028$27.98 list | $27.98 | $0.09199 | $0.120$120estimate · RTX PRO 6000 Blackwell | maker's list price |
| GLiDE 27B Pareto Open · 27B | — | Decision | $0.029$28.55estimate · DGX Spark | $28.55 estimate | $0.09388 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d3 Open · 26B | — | Decision | $0.046$46.28estimate · RTX PRO 6000 Blackwell | $46.28 estimate | $0.1522 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Sieve 31B v3 Pareto Open · 31B | — | Decision | $0.047$47.29estimate · DGX Spark | $47.29 estimate | $0.1555 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | Decision | $0.052$51.85estimate · DGX Spark | $51.85 estimate | $0.1705 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | Decision | $0.069$69.42estimate · RTX PRO 6000 Blackwell | $69.42 estimate | $0.2283 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | Decision | $0.071$71.10estimate · RTX PRO 6000 Blackwell | $71.10 estimate | $0.2338 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | Decision | $0.081$81.46 list | $81.46 | $0.2679 | $0.135$135estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-31B-Gen2 Pareto Open · 31B | — | Decision | $0.093$92.98estimate · RTX PRO 6000 Blackwell | $92.98 estimate | $0.3057 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | Decision | $0.096$95.71estimate · RTX PRO 6000 Blackwell | $95.71 estimate | $0.3147 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | Decision | $0.102$102estimate · RTX PRO 6000 Blackwell | $102.03 estimate | $0.3355 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | Decision | $0.118$118estimate · RTX PRO 6000 Blackwell | $118.01 estimate | $0.388 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | Decision | $0.212$212estimate · RTX PRO 6000 Blackwell | $212.04 estimate | $0.6972 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B Pareto chat Open · 31B | — | Chat | $0.775$775estimate · RTX PRO 6000 Blackwell | $774.76 estimate | $2.547 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | Decision | $1.025$1,025 | $1,025.11 | $3.371 | — | provider price × reported usage |
| Qwen3.8-27B Pareto low · chat Open · 27B | — | Chat | $2.020$2,020estimate · RTX PRO 6000 Blackwell | $2,019.68 estimate | $6.641 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | Chat | $3.205$3,205estimate · RTX PRO 6000 Blackwell | $3,205.06 estimate | $10.54 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $2.020$2,020estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | Chat | 97.4%96.8%–97.9% | a | 3,204 / 3,288 | Not measured | Not measured | 33.9%answered 99% | 11,200 ms | 39,560 ms | $2.020$2,020estimate · RTX PRO 6000 Blackwell |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.775$775estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% · Math 99.4% | — | Chat | 97.3%96.7%–97.8% | a | 3,200 / 3,288 | Not measured | Not measured | 17.0%answered 99% | 12,575 ms | 25,280 ms | $0.775$775estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $3.205$3,205estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% · Math 99.0% | — | Chat | 94.8%94.0%–95.5% | b | 3,117 / 3,288 | Not measured | Not measured | 44.0%answered 97% | 42,425 ms | 432,510 ms | $3.205$3,205estimate · RTX PRO 6000 Blackwell |
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.118$118estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% · Math 81.6% | — | Decision | 74.4%72.9%–75.9% | c | 2,447 / 3,288 | 1,006 right of 1,009 kept3 wrong · 2,279 skipped | 0.882 | 5.7%answered 98% | 162 ms | 198 ms | $0.118$118estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.093$92.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% · Math 80.3% | — | Decision | 73.0%71.4%–74.5% | c | 2,399 / 3,288 | 523 right of 524 kept1 wrong · 2,764 skipped | 0.882 | 0.1%answered 100% | 116 ms | 177 ms | $0.093$92.98estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.052$51.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% · Math 77.4% | — | Decision | 71.1%69.5%–72.6% | d | 2,338 / 3,288 | 553 right of 554 kept1 wrong · 2,734 skipped | 0.870 | 0.7%answered 100% | 811 ms | 1,111 ms | $0.052$51.85estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.102$102estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% · Math 77.3% | — | Decision | 71.0%69.4%–72.5% | d | 2,333 / 3,288 | 384 right of 384 kept0 mistakes · 2,904 skipped | 0.873 | 1.8%answered 99% | 128 ms | 218 ms | $0.102$102estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.047$47.29estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% · Math 77.1% | — | Decision | 70.5%68.9%–72.1% | d | 2,319 / 3,288 | 294 right of 296 kept2 wrong · 2,992 skipped | 0.865 | 3.8%answered 99% | 676 ms | 1,173 ms | $0.047$47.29estimate · DGX Spark |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.029$28.55estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% · Math 72.0% | — | Decision | 67.3%65.7%–68.9% | e | 2,214 / 3,288 | 272 right of 274 kept2 wrong · 3,014 skipped | 0.852 | 31.9%answered 89% | 450 ms | 587 ms | $0.029$28.55estimate · DGX Spark |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 74.0% | — | Decision | 67.3%65.7%–68.9% | ef | 2,213 / 3,288 | 518 right of 520 kept2 wrong · 2,768 skipped | 0.868 | 71.4%answered 69% | 150 ms | 232 ms | $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.071$71.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% · Math 72.5% | — | Decision | 66.4%64.8%–68.0% | efg | 2,184 / 3,288 | 385 right of 385 kept0 mistakes · 2,903 skipped | 0.844 | 13.5%answered 95% | 97 ms | 125 ms | $0.071$71.10estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.096$95.71estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% · Math 71.5% | — | Decision | 65.7%64.1%–67.3% | efgh | 2,161 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.835 | 7.5%answered 97% | 129 ms | 168 ms | $0.096$95.71estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.046$46.28estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 70.4% | — | Decision | 65.0%63.3%–66.6% | fgh | 2,136 / 3,288 | 216 right of 217 kept1 wrong · 3,071 skipped | 0.858 | 9.3%answered 97% | 57 ms | 86 ms | $0.046$46.28estimate · RTX PRO 6000 Blackwell |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% · Math 69.9% | — | Decision | 64.8%63.2%–66.5% | fgh | 2,132 / 3,288 | 176 right of 176 kept0 mistakes · 3,112 skipped | 0.843 | 42.3%answered 83% | 81 ms | 136 ms | $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% · Math 70.3% | — | Decision | 64.4%62.8%–66.0% | gh | 2,118 / 3,288 | 866 right of 884 kept18 wrong · 2,404 skipped | 0.841 | 5.7%answered 98% | 81 ms | 130 ms | $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.005$5.38 per 1,000 questionsper 1M questions Logic 53.3% · Math 69.8% | — | Decision | 64.1%62.5%–65.8% | h | 2,109 / 3,288 | 409 right of 411 kept2 wrong · 2,877 skipped | 0.849 | 10.3%answered 96% | 230 ms | 439 ms | $0.005$5.38 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.025$1,025 per 1,000 questionsper 1M questions Logic 52.5% · Math 69.3% | with web | Decision | 63.6%61.9%–65.2% | h | 2,090 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.854 | 25.8%answered 90% | 430 ms | 707 ms | $1.025$1,025 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.020$20.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% · Math 69.0% | — | Decision | 63.3%61.6%–64.9% | h | 2,081 / 3,288 | 50 right of 50 kept0 mistakes · 3,238 skipped | 0.844 | 1.9%answered 99% | 301 ms | 448 ms | $0.020$20.01estimate · DGX Spark |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.012$12.17 per 1,000 questionsper 1M questions Logic 41.9% · Math 59.0% | — | Decision | 53.1%51.4%–54.8% | i | 1,747 / 3,288 | 207 right of 207 kept0 mistakes · 3,081 skipped | 0.789 | 28.1%answered 85% | 339 ms | 472 ms | $0.012$12.17 |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.212$212estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% · Math 54.6% | — | Decision | 50.5%48.7%–52.2% | ij | 1,659 / 3,288 | No answers at this threshold | 0.759 | 14.5%answered 92% | 250 ms | 482 ms | $0.212$212estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.069$69.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% · Math 53.8% | — | Decision | 49.3%47.6%–51.0% | j | 1,620 / 3,288 | 30 right of 30 kept0 mistakes · 3,258 skipped | 0.770 | 1.6%answered 99% | 80 ms | 134 ms | $0.069$69.42estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% · Math 51.1% | — | Decision | 47.6%45.9%–49.3% | j | 1,564 / 3,288 | 18 right of 18 kept0 mistakes · 3,270 skipped | 0.747 | 2.4%answered 99% | 185 ms | 216 ms | $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
REAX Logits MiniMax Z.ai Xiaomi DeepSeek Thinking Machines
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
REAX 1.3: 71.6% at $0.148 per 1,000 questions ($148 per 1M) (estimate)
Logits Sigma-1 Duo: 69.3% at $0.165 per 1,000 questions ($165 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 96.8% at $0.342 per 1,000 questions ($342 per 1M)
GLM-5.3-Flash (high · chat): 96.9% at $0.471 per 1,000 questions ($471 per 1M)
MiMo-V2.6-Flash (on · chat): 98.0% at $0.615 per 1,000 questions ($615 per 1M)
DeepSeek-V4.1-Flash (high · chat · 1017): 98.4% at $0.713 per 1,000 questions ($713 per 1M)
Inkling 2 Small (high · chat): 98.2% at $1.830 per 1,000 questions ($1,830 per 1M)
Inkling-Small (high · chat · Jul 2026): 98.2% at $2.506 per 1,000 questions ($2,506 per 1M)
REAX Logits MiniMax Z.ai Xiaomi DeepSeek Thinking Machines
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| REAX 1.3 Pareto Open · 57B | — | Decision | $0.148$148estimate · RTX PRO 6000 Blackwell | $148.30 estimate | $0.4876 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | Decision | $0.165$165estimate · RTX PRO 6000 Blackwell | $164.92 estimate | $0.5423 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | Chat | $0.342$342 | $342.49 | $1.126 | — | provider price × reported usage |
| GLM-5.3-Flash Pareto high · chat Open · 320B MoE · 18B active | — | Chat | $0.471$471 | $471.05 | $1.549 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | Chat | $0.615$615 | $615.00 | $2.022 | — | provider price × reported usage |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | Chat | $0.713$713 | $713.10 | $2.345 | — | provider price × reported usage |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | Chat | $1.830$1,830 | $1,830.16 | $6.018 | — | provider price × reported usage |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | Chat | $2.506$2,506 | $2,505.67 | $8.239 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: DeepSeek-V4-Flash (chat) (missing Logic); Qwen3.5-397B-A17B (on · chat) (missing Math).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.713$713 per 1,000 questionsper 1M questions Logic 96.2% · Math 99.6% | — | Chat | 98.4%98.0%–98.8% | a | 3,237 / 3,288 | Not measured | Not measured | 6.3%answered 100% | 2,270 ms | 14,870 ms | $0.713$713 |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $1.830$1,830 per 1,000 questionsper 1M questions Logic 95.6% · Math 99.5% | — | Chat | 98.2%97.7%–98.6% | a | 3,229 / 3,288 | Not measured | Not measured | 41.7%answered 99% | 2,870 ms | 35,530 ms | $1.830$1,830 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $2.506$2,506 per 1,000 questionsper 1M questions Logic 95.4% · Math 99.6% | — | Chat | 98.2%97.7%–98.6% | a | 3,228 / 3,288 | Not measured | Not measured | 50.6%answered 99% | 2,750 ms | 34,100 ms | $2.506$2,506 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.615$615 per 1,000 questionsper 1M questions Logic 95.1% · Math 99.4% | — | Chat | 98.0%97.4%–98.4% | a | 3,221 / 3,288 | Not measured | Not measured | 27.6%answered 99% | 10,590 ms | 53,970 ms | $0.615$615 |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.471$471 per 1,000 questionsper 1M questions Logic 93.2% · Math 98.8% | — | Chat | 96.9%96.2%–97.4% | b | 3,185 / 3,288 | Not measured | Not measured | 7.4%answered 100% | 4,640 ms | 25,900 ms | $0.471$471 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.342$342 per 1,000 questionsper 1M questions Logic 94.0% · Math 98.2% | — | Chat | 96.8%96.1%–97.3% | b | 3,182 / 3,288 | Not measured | Not measured | 38.9%answered 99% | 2,170 ms | 15,760 ms | $0.342$342 |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.148$148estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% · Math 78.0% | — | Decision | 71.6%70.0%–73.1% | c | 2,354 / 3,288 | 229 right of 229 kept0 mistakes · 3,059 skipped | 0.878 | 3.8%answered 99% | 200 ms | 260 ms | $0.148$148estimate · RTX PRO 6000 Blackwell |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.165$165estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% · Math 75.2% | — | Decision | 69.3%67.7%–70.8% | d | 2,277 / 3,288 | 23 right of 23 kept0 mistakes · 3,265 skipped | 0.868 | 4.4%answered 99% | 207 ms | 324 ms | $0.165$165estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino Meta
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
celeris-1.1-decision: 62.1% at $0.001 per 1,000 questions ($1.13 per 1M)
d1.1: 58.6% at $0.004 per 1,000 questions ($3.98 per 1M)
d1: 50.5% at $0.004 per 1,000 questions ($3.98 per 1M)
Mercury Decide: 50.6% at $0.005 per 1,000 questions ($5.46 per 1M)
d1-ultra: 67.5% at $0.008 per 1,000 questions ($8.37 per 1M)
Microsoft-Decision-1: 54.3% at $0.009 per 1,000 questions ($9.07 per 1M)
SubQ Snap 1.1: 63.0% at $0.011 per 1,000 questions ($10.75 per 1M)
celeris-1-decision: 62.2% at $0.011 per 1,000 questions ($11.25 per 1M)
Drex Max v1.1: 64.8% at $0.016 per 1,000 questions ($16.13 per 1M)
Jev 1.14: 59.3% at $0.021 per 1,000 questions ($21.27 per 1M)
Jev 1.13: 58.0% at $0.021 per 1,000 questions ($21.27 per 1M)
Gemini Decision 1 (20261111): 64.1% at $0.027 per 1,000 questions ($27.07 per 1M)
GLiDE (API): 65.2% at $0.036 per 1,000 questions ($36.42 per 1M)
Muse Ember (max · chat): 98.2% at $0.863 per 1,000 questions ($863 per 1M)
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino Meta
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| celeris-1.1-decision Pareto Closed · Efficient | — | Decision | $0.001$1.13 | $1.13 | $0.003701 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | Decision | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1 Closed · Efficient | — | Decision | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | Decision | $0.005$5.46 | $5.46 | $0.01794 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | Decision | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | Decision | $0.009$9.07 | $9.07 | $0.02983 | — | provider price × reported usage |
| SubQ Snap 1.1 Closed · Efficient | — | Decision | $0.011$10.75 | $10.75 | $0.03535 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | Decision | $0.011$11.25 | $11.25 | $0.03701 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | Decision | $0.016$16.13 | $16.13 | $0.05303 | — | provider price × reported usage |
| Jev 1.14 Closed · Efficient | — | Decision | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | Decision | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | Decision | $0.027$27.07 | $27.07 | $0.08901 | — | provider price × reported usage |
| GLiDE API Closed · Efficient | — | Decision | $0.036$36.42 | $36.42 | $0.1197 | — | provider price × reported usage |
| Muse Ember Pareto max · chat Closed · Efficient | — | Chat | $0.863$863 | $862.71 | $2.837 | — | provider price × reported usage |
| GPT-6 Luna chat Closed · Efficient | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 3,276 of 3,288); shown once.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: GPT-6 Luna (max · chat) (missing Logic and Math); Claude Haiku 5.5 (max · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.863$863 per 1,000 questionsper 1M questions Logic 95.5% · Math 99.6% | — | Chat | 98.2%97.7%–98.6% | a | 3,228 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 1,460 ms | 6,300 ms | $0.863$863 |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% · Math 93.7% | — | Chat | 81.4%80.1%–82.7% | b | 2,678 / 3,288 | Not measured | Not measured | Not measured | 3,830 ms | 5,300 ms | $0.000$0.00 |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 54.9% · Math 74.1% | — | Decision | 67.5%65.9%–69.1% | c | 2,221 / 3,288 | 233 right of 233 kept0 mistakes · 3,055 skipped | 0.852 | 89.1%answered 52% | 180 ms | 426 ms | $0.008$8.37 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.036$36.42 per 1,000 questionsper 1M questions Logic 54.5% · Math 70.8% | — | Decision | 65.2%63.6%–66.8% | cd | 2,144 / 3,288 | 213 right of 213 kept0 mistakes · 3,075 skipped | 0.852 | 49.2%answered 80% | 266 ms | 284 ms | $0.036$36.42 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.016$16.13 per 1,000 questionsper 1M questions Logic 54.8% · Math 70.1% | — | Decision | 64.8%63.2%–66.5% | de | 2,132 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.846 | 48.3%answered 79% | 327 ms | 518 ms | $0.016$16.13 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.027$27.07 per 1,000 questionsper 1M questions Logic 50.0% · Math 71.3% | — | Decision | 64.1%62.4%–65.7% | de | 2,106 / 3,288 | 257 right of 257 kept0 mistakes · 3,031 skipped | 0.838 | 3.0%answered 99% | 227 ms | 347 ms | $0.027$27.07 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.011$10.75 per 1,000 questionsper 1M questions Logic 52.4% · Math 68.6% | — | Decision | 63.0%61.4%–64.7% | de | 2,072 / 3,288 | 430 right of 433 kept3 wrong · 2,855 skipped | 0.827 | 58.3%answered 73% | 382 ms | 391 ms | $0.011$10.75 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.011$11.25 per 1,000 questionsper 1M questions Logic 47.3% · Math 70.0% | — | Decision | 62.2%60.6%–63.9% | e | 2,046 / 3,288 | No answers at this threshold | 0.829 | 87.2%answered 50% | 388 ms | 392 ms | $0.011$11.25 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$1.13 per 1,000 questionsper 1M questions Logic 47.3% · Math 69.9% | — | Decision | 62.1%60.5%–63.8% | e | 2,043 / 3,288 | No answers at this threshold | 0.830 | 86.7%answered 51% | 381 ms | 388 ms | $0.001$1.13 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 48.2% · Math 65.1% | — | Decision | 59.3%57.6%–61.0% | f | 1,951 / 3,288 | 263 right of 263 kept0 mistakes · 3,025 skipped | 0.810 | 15.4%answered 93% | 164 ms | 188 ms | $0.021$21.27 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 46.0% · Math 65.2% | — | Decision | 58.6%56.9%–60.3% | f | 1,928 / 3,288 | 231 right of 231 kept0 mistakes · 3,057 skipped | 0.811 | 86.0%answered 46% | 174 ms | 285 ms | $0.004$3.98 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 46.3% · Math 64.1% | — | Decision | 58.0%56.3%–59.7% | f | 1,907 / 3,288 | 313 right of 313 kept0 mistakes · 2,975 skipped | 0.817 | 10.2%answered 96% | 166 ms | 205 ms | $0.021$21.27 |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.009$9.07 per 1,000 questionsper 1M questions Logic 44.4% · Math 59.4% | — | Decision | 54.3%52.6%–56.0% | g | 1,784 / 3,288 | 47 right of 47 kept0 mistakes · 3,241 skipped | 0.769 | 11.4%answered 94% | 312 ms | 360 ms | $0.009$9.07 |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.005$5.46 per 1,000 questionsper 1M questions Logic 45.1% · Math 53.5% | — | Decision | 50.6%48.9%–52.3% | h | 1,664 / 3,288 | 356 right of 372 kept16 wrong · 2,916 skipped | 0.777 | 10.1%answered 95% | 293 ms | 349 ms | $0.005$5.46 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 40.8% · Math 55.6% | — | Decision | 50.5%48.8%–52.3% | h | 1,662 / 3,288 | 207 right of 208 kept1 wrong · 3,080 skipped | 0.757 | 6.4%answered 97% | 185 ms | 292 ms | $0.004$3.98 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Sonnet 5.5 (chat): 98.2% at $7.042 per 1,000 questions ($7,042 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 Pareto chat Closed · Workhorse | — | Chat | $7.042$7,042 API list-price equivalent | $7,042.33 | $23.16 | — | Claude subscription |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $7.042$7,042 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% · Math 99.7% | — | Chat | 98.2%97.7%–98.6% | a | 3,230 / 3,288 | Not measured | Not measured | 43.7%answered 99% | 4,030 ms | 10,080 ms | $7.042$7,042 API list-price equivalent |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Opus 5.5 (chat): 98.5% at $13.179 per 1,000 questions ($13,179 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 Pareto chat Closed · Flagship | — | Chat | $13.179$13,179 API list-price equivalent | $13,179.29 | $43.33 | — | Claude subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 1 setup not yet: Claude Fable 5.2 (chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $13.179$13,179 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% · Math 99.8% | — | Chat | 98.5%98.1%–98.9% | a | 3,240 / 3,288 | Not measured | Not measured | 41.3%answered 99% | 3,600 ms | 8,380 ms | $13.179$13,179 API list-price equivalent |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% · Math 99.7% | — | Chat | 98.2%97.7%–98.6% | b | 3,228 / 3,288 | Not measured | Not measured | 15.7%answered 100% | 4,940 ms | 8,620 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Liquid AI Alibaba
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
d1.2: 59.9% at $0.004 per 1,000 questions ($3.98 per 1M)
d1.2-ultra: 63.6% at $0.008 per 1,000 questions ($8.37 per 1M)
Qwen3.8 Flash (off · chat): 97.6% at $0.732 per 1,000 questions ($732 per 1M)
Qwen4-Max (high · chat): 98.6% at $2.919 per 1,000 questions ($2,919 per 1M)
Liquid AI Alibaba
Cost and its basis
| Setup | Web | Models | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|---|
| d1.2 Pareto Closed · Undisclosed | — | Decision | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1.2-ultra Pareto Closed · Undisclosed | — | Decision | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
| Qwen3.8 Flash Pareto off · chat Closed · Undisclosed | — | Chat | $0.732$732 | $732.01 | $2.407 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | Chat | $2.919$2,919 | $2,919.14 | $9.598 | — | provider price × reported usage |
| GPT-6 Astra chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | Chat | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | Chat | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: Grok 4.7 (chat) (missing Logic); Gemini 4 Carbon (high · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Models | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% · Math 99.6% | — | Chat | 98.6%98.2%–99.0% | a | 3,243 / 3,288 | Not measured | Not measured | 60.9%answered 99% | 5,580 ms | 9,580 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% · Math 99.6% | — | Chat | 98.6%98.1%–98.9% | a | 3,242 / 3,288 | Not measured | Not measured | 53.3%answered 99% | 16,350 ms | 90,630 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% · Math 99.6% | — | Chat | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 66.2%answered 99% | 5,240 ms | 10,410 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $2.919$2,919 per 1,000 questionsper 1M questions Logic 96.3% · Math 99.8% | — | Chat | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 10.7%answered 100% | 5,700 ms | 19,750 ms | $2.919$2,919 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.732$732 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.5% | — | Chat | 97.6%97.0%–98.1% | b | 3,209 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 6,780 ms | 36,290 ms | $0.732$732 |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | Chat | 97.4%96.8%–97.9% | b | 3,203 / 3,288 | Not measured | Not measured | 65.0%answered 98% | 4,440 ms | 8,290 ms | $0.000$0.00 |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 51.6% · Math 69.9% | — | Decision | 63.6%61.9%–65.2% | c | 2,091 / 3,288 | 128 right of 128 kept0 mistakes · 3,160 skipped | 0.856 | 90.2%answered 48% | 217 ms | 347 ms | $0.008$8.37 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 49.2% · Math 65.5% | — | Decision | 59.9%58.2%–61.6% | d | 1,970 / 3,288 | 169 right of 169 kept0 mistakes · 3,119 skipped | 0.812 | 66.9%answered 65% | 208 ms | 390 ms | $0.004$3.98 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other OmniJev Celeris StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Apertix AI Nace.AI Cloudflare Turion.AI TypeSafe scar-ai quincyqiang Strands Agents Google newsbang Blockbrain Labs tattrongvu Patrick-18 victorbennett Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab Poindexter Labs MachinePulse REAX Logits Zefan Cai
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 21.5% at $0.001 per 1,000 questions ($0.83 per 1M) (estimate)
GLiClass v4 Edge: 23.0% at $0.001 per 1,000 questions ($1.01 per 1M) (estimate)
OneJev1.6-0.8B: 27.8% at $0.001 per 1,000 questions ($1.11 per 1M) (estimate)
celeris-1.1-decision: 62.1% at $0.001 per 1,000 questions ($1.13 per 1M)
GLiDE 0.8B: 31.1% at $0.001 per 1,000 questions ($1.38 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 28.0% at $0.002 per 1,000 questions ($1.62 per 1M) (estimate)
KnowLine-0.8B-Gen1: 27.7% at $0.002 per 1,000 questions ($1.80 per 1M) (estimate)
GLiDE 2B: 35.2% at $0.002 per 1,000 questions ($1.94 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 35.9% at $0.002 per 1,000 questions ($2.36 per 1M) (estimate)
Perplexity Decider Flash v1.2: 51.1% at $0.003 per 1,000 questions ($2.72 per 1M)
KnowLine-2B-Gen1: 34.1% at $0.003 per 1,000 questions ($3.40 per 1M) (estimate)
d1.2: 59.9% at $0.004 per 1,000 questions ($3.98 per 1M)
d1.1: 58.6% at $0.004 per 1,000 questions ($3.98 per 1M)
d1: 50.5% at $0.004 per 1,000 questions ($3.98 per 1M)
StartLux-Decision-v1.1-4B-Instant: 52.4% at $0.004 per 1,000 questions ($4.28 per 1M) (estimate)
KnowLine 4B Gen5.5: 51.3% at $0.005 per 1,000 questions ($5.02 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API): 64.1% at $0.005 per 1,000 questions ($5.38 per 1M)
Mercury Decide: 50.6% at $0.005 per 1,000 questions ($5.46 per 1M)
d1-ultra: 67.5% at $0.008 per 1,000 questions ($8.37 per 1M)
d1.2-ultra: 63.6% at $0.008 per 1,000 questions ($8.37 per 1M)
StartLux-Decision-v1.1-9B-Instant: 54.4% at $0.009 per 1,000 questions ($8.92 per 1M) (estimate)
Microsoft-Decision-1: 54.3% at $0.009 per 1,000 questions ($9.07 per 1M)
GLiDE (v1.2): 64.8% at $0.010 per 1,000 questions ($9.71 per 1M)
Elephant-1: 56.2% at $0.010 per 1,000 questions ($9.79 per 1M)
SubQ Snap 1.1: 63.0% at $0.011 per 1,000 questions ($10.75 per 1M)
Cerno 9B v1.2: 51.7% at $0.011 per 1,000 questions ($10.82 per 1M) (estimate)
celeris-1-decision: 62.2% at $0.011 per 1,000 questions ($11.25 per 1M)
GLiDE 9B: 54.5% at $0.011 per 1,000 questions ($11.45 per 1M) (estimate)
Drex 2.0: 53.1% at $0.012 per 1,000 questions ($12.17 per 1M)
Clef-Flash: 46.1% at $0.013 per 1,000 questions ($12.90 per 1M)
d1-2B: 34.9% at $0.014 per 1,000 questions ($14.30 per 1M) (estimate)
Drex Max v1.1: 64.8% at $0.016 per 1,000 questions ($16.13 per 1M)
Drex Max: 64.4% at $0.016 per 1,000 questions ($16.31 per 1M)
Turion Jev 31B: 63.3% at $0.020 per 1,000 questions ($20.01 per 1M) (estimate)
Jev 1.14: 59.3% at $0.021 per 1,000 questions ($21.27 per 1M)
Jev 1.13: 58.0% at $0.021 per 1,000 questions ($21.27 per 1M)
Perplexity Decider Flash v1.3: 52.1% at $0.024 per 1,000 questions ($23.77 per 1M) (estimate)
vera-medium-omni: 52.5% at $0.024 per 1,000 questions ($24.19 per 1M) (estimate)
Chinese Decision Pointer 4B: 47.0% at $0.025 per 1,000 questions ($25.03 per 1M) (estimate)
Strands Decider 2B: 26.5% at $0.027 per 1,000 questions ($26.72 per 1M) (estimate)
Gemini Decision 1 (20261111): 64.1% at $0.027 per 1,000 questions ($27.07 per 1M)
Mammoth-1: 67.3% at $0.028 per 1,000 questions ($27.98 per 1M)
Adjudex 9B v5: 56.7% at $0.028 per 1,000 questions ($28.19 per 1M) (estimate)
GLiDE 27B: 67.3% at $0.029 per 1,000 questions ($28.55 per 1M) (estimate)
Cygnet: 42.6% at $0.029 per 1,000 questions ($29.34 per 1M)
KnowLine-9B-Gen1: 54.5% at $0.030 per 1,000 questions ($29.87 per 1M) (estimate)
Jeffan-9B-1211: 55.8% at $0.032 per 1,000 questions ($32.40 per 1M) (estimate)
Dexy-Decision-9B-V2: 54.2% at $0.033 per 1,000 questions ($32.61 per 1M) (estimate)
Verdin-4B-v1.1: 47.5% at $0.033 per 1,000 questions ($33.45 per 1M) (estimate)
KnowLine-9B-Gen2: 54.6% at $0.034 per 1,000 questions ($34.08 per 1M) (estimate)
GLiDE (API): 65.2% at $0.036 per 1,000 questions ($36.42 per 1M)
KnowLine-12B-Gen1: 60.9% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Valen-Decision-1.5-9B: 51.6% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Winnow-12B: 44.4% at $0.043 per 1,000 questions ($42.91 per 1M) (estimate)
d3: 65.0% at $0.046 per 1,000 questions ($46.28 per 1M) (estimate)
Sieve 31B v3: 70.5% at $0.047 per 1,000 questions ($47.29 per 1M) (estimate)
KnowLine-31B-Gen1.5: 71.1% at $0.052 per 1,000 questions ($51.85 per 1M) (estimate)
OneJev-27B: 49.3% at $0.069 per 1,000 questions ($69.42 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 66.4% at $0.071 per 1,000 questions ($71.10 per 1M) (estimate)
Clef: 47.6% at $0.081 per 1,000 questions ($81.46 per 1M)
Metask-Jev-4B: 36.7% at $0.086 per 1,000 questions ($85.83 per 1M) (estimate)
KnowLine-31B-Gen2: 73.0% at $0.093 per 1,000 questions ($92.98 per 1M) (estimate)
d3-ultra: 65.7% at $0.096 per 1,000 questions ($95.71 per 1M) (estimate)
Copernicus-31B: 71.0% at $0.102 per 1,000 questions ($102 per 1M) (estimate)
Karpo Decision 31B: 74.4% at $0.118 per 1,000 questions ($118 per 1M) (estimate)
REAX 1.3: 71.6% at $0.148 per 1,000 questions ($148 per 1M) (estimate)
Logits Sigma-1 Duo: 69.3% at $0.165 per 1,000 questions ($165 per 1M) (estimate)
Open-Jev-27B-v1.1: 50.5% at $0.212 per 1,000 questions ($212 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 63.6% at $1.025 per 1,000 questions ($1,025 per 1M)
Fastino Other OmniJev Celeris StartLux PelaAI Perplexity Liquid AI Inception Microsoft Base Labs Subquadratic Apertix AI Nace.AI Cloudflare Turion.AI TypeSafe scar-ai quincyqiang Strands Agents Google newsbang Blockbrain Labs tattrongvu Patrick-18 victorbennett Valen-Team EldanRing vLLM Semantic Router Sthānika AI Metask Lab Poindexter Labs MachinePulse REAX Logits Zefan Cai
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | $0.001$0.83estimate · DGX Spark | $0.83 estimate | $0.002719 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | $0.001$1.01estimate · DGX Spark | $1.01 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | $0.001$1.11estimate · DGX Spark | $1.11 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1.1-decision Pareto Closed · Efficient | — | $0.001$1.13 | $1.13 | $0.003701 | — | provider price × reported usage |
| GLiDE 0.8B Open · 0.8B | — | $0.001$1.38estimate · DGX Spark | $1.38 estimate | $0.004552 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | $0.002$1.62estimate · DGX Spark | $1.62 estimate | $0.00532 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | $0.002$1.80estimate · DGX Spark | $1.80 estimate | $0.005912 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Open · 1.9B | — | $0.002$1.94estimate · DGX Spark | $1.94 estimate | $0.006385 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | — | $0.002$2.36estimate · DGX Spark | $2.36 estimate | $0.007744 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Open · 4.0B | — | $0.003$2.72 list | $2.72 | $0.008936 | $0.032$31.55estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | $0.003$3.40estimate · DGX Spark | $3.40 estimate | $0.01117 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1.2 Closed · Undisclosed | — | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1 Closed · Efficient | — | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | — | $0.004$4.28estimate · DGX Spark | $4.28 estimate | $0.01407 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | $0.005$5.02estimate · DGX Spark | $5.02 estimate | $0.01649 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | $0.005$5.38 | $5.38 | $0.01768 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | $0.005$5.46 | $5.46 | $0.01794 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
| d1.2-ultra Closed · Undisclosed | — | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | — | $0.009$8.92estimate · DGX Spark | $8.92 estimate | $0.02932 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Microsoft-Decision-1 Closed · Efficient | — | $0.009$9.07 | $9.07 | $0.02983 | — | provider price × reported usage |
| GLiDE v1.2 Open · 27B | — | $0.010$9.71 list | $9.71 | $0.03193 | $0.066$66.05estimate · RTX PRO 6000 Blackwell | maker's list price |
| Elephant-1 Open · 9.0B | — | $0.010$9.79 list | $9.79 | $0.0322 | $0.058$58.27estimate · RTX PRO 6000 Blackwell | maker's list price |
| SubQ Snap 1.1 Closed · Efficient | — | $0.011$10.75 | $10.75 | $0.03535 | — | provider price × reported usage |
| Cerno 9B v1.2 Open · 9.0B | — | $0.011$10.82estimate · DGX Spark | $10.82 estimate | $0.03559 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| celeris-1-decision Closed · Efficient | — | $0.011$11.25 | $11.25 | $0.03701 | — | provider price × reported usage |
| GLiDE 9B Open · 7.9B | — | $0.011$11.45estimate · DGX Spark | $11.45 estimate | $0.03766 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | $0.012$12.17 | $12.17 | $0.04001 | — | provider price × reported usage |
| Clef-Flash Open · 9.0B | — | $0.013$12.90 list | $12.90 | $0.04241 | $0.075$75.10estimate · RTX PRO 6000 Blackwell | maker's list price |
| d1-2B Open · 1.9B | — | $0.014$14.30estimate · RTX PRO 6000 Blackwell | $14.30 estimate | $0.04703 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Drex Max v1.1 Closed · Efficient (provisional) | — | $0.016$16.13 | $16.13 | $0.05303 | — | provider price × reported usage |
| Drex Max Open · 27B | — | $0.016$16.31 list | $16.31 | $0.05362 | $0.064$63.74estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | $0.020$20.01estimate · DGX Spark | $20.01 estimate | $0.0658 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Jev 1.14 Closed · Efficient | — | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | $0.024$23.77estimate · RTX PRO 6000 Blackwell | $23.77 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Open · 4.2B | — | $0.024$24.19estimate · RTX PRO 6000 Blackwell | $24.19 estimate | $0.07954 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | $0.025$25.03estimate · RTX PRO 6000 Blackwell | $25.03 estimate | $0.08231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | $0.027$26.72estimate · RTX PRO 6000 Blackwell | $26.72 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemini Decision 1 20261111 Closed · Efficient | — | $0.027$27.07 | $27.07 | $0.08901 | — | provider price × reported usage |
| Mammoth-1 Open · 27B | — | $0.028$27.98 list | $27.98 | $0.09199 | $0.120$120estimate · RTX PRO 6000 Blackwell | maker's list price |
| Adjudex 9B v5 Open · 9.0B | — | $0.028$28.19estimate · RTX PRO 6000 Blackwell | $28.19 estimate | $0.09268 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE 27B Open · 27B | — | $0.029$28.55estimate · DGX Spark | $28.55 estimate | $0.09388 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Cygnet Open · 12B | — | $0.029$29.34 list | $29.34 | $0.09647 | $0.029$28.61estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen1 Open · 9.0B | — | $0.030$29.87estimate · RTX PRO 6000 Blackwell | $29.87 estimate | $0.09822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Open · 9.0B | — | $0.032$32.40estimate · RTX PRO 6000 Blackwell | $32.40 estimate | $0.1065 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | $0.033$32.61estimate · RTX PRO 6000 Blackwell | $32.61 estimate | $0.1072 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | $0.033$33.45estimate · RTX PRO 6000 Blackwell | $33.45 estimate | $0.11 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | $0.034$34.08estimate · RTX PRO 6000 Blackwell | $34.08 estimate | $0.1121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLiDE API Closed · Efficient | — | $0.036$36.42 | $36.42 | $0.1197 | — | provider price × reported usage |
| KnowLine-12B-Gen1 Open · 12B | — | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | $0.043$42.91estimate · RTX PRO 6000 Blackwell | $42.91 estimate | $0.1411 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3 Open · 26B | — | $0.046$46.28estimate · RTX PRO 6000 Blackwell | $46.28 estimate | $0.1522 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Sieve 31B v3 Pareto Open · 31B | — | $0.047$47.29estimate · DGX Spark | $47.29 estimate | $0.1555 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | $0.052$51.85estimate · DGX Spark | $51.85 estimate | $0.1705 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | $0.069$69.42estimate · RTX PRO 6000 Blackwell | $69.42 estimate | $0.2283 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | $0.071$71.10estimate · RTX PRO 6000 Blackwell | $71.10 estimate | $0.2338 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | $0.081$81.46 list | $81.46 | $0.2679 | $0.135$135estimate · RTX PRO 6000 Blackwell | maker's list price |
| Metask-Jev-4B Open · 4.5B | — | $0.086$85.83estimate · RTX PRO 6000 Blackwell | $85.83 estimate | $0.2822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-31B-Gen2 Pareto Open · 31B | — | $0.093$92.98estimate · RTX PRO 6000 Blackwell | $92.98 estimate | $0.3057 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | $0.096$95.71estimate · RTX PRO 6000 Blackwell | $95.71 estimate | $0.3147 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | $0.102$102estimate · RTX PRO 6000 Blackwell | $102.03 estimate | $0.3355 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | $0.118$118estimate · RTX PRO 6000 Blackwell | $118.01 estimate | $0.388 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| REAX 1.3 Open · 57B | — | $0.148$148estimate · RTX PRO 6000 Blackwell | $148.30 estimate | $0.4876 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | $0.165$165estimate · RTX PRO 6000 Blackwell | $164.92 estimate | $0.5423 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | $0.212$212estimate · RTX PRO 6000 Blackwell | $212.04 estimate | $0.6972 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | $1.025$1,025 | $1,025.11 | $3.371 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 3,276 of 3,288); shown once.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.118$118estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% · Math 81.6% | — | 74.4%72.9%–75.9% | a | 2,447 / 3,288 | 1,006 right of 1,009 kept3 wrong · 2,279 skipped | 0.882 | 5.7%answered 98% | 162 ms | 198 ms | $0.118$118estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.093$92.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% · Math 80.3% | — | 73.0%71.4%–74.5% | ab | 2,399 / 3,288 | 523 right of 524 kept1 wrong · 2,764 skipped | 0.882 | 0.1%answered 100% | 116 ms | 177 ms | $0.093$92.98estimate · RTX PRO 6000 Blackwell |
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.148$148estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% · Math 78.0% | — | 71.6%70.0%–73.1% | b | 2,354 / 3,288 | 229 right of 229 kept0 mistakes · 3,059 skipped | 0.878 | 3.8%answered 99% | 200 ms | 260 ms | $0.148$148estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.052$51.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% · Math 77.4% | — | 71.1%69.5%–72.6% | bc | 2,338 / 3,288 | 553 right of 554 kept1 wrong · 2,734 skipped | 0.870 | 0.7%answered 100% | 811 ms | 1,111 ms | $0.052$51.85estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.102$102estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% · Math 77.3% | — | 71.0%69.4%–72.5% | bc | 2,333 / 3,288 | 384 right of 384 kept0 mistakes · 2,904 skipped | 0.873 | 1.8%answered 99% | 128 ms | 218 ms | $0.102$102estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.047$47.29estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% · Math 77.1% | — | 70.5%68.9%–72.1% | bcd | 2,319 / 3,288 | 294 right of 296 kept2 wrong · 2,992 skipped | 0.865 | 3.8%answered 99% | 676 ms | 1,173 ms | $0.047$47.29estimate · DGX Spark |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.165$165estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% · Math 75.2% | — | 69.3%67.7%–70.8% | cde | 2,277 / 3,288 | 23 right of 23 kept0 mistakes · 3,265 skipped | 0.868 | 4.4%answered 99% | 207 ms | 324 ms | $0.165$165estimate · RTX PRO 6000 Blackwell |
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 54.9% · Math 74.1% | — | 67.5%65.9%–69.1% | defg | 2,221 / 3,288 | 233 right of 233 kept0 mistakes · 3,055 skipped | 0.852 | 89.1%answered 52% | 180 ms | 426 ms | $0.008$8.37 |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.029$28.55estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% · Math 72.0% | — | 67.3%65.7%–68.9% | efh | 2,214 / 3,288 | 272 right of 274 kept2 wrong · 3,014 skipped | 0.852 | 31.9%answered 89% | 450 ms | 587 ms | $0.029$28.55estimate · DGX Spark |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 74.0% | — | 67.3%65.7%–68.9% | efghi | 2,213 / 3,288 | 518 right of 520 kept2 wrong · 2,768 skipped | 0.868 | 71.4%answered 69% | 150 ms | 232 ms | $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.071$71.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% · Math 72.5% | — | 66.4%64.8%–68.0% | fghij | 2,184 / 3,288 | 385 right of 385 kept0 mistakes · 2,903 skipped | 0.844 | 13.5%answered 95% | 97 ms | 125 ms | $0.071$71.10estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.096$95.71estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% · Math 71.5% | — | 65.7%64.1%–67.3% | fghijk | 2,161 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.835 | 7.5%answered 97% | 129 ms | 168 ms | $0.096$95.71estimate · RTX PRO 6000 Blackwell |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.036$36.42 per 1,000 questionsper 1M questions Logic 54.5% · Math 70.8% | — | 65.2%63.6%–66.8% | fghijkl | 2,144 / 3,288 | 213 right of 213 kept0 mistakes · 3,075 skipped | 0.852 | 49.2%answered 80% | 266 ms | 284 ms | $0.036$36.42 |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.046$46.28estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 70.4% | — | 65.0%63.3%–66.6% | fghijkl | 2,136 / 3,288 | 216 right of 217 kept1 wrong · 3,071 skipped | 0.858 | 9.3%answered 97% | 57 ms | 86 ms | $0.046$46.28estimate · RTX PRO 6000 Blackwell |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.016$16.13 per 1,000 questionsper 1M questions Logic 54.8% · Math 70.1% | — | 64.8%63.2%–66.5% | fghijkl | 2,132 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.846 | 48.3%answered 79% | 327 ms | 518 ms | $0.016$16.13 |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% · Math 69.9% | — | 64.8%63.2%–66.5% | gijkl | 2,132 / 3,288 | 176 right of 176 kept0 mistakes · 3,112 skipped | 0.843 | 42.3%answered 83% | 81 ms | 136 ms | $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% · Math 70.3% | — | 64.4%62.8%–66.0% | gijkl | 2,118 / 3,288 | 866 right of 884 kept18 wrong · 2,404 skipped | 0.841 | 5.7%answered 98% | 81 ms | 130 ms | $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.005$5.38 per 1,000 questionsper 1M questions Logic 53.3% · Math 69.8% | — | 64.1%62.5%–65.8% | jklm | 2,109 / 3,288 | 409 right of 411 kept2 wrong · 2,877 skipped | 0.849 | 10.3%answered 96% | 230 ms | 439 ms | $0.005$5.38 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.027$27.07 per 1,000 questionsper 1M questions Logic 50.0% · Math 71.3% | — | 64.1%62.4%–65.7% | hijklm | 2,106 / 3,288 | 257 right of 257 kept0 mistakes · 3,031 skipped | 0.838 | 3.0%answered 99% | 227 ms | 347 ms | $0.027$27.07 |
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 51.6% · Math 69.9% | — | 63.6%61.9%–65.2% | jklm | 2,091 / 3,288 | 128 right of 128 kept0 mistakes · 3,160 skipped | 0.856 | 90.2%answered 48% | 217 ms | 347 ms | $0.008$8.37 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.025$1,025 per 1,000 questionsper 1M questions Logic 52.5% · Math 69.3% | with web | 63.6%61.9%–65.2% | klm | 2,090 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.854 | 25.8%answered 90% | 430 ms | 707 ms | $1.025$1,025 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.020$20.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% · Math 69.0% | — | 63.3%61.6%–64.9% | jklmn | 2,081 / 3,288 | 50 right of 50 kept0 mistakes · 3,238 skipped | 0.844 | 1.9%answered 99% | 301 ms | 448 ms | $0.020$20.01estimate · DGX Spark |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.011$10.75 per 1,000 questionsper 1M questions Logic 52.4% · Math 68.6% | — | 63.0%61.4%–64.7% | klmn | 2,072 / 3,288 | 430 right of 433 kept3 wrong · 2,855 skipped | 0.827 | 58.3%answered 73% | 382 ms | 391 ms | $0.011$10.75 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.011$11.25 per 1,000 questionsper 1M questions Logic 47.3% · Math 70.0% | — | 62.2%60.6%–63.9% | lmno | 2,046 / 3,288 | No answers at this threshold | 0.829 | 87.2%answered 50% | 388 ms | 392 ms | $0.011$11.25 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$1.13 per 1,000 questionsper 1M questions Logic 47.3% · Math 69.9% | — | 62.1%60.5%–63.8% | lmnop | 2,043 / 3,288 | No answers at this threshold | 0.830 | 86.7%answered 51% | 381 ms | 388 ms | $0.001$1.13 |
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% · Math 66.3% | — | 60.9%59.2%–62.5% | mnopq | 2,002 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.810 | 4.5%answered 98% | 51 ms | 76 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 49.2% · Math 65.5% | — | 59.9%58.2%–61.6% | nopqr | 1,970 / 3,288 | 169 right of 169 kept0 mistakes · 3,119 skipped | 0.812 | 66.9%answered 65% | 208 ms | 390 ms | $0.004$3.98 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 48.2% · Math 65.1% | — | 59.3%57.6%–61.0% | opqrs | 1,951 / 3,288 | 263 right of 263 kept0 mistakes · 3,025 skipped | 0.810 | 15.4%answered 93% | 164 ms | 188 ms | $0.021$21.27 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 46.0% · Math 65.2% | — | 58.6%56.9%–60.3% | pqrs | 1,928 / 3,288 | 231 right of 231 kept0 mistakes · 3,057 skipped | 0.811 | 86.0%answered 46% | 174 ms | 285 ms | $0.004$3.98 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 46.3% · Math 64.1% | — | 58.0%56.3%–59.7% | qrst | 1,907 / 3,288 | 313 right of 313 kept0 mistakes · 2,975 skipped | 0.817 | 10.2%answered 96% | 166 ms | 205 ms | $0.021$21.27 |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.028$28.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 62.1% | — | 56.7%55.0%–58.4% | rstu | 1,865 / 3,288 | 251 right of 253 kept2 wrong · 3,035 skipped | 0.802 | 7.0%answered 97% | 38 ms | 45 ms | $0.028$28.19estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 61.3% | — | 56.2%54.5%–57.9% | stu | 1,849 / 3,288 | 235 right of 236 kept1 wrong · 3,052 skipped | 0.797 | 69.4%answered 60% | 80 ms | 94 ms | $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.032$32.40estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% · Math 59.8% | — | 55.8%54.1%–57.5% | stu | 1,836 / 3,288 | 74 right of 74 kept0 mistakes · 3,214 skipped | 0.798 | 13.5%answered 94% | 39 ms | 53 ms | $0.032$32.40estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.034$34.08estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% · Math 60.1% | — | 54.6%52.9%–56.3% | tuv | 1,794 / 3,288 | No answers at this threshold | 0.786 | 10.4%answered 95% | 42 ms | 57 ms | $0.034$34.08estimate · RTX PRO 6000 Blackwell |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.011$11.45estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% · Math 57.9% | — | 54.5%52.8%–56.2% | uv | 1,791 / 3,288 | 105 right of 105 kept0 mistakes · 3,183 skipped | 0.786 | 26.9%answered 86% | 177 ms | 222 ms | $0.011$11.45estimate · DGX Spark |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.030$29.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% · Math 59.8% | — | 54.5%52.8%–56.2% | uv | 1,791 / 3,288 | 153 right of 153 kept0 mistakes · 3,135 skipped | 0.782 | 3.4%answered 98% | 38 ms | 55 ms | $0.030$29.87estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.009$8.92estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% · Math 58.4% | — | 54.4%52.7%–56.1% | uv | 1,788 / 3,288 | 63 right of 63 kept0 mistakes · 3,225 skipped | 0.779 | 9.6%answered 95% | 142 ms | 165 ms | $0.009$8.92estimate · DGX Spark |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.009$9.07 per 1,000 questionsper 1M questions Logic 44.4% · Math 59.4% | — | 54.3%52.6%–56.0% | uvw | 1,784 / 3,288 | 47 right of 47 kept0 mistakes · 3,241 skipped | 0.769 | 11.4%answered 94% | 312 ms | 360 ms | $0.009$9.07 |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.033$32.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% · Math 58.8% | — | 54.2%52.5%–55.9% | uvw | 1,782 / 3,288 | 263 right of 266 kept3 wrong · 3,022 skipped | 0.795 | 6.1%answered 97% | 46 ms | 51 ms | $0.033$32.61estimate · RTX PRO 6000 Blackwell |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.012$12.17 per 1,000 questionsper 1M questions Logic 41.9% · Math 59.0% | — | 53.1%51.4%–54.8% | uvwx | 1,747 / 3,288 | 207 right of 207 kept0 mistakes · 3,081 skipped | 0.789 | 28.1%answered 85% | 339 ms | 472 ms | $0.012$12.17 |
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.024$24.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% · Math 56.8% | — | 52.5%50.8%–54.2% | vwxy | 1,725 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.777 | 7.9%answered 96% | 31 ms | 36 ms | $0.024$24.19estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% · Math 55.8% | — | 52.4%50.7%–54.1% | vwxy | 1,724 / 3,288 | 117 right of 117 kept0 mistakes · 3,171 skipped | 0.763 | 14.4%answered 93% | 68 ms | 80 ms | $0.004$4.28estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.024$23.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% · Math 55.9% | — | 52.1%50.4%–53.8% | vwxy | 1,713 / 3,288 | 130 right of 131 kept1 wrong · 3,157 skipped | 0.782 | 15.0%answered 92% | 29 ms | 35 ms | $0.024$23.77estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.011$10.82estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% · Math 56.0% | — | 51.7%50.0%–53.4% | vwxy | 1,699 / 3,288 | 270 right of 281 kept11 wrong · 3,007 skipped | 0.776 | 3.1%answered 99% | 163 ms | 240 ms | $0.011$10.82estimate · DGX Spark |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% · Math 55.3% | — | 51.6%49.9%–53.3% | vwxy | 1,696 / 3,288 | 165 right of 167 kept2 wrong · 3,121 skipped | 0.765 | 7.1%answered 96% | 59 ms | 65 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$5.02estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% · Math 56.0% | — | 51.3%49.6%–53.0% | vwxyz | 1,687 / 3,288 | 71 right of 72 kept1 wrong · 3,216 skipped | 0.752 | 4.4%answered 98% | 76 ms | 104 ms | $0.005$5.02estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% · Math 55.2% | — | 51.1%49.4%–52.8% | vwxyz | 1,680 / 3,288 | 185 right of 190 kept5 wrong · 3,098 skipped | 0.769 | 18.9%answered 90% | 41 ms | 51 ms | $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.005$5.46 per 1,000 questionsper 1M questions Logic 45.1% · Math 53.5% | — | 50.6%48.9%–52.3% | wxyzA | 1,664 / 3,288 | 356 right of 372 kept16 wrong · 2,916 skipped | 0.777 | 10.1%answered 95% | 293 ms | 349 ms | $0.005$5.46 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 40.8% · Math 55.6% | — | 50.5%48.8%–52.3% | wxyzA | 1,662 / 3,288 | 207 right of 208 kept1 wrong · 3,080 skipped | 0.757 | 6.4%answered 97% | 185 ms | 292 ms | $0.004$3.98 |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.212$212estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% · Math 54.6% | — | 50.5%48.7%–52.2% | xyzA | 1,659 / 3,288 | No answers at this threshold | 0.759 | 14.5%answered 92% | 250 ms | 482 ms | $0.212$212estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.069$69.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% · Math 53.8% | — | 49.3%47.6%–51.0% | yzAB | 1,620 / 3,288 | 30 right of 30 kept0 mistakes · 3,258 skipped | 0.770 | 1.6%answered 99% | 80 ms | 134 ms | $0.069$69.42estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% · Math 51.1% | — | 47.6%45.9%–49.3% | zABC | 1,564 / 3,288 | 18 right of 18 kept0 mistakes · 3,270 skipped | 0.747 | 2.4%answered 99% | 185 ms | 216 ms | $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.45estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 51.5% | — | 47.5%45.8%–49.2% | ABC | 1,561 / 3,288 | 161 right of 165 kept4 wrong · 3,123 skipped | 0.747 | 8.9%answered 95% | 46 ms | 52 ms | $0.033$33.45estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.025$25.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% · Math 50.4% | — | 47.0%45.3%–48.7% | ABC | 1,545 / 3,288 | 188 right of 194 kept6 wrong · 3,094 skipped | 0.743 | 7.8%answered 96% | 30 ms | 42 ms | $0.025$25.03estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% · Math 49.8% | — | 46.1%44.4%–47.8% | BCD | 1,515 / 3,288 | No answers at this threshold | 0.707 | 6.9%answered 96% | 103 ms | 131 ms | $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.043$42.91estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 46.9% | — | 44.4%42.7%–46.1% | CD | 1,461 / 3,288 | 52 right of 53 kept1 wrong · 3,235 skipped | 0.737 | 17.1%answered 90% | 55 ms | 72 ms | $0.043$42.91estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% · Math 44.3% | — | 42.6%40.9%–44.3% | D | 1,401 / 3,288 | No answers at this threshold | 0.729 | 7.3%answered 96% | 38 ms | 44 ms | $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.086$85.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% · Math 39.2% | — | 36.7%35.1%–38.4% | E | 1,208 / 3,288 | No answers at this threshold | 0.659 | 2.0%answered 99% | 117 ms | 146 ms | $0.086$85.83estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.36estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% · Math 38.7% | — | 35.9%34.3%–37.6% | E | 1,182 / 3,288 | 4 right of 4 kept0 mistakes · 3,284 skipped | 0.675 | 16.4%answered 89% | 36 ms | 42 ms | $0.002$2.36estimate · DGX Spark |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% · Math 36.8% | — | 35.2%33.6%–36.9% | E | 1,158 / 3,288 | 29 right of 30 kept1 wrong · 3,258 skipped | 0.637 | 23.1%answered 84% | 28 ms | 37 ms | $0.002$1.94estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.30estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% · Math 37.1% | — | 34.9%33.3%–36.5% | EF | 1,147 / 3,288 | 7 right of 7 kept0 mistakes · 3,281 skipped | 0.648 | 82.5%answered 30% | 17 ms | 21 ms | $0.014$14.30estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.40estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% · Math 35.3% | — | 34.1%32.5%–35.7% | EF | 1,120 / 3,288 | 16 right of 16 kept0 mistakes · 3,272 skipped | 0.626 | 6.8%answered 95% | 51 ms | 63 ms | $0.003$3.40estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.38estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% · Math 32.1% | — | 31.1%29.5%–32.7% | FG | 1,022 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.602 | 22.7%answered 83% | 20 ms | 25 ms | $0.001$1.38estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.62estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% · Math 29.7% | — | 28.0%26.5%–29.5% | GH | 920 / 3,288 | No answers at this threshold | 0.605 | 14.9%answered 89% | 24 ms | 29 ms | $0.002$1.62estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.11estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% · Math 28.2% | — | 27.8%26.3%–29.4% | GH | 915 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.576 | 2.0%answered 99% | 15 ms | 17 ms | $0.001$1.11estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.80estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% · Math 28.2% | — | 27.7%26.2%–29.2% | GH | 910 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.590 | 0.6%answered 100% | 26 ms | 35 ms | $0.002$1.80estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.027$26.72estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% · Math 28.7% | — | 26.5%25.0%–28.1% | HI | 872 / 3,288 | No answers at this threshold | 0.573 | 5.0%answered 96% | 35 ms | 42 ms | $0.027$26.72estimate · RTX PRO 6000 Blackwell |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% · Math 22.7% | — | 23.0%21.6%–24.4% | IJ | 755 / 3,288 | No answers at this threshold | 0.511 | 15.6%answered 87% | 12 ms | 23 ms | $0.001$1.01estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.83estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% · Math 20.6% | — | 21.5%20.1%–22.9% | J | 706 / 3,288 | No answers at this threshold | 0.520 | 73.9%answered 32% | 11 ms | 14 ms | $0.001$0.83estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI scar-ai quincyqiang Strands Agents victorbennett Metask Lab
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
GLiDE 0.4B: 21.5% at $0.001 per 1,000 questions ($0.83 per 1M) (estimate)
GLiClass v4 Edge: 23.0% at $0.001 per 1,000 questions ($1.01 per 1M) (estimate)
OneJev1.6-0.8B: 27.8% at $0.001 per 1,000 questions ($1.11 per 1M) (estimate)
GLiDE 0.8B: 31.1% at $0.001 per 1,000 questions ($1.38 per 1M) (estimate)
StartLux-Decision-v1.1-0.8B-Instant: 28.0% at $0.002 per 1,000 questions ($1.62 per 1M) (estimate)
KnowLine-0.8B-Gen1: 27.7% at $0.002 per 1,000 questions ($1.80 per 1M) (estimate)
GLiDE 2B: 35.2% at $0.002 per 1,000 questions ($1.94 per 1M) (estimate)
StartLux-Decision-v1.1-2B-Instant: 35.9% at $0.002 per 1,000 questions ($2.36 per 1M) (estimate)
Perplexity Decider Flash v1.2: 51.1% at $0.003 per 1,000 questions ($2.72 per 1M)
KnowLine-2B-Gen1: 34.1% at $0.003 per 1,000 questions ($3.40 per 1M) (estimate)
StartLux-Decision-v1.1-4B-Instant: 52.4% at $0.004 per 1,000 questions ($4.28 per 1M) (estimate)
KnowLine 4B Gen5.5: 51.3% at $0.005 per 1,000 questions ($5.02 per 1M) (estimate)
d1-2B: 34.9% at $0.014 per 1,000 questions ($14.30 per 1M) (estimate)
Perplexity Decider Flash v1.3: 52.1% at $0.024 per 1,000 questions ($23.77 per 1M) (estimate)
vera-medium-omni: 52.5% at $0.024 per 1,000 questions ($24.19 per 1M) (estimate)
Chinese Decision Pointer 4B: 47.0% at $0.025 per 1,000 questions ($25.03 per 1M) (estimate)
Strands Decider 2B: 26.5% at $0.027 per 1,000 questions ($26.72 per 1M) (estimate)
Verdin-4B-v1.1: 47.5% at $0.033 per 1,000 questions ($33.45 per 1M) (estimate)
Metask-Jev-4B: 36.7% at $0.086 per 1,000 questions ($85.83 per 1M) (estimate)
Fastino Other OmniJev StartLux PelaAI Perplexity Liquid AI scar-ai quincyqiang Strands Agents victorbennett Metask Lab
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| GLiDE 0.4B Pareto Open · 0.4B | — | $0.001$0.83estimate · DGX Spark | $0.83 estimate | $0.002719 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiClass v4 Edge Pareto Open · 0.4B | — | $0.001$1.01estimate · DGX Spark | $1.01 estimate | $0.00331 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev1.6-0.8B Pareto Open · 0.8B | — | $0.001$1.11estimate · DGX Spark | $1.11 estimate | $0.003665 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 0.8B Pareto Open · 0.8B | — | $0.001$1.38estimate · DGX Spark | $1.38 estimate | $0.004552 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | — | $0.002$1.62estimate · DGX Spark | $1.62 estimate | $0.00532 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-0.8B-Gen1 Open · 0.8B | — | $0.002$1.80estimate · DGX Spark | $1.80 estimate | $0.005912 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 2B Pareto Open · 1.9B | — | $0.002$1.94estimate · DGX Spark | $1.94 estimate | $0.006385 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-2B-Instant Pareto Open · 2.0B | — | $0.002$2.36estimate · DGX Spark | $2.36 estimate | $0.007744 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Perplexity Decider Flash v1.2 Pareto Open · 4.0B | — | $0.003$2.72 list | $2.72 | $0.008936 | $0.032$31.55estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-2B-Gen1 Open · 2.0B | — | $0.003$3.40estimate · DGX Spark | $3.40 estimate | $0.01117 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| StartLux-Decision-v1.1-4B-Instant Pareto Open · 4.0B | — | $0.004$4.28estimate · DGX Spark | $4.28 estimate | $0.01407 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine 4B Gen5.5 Open · 4.0B | — | $0.005$5.02estimate · DGX Spark | $5.02 estimate | $0.01649 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d1-2B Open · 1.9B | — | $0.014$14.30estimate · RTX PRO 6000 Blackwell | $14.30 estimate | $0.04703 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider Flash v1.3 Open · 4.0B | — | $0.024$23.77estimate · RTX PRO 6000 Blackwell | $23.77 estimate | $0.07816 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| vera-medium-omni Pareto Open · 4.2B | — | $0.024$24.19estimate · RTX PRO 6000 Blackwell | $24.19 estimate | $0.07954 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Chinese Decision Pointer 4B Open · 4.0B | — | $0.025$25.03estimate · RTX PRO 6000 Blackwell | $25.03 estimate | $0.08231 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Strands Decider 2B Open · 2.0B | — | $0.027$26.72estimate · RTX PRO 6000 Blackwell | $26.72 estimate | $0.08784 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Verdin-4B-v1.1 Open · 4.0B | — | $0.033$33.45estimate · RTX PRO 6000 Blackwell | $33.45 estimate | $0.11 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Metask-Jev-4B Open · 4.5B | — | $0.086$85.83estimate · RTX PRO 6000 Blackwell | $85.83 estimate | $0.2822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Cost $0.024$24.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.1% · Math 56.8% | — | 52.5%50.8%–54.2% | a | 1,725 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.777 | 7.9%answered 96% | 31 ms | 36 ms | $0.024$24.19estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Cost $0.004$4.28estimate · DGX Spark per 1,000 questionsper 1M questions Logic 45.9% · Math 55.8% | — | 52.4%50.7%–54.1% | a | 1,724 / 3,288 | 117 right of 117 kept0 mistakes · 3,171 skipped | 0.763 | 14.4%answered 93% | 68 ms | 80 ms | $0.004$4.28estimate · DGX Spark |
| Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Cost $0.024$23.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.7% · Math 55.9% | — | 52.1%50.4%–53.8% | a | 1,713 / 3,288 | 130 right of 131 kept1 wrong · 3,157 skipped | 0.782 | 15.0%answered 92% | 29 ms | 35 ms | $0.024$23.77estimate · RTX PRO 6000 Blackwell |
| KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Cost $0.005$5.02estimate · DGX Spark per 1,000 questionsper 1M questions Logic 42.3% · Math 56.0% | — | 51.3%49.6%–53.0% | a | 1,687 / 3,288 | 71 right of 72 kept1 wrong · 3,216 skipped | 0.752 | 4.4%answered 98% | 76 ms | 104 ms | $0.005$5.02estimate · DGX Spark |
| Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Cost $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.1% · Math 55.2% | — | 51.1%49.4%–52.8% | a | 1,680 / 3,288 | 185 right of 190 kept5 wrong · 3,098 skipped | 0.769 | 18.9%answered 90% | 41 ms | 51 ms | $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell |
| Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.033$33.45estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 51.5% | — | 47.5%45.8%–49.2% | b | 1,561 / 3,288 | 161 right of 165 kept4 wrong · 3,123 skipped | 0.747 | 8.9%answered 95% | 46 ms | 52 ms | $0.033$33.45estimate · RTX PRO 6000 Blackwell |
| Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Cost $0.025$25.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.4% · Math 50.4% | — | 47.0%45.3%–48.7% | b | 1,545 / 3,288 | 188 right of 194 kept6 wrong · 3,094 skipped | 0.743 | 7.8%answered 96% | 30 ms | 42 ms | $0.025$25.03estimate · RTX PRO 6000 Blackwell |
| Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Cost $0.086$85.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 32.1% · Math 39.2% | — | 36.7%35.1%–38.4% | c | 1,208 / 3,288 | No answers at this threshold | 0.659 | 2.0%answered 99% | 117 ms | 146 ms | $0.086$85.83estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Cost $0.002$2.36estimate · DGX Spark per 1,000 questionsper 1M questions Logic 30.8% · Math 38.7% | — | 35.9%34.3%–37.6% | c | 1,182 / 3,288 | 4 right of 4 kept0 mistakes · 3,284 skipped | 0.675 | 16.4%answered 89% | 36 ms | 42 ms | $0.002$2.36estimate · DGX Spark |
| GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Cost $0.002$1.94estimate · DGX Spark per 1,000 questionsper 1M questions Logic 32.3% · Math 36.8% | — | 35.2%33.6%–36.9% | c | 1,158 / 3,288 | 29 right of 30 kept1 wrong · 3,258 skipped | 0.637 | 23.1%answered 84% | 28 ms | 37 ms | $0.002$1.94estimate · DGX Spark |
| d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Cost $0.014$14.30estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 30.7% · Math 37.1% | — | 34.9%33.3%–36.5% | c | 1,147 / 3,288 | 7 right of 7 kept0 mistakes · 3,281 skipped | 0.648 | 82.5%answered 30% | 17 ms | 21 ms | $0.014$14.30estimate · RTX PRO 6000 Blackwell |
| KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Cost $0.003$3.40estimate · DGX Spark per 1,000 questionsper 1M questions Logic 31.7% · Math 35.3% | — | 34.1%32.5%–35.7% | cd | 1,120 / 3,288 | 16 right of 16 kept0 mistakes · 3,272 skipped | 0.626 | 6.8%answered 95% | 51 ms | 63 ms | $0.003$3.40estimate · DGX Spark |
| GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Cost $0.001$1.38estimate · DGX Spark per 1,000 questionsper 1M questions Logic 29.2% · Math 32.1% | — | 31.1%29.5%–32.7% | d | 1,022 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.602 | 22.7%answered 83% | 20 ms | 25 ms | $0.001$1.38estimate · DGX Spark |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.62estimate · DGX Spark per 1,000 questionsper 1M questions Logic 24.7% · Math 29.7% | — | 28.0%26.5%–29.5% | e | 920 / 3,288 | No answers at this threshold | 0.605 | 14.9%answered 89% | 24 ms | 29 ms | $0.002$1.62estimate · DGX Spark |
| OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Cost $0.001$1.11estimate · DGX Spark per 1,000 questionsper 1M questions Logic 27.0% · Math 28.2% | — | 27.8%26.3%–29.4% | e | 915 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.576 | 2.0%answered 99% | 15 ms | 17 ms | $0.001$1.11estimate · DGX Spark |
| KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Cost $0.002$1.80estimate · DGX Spark per 1,000 questionsper 1M questions Logic 26.7% · Math 28.2% | — | 27.7%26.2%–29.2% | e | 910 / 3,288 | 2 right of 2 kept0 mistakes · 3,286 skipped | 0.590 | 0.6%answered 100% | 26 ms | 35 ms | $0.002$1.80estimate · DGX Spark |
| Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Cost $0.027$26.72estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 22.4% · Math 28.7% | — | 26.5%25.0%–28.1% | e | 872 / 3,288 | No answers at this threshold | 0.573 | 5.0%answered 96% | 35 ms | 42 ms | $0.027$26.72estimate · RTX PRO 6000 Blackwell |
| GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Cost $0.001$1.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.6% · Math 22.7% | — | 23.0%21.6%–24.4% | f | 755 / 3,288 | No answers at this threshold | 0.511 | 15.6%answered 87% | 12 ms | 23 ms | $0.001$1.01estimate · DGX Spark |
| GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Cost $0.001$0.83estimate · DGX Spark per 1,000 questionsper 1M questions Logic 23.2% · Math 20.6% | — | 21.5%20.1%–22.9% | f | 706 / 3,288 | No answers at this threshold | 0.520 | 73.9%answered 32% | 11 ms | 14 ms | $0.001$0.83estimate · DGX Spark |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
StartLux Base Labs Apertix AI Fastino Cloudflare newsbang Blockbrain Labs PelaAI tattrongvu Patrick-18 Valen-Team EldanRing
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
StartLux-Decision-v1.1-9B-Instant: 54.4% at $0.009 per 1,000 questions ($8.92 per 1M) (estimate)
Elephant-1: 56.2% at $0.010 per 1,000 questions ($9.79 per 1M)
Cerno 9B v1.2: 51.7% at $0.011 per 1,000 questions ($10.82 per 1M) (estimate)
GLiDE 9B: 54.5% at $0.011 per 1,000 questions ($11.45 per 1M) (estimate)
Clef-Flash: 46.1% at $0.013 per 1,000 questions ($12.90 per 1M)
Adjudex 9B v5: 56.7% at $0.028 per 1,000 questions ($28.19 per 1M) (estimate)
Cygnet: 42.6% at $0.029 per 1,000 questions ($29.34 per 1M)
KnowLine-9B-Gen1: 54.5% at $0.030 per 1,000 questions ($29.87 per 1M) (estimate)
Jeffan-9B-1211: 55.8% at $0.032 per 1,000 questions ($32.40 per 1M) (estimate)
Dexy-Decision-9B-V2: 54.2% at $0.033 per 1,000 questions ($32.61 per 1M) (estimate)
KnowLine-9B-Gen2: 54.6% at $0.034 per 1,000 questions ($34.08 per 1M) (estimate)
KnowLine-12B-Gen1: 60.9% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Valen-Decision-1.5-9B: 51.6% at $0.041 per 1,000 questions ($41.44 per 1M) (estimate)
Winnow-12B: 44.4% at $0.043 per 1,000 questions ($42.91 per 1M) (estimate)
StartLux Base Labs Apertix AI Fastino Cloudflare newsbang Blockbrain Labs PelaAI tattrongvu Patrick-18 Valen-Team EldanRing
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| StartLux-Decision-v1.1-9B-Instant Pareto Open · 9.0B | — | $0.009$8.92estimate · DGX Spark | $8.92 estimate | $0.02932 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Elephant-1 Pareto Open · 9.0B | — | $0.010$9.79 list | $9.79 | $0.0322 | $0.058$58.27estimate · RTX PRO 6000 Blackwell | maker's list price |
| Cerno 9B v1.2 Open · 9.0B | — | $0.011$10.82estimate · DGX Spark | $10.82 estimate | $0.03559 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| GLiDE 9B Open · 7.9B | — | $0.011$11.45estimate · DGX Spark | $11.45 estimate | $0.03766 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Clef-Flash Open · 9.0B | — | $0.013$12.90 list | $12.90 | $0.04241 | $0.075$75.10estimate · RTX PRO 6000 Blackwell | maker's list price |
| Adjudex 9B v5 Pareto Open · 9.0B | — | $0.028$28.19estimate · RTX PRO 6000 Blackwell | $28.19 estimate | $0.09268 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Cygnet Open · 12B | — | $0.029$29.34 list | $29.34 | $0.09647 | $0.029$28.61estimate · RTX PRO 6000 Blackwell | public host's list price (same weights) |
| KnowLine-9B-Gen1 Open · 9.0B | — | $0.030$29.87estimate · RTX PRO 6000 Blackwell | $29.87 estimate | $0.09822 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Jeffan-9B-1211 Open · 9.0B | — | $0.032$32.40estimate · RTX PRO 6000 Blackwell | $32.40 estimate | $0.1065 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Dexy-Decision-9B-V2 Open · 9.0B | — | $0.033$32.61estimate · RTX PRO 6000 Blackwell | $32.61 estimate | $0.1072 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-9B-Gen2 Open · 9.0B | — | $0.034$34.08estimate · RTX PRO 6000 Blackwell | $34.08 estimate | $0.1121 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| KnowLine-12B-Gen1 Pareto Open · 12B | — | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Valen-Decision-1.5-9B Open · 9.0B | — | $0.041$41.44estimate · RTX PRO 6000 Blackwell | $41.44 estimate | $0.1363 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Winnow-12B Open · 12B | — | $0.043$42.91estimate · RTX PRO 6000 Blackwell | $42.91 estimate | $0.1411 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 50.6% · Math 66.3% | — | 60.9%59.2%–62.5% | a | 2,002 / 3,288 | 13 right of 13 kept0 mistakes · 3,275 skipped | 0.810 | 4.5%answered 98% | 51 ms | 76 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.028$28.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 62.1% | — | 56.7%55.0%–58.4% | b | 1,865 / 3,288 | 251 right of 253 kept2 wrong · 3,035 skipped | 0.802 | 7.0%answered 97% | 38 ms | 45 ms | $0.028$28.19estimate · RTX PRO 6000 Blackwell |
| Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Cost $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 46.4% · Math 61.3% | — | 56.2%54.5%–57.9% | b | 1,849 / 3,288 | 235 right of 236 kept1 wrong · 3,052 skipped | 0.797 | 69.4%answered 60% | 80 ms | 94 ms | $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell |
| Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.032$32.40estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 48.2% · Math 59.8% | — | 55.8%54.1%–57.5% | b | 1,836 / 3,288 | 74 right of 74 kept0 mistakes · 3,214 skipped | 0.798 | 13.5%answered 94% | 39 ms | 53 ms | $0.032$32.40estimate · RTX PRO 6000 Blackwell |
| KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.034$34.08estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 43.9% · Math 60.1% | — | 54.6%52.9%–56.3% | b | 1,794 / 3,288 | No answers at this threshold | 0.786 | 10.4%answered 95% | 42 ms | 57 ms | $0.034$34.08estimate · RTX PRO 6000 Blackwell |
| GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Cost $0.011$11.45estimate · DGX Spark per 1,000 questionsper 1M questions Logic 47.8% · Math 57.9% | — | 54.5%52.8%–56.2% | bc | 1,791 / 3,288 | 105 right of 105 kept0 mistakes · 3,183 skipped | 0.786 | 26.9%answered 86% | 177 ms | 222 ms | $0.011$11.45estimate · DGX Spark |
| KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.030$29.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.3% · Math 59.8% | — | 54.5%52.8%–56.2% | b | 1,791 / 3,288 | 153 right of 153 kept0 mistakes · 3,135 skipped | 0.782 | 3.4%answered 98% | 38 ms | 55 ms | $0.030$29.87estimate · RTX PRO 6000 Blackwell |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Cost $0.009$8.92estimate · DGX Spark per 1,000 questionsper 1M questions Logic 46.6% · Math 58.4% | — | 54.4%52.7%–56.1% | bc | 1,788 / 3,288 | 63 right of 63 kept0 mistakes · 3,225 skipped | 0.779 | 9.6%answered 95% | 142 ms | 165 ms | $0.009$8.92estimate · DGX Spark |
| Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Cost $0.033$32.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 45.3% · Math 58.8% | — | 54.2%52.5%–55.9% | bc | 1,782 / 3,288 | 263 right of 266 kept3 wrong · 3,022 skipped | 0.795 | 6.1%answered 97% | 46 ms | 51 ms | $0.033$32.61estimate · RTX PRO 6000 Blackwell |
| Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Cost $0.011$10.82estimate · DGX Spark per 1,000 questionsper 1M questions Logic 43.3% · Math 56.0% | — | 51.7%50.0%–53.4% | c | 1,699 / 3,288 | 270 right of 281 kept11 wrong · 3,007 skipped | 0.776 | 3.1%answered 99% | 163 ms | 240 ms | $0.011$10.82estimate · DGX Spark |
| Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 44.4% · Math 55.3% | — | 51.6%49.9%–53.3% | c | 1,696 / 3,288 | 165 right of 167 kept2 wrong · 3,121 skipped | 0.765 | 7.1%answered 96% | 59 ms | 65 ms | $0.041$41.44estimate · RTX PRO 6000 Blackwell |
| Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Cost $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 38.9% · Math 49.8% | — | 46.1%44.4%–47.8% | d | 1,515 / 3,288 | No answers at this threshold | 0.707 | 6.9%answered 96% | 103 ms | 131 ms | $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell |
| Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Cost $0.043$42.91estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.7% · Math 46.9% | — | 44.4%42.7%–46.1% | de | 1,461 / 3,288 | 52 right of 53 kept1 wrong · 3,235 skipped | 0.737 | 17.1%answered 90% | 55 ms | 72 ms | $0.043$42.91estimate · RTX PRO 6000 Blackwell |
| Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Cost $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 39.3% · Math 44.3% | — | 42.6%40.9%–44.3% | e | 1,401 / 3,288 | No answers at this threshold | 0.729 | 7.3%answered 96% | 38 ms | 44 ms | $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router Sthānika AI PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Perplexity Decider 27B v1.2 (API): 64.1% at $0.005 per 1,000 questions ($5.38 per 1M)
GLiDE (v1.2): 64.8% at $0.010 per 1,000 questions ($9.71 per 1M)
Drex 2.0: 53.1% at $0.012 per 1,000 questions ($12.17 per 1M)
Drex Max: 64.4% at $0.016 per 1,000 questions ($16.31 per 1M)
Turion Jev 31B: 63.3% at $0.020 per 1,000 questions ($20.01 per 1M) (estimate)
Mammoth-1: 67.3% at $0.028 per 1,000 questions ($27.98 per 1M)
GLiDE 27B: 67.3% at $0.029 per 1,000 questions ($28.55 per 1M) (estimate)
d3: 65.0% at $0.046 per 1,000 questions ($46.28 per 1M) (estimate)
Sieve 31B v3: 70.5% at $0.047 per 1,000 questions ($47.29 per 1M) (estimate)
KnowLine-31B-Gen1.5: 71.1% at $0.052 per 1,000 questions ($51.85 per 1M) (estimate)
OneJev-27B: 49.3% at $0.069 per 1,000 questions ($69.42 per 1M) (estimate)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): 66.4% at $0.071 per 1,000 questions ($71.10 per 1M) (estimate)
Clef: 47.6% at $0.081 per 1,000 questions ($81.46 per 1M)
KnowLine-31B-Gen2: 73.0% at $0.093 per 1,000 questions ($92.98 per 1M) (estimate)
d3-ultra: 65.7% at $0.096 per 1,000 questions ($95.71 per 1M) (estimate)
Copernicus-31B: 71.0% at $0.102 per 1,000 questions ($102 per 1M) (estimate)
Karpo Decision 31B: 74.4% at $0.118 per 1,000 questions ($118 per 1M) (estimate)
Open-Jev-27B-v1.1: 50.5% at $0.212 per 1,000 questions ($212 per 1M) (estimate)
Perplexity Decider 27B v1.2 (API · with web): 63.6% at $1.025 per 1,000 questions ($1,025 per 1M)
Perplexity Fastino Nace.AI Turion.AI Base Labs vLLM Semantic Router Sthānika AI PelaAI OmniJev Cloudflare Poindexter Labs MachinePulse Zefan Cai
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Perplexity Decider 27B v1.2 Pareto API Open · 27B | — | $0.005$5.38 | $5.38 | $0.01768 | — | provider price × reported usage |
| GLiDE Pareto v1.2 Open · 27B | — | $0.010$9.71 list | $9.71 | $0.03193 | $0.066$66.05estimate · RTX PRO 6000 Blackwell | maker's list price |
| Drex 2.0 Open · 35B MoE · 3.0B active | — | $0.012$12.17 | $12.17 | $0.04001 | — | provider price × reported usage |
| Drex Max Open · 27B | — | $0.016$16.31 list | $16.31 | $0.05362 | $0.064$63.74estimate · RTX PRO 6000 Blackwell | maker's list price |
| Turion Jev 31B Open · 31B | — | $0.020$20.01estimate · DGX Spark | $20.01 estimate | $0.0658 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Mammoth-1 Pareto Open · 27B | — | $0.028$27.98 list | $27.98 | $0.09199 | $0.120$120estimate · RTX PRO 6000 Blackwell | maker's list price |
| GLiDE 27B Pareto Open · 27B | — | $0.029$28.55estimate · DGX Spark | $28.55 estimate | $0.09388 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| d3 Open · 26B | — | $0.046$46.28estimate · RTX PRO 6000 Blackwell | $46.28 estimate | $0.1522 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Sieve 31B v3 Pareto Open · 31B | — | $0.047$47.29estimate · DGX Spark | $47.29 estimate | $0.1555 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| KnowLine-31B-Gen1.5 Pareto Open · 31B | — | $0.052$51.85estimate · DGX Spark | $51.85 estimate | $0.1705 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| OneJev-27B Open · 27B | — | $0.069$69.42estimate · RTX PRO 6000 Blackwell | $69.42 estimate | $0.2283 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B | — | $0.071$71.10estimate · RTX PRO 6000 Blackwell | $71.10 estimate | $0.2338 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Clef Open · 27B | — | $0.081$81.46 list | $81.46 | $0.2679 | $0.135$135estimate · RTX PRO 6000 Blackwell | maker's list price |
| KnowLine-31B-Gen2 Pareto Open · 31B | — | $0.093$92.98estimate · RTX PRO 6000 Blackwell | $92.98 estimate | $0.3057 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| d3-ultra Open · 26B | — | $0.096$95.71estimate · RTX PRO 6000 Blackwell | $95.71 estimate | $0.3147 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Copernicus-31B Open · 31B | — | $0.102$102estimate · RTX PRO 6000 Blackwell | $102.03 estimate | $0.3355 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Karpo Decision 31B Pareto Open · 31B | — | $0.118$118estimate · RTX PRO 6000 Blackwell | $118.01 estimate | $0.388 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Open-Jev-27B-v1.1 Open · 27B | — | $0.212$212estimate · RTX PRO 6000 Blackwell | $212.04 estimate | $0.6972 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | with web | $1.025$1,025 | $1,025.11 | $3.371 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Perplexity Decider 27B v1.2 (API) and Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): two setups of the same model, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings. Its row is the API setup on every test; Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) is under the row's other settings and All setups.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Cost $0.118$118estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 60.7% · Math 81.6% | — | 74.4%72.9%–75.9% | a | 2,447 / 3,288 | 1,006 right of 1,009 kept3 wrong · 2,279 skipped | 0.882 | 5.7%answered 98% | 162 ms | 198 ms | $0.118$118estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Cost $0.093$92.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.9% · Math 80.3% | — | 73.0%71.4%–74.5% | a | 2,399 / 3,288 | 523 right of 524 kept1 wrong · 2,764 skipped | 0.882 | 0.1%answered 100% | 116 ms | 177 ms | $0.093$92.98estimate · RTX PRO 6000 Blackwell |
| KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Cost $0.052$51.85estimate · DGX Spark per 1,000 questionsper 1M questions Logic 59.0% · Math 77.4% | — | 71.1%69.5%–72.6% | b | 2,338 / 3,288 | 553 right of 554 kept1 wrong · 2,734 skipped | 0.870 | 0.7%answered 100% | 811 ms | 1,111 ms | $0.052$51.85estimate · DGX Spark |
| Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Cost $0.102$102estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 58.7% · Math 77.3% | — | 71.0%69.4%–72.5% | b | 2,333 / 3,288 | 384 right of 384 kept0 mistakes · 2,904 skipped | 0.873 | 1.8%answered 99% | 128 ms | 218 ms | $0.102$102estimate · RTX PRO 6000 Blackwell |
| Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Cost $0.047$47.29estimate · DGX Spark per 1,000 questionsper 1M questions Logic 57.9% · Math 77.1% | — | 70.5%68.9%–72.1% | b | 2,319 / 3,288 | 294 right of 296 kept2 wrong · 2,992 skipped | 0.865 | 3.8%answered 99% | 676 ms | 1,173 ms | $0.047$47.29estimate · DGX Spark |
| GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Cost $0.029$28.55estimate · DGX Spark per 1,000 questionsper 1M questions Logic 58.4% · Math 72.0% | — | 67.3%65.7%–68.9% | c | 2,214 / 3,288 | 272 right of 274 kept2 wrong · 3,014 skipped | 0.852 | 31.9%answered 89% | 450 ms | 587 ms | $0.029$28.55estimate · DGX Spark |
| Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Cost $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 74.0% | — | 67.3%65.7%–68.9% | cd | 2,213 / 3,288 | 518 right of 520 kept2 wrong · 2,768 skipped | 0.868 | 71.4%answered 69% | 150 ms | 232 ms | $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Cost $0.071$71.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.8% · Math 72.5% | — | 66.4%64.8%–68.0% | cde | 2,184 / 3,288 | 385 right of 385 kept0 mistakes · 2,903 skipped | 0.844 | 13.5%answered 95% | 97 ms | 125 ms | $0.071$71.10estimate · RTX PRO 6000 Blackwell |
| d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.096$95.71estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.6% · Math 71.5% | — | 65.7%64.1%–67.3% | cdef | 2,161 / 3,288 | 39 right of 39 kept0 mistakes · 3,249 skipped | 0.835 | 7.5%answered 97% | 129 ms | 168 ms | $0.096$95.71estimate · RTX PRO 6000 Blackwell |
| d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Cost $0.046$46.28estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.5% · Math 70.4% | — | 65.0%63.3%–66.6% | def | 2,136 / 3,288 | 216 right of 217 kept1 wrong · 3,071 skipped | 0.858 | 9.3%answered 97% | 57 ms | 86 ms | $0.046$46.28estimate · RTX PRO 6000 Blackwell |
| GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Cost $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 55.2% · Math 69.9% | — | 64.8%63.2%–66.5% | def | 2,132 / 3,288 | 176 right of 176 kept0 mistakes · 3,112 skipped | 0.843 | 42.3%answered 83% | 81 ms | 136 ms | $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell |
| Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Cost $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 53.1% · Math 70.3% | — | 64.4%62.8%–66.0% | ef | 2,118 / 3,288 | 866 right of 884 kept18 wrong · 2,404 skipped | 0.841 | 5.7%answered 98% | 81 ms | 130 ms | $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell |
| Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Cost $0.005$5.38 per 1,000 questionsper 1M questions Logic 53.3% · Math 69.8% | — | 64.1%62.5%–65.8% | f | 2,109 / 3,288 | 409 right of 411 kept2 wrong · 2,877 skipped | 0.849 | 10.3%answered 96% | 230 ms | 439 ms | $0.005$5.38 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Cost $1.025$1,025 per 1,000 questionsper 1M questions Logic 52.5% · Math 69.3% | with web | 63.6%61.9%–65.2% | f | 2,090 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.854 | 25.8%answered 90% | 430 ms | 707 ms | $1.025$1,025 |
| Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Cost $0.020$20.01estimate · DGX Spark per 1,000 questionsper 1M questions Logic 52.3% · Math 69.0% | — | 63.3%61.6%–64.9% | f | 2,081 / 3,288 | 50 right of 50 kept0 mistakes · 3,238 skipped | 0.844 | 1.9%answered 99% | 301 ms | 448 ms | $0.020$20.01estimate · DGX Spark |
| Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active Cost $0.012$12.17 per 1,000 questionsper 1M questions Logic 41.9% · Math 59.0% | — | 53.1%51.4%–54.8% | g | 1,747 / 3,288 | 207 right of 207 kept0 mistakes · 3,081 skipped | 0.789 | 28.1%answered 85% | 339 ms | 472 ms | $0.012$12.17 |
| Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Cost $0.212$212estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 42.6% · Math 54.6% | — | 50.5%48.7%–52.2% | gh | 1,659 / 3,288 | No answers at this threshold | 0.759 | 14.5%answered 92% | 250 ms | 482 ms | $0.212$212estimate · RTX PRO 6000 Blackwell |
| OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Cost $0.069$69.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.6% · Math 53.8% | — | 49.3%47.6%–51.0% | h | 1,620 / 3,288 | 30 right of 30 kept0 mistakes · 3,258 skipped | 0.770 | 1.6%answered 99% | 80 ms | 134 ms | $0.069$69.42estimate · RTX PRO 6000 Blackwell |
| Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Cost $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 40.8% · Math 51.1% | — | 47.6%45.9%–49.3% | h | 1,564 / 3,288 | 18 right of 18 kept0 mistakes · 3,270 skipped | 0.747 | 2.4%answered 99% | 185 ms | 216 ms | $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
REAX Logits
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
REAX 1.3: 71.6% at $0.148 per 1,000 questions ($148 per 1M) (estimate)
Logits Sigma-1 Duo: 69.3% at $0.165 per 1,000 questions ($165 per 1M) (estimate)
REAX Logits
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| REAX 1.3 Pareto Open · 57B | — | $0.148$148estimate · RTX PRO 6000 Blackwell | $148.30 estimate | $0.4876 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Logits Sigma-1 Duo Open · 57B | — | $0.165$165estimate · RTX PRO 6000 Blackwell | $164.92 estimate | $0.5423 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense Cost $0.148$148estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 59.3% · Math 78.0% | — | 71.6%70.0%–73.1% | a | 2,354 / 3,288 | 229 right of 229 kept0 mistakes · 3,059 skipped | 0.878 | 3.8%answered 99% | 200 ms | 260 ms | $0.148$148estimate · RTX PRO 6000 Blackwell |
| Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Cost $0.165$165estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 57.8% · Math 75.2% | — | 69.3%67.7%–70.8% | b | 2,277 / 3,288 | 23 right of 23 kept0 mistakes · 3,265 skipped | 0.868 | 4.4%answered 99% | 207 ms | 324 ms | $0.165$165estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
celeris-1.1-decision: 62.1% at $0.001 per 1,000 questions ($1.13 per 1M)
d1.1: 58.6% at $0.004 per 1,000 questions ($3.98 per 1M)
d1: 50.5% at $0.004 per 1,000 questions ($3.98 per 1M)
Mercury Decide: 50.6% at $0.005 per 1,000 questions ($5.46 per 1M)
d1-ultra: 67.5% at $0.008 per 1,000 questions ($8.37 per 1M)
Microsoft-Decision-1: 54.3% at $0.009 per 1,000 questions ($9.07 per 1M)
SubQ Snap 1.1: 63.0% at $0.011 per 1,000 questions ($10.75 per 1M)
celeris-1-decision: 62.2% at $0.011 per 1,000 questions ($11.25 per 1M)
Drex Max v1.1: 64.8% at $0.016 per 1,000 questions ($16.13 per 1M)
Jev 1.14: 59.3% at $0.021 per 1,000 questions ($21.27 per 1M)
Jev 1.13: 58.0% at $0.021 per 1,000 questions ($21.27 per 1M)
Gemini Decision 1 (20261111): 64.1% at $0.027 per 1,000 questions ($27.07 per 1M)
GLiDE (API): 65.2% at $0.036 per 1,000 questions ($36.42 per 1M)
Celeris Liquid AI Inception Microsoft Subquadratic Nace.AI TypeSafe Google Fastino
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| celeris-1.1-decision Pareto Closed · Efficient | — | $0.001$1.13 | $1.13 | $0.003701 | — | provider price × reported usage |
| d1.1 Closed · Efficient | — | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1 Closed · Efficient | — | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| Mercury Decide Closed · Efficient | — | $0.005$5.46 | $5.46 | $0.01794 | — | provider price × reported usage |
| d1-ultra Pareto Closed · Efficient | — | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
| Microsoft-Decision-1 Closed · Efficient | — | $0.009$9.07 | $9.07 | $0.02983 | — | provider price × reported usage |
| SubQ Snap 1.1 Closed · Efficient | — | $0.011$10.75 | $10.75 | $0.03535 | — | provider price × reported usage |
| celeris-1-decision Closed · Efficient | — | $0.011$11.25 | $11.25 | $0.03701 | — | provider price × reported usage |
| Drex Max v1.1 Closed · Efficient (provisional) | — | $0.016$16.13 | $16.13 | $0.05303 | — | provider price × reported usage |
| Jev 1.14 Closed · Efficient | — | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Jev 1.13 Closed · Efficient | — | $0.021$21.27 | $21.27 | $0.06993 | — | provider price × reported usage |
| Gemini Decision 1 20261111 Closed · Efficient | — | $0.027$27.07 | $27.07 | $0.08901 | — | provider price × reported usage |
| GLiDE API Closed · Efficient | — | $0.036$36.42 | $36.42 | $0.1197 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
SubQ Snap Preview and SubQ Snap 1.1 returned the same answer to every question (and the same probabilities on 3,276 of 3,288); shown once.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 54.9% · Math 74.1% | — | 67.5%65.9%–69.1% | a | 2,221 / 3,288 | 233 right of 233 kept0 mistakes · 3,055 skipped | 0.852 | 89.1%answered 52% | 180 ms | 426 ms | $0.008$8.37 |
| GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker Cost $0.036$36.42 per 1,000 questionsper 1M questions Logic 54.5% · Math 70.8% | — | 65.2%63.6%–66.8% | ab | 2,144 / 3,288 | 213 right of 213 kept0 mistakes · 3,075 skipped | 0.852 | 49.2%answered 80% | 266 ms | 284 ms | $0.036$36.42 |
| Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker Cost $0.016$16.13 per 1,000 questionsper 1M questions Logic 54.8% · Math 70.1% | — | 64.8%63.2%–66.5% | bc | 2,132 / 3,288 | 404 right of 405 kept1 wrong · 2,883 skipped | 0.846 | 48.3%answered 79% | 327 ms | 518 ms | $0.016$16.13 |
| Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) Cost $0.027$27.07 per 1,000 questionsper 1M questions Logic 50.0% · Math 71.3% | — | 64.1%62.4%–65.7% | bc | 2,106 / 3,288 | 257 right of 257 kept0 mistakes · 3,031 skipped | 0.838 | 3.0%answered 99% | 227 ms | 347 ms | $0.027$27.07 |
| SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker Shown once with SubQ Snap Preview: the same answer to every question Cost $0.011$10.75 per 1,000 questionsper 1M questions Logic 52.4% · Math 68.6% | — | 63.0%61.4%–64.7% | bc | 2,072 / 3,288 | 430 right of 433 kept3 wrong · 2,855 skipped | 0.827 | 58.3%answered 73% | 382 ms | 391 ms | $0.011$10.75 |
| celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.011$11.25 per 1,000 questionsper 1M questions Logic 47.3% · Math 70.0% | — | 62.2%60.6%–63.9% | c | 2,046 / 3,288 | No answers at this threshold | 0.829 | 87.2%answered 50% | 388 ms | 392 ms | $0.011$11.25 |
| celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker Cost $0.001$1.13 per 1,000 questionsper 1M questions Logic 47.3% · Math 69.9% | — | 62.1%60.5%–63.8% | c | 2,043 / 3,288 | No answers at this threshold | 0.830 | 86.7%answered 51% | 381 ms | 388 ms | $0.001$1.13 |
| Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 48.2% · Math 65.1% | — | 59.3%57.6%–61.0% | d | 1,951 / 3,288 | 263 right of 263 kept0 mistakes · 3,025 skipped | 0.810 | 15.4%answered 93% | 164 ms | 188 ms | $0.021$21.27 |
| d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 46.0% · Math 65.2% | — | 58.6%56.9%–60.3% | d | 1,928 / 3,288 | 231 right of 231 kept0 mistakes · 3,057 skipped | 0.811 | 86.0%answered 46% | 174 ms | 285 ms | $0.004$3.98 |
| Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker Cost $0.021$21.27 per 1,000 questionsper 1M questions Logic 46.3% · Math 64.1% | — | 58.0%56.3%–59.7% | d | 1,907 / 3,288 | 313 right of 313 kept0 mistakes · 2,975 skipped | 0.817 | 10.2%answered 96% | 166 ms | 205 ms | $0.021$21.27 |
| Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker Base model: Qwen3.5-9B Cost $0.009$9.07 per 1,000 questionsper 1M questions Logic 44.4% · Math 59.4% | — | 54.3%52.6%–56.0% | e | 1,784 / 3,288 | 47 right of 47 kept0 mistakes · 3,241 skipped | 0.769 | 11.4%answered 94% | 312 ms | 360 ms | $0.009$9.07 |
| Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker Cost $0.005$5.46 per 1,000 questionsper 1M questions Logic 45.1% · Math 53.5% | — | 50.6%48.9%–52.3% | f | 1,664 / 3,288 | 356 right of 372 kept16 wrong · 2,916 skipped | 0.777 | 10.1%answered 95% | 293 ms | 349 ms | $0.005$5.46 |
| d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 40.8% · Math 55.6% | — | 50.5%48.8%–52.3% | f | 1,662 / 3,288 | 207 right of 208 kept1 wrong · 3,080 skipped | 0.757 | 6.4%answered 97% | 185 ms | 292 ms | $0.004$3.98 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
No workhorse decision-model setup has answers on Logic and Math yet. Decision-model runs appear here as soon as the first one stores answers.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
No flagship decision-model setup has answers on Logic and Math yet. Decision-model runs appear here as soon as the first one stores answers.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
d1.2: 59.9% at $0.004 per 1,000 questions ($3.98 per 1M)
d1.2-ultra: 63.6% at $0.008 per 1,000 questions ($8.37 per 1M)
Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| d1.2 Pareto Closed · Undisclosed | — | $0.004$3.98 | $3.98 | $0.0131 | — | provider price × reported usage |
| d1.2-ultra Pareto Closed · Undisclosed | — | $0.008$8.37 | $8.37 | $0.02751 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.008$8.37 per 1,000 questionsper 1M questions Logic 51.6% · Math 69.9% | — | 63.6%61.9%–65.2% | a | 2,091 / 3,288 | 128 right of 128 kept0 mistakes · 3,160 skipped | 0.856 | 90.2%answered 48% | 217 ms | 347 ms | $0.008$8.37 |
| d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker Cost $0.004$3.98 per 1,000 questionsper 1M questions Logic 49.2% · Math 65.5% | — | 59.9%58.2%–61.6% | b | 1,970 / 3,288 | 169 right of 169 kept0 mistakes · 3,119 skipped | 0.812 | 66.9%answered 65% | 208 ms | 390 ms | $0.004$3.98 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Other Google eCloud Technology MiniMax Z.ai OpenBMB Xiaomi Alibaba DeepSeek Meta Liquid AI Thinking Machines IBM KRAFTON Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
LFM3-1.7B (off · chat): 94.2% at $0.206 per 1,000 questions ($206 per 1M) (estimate)
Gemma 4 E2B (on · chat): 89.3% at $0.215 per 1,000 questions ($215 per 1M) (estimate)
Erk-14B (off · chat): 78.3% at $0.308 per 1,000 questions ($308 per 1M) (estimate)
Gemma 4 E2B (chat): 88.6% at $0.313 per 1,000 questions ($313 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 90.5% at $0.325 per 1,000 questions ($325 per 1M) (estimate)
MiniMax-M3.1-Flash (high · chat): 96.8% at $0.342 per 1,000 questions ($342 per 1M)
Gemma 4 E4B (on · chat): 92.9% at $0.375 per 1,000 questions ($375 per 1M) (estimate)
LFM3-1.7B (on · chat): 93.9% at $0.435 per 1,000 questions ($435 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 94.0% at $0.449 per 1,000 questions ($449 per 1M) (estimate)
Gemma 4 E4B (chat): 93.0% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
GLM-5.3-Flash (high · chat): 96.9% at $0.471 per 1,000 questions ($471 per 1M)
Gemma 4 E4B (on · chat · 32k cap · sampled): 94.2% at $0.549 per 1,000 questions ($549 per 1M) (estimate)
MiniCPM5-2B (off · chat): 93.2% at $0.598 per 1,000 questions ($598 per 1M) (estimate)
MiMo-V2.6-Flash (on · chat): 98.0% at $0.615 per 1,000 questions ($615 per 1M)
Gemma 4 E2B (on · chat · 32k cap): 90.3% at $0.678 per 1,000 questions ($678 per 1M) (estimate)
Qwen3.5-4B (off · chat): 95.6% at $0.683 per 1,000 questions ($683 per 1M) (estimate)
DeepSeek-V4.1-Flash (high · chat · 1017): 98.4% at $0.713 per 1,000 questions ($713 per 1M)
Qwen3.8 Flash (off · chat): 97.6% at $0.732 per 1,000 questions ($732 per 1M)
Gemma 4 12B Unified (chat): 95.0% at $0.749 per 1,000 questions ($749 per 1M) (estimate)
Gemma 4 31B (chat): 97.3% at $0.775 per 1,000 questions ($775 per 1M) (estimate)
Qwen3.5-2B (off · chat): 84.4% at $0.788 per 1,000 questions ($788 per 1M) (estimate)
Qwen3-14B (off · chat): 85.4% at $0.840 per 1,000 questions ($840 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 93.9% at $0.855 per 1,000 questions ($855 per 1M) (estimate)
Muse Ember (max · chat): 98.2% at $0.863 per 1,000 questions ($863 per 1M)
LFM2.5-2.6B (on · chat): 87.6% at $1.046 per 1,000 questions ($1,046 per 1M) (estimate)
MiniCPM5-2B (on · chat): 95.7% at $1.410 per 1,000 questions ($1,410 per 1M) (estimate)
Inkling 2 Small (high · chat): 98.2% at $1.830 per 1,000 questions ($1,830 per 1M)
LFM2.5-8B-A1B (on · chat): 83.9% at $1.913 per 1,000 questions ($1,913 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
Qwen3.8-27B (low · chat): 97.4% at $2.020 per 1,000 questions ($2,020 per 1M) (estimate)
Qwen3.5-4B (on · chat): 97.0% at $2.042 per 1,000 questions ($2,042 per 1M) (estimate)
Inkling-Small (high · chat · Jul 2026): 98.2% at $2.506 per 1,000 questions ($2,506 per 1M)
Qwen4-Max (high · chat): 98.6% at $2.919 per 1,000 questions ($2,919 per 1M)
Qwen3.5-2B (on · chat): 86.4% at $3.106 per 1,000 questions ($3,106 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 94.8% at $3.205 per 1,000 questions ($3,205 per 1M) (estimate)
Granite-4.2-3B (high · chat): 96.6% at $4.241 per 1,000 questions ($4,241 per 1M) (estimate)
Raon-1 nano (off · chat): 81.2% at $4.422 per 1,000 questions ($4,422 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (on · chat): 88.1% at $6.344 per 1,000 questions ($6,344 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Claude Sonnet 5.5 (chat): 98.2% at $7.042 per 1,000 questions ($7,042 per 1M)
Claude Opus 5.5 (chat): 98.5% at $13.179 per 1,000 questions ($13,179 per 1M)
Other Google eCloud Technology MiniMax Z.ai OpenBMB Xiaomi Alibaba DeepSeek Meta Liquid AI Thinking Machines IBM KRAFTON Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | $0.206$206estimate · DGX Spark | $205.97 estimate | $0.6772 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | $0.215$215estimate · DGX Spark | $215.10 estimate | $0.7073 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Erk-14B off · chat Open · 15B | — | $0.308$308estimate · RTX PRO 6000 Blackwell | $308.39 estimate | $1.014 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B chat Open · 5.1B | — | $0.313$313estimate · RTX PRO 6000 Blackwell | $312.60 estimate | $1.028 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | $0.325$325estimate · RTX PRO 6000 Blackwell | $325.22 estimate | $1.064 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | $0.342$342 | $342.49 | $1.126 | — | provider price × reported usage |
| Gemma 4 E4B on · chat Open · 8.0B | — | $0.375$375estimate · DGX Spark | $375.43 estimate | $1.234 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| LFM3-1.7B on · chat Open · 1.7B | — | $0.435$435estimate · RTX PRO 6000 Blackwell | $435.17 estimate | $1.408 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | $0.449$449estimate · RTX PRO 6000 Blackwell | $448.98 estimate | $1.499 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B chat Open · 8.0B | — | $0.467$467estimate · RTX PRO 6000 Blackwell | $466.58 estimate | $1.534 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| GLM-5.3-Flash Pareto high · chat Open · 320B MoE · 18B active | — | $0.471$471 | $471.05 | $1.549 | — | provider price × reported usage |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B | — | $0.549$549estimate · RTX PRO 6000 Blackwell | $548.55 estimate | $1.795 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | $0.598$598estimate · RTX PRO 6000 Blackwell | $597.99 estimate | $2.074 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | $0.615$615 | $615.00 | $2.022 | — | provider price × reported usage |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | $0.678$678estimate · RTX PRO 6000 Blackwell | $677.57 estimate | $2.228 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B off · chat Open · 4.7B | — | $0.683$683estimate · RTX PRO 6000 Blackwell | $683.25 estimate | $2.247 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | $0.713$713 | $713.10 | $2.345 | — | provider price × reported usage |
| Qwen3.8 Flash off · chat Closed · Undisclosed | — | $0.732$732 | $732.01 | $2.407 | — | provider price × reported usage |
| Gemma 4 12B Unified chat Open · 12B | — | $0.749$749estimate · RTX PRO 6000 Blackwell | $749.31 estimate | $2.464 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 31B chat Open · 31B | — | $0.775$775estimate · RTX PRO 6000 Blackwell | $774.76 estimate | $2.547 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | $0.788$788estimate · RTX PRO 6000 Blackwell | $788.01 estimate | $2.591 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | $0.840$840estimate · RTX PRO 6000 Blackwell | $839.76 estimate | $2.761 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | $0.855$855estimate · RTX PRO 6000 Blackwell | $854.91 estimate | $2.811 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Muse Ember max · chat Closed · Efficient | — | $0.863$863 | $862.71 | $2.837 | — | provider price × reported usage |
| LFM2.5-2.6B on · chat Open · 2.7B | — | $1.046$1,046estimate · RTX PRO 6000 Blackwell | $1,045.70 estimate | $3.438 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B on · chat Open · 2.5B | — | $1.410$1,410estimate · RTX PRO 6000 Blackwell | $1,409.70 estimate | $4.726 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | $1.830$1,830 | $1,830.16 | $6.018 | — | provider price × reported usage |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | $1.913$1,913estimate · RTX PRO 6000 Blackwell† | $1,913.02 estimate† | $6.29 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B low · chat Open · 27B | — | $2.020$2,020estimate · RTX PRO 6000 Blackwell | $2,019.68 estimate | $6.641 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B on · chat Open · 4.7B | — | $2.042$2,042estimate · RTX PRO 6000 Blackwell | $2,042.18 estimate | $6.715 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | $2.506$2,506 | $2,505.67 | $8.239 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | $2.919$2,919 | $2,919.14 | $9.598 | — | provider price × reported usage |
| Qwen3.5-2B on · chat Open · 2.0B | — | $3.106$3,106estimate · RTX PRO 6000 Blackwell | $3,106.40 estimate | $10.21 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | $3.205$3,205estimate · RTX PRO 6000 Blackwell | $3,205.06 estimate | $10.54 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | $4.241$4,241estimate · RTX PRO 6000 Blackwell | $4,240.67 estimate | $13.94 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | $4.422$4,422estimate · RTX PRO 6000 Blackwell† | $4,422.21 estimate† | $14.54 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | $6.344$6,344estimate · RTX PRO 6000 Blackwell† | $6,344.06 estimate† | $20.86 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Claude Sonnet 5.5 chat Closed · Workhorse | — | $7.042$7,042 API list-price equivalent | $7,042.33 | $23.16 | — | Claude subscription |
| Claude Opus 5.5 chat Closed · Flagship | — | $13.179$13,179 API list-price equivalent | $13,179.29 | $43.33 | — | Claude subscription |
| GPT-6 Astra chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Hydrogen chat | — | — | — | — | — | |
| Claude Fable 5.1 chat Closed · Flagship | — | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6 Luna chat Closed · Efficient | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 9 setups not yet: DeepSeek-V4-Flash (chat) (missing Logic); Qwen3.5-397B-A17B (on · chat) (missing Math); GPT-6 Luna (max · chat) (missing Logic and Math); Claude Haiku 5.5 (max · chat) (missing Logic and Math); Claude Fable 5.2 (chat) (missing Logic and Math); Grok 4.7 (chat) (missing Logic); Spark-X2.6-4B (high · chat) (missing Logic and Math); Granite-4.2-3B (off · chat) (missing Logic and Math); Gemini 4 Carbon (high · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% · Math 99.6% | — | 98.6%98.2%–99.0% | a | 3,243 / 3,288 | Not measured | Not measured | 60.9%answered 99% | 5,580 ms | 9,580 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% · Math 99.6% | — | 98.6%98.1%–98.9% | a | 3,242 / 3,288 | Not measured | Not measured | 53.3%answered 99% | 16,350 ms | 90,630 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% · Math 99.6% | — | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 66.2%answered 99% | 5,240 ms | 10,410 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $2.919$2,919 per 1,000 questionsper 1M questions Logic 96.3% · Math 99.8% | — | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 10.7%answered 100% | 5,700 ms | 19,750 ms | $2.919$2,919 |
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $13.179$13,179 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% · Math 99.8% | — | 98.5%98.1%–98.9% | a | 3,240 / 3,288 | Not measured | Not measured | 41.3%answered 99% | 3,600 ms | 8,380 ms | $13.179$13,179 API list-price equivalent |
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.713$713 per 1,000 questionsper 1M questions Logic 96.2% · Math 99.6% | — | 98.4%98.0%–98.8% | a | 3,237 / 3,288 | Not measured | Not measured | 6.3%answered 100% | 2,270 ms | 14,870 ms | $0.713$713 |
| Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Logic 96.1% · Math 99.6% | — | 98.4%97.9%–98.8% | ab | 3,235 / 3,288 | Not measured | Not measured | 78.3%answered 98% | 4,920 ms | 12,150 ms | — |
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $7.042$7,042 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% · Math 99.7% | — | 98.2%97.7%–98.6% | abc | 3,230 / 3,288 | Not measured | Not measured | 43.7%answered 99% | 4,030 ms | 10,080 ms | $7.042$7,042 API list-price equivalent |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $1.830$1,830 per 1,000 questionsper 1M questions Logic 95.6% · Math 99.5% | — | 98.2%97.7%–98.6% | abcd | 3,229 / 3,288 | Not measured | Not measured | 41.7%answered 99% | 2,870 ms | 35,530 ms | $1.830$1,830 |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% · Math 99.7% | — | 98.2%97.7%–98.6% | abcd | 3,228 / 3,288 | Not measured | Not measured | 15.7%answered 100% | 4,940 ms | 8,620 ms | $0.000$0.00 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $2.506$2,506 per 1,000 questionsper 1M questions Logic 95.4% · Math 99.6% | — | 98.2%97.7%–98.6% | abcd | 3,228 / 3,288 | Not measured | Not measured | 50.6%answered 99% | 2,750 ms | 34,100 ms | $2.506$2,506 |
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.863$863 per 1,000 questionsper 1M questions Logic 95.5% · Math 99.6% | — | 98.2%97.7%–98.6% | abcd | 3,228 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 1,460 ms | 6,300 ms | $0.863$863 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.615$615 per 1,000 questionsper 1M questions Logic 95.1% · Math 99.4% | — | 98.0%97.4%–98.4% | abcde | 3,221 / 3,288 | Not measured | Not measured | 27.6%answered 99% | 10,590 ms | 53,970 ms | $0.615$615 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.732$732 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.5% | — | 97.6%97.0%–98.1% | bcdef | 3,209 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 6,780 ms | 36,290 ms | $0.732$732 |
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $2.020$2,020estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | 97.4%96.8%–97.9% | cdef | 3,204 / 3,288 | Not measured | Not measured | 33.9%answered 99% | 11,200 ms | 39,560 ms | $2.020$2,020estimate · RTX PRO 6000 Blackwell |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | 97.4%96.8%–97.9% | cdef | 3,203 / 3,288 | Not measured | Not measured | 65.0%answered 98% | 4,440 ms | 8,290 ms | $0.000$0.00 |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.775$775estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% · Math 99.4% | — | 97.3%96.7%–97.8% | def | 3,200 / 3,288 | Not measured | Not measured | 17.0%answered 99% | 12,575 ms | 25,280 ms | $0.775$775estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $2.042$2,042estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% · Math 99.4% | — | 97.0%96.4%–97.5% | ef | 3,190 / 3,288 | Not measured | Not measured | 34.2%answered 99% | 63,270 ms | 193,950 ms | $2.042$2,042estimate · RTX PRO 6000 Blackwell |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.471$471 per 1,000 questionsper 1M questions Logic 93.2% · Math 98.8% | — | 96.9%96.2%–97.4% | fg | 3,185 / 3,288 | Not measured | Not measured | 7.4%answered 100% | 4,640 ms | 25,900 ms | $0.471$471 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.342$342 per 1,000 questionsper 1M questions Logic 94.0% · Math 98.2% | — | 96.8%96.1%–97.3% | fg | 3,182 / 3,288 | Not measured | Not measured | 38.9%answered 99% | 2,170 ms | 15,760 ms | $0.342$342 |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $4.241$4,241estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% · Math 99.2% | — | 96.6%96.0%–97.2% | fg | 3,177 / 3,288 | Not measured | Not measured | 54.7%answered 98% | 25,615 ms | 523,100 ms | $4.241$4,241estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $1.410$1,410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% · Math 99.2% | — | 95.7%95.4%–96.0% min–max of 3 seeds | gh | 3,145 / 3,288seed 0 run | Not measured | Not measured | 51.8%answered 97% | 19,260 ms | 194,917 ms | $1.410$1,410estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.683$683estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% · Math 98.5% | — | 95.6%94.9%–96.3% | gh | 3,144 / 3,288 | Not measured | Not measured | 34.0%answered 98% | 14,295 ms | 78,230 ms | $0.683$683estimate · RTX PRO 6000 Blackwell |
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.749$749estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% · Math 98.8% | — | 95.0%94.2%–95.7% | hi | 3,125 / 3,288 | Not measured | Not measured | 15.3%answered 99% | 22,345 ms | 53,390 ms | $0.749$749estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $3.205$3,205estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% · Math 99.0% | — | 94.8%94.0%–95.5% | hij | 3,117 / 3,288 | Not measured | Not measured | 44.0%answered 97% | 42,425 ms | 432,510 ms | $3.205$3,205estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.549$549estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% · Math 98.5% | — | 94.2%94.0%–94.4% min–max of 3 seeds | hijk | 3,104 / 3,288seed 0 run | Not measured | Not measured | 54.1%answered 96% | 13,945 ms | 51,987 ms | $0.549$549estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.206$206estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% · Math 98.7% | — | 94.2%93.3%–94.9% | hijk | 3,097 / 3,288 | Not measured | Not measured | 55.9%answered 96% | 29,040 ms | 343,820 ms | $0.206$206estimate · DGX Spark |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.449$449estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% · Math 98.9% | — | 94.0%93.8%–94.2% min–max of 3 seeds | hijk | 3,098 / 3,288seed 0 run | Not measured | Not measured | Not measured | 5,667 ms | 58,313 ms | $0.449$449estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.435$435estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% · Math 98.9% | — | 93.9%93.89%–93.95% min–max of 3 seeds | ijk | 3,089 / 3,288seed 0 run | Not measured | Not measured | 46.7%answered 96% | 5,630 ms | 63,067 ms | $0.435$435estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.855$855estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% · Math 98.5% | — | 93.9%93.0%–94.7% | ijk | 3,087 / 3,288 | Not measured | Not measured | Not measured | 25,530 ms | 77,380 ms | $0.855$855estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.598$598estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% · Math 98.2% | — | 93.2%92.9%–93.3% min–max of 3 seeds | jk | 3,068 / 3,288seed 0 run | Not measured | Not measured | 41.0%answered 95% | 9,065 ms | 75,703 ms | $0.598$598estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% · Math 98.1% | — | 93.0%92.1%–93.8% | k | 3,058 / 3,288 | Not measured | Not measured | 51.0%answered 95% | 13,150 ms | 44,990 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.375$375estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% · Math 98.2% | — | 92.9%92.0%–93.7% | k | 3,054 / 3,288 | Not measured | Not measured | 43.7%answered 96% | 131,535 ms | 392,620 ms | $0.375$375estimate · DGX Spark |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.325$325estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% · Math 97.7% | — | 90.5%90.2%–90.9% min–max of 3 seeds | l | 2,966 / 3,288seed 0 run | Not measured | Not measured | 31.0%answered 97% | 8,873 ms | 32,503 ms | $0.325$325estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.678$678estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% · Math 97.8% | — | 90.3%89.3%–91.3% | l | 2,970 / 3,288 | Not measured | Not measured | Not measured | 19,100 ms | 54,630 ms | $0.678$678estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.215$215estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% · Math 96.9% | — | 89.3%88.2%–90.3% | lm | 2,936 / 3,288 | Not measured | Not measured | 24.3%answered 97% | 77,885 ms | 232,800 ms | $0.215$215estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.313$313estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% · Math 96.8% | — | 88.6%87.5%–89.6% | lmn | 2,913 / 3,288 | Not measured | Not measured | 32.3%answered 96% | 9,935 ms | 27,690 ms | $0.313$313estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $6.344$6,344estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% · Math 97.0% | — | 88.1%86.9%–89.1% | mn | 2,896 / 3,288 | Not measured | Not measured | Not measured | 54,985 ms† | 987,660 ms† | $6.344$6,344estimate · RTX PRO 6000 Blackwell† |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $1.046$1,046estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% · Math 94.0% | — | 87.6%86.5%–88.7% | mno | 2,881 / 3,288 | Not measured | Not measured | 40.6%answered 95% | 14,035 ms | 142,970 ms | $1.046$1,046estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $3.106$3,106estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% · Math 94.7% | — | 86.4%85.2%–87.5% | nop | 2,840 / 3,288 | Not measured | Not measured | 30.9%answered 97% | 121,840 ms | 241,590 ms | $3.106$3,106estimate · RTX PRO 6000 Blackwell |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.840$840estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% · Math 95.8% | — | 85.4%84.1%–86.5% | op | 2,807 / 3,288 | Not measured | Not measured | 20.9%answered 96% | 9,380 ms | 35,110 ms | $0.840$840estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.788$788estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% · Math 94.6% | — | 84.4%83.1%–85.6% | p | 2,776 / 3,288 | Not measured | Not measured | 34.3%answered 94% | 17,805 ms | 92,100 ms | $0.788$788estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $1.913$1,913estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% · Math 96.3% | — | 83.9%82.6%–85.1% | pq | 2,759 / 3,288 | Not measured | Not measured | 55.9%answered 88% | 20,095 ms† | 230,570 ms† | $1.913$1,913estimate · RTX PRO 6000 Blackwell† |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% · Math 93.7% | — | 81.4%80.1%–82.7% | q | 2,678 / 3,288 | Not measured | Not measured | Not measured | 3,830 ms | 5,300 ms | $0.000$0.00 |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $4.422$4,422estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% · Math 89.6% | — | 81.2%79.8%–82.5% | qr | 2,669 / 3,288 | Not measured | Not measured | 31.1%answered 94% | 42,585 ms† | 593,950 ms† | $4.422$4,422estimate · RTX PRO 6000 Blackwell† |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.308$308estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% · Math 90.8% | — | 78.3%76.8%–79.6% | r | 2,573 / 3,288 | Not measured | Not measured | 32.3%answered 92% | 8,740 ms | 16,420 ms | $0.308$308estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Other Google OpenBMB Alibaba Liquid AI IBM KRAFTON
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
LFM3-1.7B (off · chat): 94.2% at $0.206 per 1,000 questions ($206 per 1M) (estimate)
Gemma 4 E2B (on · chat): 89.3% at $0.215 per 1,000 questions ($215 per 1M) (estimate)
Gemma 4 E2B (chat): 88.6% at $0.313 per 1,000 questions ($313 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap · sampled): 90.5% at $0.325 per 1,000 questions ($325 per 1M) (estimate)
LFM3-1.7B (on · chat): 93.9% at $0.435 per 1,000 questions ($435 per 1M) (estimate)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): 94.0% at $0.449 per 1,000 questions ($449 per 1M) (estimate)
MiniCPM5-2B (off · chat): 93.2% at $0.598 per 1,000 questions ($598 per 1M) (estimate)
Gemma 4 E2B (on · chat · 32k cap): 90.3% at $0.678 per 1,000 questions ($678 per 1M) (estimate)
Qwen3.5-4B (off · chat): 95.6% at $0.683 per 1,000 questions ($683 per 1M) (estimate)
Qwen3.5-2B (off · chat): 84.4% at $0.788 per 1,000 questions ($788 per 1M) (estimate)
LFM2.5-2.6B (on · chat): 87.6% at $1.046 per 1,000 questions ($1,046 per 1M) (estimate)
MiniCPM5-2B (on · chat): 95.7% at $1.410 per 1,000 questions ($1,410 per 1M) (estimate)
Qwen3.5-4B (on · chat): 97.0% at $2.042 per 1,000 questions ($2,042 per 1M) (estimate)
Qwen3.5-2B (on · chat): 86.4% at $3.106 per 1,000 questions ($3,106 per 1M) (estimate)
Granite-4.2-3B (high · chat): 96.6% at $4.241 per 1,000 questions ($4,241 per 1M) (estimate)
Raon-1 nano (off · chat): 81.2% at $4.422 per 1,000 questions ($4,422 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Raon-1 nano (on · chat): 88.1% at $6.344 per 1,000 questions ($6,344 per 1M) (estimate) (other engine: Logic: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; Math: served on KRAFTON's vLLM fork 13a5dd87, not stock vLLM: cost and speed not comparable; see its record on Method)
Other Google OpenBMB Alibaba Liquid AI IBM KRAFTON
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| LFM3-1.7B Pareto off · chat Open · 1.7B | — | $0.206$206estimate · DGX Spark | $205.97 estimate | $0.6772 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B on · chat Open · 5.1B | — | $0.215$215estimate · DGX Spark | $215.10 estimate | $0.7073 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E2B chat Open · 5.1B | — | $0.313$313estimate · RTX PRO 6000 Blackwell | $312.60 estimate | $1.028 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B | — | $0.325$325estimate · RTX PRO 6000 Blackwell | $325.22 estimate | $1.064 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B on · chat Open · 1.7B | — | $0.435$435estimate · RTX PRO 6000 Blackwell | $435.17 estimate | $1.408 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B | — | $0.449$449estimate · RTX PRO 6000 Blackwell | $448.98 estimate | $1.499 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B off · chat Open · 2.5B | — | $0.598$598estimate · RTX PRO 6000 Blackwell | $597.99 estimate | $2.074 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B | — | $0.678$678estimate · RTX PRO 6000 Blackwell | $677.57 estimate | $2.228 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto off · chat Open · 4.7B | — | $0.683$683estimate · RTX PRO 6000 Blackwell | $683.25 estimate | $2.247 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B off · chat Open · 2.0B | — | $0.788$788estimate · RTX PRO 6000 Blackwell | $788.01 estimate | $2.591 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-2.6B on · chat Open · 2.7B | — | $1.046$1,046estimate · RTX PRO 6000 Blackwell | $1,045.70 estimate | $3.438 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| MiniCPM5-2B Pareto on · chat Open · 2.5B | — | $1.410$1,410estimate · RTX PRO 6000 Blackwell | $1,409.70 estimate | $4.726 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-4B Pareto on · chat Open · 4.7B | — | $2.042$2,042estimate · RTX PRO 6000 Blackwell | $2,042.18 estimate | $6.715 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.5-2B on · chat Open · 2.0B | — | $3.106$3,106estimate · RTX PRO 6000 Blackwell | $3,106.40 estimate | $10.21 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Granite-4.2-3B high · chat Open · 3.0B | — | $4.241$4,241estimate · RTX PRO 6000 Blackwell | $4,240.67 estimate | $13.94 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano off · chat Open · 3.1B | — | $4.422$4,422estimate · RTX PRO 6000 Blackwell† | $4,422.21 estimate† | $14.54 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Raon-1 nano on · chat Open · 3.1B | — | $6.344$6,344estimate · RTX PRO 6000 Blackwell† | $6,344.06 estimate† | $20.86 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: Spark-X2.6-4B (high · chat) (missing Logic and Math); Granite-4.2-3B (off · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Cost $2.042$2,042estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 92.4% · Math 99.4% | — | 97.0%96.4%–97.5% | a | 3,190 / 3,288 | Not measured | Not measured | 34.2%answered 99% | 63,270 ms | 193,950 ms | $2.042$2,042estimate · RTX PRO 6000 Blackwell |
| Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Cost $4.241$4,241estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 91.6% · Math 99.2% | — | 96.6%96.0%–97.2% | ab | 3,177 / 3,288 | Not measured | Not measured | 54.7%answered 98% | 25,615 ms | 523,100 ms | $4.241$4,241estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Cost $1.410$1,410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 88.9% · Math 99.2% | — | 95.7%95.4%–96.0% min–max of 3 seeds | b | 3,145 / 3,288seed 0 run | Not measured | Not measured | 51.8%answered 97% | 19,260 ms | 194,917 ms | $1.410$1,410estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Cost $0.683$683estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 90.1% · Math 98.5% | — | 95.6%94.9%–96.3% | b | 3,144 / 3,288 | Not measured | Not measured | 34.0%answered 98% | 14,295 ms | 78,230 ms | $0.683$683estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Cost $0.206$206estimate · DGX Spark per 1,000 questionsper 1M questions Logic 85.5% · Math 98.7% | — | 94.2%93.3%–94.9% | c | 3,097 / 3,288 | Not measured | Not measured | 55.9%answered 96% | 29,040 ms | 343,820 ms | $0.206$206estimate · DGX Spark |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Cost $0.449$449estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.6% · Math 98.9% | — | 94.0%93.8%–94.2% min–max of 3 seeds | c | 3,098 / 3,288seed 0 run | Not measured | Not measured | Not measured | 5,667 ms | 58,313 ms | $0.449$449estimate · RTX PRO 6000 Blackwell |
| LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Cost $0.435$435estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 84.2% · Math 98.9% | — | 93.9%93.89%–93.95% min–max of 3 seeds | c | 3,089 / 3,288seed 0 run | Not measured | Not measured | 46.7%answered 96% | 5,630 ms | 63,067 ms | $0.435$435estimate · RTX PRO 6000 Blackwell |
| MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Cost $0.598$598estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.6% · Math 98.2% | — | 93.2%92.9%–93.3% min–max of 3 seeds | c | 3,068 / 3,288seed 0 run | Not measured | Not measured | 41.0%answered 95% | 9,065 ms | 75,703 ms | $0.598$598estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.325$325estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 76.8% · Math 97.7% | — | 90.5%90.2%–90.9% min–max of 3 seeds | de | 2,966 / 3,288seed 0 run | Not measured | Not measured | 31.0%answered 97% | 8,873 ms | 32,503 ms | $0.325$325estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.678$678estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.9% · Math 97.8% | — | 90.3%89.3%–91.3% | d | 2,970 / 3,288 | Not measured | Not measured | Not measured | 19,100 ms | 54,630 ms | $0.678$678estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.215$215estimate · DGX Spark per 1,000 questionsper 1M questions Logic 74.6% · Math 96.9% | — | 89.3%88.2%–90.3% | def | 2,936 / 3,288 | Not measured | Not measured | 24.3%answered 97% | 77,885 ms | 232,800 ms | $0.215$215estimate · DGX Spark |
| Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Cost $0.313$313estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 72.9% · Math 96.8% | — | 88.6%87.5%–89.6% | ef | 2,913 / 3,288 | Not measured | Not measured | 32.3%answered 96% | 9,935 ms | 27,690 ms | $0.313$313estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $6.344$6,344estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 70.9% · Math 97.0% | — | 88.1%86.9%–89.1% | fg | 2,896 / 3,288 | Not measured | Not measured | Not measured | 54,985 ms† | 987,660 ms† | $6.344$6,344estimate · RTX PRO 6000 Blackwell† |
| LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Cost $1.046$1,046estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 75.4% · Math 94.0% | — | 87.6%86.5%–88.7% | fg | 2,881 / 3,288 | Not measured | Not measured | 40.6%answered 95% | 14,035 ms | 142,970 ms | $1.046$1,046estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Cost $3.106$3,106estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 70.3% · Math 94.7% | — | 86.4%85.2%–87.5% | gh | 2,840 / 3,288 | Not measured | Not measured | 30.9%answered 97% | 121,840 ms | 241,590 ms | $3.106$3,106estimate · RTX PRO 6000 Blackwell |
| Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Cost $0.788$788estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 64.8% · Math 94.6% | — | 84.4%83.1%–85.6% | h | 2,776 / 3,288 | Not measured | Not measured | 34.3%answered 94% | 17,805 ms | 92,100 ms | $0.788$788estimate · RTX PRO 6000 Blackwell |
| Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Cost $4.422$4,422estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 65.1% · Math 89.6% | — | 81.2%79.8%–82.5% | i | 2,669 / 3,288 | Not measured | Not measured | 31.1%answered 94% | 42,585 ms† | 593,950 ms† | $4.422$4,422estimate · RTX PRO 6000 Blackwell† |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
eCloud Technology Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Erk-14B (off · chat): 78.3% at $0.308 per 1,000 questions ($308 per 1M) (estimate)
Gemma 4 E4B (on · chat): 92.9% at $0.375 per 1,000 questions ($375 per 1M) (estimate)
Gemma 4 E4B (chat): 93.0% at $0.467 per 1,000 questions ($467 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap · sampled): 94.2% at $0.549 per 1,000 questions ($549 per 1M) (estimate)
Gemma 4 12B Unified (chat): 95.0% at $0.749 per 1,000 questions ($749 per 1M) (estimate)
Qwen3-14B (off · chat): 85.4% at $0.840 per 1,000 questions ($840 per 1M) (estimate)
Gemma 4 E4B (on · chat · 32k cap): 93.9% at $0.855 per 1,000 questions ($855 per 1M) (estimate)
LFM2.5-8B-A1B (on · chat): 83.9% at $1.913 per 1,000 questions ($1,913 per 1M) (estimate) (mixed concurrency: This run changed the number of requests in flight part-way; see its record on Method)
eCloud Technology Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Erk-14B Pareto off · chat Open · 15B | — | $0.308$308estimate · RTX PRO 6000 Blackwell | $308.39 estimate | $1.014 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat Open · 8.0B | — | $0.375$375estimate · DGX Spark | $375.43 estimate | $1.234 estimate | — | estimate: NVIDIA DGX Spark (GB10) time while the run was running, first to last answer × $0.213/h |
| Gemma 4 E4B Pareto chat Open · 8.0B | — | $0.467$467estimate · RTX PRO 6000 Blackwell | $466.58 estimate | $1.534 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B Pareto on · chat · 32k cap · sampled Open · 8.0B | — | $0.549$549estimate · RTX PRO 6000 Blackwell | $548.55 estimate | $1.795 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 12B Unified Pareto chat Open · 12B | — | $0.749$749estimate · RTX PRO 6000 Blackwell | $749.31 estimate | $2.464 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3-14B off · chat Open · 15B | — | $0.840$840estimate · RTX PRO 6000 Blackwell | $839.76 estimate | $2.761 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B | — | $0.855$855estimate · RTX PRO 6000 Blackwell | $854.91 estimate | $2.811 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active | — | $1.913$1,913estimate · RTX PRO 6000 Blackwell† | $1,913.02 estimate† | $6.29 estimate† | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95%; min–max for 3-seed rows | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Cost $0.749$749estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 87.7% · Math 98.8% | — | 95.0%94.2%–95.7% | a | 3,125 / 3,288 | Not measured | Not measured | 15.3%answered 99% | 22,345 ms | 53,390 ms | $0.749$749estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.549$549estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.9% · Math 98.5% | — | 94.2%94.0%–94.4% min–max of 3 seeds | ab | 3,104 / 3,288seed 0 run | Not measured | Not measured | 54.1%answered 96% | 13,945 ms | 51,987 ms | $0.549$549estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.855$855estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 85.0% · Math 98.5% | — | 93.9%93.0%–94.7% | bc | 3,087 / 3,288 | Not measured | Not measured | Not measured | 25,530 ms | 77,380 ms | $0.855$855estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 83.2% · Math 98.1% | — | 93.0%92.1%–93.8% | c | 3,058 / 3,288 | Not measured | Not measured | 51.0%answered 95% | 13,150 ms | 44,990 ms | $0.467$467estimate · RTX PRO 6000 Blackwell |
| Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Cost $0.375$375estimate · DGX Spark per 1,000 questionsper 1M questions Logic 82.6% · Math 98.2% | — | 92.9%92.0%–93.7% | c | 3,054 / 3,288 | Not measured | Not measured | 43.7%answered 96% | 131,535 ms | 392,620 ms | $0.375$375estimate · DGX Spark |
| Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Cost $0.840$840estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 65.3% · Math 95.8% | — | 85.4%84.1%–86.5% | d | 2,807 / 3,288 | Not measured | Not measured | 20.9%answered 96% | 9,380 ms | 35,110 ms | $0.840$840estimate · RTX PRO 6000 Blackwell |
| LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active Cost $1.913$1,913estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† Logic 60.2% · Math 96.3% | — | 83.9%82.6%–85.1% | d | 2,759 / 3,288 | Not measured | Not measured | 55.9%answered 88% | 20,095 ms† | 230,570 ms† | $1.913$1,913estimate · RTX PRO 6000 Blackwell† |
| Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Cost $0.308$308estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 54.2% · Math 90.8% | — | 78.3%76.8%–79.6% | e | 2,573 / 3,288 | Not measured | Not measured | 32.3%answered 92% | 8,740 ms | 16,420 ms | $0.308$308estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Google Alibaba Liquid AI
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Gemma 4 31B (chat): 97.3% at $0.775 per 1,000 questions ($775 per 1M) (estimate)
Qwen3.8-27B (low · chat): 97.4% at $2.020 per 1,000 questions ($2,020 per 1M) (estimate)
LFM2.5-24B-A2B (on · chat): 94.8% at $3.205 per 1,000 questions ($3,205 per 1M) (estimate)
Google Alibaba Liquid AI
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Gemma 4 31B Pareto chat Open · 31B | — | $0.775$775estimate · RTX PRO 6000 Blackwell | $774.76 estimate | $2.547 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| Qwen3.8-27B Pareto low · chat Open · 27B | — | $2.020$2,020estimate · RTX PRO 6000 Blackwell | $2,019.68 estimate | $6.641 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active | — | $3.205$3,205estimate · RTX PRO 6000 Blackwell | $3,205.06 estimate | $10.54 estimate | — | estimate: NVIDIA RTX PRO 6000 Blackwell (96 GB) time while the run was running, first to last answer × $2.49/h |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Cost $2.020$2,020estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | 97.4%96.8%–97.9% | a | 3,204 / 3,288 | Not measured | Not measured | 33.9%answered 99% | 11,200 ms | 39,560 ms | $2.020$2,020estimate · RTX PRO 6000 Blackwell |
| Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Cost $0.775$775estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 93.3% · Math 99.4% | — | 97.3%96.7%–97.8% | a | 3,200 / 3,288 | Not measured | Not measured | 17.0%answered 99% | 12,575 ms | 25,280 ms | $0.775$775estimate · RTX PRO 6000 Blackwell |
| LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active Cost $3.205$3,205estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions Logic 86.7% · Math 99.0% | — | 94.8%94.0%–95.5% | b | 3,117 / 3,288 | Not measured | Not measured | 44.0%answered 97% | 42,425 ms | 432,510 ms | $3.205$3,205estimate · RTX PRO 6000 Blackwell |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
MiniMax Z.ai Xiaomi DeepSeek Thinking Machines
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
MiniMax-M3.1-Flash (high · chat): 96.8% at $0.342 per 1,000 questions ($342 per 1M)
GLM-5.3-Flash (high · chat): 96.9% at $0.471 per 1,000 questions ($471 per 1M)
MiMo-V2.6-Flash (on · chat): 98.0% at $0.615 per 1,000 questions ($615 per 1M)
DeepSeek-V4.1-Flash (high · chat · 1017): 98.4% at $0.713 per 1,000 questions ($713 per 1M)
Inkling 2 Small (high · chat): 98.2% at $1.830 per 1,000 questions ($1,830 per 1M)
Inkling-Small (high · chat · Jul 2026): 98.2% at $2.506 per 1,000 questions ($2,506 per 1M)
MiniMax Z.ai Xiaomi DeepSeek Thinking Machines
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| MiniMax-M3.1-Flash Pareto high · chat Open · 428B MoE · 23B active | — | $0.342$342 | $342.49 | $1.126 | — | provider price × reported usage |
| GLM-5.3-Flash Pareto high · chat Open · 320B MoE · 18B active | — | $0.471$471 | $471.05 | $1.549 | — | provider price × reported usage |
| MiMo-V2.6-Flash Pareto on · chat Open · 309B MoE · 15B active | — | $0.615$615 | $615.00 | $2.022 | — | provider price × reported usage |
| DeepSeek-V4.1-Flash Pareto high · chat · 1017 Open · 552B MoE · 16B active | — | $0.713$713 | $713.10 | $2.345 | — | provider price × reported usage |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active | — | $1.830$1,830 | $1,830.16 | $6.018 | — | provider price × reported usage |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | — | $2.506$2,506 | $2,505.67 | $8.239 | — | provider price × reported usage |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: DeepSeek-V4-Flash (chat) (missing Logic); Qwen3.5-397B-A17B (on · chat) (missing Math).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active Cost $0.713$713 per 1,000 questionsper 1M questions Logic 96.2% · Math 99.6% | — | 98.4%98.0%–98.8% | a | 3,237 / 3,288 | Not measured | Not measured | 6.3%answered 100% | 2,270 ms | 14,870 ms | $0.713$713 |
| Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active Cost $1.830$1,830 per 1,000 questionsper 1M questions Logic 95.6% · Math 99.5% | — | 98.2%97.7%–98.6% | a | 3,229 / 3,288 | Not measured | Not measured | 41.7%answered 99% | 2,870 ms | 35,530 ms | $1.830$1,830 |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active Cost $2.506$2,506 per 1,000 questionsper 1M questions Logic 95.4% · Math 99.6% | — | 98.2%97.7%–98.6% | a | 3,228 / 3,288 | Not measured | Not measured | 50.6%answered 99% | 2,750 ms | 34,100 ms | $2.506$2,506 |
| MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active Cost $0.615$615 per 1,000 questionsper 1M questions Logic 95.1% · Math 99.4% | — | 98.0%97.4%–98.4% | a | 3,221 / 3,288 | Not measured | Not measured | 27.6%answered 99% | 10,590 ms | 53,970 ms | $0.615$615 |
| GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active Cost $0.471$471 per 1,000 questionsper 1M questions Logic 93.2% · Math 98.8% | — | 96.9%96.2%–97.4% | b | 3,185 / 3,288 | Not measured | Not measured | 7.4%answered 100% | 4,640 ms | 25,900 ms | $0.471$471 |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active Cost $0.342$342 per 1,000 questionsper 1M questions Logic 94.0% · Math 98.2% | — | 96.8%96.1%–97.3% | b | 3,182 / 3,288 | Not measured | Not measured | 38.9%answered 99% | 2,170 ms | 15,760 ms | $0.342$342 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Meta
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Muse Ember (max · chat): 98.2% at $0.863 per 1,000 questions ($863 per 1M)
Meta
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Muse Ember Pareto max · chat Closed · Efficient | — | $0.863$863 | $862.71 | $2.837 | — | provider price × reported usage |
| GPT-6 Luna chat Closed · Efficient | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: GPT-6 Luna (max · chat) (missing Logic and Math); Claude Haiku 5.5 (max · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) Cost $0.863$863 per 1,000 questionsper 1M questions Logic 95.5% · Math 99.6% | — | 98.2%97.7%–98.6% | a | 3,228 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 1,460 ms | 6,300 ms | $0.863$863 |
| GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 58.0% · Math 93.7% | — | 81.4%80.1%–82.7% | b | 2,678 / 3,288 | Not measured | Not measured | Not measured | 3,830 ms | 5,300 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Sonnet 5.5 (chat): 98.2% at $7.042 per 1,000 questions ($7,042 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 Pareto chat Closed · Workhorse | — | $7.042$7,042 API list-price equivalent | $7,042.33 | $23.16 | — | Claude subscription |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $7.042$7,042 API list-price equivalent per 1,000 questionsper 1M questions Logic 95.5% · Math 99.7% | — | 98.2%97.7%–98.6% | a | 3,230 / 3,288 | Not measured | Not measured | 43.7%answered 99% | 4,030 ms | 10,080 ms | $7.042$7,042 API list-price equivalent |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Anthropic
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Claude Opus 5.5 (chat): 98.5% at $13.179 per 1,000 questions ($13,179 per 1M)
Anthropic
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Claude Opus 5.5 Pareto chat Closed · Flagship | — | $13.179$13,179 API list-price equivalent | $13,179.29 | $43.33 | — | Claude subscription |
| Claude Fable 5.1 chat Closed · Flagship | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 1 setup not yet: Claude Fable 5.2 (chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $13.179$13,179 API list-price equivalent per 1,000 questionsper 1M questions Logic 96.2% · Math 99.8% | — | 98.5%98.1%–98.9% | a | 3,240 / 3,288 | Not measured | Not measured | 41.3%answered 99% | 3,600 ms | 8,380 ms | $13.179$13,179 API list-price equivalent |
| Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 95.2% · Math 99.7% | — | 98.2%97.7%–98.6% | b | 3,228 / 3,288 | Not measured | Not measured | 15.7%answered 100% | 4,940 ms | 8,620 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Cost
Dollars per 1,000 questionsper 1M questions (log scale), the run's cost over the questions it answered; “estimate”: a run on our own hardware, its GPU time × the machine's hourly rate (hover, focus or tap a name in the tables for how it was counted); the cost basis of each is in the table below · lower is better
Alibaba
Accuracy across cost
Each setup's accuracy against its cost per 1,000 questions (log scale); the dotted line joins the Pareto front: no other setup is both cheaper and more accurate; shaded: cheaper and more accurate than the median setup
Qwen3.8 Flash (off · chat): 97.6% at $0.732 per 1,000 questions ($732 per 1M)
Qwen4-Max (high · chat): 98.6% at $2.919 per 1,000 questions ($2,919 per 1M)
Alibaba
Cost and its basis
| Setup | Web | Costper 1,000 or 1M questions (switch) | Cost$ per 1M questions | Run total$ for the questions answered | On our hardwareGPU-time cost beside a list price (switch) | Basisprovider price × usage, list price, GPU time, … |
|---|---|---|---|---|---|---|
| Qwen3.8 Flash Pareto off · chat Closed · Undisclosed | — | $0.732$732 | $732.01 | $2.407 | — | provider price × reported usage |
| Qwen4-Max Pareto high · chat Closed · Undisclosed | — | $2.919$2,919 | $2,919.14 | $9.598 | — | provider price × reported usage |
| GPT-6 Astra chat Closed · Undisclosed | — | — | — | — | — | |
| Gemini 4 Carbon chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Sol chat Closed · Undisclosed | — | — | — | — | — | |
| GPT-6.1 Terra chat Closed · Undisclosed | — | — | — | — | — |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
Not in Logic and Math yet
Logic and Math pools only setups with a complete English run on both benchmarks. 2 setups not yet: Grok 4.7 (chat) (missing Logic); Gemini 4 Carbon (high · chat) (missing Logic and Math).
Every setup, every measure
| Setup | Web | Accuracy% of N (N ≤ 3,288) · Wilson 95% | Tie groupHolm · McNemar | Right / Nquestions | Kept at ≥99%right / wrong | AUROCright vs wrong | Non-hallucinationof answers not right, % “I don't know” · higher is better | Median latencyms | p90ms, nearest rank | Cost$ per 1,000 qper 1M q |
|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.7% · Math 99.6% | — | 98.6%98.2%–99.0% | a | 3,243 / 3,288 | Not measured | Not measured | 60.9%answered 99% | 5,580 ms | 9,580 ms | $0.000$0.00 |
| Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) Logic 96.7% · Math 99.6% | — | 98.6%98.1%–98.9% | a | 3,242 / 3,288 | Not measured | Not measured | 53.3%answered 99% | 16,350 ms | 90,630 ms | — |
| GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 96.5% · Math 99.6% | — | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 66.2%answered 99% | 5,240 ms | 10,410 ms | $0.000$0.00 |
| Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) Cost $2.919$2,919 per 1,000 questionsper 1M questions Logic 96.3% · Math 99.8% | — | 98.6%98.1%–98.9% | a | 3,241 / 3,288 | Not measured | Not measured | 10.7%answered 100% | 5,700 ms | 19,750 ms | $2.919$2,919 |
| Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) Cost $0.732$732 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.5% | — | 97.6%97.0%–98.1% | b | 3,209 / 3,288 | Not measured | Not measured | 31.3%answered 99% | 6,780 ms | 36,290 ms | $0.732$732 |
| GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet Cost $0.000$0.00 per 1,000 questionsper 1M questions Logic 94.0% · Math 99.2% | — | 97.4%96.8%–97.9% | b | 3,203 / 3,288 | Not measured | Not measured | 65.0%answered 98% | 4,440 ms | 8,290 ms | $0.000$0.00 |
estimate · RTX PRO 6000 Blackwell / DGX Spark = run on our own hardware; its cost is an estimate, explained on hover, focus or tap of the amount: the machine's wall-clock time on the run, first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out and listed), × $2.49/h per NVIDIA RTX PRO 6000 Blackwell (96 GB) (Runpod's price for the card) or × $0.213/h per NVIDIA DGX Spark (GB10) (its purchase over three years plus electricity; see Method), each run charged in full even when runs shared the machine, so at most; sorting and the cost chart use the estimate; list = maker's list price × our token counts; unmarked = provider's price × usage the API reported.