Logic & Math bench
Leaderboard
| Tier | Model | Logic accuracy 1,125 questions, 95% interval; min–max for 3-seed rows | Math of 2,163 questions | Non-hallucination of answers not right, % “I don't know” | “I don't know” share declined | Calibration error, points; decision models | $ per 1,000 q$ per 1M q list price or GPU time | GPU time s per 1,000 q, our hardware | Latency median, p90 under it |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Logic 96.7% (95%: 95.5%–97.6%) · Math 99.6% Non-hallucination 56.4% · “I don't know” on 2.8% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 6.59 s per question (median latency) | 96.7%95.5%–97.6% | 99.6% | 56.4% | 2.8% | – | $0.000$0.00 | – | 6.59 sp90 14.3 s |
| 1 | Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Logic 96.7% (95%: 95.5%–97.6%) · Math 99.6% Non-hallucination 45.5% · “I don't know” on 1.8% of questions Runs on an API · 44.5 s per question (median latency) | 96.7%95.5%–97.6% | 99.6% | 45.5% | 1.8% | – | – | – | 44.5 sp90 206.7 s |
| 1 | GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Logic 96.5% (95%: 95.3%–97.5%) · Math 99.6% Non-hallucination 63.2% · “I don't know” on 3.2% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 7.07 s per question (median latency) | 96.5%95.3%–97.5% | 99.6% | 63.2% | 3.2% | – | $0.000$0.00 | – | 7.07 sp90 16.2 s |
| 1 | Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Logic 96.3% (95%: 95.0%–97.2%) · Math 99.8% Non-hallucination 0.0% · “I don't know” on 0.0% of questions Runs on an API · cost $5.690$5,690 per 1,000 questionsper 1M questions · 11.6 s per question (median latency) | 96.3%95.0%–97.2% | 99.8% | 0.0% | 0.0% | – | $5.690$5,690 | – | 11.6 sp90 38.6 s |
| 1 | Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $4.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Logic 96.2% (95%: 94.9%–97.2%) · Math 99.8% Non-hallucination 38.2% · “I don't know” on 1.9% of questions Runs on an API · cost $19.540$19,540 API list-price equivalent per 1,000 questionsper 1M questions · 5.83 s per question (median latency) | 96.2%94.9%–97.2% | 99.8% | 38.2% | 1.9% | – | $19.540$19,540 API list-price equivalent | – | 5.83 sp90 12.2 s |
| 1 | DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active · Mixture of experts: 552B total, about 16B active per step. Size class uses 552B. Logic 96.2% (95%: 94.9%–97.2%) · Math 99.6% Non-hallucination 3.8% · “I don't know” on 0.2% of questions Runs on an API · cost $1.647$1,647 per 1,000 questionsper 1M questions · 6.68 s per question (median latency) | 96.2%94.9%–97.2% | 99.6% | 3.8% | 0.2% | – | $1.647$1,647 | – | 6.68 sp90 40.0 s |
| 1 | Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Logic 96.1% (95%: 94.8%–97.1%) · Math 99.6% Non-hallucination 76.1% · “I don't know” on 4.8% of questions Runs on an API · 8.21 s per question (median latency) | 96.1%94.8%–97.1% | 99.6% | 76.1% | 4.8% | – | – | – | 8.21 sp90 24.1 s |
| 1 | Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active · Mixture of experts: 594B total, about 17B active per step. Size class uses 594B. Logic 95.6% (95%: 94.3%–96.7%) · Math 99.5% Non-hallucination 37.3% · “I don't know” on 2.0% of questions Runs on an API · cost $4.121$4,121 per 1,000 questionsper 1M questions · 13.2 s per question (median latency) | 95.6%94.3%–96.7% | 99.5% | 37.3% | 2.0% | – | $4.121$4,121 | – | 13.2 sp90 144.2 s |
| 1 | Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $2.00 per 1M input tokens: Workhorse (source: https://platform.claude.com/docs/en/about-claude/pricing) Logic 95.5% (95%: 94.1%–96.5%) · Math 99.7% Non-hallucination 41.3% · “I don't know” on 2.3% of questions Runs on an API · cost $11.480$11,480 API list-price equivalent per 1,000 questionsper 1M questions · 6.71 s per question (median latency) | 95.5%94.1%–96.5% | 99.7% | 41.3% | 2.3% | – | $11.480$11,480 API list-price equivalent | – | 6.71 sp90 14.9 s |
| 1 | Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/meta/muse-ember) Logic 95.5% (95%: 94.1%–96.5%) · Math 99.6% Non-hallucination 22.6% · “I don't know” on 1.1% of questions Runs on an API · cost $1.820$1,820 per 1,000 questionsper 1M questions · 3.11 s per question (median latency) | 95.5%94.1%–96.5% | 99.6% | 22.6% | 1.1% | – | $1.820$1,820 | – | 3.11 sp90 14.0 s |
| 1 | Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active · Mixture of experts: 276B total, about 12B active per step. Size class uses 276B. Logic 95.4% (95%: 94.0%–96.5%) · Math 99.6% Non-hallucination 50.0% · “I don't know” on 3.2% of questions Runs on an API · cost $5.489$5,489 per 1,000 questionsper 1M questions · 13.3 s per question (median latency) | 95.4%94.0%–96.5% | 99.6% | 50.0% | 3.2% | – | $5.489$5,489 | – | 13.3 sp90 78.5 s |
| 1 | Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $10.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Logic 95.2% (95%: 93.8%–96.3%) · Math 99.7% Non-hallucination 8.3% · “I don't know” on 0.4% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.82 s per question (median latency) | 95.2%93.8%–96.3% | 99.7% | 8.3% | 0.4% | – | $0.000$0.00 | – | 3.82 sp90 13.5 s |
| 1 | MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active · Mixture of experts: 309B total, about 15B active per step. Size class uses 309B. Logic 95.1% (95%: 93.7%–96.2%) · Math 99.4% Non-hallucination 23.3% · “I don't know” on 1.5% of questions Runs on an API · cost $1.390$1,390 per 1,000 questionsper 1M questions · 24.7 s per question (median latency) | 95.1%93.7%–96.2% | 99.4% | 23.3% | 1.5% | – | $1.390$1,390 | – | 24.7 sp90 179.9 s |
| 1 | Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active Qwen3.5-397B-A17B on · chat · Alibaba · chat model · reasoning on Parameters: 397B MoE · 17B active · Mixture of experts: 397B total, about 17B active per step. Size class uses 397B. Logic 95.1% (95%: 93.7%–96.2%) · Math running Non-hallucination 31.7% · “I don't know” on 1.8% of questions Runs on an API · cost $13.426$13,426 per 1,000 questionsper 1M questions · 84.7 s per question (median latency) | 95.1%93.7%–96.2% | running | 31.7% | 1.8% | – | $13.426$13,426 | – | 84.7 sp90 167.5 s |
| 2 | Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Logic 94.0% (95%: 92.5%–95.3%) · Math 99.2% Non-hallucination 39.1% · “I don't know” on 3.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.884$4,884estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 18.7 s per question (median latency) | 94.0%92.5%–95.3% | 99.2% | 39.1% | 3.2% | – | $4.884$4,884estimate · RTX PRO 6000 Blackwell | 7,061 sRTX PRO 6000 | 18.7 sp90 101.0 s |
| 2 | GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Logic 94.0% (95%: 92.4%–95.2%) · Math 99.2% Non-hallucination 61.1% · “I don't know” on 4.9% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.64 s per question (median latency) | 94.0%92.4%–95.2% | 99.2% | 61.1% | 4.9% | – | $0.000$0.00 | – | 5.64 sp90 10.3 s |
| 2 | MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active · Mixture of experts: 428B total, about 23B active per step. Size class uses 428B. Logic 94.0% (95%: 92.4%–95.2%) · Math 98.2% Non-hallucination 38.3% · “I don't know” on 3.2% of questions Runs on an API · cost $0.803$803 per 1,000 questionsper 1M questions · 5.61 s per question (median latency) | 94.0%92.4%–95.2% | 98.2% | 38.3% | 3.2% | – | $0.803$803 | – | 5.61 sp90 37.4 s |
| 2 | Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Logic 94.0% (95%: 92.4%–95.2%) · Math 99.5% Non-hallucination 34.6% · “I don't know” on 2.5% of questions Runs on an API · cost $1.376$1,376 per 1,000 questionsper 1M questions · 17.6 s per question (median latency) | 94.0%92.4%–95.2% | 99.5% | 34.6% | 2.5% | – | $1.376$1,376 | – | 17.6 sp90 73.2 s |
| 2 | Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Logic 93.3% (95%: 91.7%–94.6%) · Math 99.4% Non-hallucination 16.3% · “I don't know” on 1.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.244$1,244estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 18.3 s per question (median latency) | 93.3%91.7%–94.6% | 99.4% | 16.3% | 1.4% | – | $1.244$1,244estimate · RTX PRO 6000 Blackwell | 1,798 sRTX PRO 6000 | 18.3 sp90 39.8 s |
| 2 | GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active · Mixture of experts: 320B total, about 18B active per step. Size class uses 320B. Logic 93.2% (95%: 91.5%–94.5%) · Math 98.8% Non-hallucination 4.2% · “I don't know” on 0.3% of questions Runs on an API · cost $1.057$1,057 per 1,000 questionsper 1M questions · 12.1 s per question (median latency) | 93.2%91.5%–94.5% | 98.8% | 4.2% | 0.3% | – | $1.057$1,057 | – | 12.1 sp90 82.8 s |
| 2 | Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Logic 92.4% (95%: 90.7%–93.8%) · Math 99.4% Non-hallucination 36.1% · “I don't know” on 3.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.525$3,525estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 142.5 s per question (median latency) Other settings: Qwen3.5-4B (off · chat) | 92.4%90.7%–93.8% | 99.4% | 36.1% | 3.1% | – | $3.525$3,525estimate · RTX PRO 6000 Blackwell | 5,096 sRTX PRO 6000 | 142.5 sp90 246.3 s |
| Qwen3.5-4B off · chat Open · 4.7Bother setting of Qwen3.5-4B (on · chat) Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Logic 90.1% (95%: 88.3%–91.7%) · Math 98.5% Non-hallucination 33.6% · “I don't know” on 4.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.420$1,420estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 45.4 s per question (median latency) | 90.1%88.3%–91.7% | 98.5% | 33.6% | 4.1% | – | $1.420$1,420estimate · RTX PRO 6000 Blackwell | 2,053 sRTX PRO 6000 | 45.4 sp90 127.5 s | |
| 2 | Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Logic 91.6% (95%: 89.9%–93.1%) · Math 99.2% Non-hallucination 53.2% · “I don't know” on 5.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $10.893$10,893estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 221.3 s per question (median latency) | 91.6%89.9%–93.1% | 99.2% | 53.2% | 5.9% | – | $10.893$10,893estimate · RTX PRO 6000 Blackwell | 15,748 sRTX PRO 6000 | 221.3 sp90 1229.5 s |
| 3 | MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Logic 88.9% (mean of 3 seeds; min–max 88.3%–89.6%) · Math 99.2% Seed runs: seed 0: 88.7% (998 of 1,125, run minicpm5-2b_en) · seed 1: 88.3% (993 of 1,125, run minicpm5-2b-seed1_en) · seed 2: 89.6% (1,008 of 1,125, run minicpm5-2b-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b_en, not the mean Non-hallucination 51.5% · “I don't know” on 7.6% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.335$3,335estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 101.1 s per question (median latency) Other settings: MiniCPM5-2B (off · chat) | 88.9%88.3%–89.6% min–max of 3 seeds | 99.2% | 51.5%seed 0 only | 7.6%seed 0 only | – | $3.335$3,335estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 4,821 sRTX PRO 6000mean of 3 seeds | 101.1 sp90 333.4 smean of 3 seeds |
| MiniCPM5-2B off · chat Open · 2.5Bother setting of MiniCPM5-2B (on · chat) MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Logic 83.6% (mean of 3 seeds; min–max 83.1%–83.9%) · Math 98.2% Seed runs: seed 0: 83.7% (942 of 1,125, run minicpm5-2b-nothink_en) · seed 1: 83.9% (944 of 1,125, run minicpm5-2b-nothink-seed1_en) · seed 2: 83.1% (935 of 1,125, run minicpm5-2b-nothink-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b-nothink_en, not the mean Non-hallucination 43.9% · “I don't know” on 11.8% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.327$1,327estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 46.5 s per question (median latency) | 83.6%83.1%–83.9% min–max of 3 seeds | 98.2% | 43.9%seed 0 only | 11.8%seed 0 only | – | $1.327$1,327estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,919 sRTX PRO 6000mean of 3 seeds | 46.5 sp90 111.6 smean of 3 seeds | |
| 3 | Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Logic 87.7% (95%: 85.7%–89.5%) · Math 98.8% Non-hallucination 15.5% · “I don't know” on 2.0% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.292$1,292estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 42.1 s per question (median latency) | 87.7%85.7%–89.5% | 98.8% | 15.5% | 2.0% | – | $1.292$1,292estimate · RTX PRO 6000 Blackwell | 1,868 sRTX PRO 6000 | 42.1 sp90 82.9 s |
| 3 | LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active · Mixture of experts: 24.0B total, about 2.45B active per step. Size class uses 24.0B. Logic 86.7% (95%: 84.6%–88.5%) · Math 99.0% Non-hallucination 37.8% · “I don't know” on 5.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $7.562$7,562estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 229.0 s per question (median latency) | 86.7%84.6%–88.5% | 99.0% | 37.8% | 5.2% | – | $7.562$7,562estimate · RTX PRO 6000 Blackwell | 10,932 sRTX PRO 6000 | 229.0 sp90 776.9 s |
| 3 | LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Logic 85.5% (95%: 83.3%–87.4%) · Math 98.7% Non-hallucination 54.2% · “I don't know” on 10.4% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.487$487estimate · DGX Spark per 1,000 questionsper 1M questions · 163.9 s per question (median latency) Other settings: LFM3-1.7B (on · chat), LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell) | 85.5%83.3%–87.4% | 98.7% | 54.2% | 10.4% | – | $0.487$487estimate · DGX Spark | 8,234 sDGX Spark | 163.9 sp90 656.8 s |
| LFM3-1.7B on · chat Open · 1.7Bother setting of LFM3-1.7B (off · chat) LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Logic 84.2% (mean of 3 seeds; min–max 83.9%–84.4%) · Math 98.9% Seed runs: seed 0: 84.4% (950 of 1,125, run lfm3-1.7b-vllm_en) · seed 1: 83.9% (944 of 1,125, run lfm3-1.7b-vllm-seed1_en) · seed 2: 84.4% (949 of 1,125, run lfm3-1.7b-vllm-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm_en, not the mean Non-hallucination 43.6% · “I don't know” on 8.8% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.017$1,017estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 29.8 s per question (median latency) | 84.2%83.9%–84.4% min–max of 3 seeds | 98.9% | 43.6%seed 0 only | 8.8%seed 0 only | – | $1.017$1,017estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,471 sRTX PRO 6000mean of 3 seeds | 29.8 sp90 113.2 smean of 3 seeds | |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7Bother setting of LFM3-1.7B (off · chat) LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Logic 84.6% (mean of 3 seeds; min–max 84.4%–85.0%) · Math 98.9% Seed runs: seed 0: 85.0% (956 of 1,125, run lfm3-1.7b-vllm-nothink-rtx-seed0_en) · seed 1: 84.4% (949 of 1,125, run lfm3-1.7b-vllm-nothink-rtx-seed1_en) · seed 2: 84.4% (949 of 1,125, run lfm3-1.7b-vllm-nothink-rtx-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm-nothink-rtx-seed0_en, not the mean Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.040$1,040estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 27.9 s per question (median latency) | 84.6%84.4%–85.0% min–max of 3 seeds | 98.9% | – | – | – | $1.040$1,040estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,503 sRTX PRO 6000mean of 3 seeds | 27.9 sp90 107.5 smean of 3 seeds | |
| – | DeepSeek-V4-Flash chat Open · 285B MoE · 13B active sample DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active · Mixture of experts: 285B total, about 13B active per step. Size class uses 285B. Not tiered: sample (317 questions) Logic 83.6% (95%: 79.1%–87.3%) · Math 99.3% Runs on NVIDIA DGX Spark (GB10) · cost $1.154$1,154estimate · DGX Spark per 1,000 questionsper 1M questions · 24.8 s per question (median latency) | 83.6%79.1%–87.3% | 99.3% | – | – | – | $1.154$1,154estimate · DGX Spark | 19,527 sDGX Spark | 24.8 sp90 91.1 s |
| 4 | Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Logic 83.2% (95%: 80.9%–85.3%) · Math 98.1% Non-hallucination 52.4% · “I don't know” on 11.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.842$842estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 32.7 s per question (median latency) Other settings: Gemma 4 E4B (on · chat), Gemma 4 E4B (on · chat · 32k cap), Gemma 4 E4B (on · chat · 32k cap · sampled) | 83.2%80.9%–85.3% | 98.1% | 52.4% | 11.7% | – | $0.842$842estimate · RTX PRO 6000 Blackwell | 1,218 sRTX PRO 6000 | 32.7 sp90 64.1 s |
| Gemma 4 E4B on · chat Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Logic 82.6% (95%: 80.3%–84.7%) · Math 98.2% Non-hallucination 45.1% · “I don't know” on 9.3% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.650$650estimate · DGX Spark per 1,000 questionsper 1M questions · 284.1 s per question (median latency) | 82.6%80.3%–84.7% | 98.2% | 45.1% | 9.3% | – | $0.650$650estimate · DGX Spark | 10,988 sDGX Spark | 284.1 sp90 540.0 s | |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Logic 85.0% (95%: 82.8%–86.9%) · Math 98.5% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.503$1,503estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 57.9 s per question (median latency) | 85.0%82.8%–86.9% | 98.5% | – | – | – | $1.503$1,503estimate · RTX PRO 6000 Blackwell | 2,172 sRTX PRO 6000 | 57.9 sp90 109.4 s | |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Logic 85.9% (mean of 3 seeds; min–max 85.5%–86.1%) · Math 98.5% Seed runs: seed 0: 86.0% (968 of 1,125, run gemma-4-e4b-it-think-32k-sampled-seed0_en) · seed 1: 86.1% (969 of 1,125, run gemma-4-e4b-it-think-32k-sampled-seed1_en) · seed 2: 85.5% (962 of 1,125, run gemma-4-e4b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e4b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 55.9% · “I don't know” on 10.5% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.993$993estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 31.6 s per question (median latency) | 85.9%85.5%–86.1% min–max of 3 seeds | 98.5% | 55.9%seed 0 only | 10.5%seed 0 only | – | $0.993$993estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,435 sRTX PRO 6000mean of 3 seeds | 31.6 sp90 83.5 smean of 3 seeds | |
| 5 | LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Logic 75.4% (95%: 72.8%–77.8%) · Math 94.0% Non-hallucination 44.3% · “I don't know” on 13.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.480$2,480estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 81.3 s per question (median latency) | 75.4%72.8%–77.8% | 94.0% | 44.3% | 13.6% | – | $2.480$2,480estimate · RTX PRO 6000 Blackwell | 3,585 sRTX PRO 6000 | 81.3 sp90 224.0 s |
| 5 | Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 72.9% (95%: 70.2%–75.4%) · Math 96.8% Non-hallucination 34.0% · “I don't know” on 9.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.513$513estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 21.4 s per question (median latency) Other settings: Gemma 4 E2B (on · chat), Gemma 4 E2B (on · chat · 32k cap), Gemma 4 E2B (on · chat · 32k cap · sampled) | 72.9%70.2%–75.4% | 96.8% | 34.0% | 9.8% | – | $0.513$513estimate · RTX PRO 6000 Blackwell | 742 sRTX PRO 6000 | 21.4 sp90 35.2 s |
| Gemma 4 E2B on · chat Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 74.6% (95%: 72.0%–77.0%) · Math 96.9% Non-hallucination 26.4% · “I don't know” on 6.9% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.363$363estimate · DGX Spark per 1,000 questionsper 1M questions · 173.2 s per question (median latency) | 74.6%72.0%–77.0% | 96.9% | 26.4% | 6.9% | – | $0.363$363estimate · DGX Spark | 6,145 sDGX Spark | 173.2 sp90 297.1 s | |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 75.9% (95%: 73.3%–78.3%) · Math 97.8% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.193$1,193estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 41.4 s per question (median latency) | 75.9%73.3%–78.3% | 97.8% | – | – | – | $1.193$1,193estimate · RTX PRO 6000 Blackwell | 1,724 sRTX PRO 6000 | 41.4 sp90 70.7 s | |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 76.8% (mean of 3 seeds; min–max 76.0%–77.6%) · Math 97.7% Seed runs: seed 0: 76.0% (855 of 1,125, run gemma-4-e2b-it-think-32k-sampled-seed0_en) · seed 1: 77.6% (873 of 1,125, run gemma-4-e2b-it-think-32k-sampled-seed1_en) · seed 2: 76.7% (863 of 1,125, run gemma-4-e2b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e2b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 32.6% · “I don't know” on 8.8% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.574$574estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 21.2 s per question (median latency) | 76.8%76.0%–77.6% min–max of 3 seeds | 97.7% | 32.6%seed 0 only | 8.8%seed 0 only | – | $0.574$574estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 830 sRTX PRO 6000mean of 3 seeds | 21.2 sp90 45.1 smean of 3 seeds | |
| 6 | Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Logic 65.3% (95%: 62.5%–68.1%) · Math 95.8% Non-hallucination 16.1% · “I don't know” on 6.3% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.772$1,772estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 16.6 s per question (median latency) | 65.3%62.5%–68.1% | 95.8% | 16.1% | 6.3% | – | $1.772$1,772estimate · RTX PRO 6000 Blackwell | 2,562 sRTX PRO 6000 | 16.6 sp90 178.7 s |
| 6 | Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Logic 65.1% (95%: 62.2%–67.8%) · Math 89.6% Non-hallucination 35.7% · “I don't know” on 12.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $9.270$9,270estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 276.7 s† per question (median latency) Other settings: Raon-1 nano (on · chat) | 65.1%62.2%–67.8% | 89.6% | 35.7% | 12.8% | – | $9.270$9,270estimate · RTX PRO 6000 Blackwell† | 13,403 sRTX PRO 6000 | 276.7 s†p90 965.6 s |
| Raon-1 nano on · chat Open · 3.1Bother setting of Raon-1 nano (off · chat) Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Logic 70.9% (95%: 68.2%–73.5%) · Math 97.0% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $15.330$15,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 493.2 s† per question (median latency) | 70.9%68.2%–73.5% | 97.0% | – | – | – | $15.330$15,330estimate · RTX PRO 6000 Blackwell† | 22,164 sRTX PRO 6000 | 493.2 s†p90 1396.5 s | |
| 6 | Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Logic 64.8% (95%: 62.0%–67.5%) · Math 94.6% Non-hallucination 35.2% · “I don't know” on 14.0% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.605$1,605estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 71.3 s per question (median latency) Other settings: Qwen3.5-2B (on · chat) | 64.8%62.0%–67.5% | 94.6% | 35.2% | 14.0% | – | $1.605$1,605estimate · RTX PRO 6000 Blackwell | 2,320 sRTX PRO 6000 | 71.3 sp90 113.5 s |
| Qwen3.5-2B on · chat Open · 2.0Bother setting of Qwen3.5-2B (off · chat) Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Logic 70.3% (95%: 67.6%–72.9%) · Math 94.7% Non-hallucination 33.1% · “I don't know” on 9.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.330$4,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 186.3 s per question (median latency) | 70.3%67.6%–72.9% | 94.7% | 33.1% | 9.1% | – | $4.330$4,330estimate · RTX PRO 6000 Blackwell | 6,260 sRTX PRO 6000 | 186.3 sp90 272.2 s | |
| 6 | Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Logic 60.7% (95%: 57.8%–63.5%) · Math 81.6% Non-hallucination 6.6% · “I don't know” on 2.7% of questions Confidence: 172 answers kept at ≥99%, 169 right · 3 wrong · AUROC 0.816 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.128$128estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.173 s per question (median latency) | 60.7%57.8%–63.5% | 81.6% | 6.6% | 2.7% | 5.7 | $0.128$128estimate · RTX PRO 6000 Blackwell | 186 sRTX PRO 6000 | 0.173 sp90 0.211 s |
| 6 | LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active · Mixture of experts: 8.3B total, about 1.5B active per step. Size class uses 8.3B. Logic 60.2% (95%: 57.3%–63.0%) · Math 96.3% Non-hallucination 58.3% · “I don't know” on 29.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.664$4,664estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 138.1 s† per question (median latency) | 60.2%57.3%–63.0% | 96.3% | 58.3% | 29.8% | – | $4.664$4,664estimate · RTX PRO 6000 Blackwell† | 6,743 sRTX PRO 6000 | 138.1 s†p90 403.4 s |
| 6 | REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense · Two models run together, option probabilities averaged 50/50: 57.36B in all, all used on every question. Logic 59.3% (95%: 56.4%–62.1%) · Math 78.0% Non-hallucination 5.3% · “I don't know” on 2.2% of questions Confidence: 51 answers kept at ≥99%, 51 right · 0 wrong · AUROC 0.816 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.160$160estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.215 s per question (median latency) | 59.3%56.4%–62.1% | 78.0% | 5.3% | 2.2% | 3.7 | $0.160$160estimate · RTX PRO 6000 Blackwell | 232 sRTX PRO 6000 | 0.215 sp90 0.278 s |
| 6 | KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Logic 59.0% (95%: 56.1%–61.9%) · Math 77.4% Non-hallucination 0.4% · “I don't know” on 0.2% of questions Confidence: 89 answers kept at ≥99%, 89 right · 0 wrong · AUROC 0.804 Runs on NVIDIA DGX Spark (GB10) · cost $0.061$61.43estimate · DGX Spark per 1,000 questionsper 1M questions · 0.983 s per question (median latency) | 59.0%56.1%–61.9% | 77.4% | 0.4% | 0.2% | 2.6 | $0.061$61.43estimate · DGX Spark | 1,039 sDGX Spark | 0.983 sp90 1.28 s |
| 6 | KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Logic 58.9% (95%: 56.0%–61.8%) · Math 80.3% Non-hallucination 0.0% · “I don't know” on 0.0% of questions Confidence: 54 answers kept at ≥99%, 54 right · 0 wrong · AUROC 0.825 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.112$112estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.151 s per question (median latency) | 58.9%56.0%–61.8% | 80.3% | 0.0% | 0.0% | 3.7 | $0.112$112estimate · RTX PRO 6000 Blackwell | 162 sRTX PRO 6000 | 0.151 sp90 0.204 s |
| 6 | Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Logic 58.7% (95%: 55.8%–61.5%) · Math 77.3% Non-hallucination 1.9% · “I don't know” on 0.8% of questions Confidence: 66 answers kept at ≥99%, 66 right · 0 wrong · AUROC 0.801 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.136$136estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.185 s per question (median latency) | 58.7%55.8%–61.5% | 77.3% | 1.9% | 0.8% | 3.9 | $0.136$136estimate · RTX PRO 6000 Blackwell | 196 sRTX PRO 6000 | 0.185 sp90 0.255 s |
| 6 | GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Logic 58.4% (95%: 55.5%–61.2%) · Math 72.0% Non-hallucination 44.3% · “I don't know” on 21.1% of questions Confidence: 75 answers kept at ≥99%, 74 right · 1 wrong · AUROC 0.822 Runs on NVIDIA DGX Spark (GB10) · cost $0.033$33.16estimate · DGX Spark per 1,000 questionsper 1M questions · 0.544 s per question (median latency) | 58.4%55.5%–61.2% | 72.0% | 44.3% | 21.1% | 4.4 | $0.033$33.16estimate · DGX Spark | 561 sDGX Spark | 0.544 sp90 0.655 s |
| 7 | GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/openai/gpt-6-luna) Logic 58.0% (95%: 55.0%–60.8%) · Math 93.7% Non-hallucination 55.1% · “I don't know” on 28.1% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.86 s per question (median latency) | 58.0%55.0%–60.8% | 93.7% | 55.1% | 28.1% | – | $0.000$0.00 | – | 3.86 sp90 5.82 s |
| 7 | Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Logic 57.9% (95%: 55.0%–60.7%) · Math 77.1% Non-hallucination 5.5% · “I don't know” on 2.4% of questions Confidence: 81 answers kept at ≥99%, 79 right · 2 wrong · AUROC 0.797 Runs on NVIDIA DGX Spark (GB10) · cost $0.063$62.95estimate · DGX Spark per 1,000 questionsper 1M questions · 1.05 s per question (median latency) | 57.9%55.0%–60.7% | 77.1% | 5.5% | 2.4% | 2.1 | $0.063$62.95estimate · DGX Spark | 1,065 sDGX Spark | 1.05 sp90 1.33 s |
| 7 | Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Logic 57.8% (95%: 54.9%–60.6%) · Math 75.2% Non-hallucination 7.1% · “I don't know” on 3.0% of questions Confidence: 8 answers kept at ≥99%, 8 right · 0 wrong · AUROC 0.796 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.211$211estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.285 s per question (median latency) | 57.8%54.9%–60.6% | 75.2% | 7.1% | 3.0% | 5.0 | $0.211$211estimate · RTX PRO 6000 Blackwell | 306 sRTX PRO 6000 | 0.285 sp90 0.399 s |
| 7 | GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Logic 55.2% (95%: 52.3%–58.1%) · Math 69.9% Non-hallucination 50.2% · “I don't know” on 26.8% of questions Confidence: 45 answers kept at ≥99%, 45 right · 0 wrong · AUROC 0.793 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.111 s per question (median latency) | 55.2%52.3%–58.1% | 69.9% | 50.2% | 26.8% | 5.9 | $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell | 118 sRTX PRO 6000 | 0.111 sp90 0.140 s |
| 7 | d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://www.liquid.ai/blog/d1-ultra) Logic 54.9% (95%: 52.0%–57.8%) · Math 74.1% Non-hallucination 91.0% · “I don't know” on 64.9% of questions Confidence: 20 answers kept at ≥99%, 20 right · 0 wrong · AUROC 0.783 Runs on an API · cost $0.013$12.64 per 1,000 questionsper 1M questions · 0.182 s per question (median latency) | 54.9%52.0%–57.8% | 74.1% | 91.0% | 64.9% | 3.2 | $0.013$12.64 | – | 0.182 sp90 0.425 s |
| 7 | Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker · Closed; introductory list price (the only one published) $0.03 per 1M input tokens: Efficient (provisional) (source: https://nace.ai/drex) Logic 54.8% (95%: 51.8%–57.6%) · Math 70.1% Non-hallucination 54.1% · “I don't know” on 30.8% of questions Confidence: 90 answers kept at ≥99%, 89 right · 1 wrong · AUROC 0.793 Runs on an API · cost $0.022$22.37 per 1,000 questionsper 1M questions · 0.329 s per question (median latency) | 54.8%51.8%–57.6% | 70.1% | 54.1% | 30.8% | 6.1 | $0.022$22.37 | – | 0.329 sp90 1.54 s |
| 7 | Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Logic 54.8% (95%: 51.8%–57.6%) · Math 72.5% Non-hallucination 18.1% · “I don't know” on 9.1% of questions Confidence: 62 answers kept at ≥99%, 62 right · 0 wrong · AUROC 0.803 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.108 s per question (median latency) | 54.8%51.8%–57.6% | 72.5% | 18.1% | 9.1% | 9.0 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | 117 sRTX PRO 6000 | 0.108 sp90 0.131 s |
| 7 | d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Logic 54.6% (95%: 51.7%–57.5%) · Math 71.5% Non-hallucination 10.1% · “I don't know” on 4.7% of questions Confidence: 28 answers kept at ≥99%, 28 right · 0 wrong · AUROC 0.778 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.109$109estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.156 s per question (median latency) | 54.6%51.7%–57.5% | 71.5% | 10.1% | 4.7% | 15.1 | $0.109$109estimate · RTX PRO 6000 Blackwell | 158 sRTX PRO 6000 | 0.156 sp90 0.185 s |
| 7 | GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker · Closed; list price $0.15 per 1M input tokens: Efficient (source: https://docs.fastino.ai/pricing) Logic 54.5% (95%: 51.6%–57.4%) · Math 70.8% Non-hallucination 60.0% · “I don't know” on 32.5% of questions Confidence: 52 answers kept at ≥99%, 52 right · 0 wrong · AUROC 0.814 Runs on an API · cost $0.052$52.03 per 1,000 questionsper 1M questions · 0.271 s per question (median latency) | 54.5%51.6%–57.4% | 70.8% | 60.0% | 32.5% | 6.6 | $0.052$52.03 | – | 0.271 sp90 0.291 s |
| 7 | Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Logic 54.5% (95%: 51.6%–57.4%) · Math 74.0% Non-hallucination 72.3% · “I don't know” on 42.4% of questions Confidence: 108 answers kept at ≥99%, 106 right · 2 wrong · AUROC 0.795 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.221 s per question (median latency) | 54.5%51.6%–57.4% | 74.0% | 72.3% | 42.4% | 5.3 | $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell | 218 sRTX PRO 6000 | 0.221 sp90 0.244 s |
| 7 | d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Logic 54.5% (95%: 51.6%–57.4%) · Math 70.4% Non-hallucination 14.8% · “I don't know” on 7.2% of questions Confidence: 54 answers kept at ≥99%, 54 right · 0 wrong · AUROC 0.806 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.057$57.18estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.076 s per question (median latency) | 54.5%51.6%–57.4% | 70.4% | 14.8% | 7.2% | 4.6 | $0.057$57.18estimate · RTX PRO 6000 Blackwell | 83 sRTX PRO 6000 | 0.076 sp90 0.098 s |
| 7 | Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Logic 54.2% (95%: 51.3%–57.1%) · Math 90.8% Non-hallucination 30.7% · “I don't know” on 16.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.390$390estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 10.2 s per question (median latency) | 54.2%51.3%–57.1% | 90.8% | 30.7% | 16.2% | – | $0.390$390estimate · RTX PRO 6000 Blackwell | 564 sRTX PRO 6000 | 10.2 sp90 18.4 s |
| 7 | Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Logic 53.3% (95%: 50.4%–56.2%) · Math 69.8% Non-hallucination 14.4% · “I don't know” on 7.1% of questions Confidence: 80 answers kept at ≥99%, 78 right · 2 wrong · AUROC 0.798 Runs on an API · cost $0.007$7.46 per 1,000 questionsper 1M questions · 0.238 s per question (median latency) Other settings: Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) | 53.3%50.4%–56.2% | 69.8% | 14.4% | 7.1% | 3.4 | $0.007$7.46 | – | 0.238 sp90 0.552 s |
| Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell Open · 27Bother setting of Perplexity Decider 27B v1.2 (API) Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Logic 53.2% (95%: 50.3%–56.1%) · Math 70.3% Non-hallucination 13.4% · “I don't know” on 6.6% of questions Confidence: 90 answers kept at ≥99%, 88 right · 2 wrong · AUROC 0.799 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.008$7.52 list$0.105$105estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.136 s per question (median latency) Shown under Perplexity Decider 27B v1.2 (API): the same model name, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings | 53.2%50.3%–56.1% | 70.3% | 13.4% | 6.6% | 4.6 | $0.008$7.52 list$0.105$105estimate · RTX PRO 6000 Blackwell | 151 sRTX PRO 6000 | 0.136 sp90 0.175 s | |
| 7 | Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Logic 53.1% (95%: 50.1%–56.0%) · Math 70.3% Non-hallucination 7.3% · “I don't know” on 3.5% of questions Confidence: 155 answers kept at ≥99%, 147 right · 8 wrong · AUROC 0.786 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.106 s per question (median latency) | 53.1%50.1%–56.0% | 70.3% | 7.3% | 3.5% | 16.2 | $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell | 117 sRTX PRO 6000 | 0.106 sp90 0.143 s |
| – | Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Not tiered: with web Logic 52.5% (95%: 49.6%–55.4%) · Math 69.3% Non-hallucination 37.7% · “I don't know” on 19.9% of questions Confidence: 79 answers kept at ≥99%, 78 right · 1 wrong · AUROC 0.800 Runs on an API · cost $1.030$1,030 per 1,000 questionsper 1M questions · 0.448 s per question (median latency) | 52.5%49.6%–55.4% | 69.3% | 37.7% | 19.9% | 4.2 | $1.030$1,030 | – | 0.448 sp90 0.827 s |
| 7 | SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Logic 52.4% (95%: 49.4%–55.3%) · Math 68.6% Non-hallucination 65.7% · “I don't know” on 39.0% of questions Confidence: 72 answers kept at ≥99%, 70 right · 2 wrong · AUROC 0.787 Runs on an API · cost $0.015$14.92 per 1,000 questionsper 1M questions · 0.384 s per question (median latency) Shown once with SubQ Snap Preview: the same answer to every question | 52.4%49.4%–55.3% | 68.6% | 65.7% | 39.0% | 12.9 | $0.015$14.92 | – | 0.384 sp90 0.393 s |
| 7 | Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Logic 52.3% (95%: 49.3%–55.2%) · Math 69.0% Non-hallucination 1.5% · “I don't know” on 0.7% of questions Confidence: 7 answers kept at ≥99%, 7 right · 0 wrong · AUROC 0.782 Runs on NVIDIA DGX Spark (GB10) · cost $0.025$24.91estimate · DGX Spark per 1,000 questionsper 1M questions · 0.385 s per question (median latency) | 52.3%49.3%–55.2% | 69.0% | 1.5% | 0.7% | 2.9 | $0.025$24.91estimate · DGX Spark | 421 sDGX Spark | 0.385 sp90 0.533 s |
| 7 | d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Logic 51.6% (95%: 48.6%–54.5%) · Math 69.9% Non-hallucination 94.5% · “I don't know” on 71.8% of questions Confidence: 13 answers kept at ≥99%, 13 right · 0 wrong · AUROC 0.800 Runs on an API · cost $0.013$12.64 per 1,000 questionsper 1M questions · 0.220 s per question (median latency) | 51.6%48.6%–54.5% | 69.9% | 94.5% | 71.8% | 3.0 | $0.013$12.64 | – | 0.220 sp90 0.341 s |
| 7 | KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Logic 50.6% (95%: 47.7%–53.5%) · Math 66.3% Non-hallucination 8.8% · “I don't know” on 4.5% of questions Confidence: 6 answers kept at ≥99%, 6 right · 0 wrong · AUROC 0.774 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.068 s per question (median latency) | 50.6%47.7%–53.5% | 66.3% | 8.8% | 4.5% | 4.8 | $0.052$51.64estimate · RTX PRO 6000 Blackwell | 75 sRTX PRO 6000 | 0.068 sp90 0.088 s |
| 8 | Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/google/gemini-decision-1) Logic 50.0% (95%: 47.1%–53.0%) · Math 71.3% Non-hallucination 2.0% · “I don't know” on 1.0% of questions Confidence: 43 answers kept at ≥99%, 43 right · 0 wrong · AUROC 0.768 Runs on an API · cost $0.037$36.77 per 1,000 questionsper 1M questions · 0.228 s per question (median latency) | 50.0%47.1%–53.0% | 71.3% | 2.0% | 1.0% | 3.1 | $0.037$36.77 | – | 0.228 sp90 0.344 s |
| 8 | d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Logic 49.2% (95%: 46.2%–52.1%) · Math 65.5% Non-hallucination 78.8% · “I don't know” on 56.6% of questions Confidence: 28 answers kept at ≥99%, 28 right · 0 wrong · AUROC 0.742 Runs on an API · cost $0.006$6.02 per 1,000 questionsper 1M questions · 0.222 s per question (median latency) | 49.2%46.2%–52.1% | 65.5% | 78.8% | 56.6% | 2.6 | $0.006$6.02 | – | 0.222 sp90 0.390 s |
| 8 | Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 48.2% (95%: 45.3%–51.1%) · Math 59.8% Non-hallucination 23.4% · “I don't know” on 13.1% of questions Confidence: 10 answers kept at ≥99%, 10 right · 0 wrong · AUROC 0.753 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.039$39.35estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.052 s per question (median latency) | 48.2%45.3%–51.1% | 59.8% | 23.4% | 13.1% | 4.0 | $0.039$39.35estimate · RTX PRO 6000 Blackwell | 57 sRTX PRO 6000 | 0.052 sp90 0.066 s |
| 8 | Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://docs.typesafe.ai/models) Logic 48.2% (95%: 45.3%–51.1%) · Math 65.1% Non-hallucination 24.5% · “I don't know” on 14.1% of questions Confidence: 61 answers kept at ≥99%, 61 right · 0 wrong · AUROC 0.750 Runs on an API · cost $0.026$25.64 per 1,000 questionsper 1M questions · 0.166 s per question (median latency) | 48.2%45.3%–51.1% | 65.1% | 24.5% | 14.1% | 3.3 | $0.026$25.64 | – | 0.166 sp90 0.193 s |
| 8 | GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Logic 47.8% (95%: 44.9%–50.7%) · Math 57.9% Non-hallucination 30.4% · “I don't know” on 17.7% of questions Confidence: 32 answers kept at ≥99%, 32 right · 0 wrong · AUROC 0.749 Runs on NVIDIA DGX Spark (GB10) · cost $0.013$13.29estimate · DGX Spark per 1,000 questionsper 1M questions · 0.208 s per question (median latency) | 47.8%44.9%–50.7% | 57.9% | 30.4% | 17.7% | 6.7 | $0.013$13.29estimate · DGX Spark | 225 sDGX Spark | 0.208 sp90 0.251 s |
| 8 | celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Logic 47.3% (95%: 44.4%–50.2%) · Math 70.0% Non-hallucination 92.3% · “I don't know” on 72.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.735 Runs on an API · cost $0.015$15.40 per 1,000 questionsper 1M questions · 0.390 s per question (median latency) | 47.3%44.4%–50.2% | 70.0% | 92.3% | 72.2% | 10.7 | $0.015$15.40 | – | 0.390 sp90 0.394 s |
| 8 | celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.004 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Logic 47.3% (95%: 44.4%–50.2%) · Math 69.9% Non-hallucination 91.0% · “I don't know” on 71.0% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.733 Runs on an API · cost $0.002$1.54 per 1,000 questionsper 1M questions · 0.383 s per question (median latency) | 47.3%44.4%–50.2% | 69.9% | 91.0% | 71.0% | 9.8 | $0.002$1.54 | – | 0.383 sp90 0.390 s |
| 8 | StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Logic 46.6% (95%: 43.7%–49.5%) · Math 58.4% Non-hallucination 17.4% · “I don't know” on 10.0% of questions Confidence: 14 answers kept at ≥99%, 14 right · 0 wrong · AUROC 0.762 Runs on NVIDIA DGX Spark (GB10) · cost $0.010$10.04estimate · DGX Spark per 1,000 questionsper 1M questions · 0.160 s per question (median latency) | 46.6%43.7%–49.5% | 58.4% | 17.4% | 10.0% | 3.8 | $0.010$10.04estimate · DGX Spark | 170 sDGX Spark | 0.160 sp90 0.175 s |
| 8 | Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 46.4% (95%: 43.5%–49.3%) · Math 62.1% Non-hallucination 12.2% · “I don't know” on 6.8% of questions Confidence: 55 answers kept at ≥99%, 54 right · 1 wrong · AUROC 0.771 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.031$31.36estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.041 s per question (median latency) | 46.4%43.5%–49.3% | 62.1% | 12.2% | 6.8% | 7.2 | $0.031$31.36estimate · RTX PRO 6000 Blackwell | 45 sRTX PRO 6000 | 0.041 sp90 0.046 s |
| 8 | Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Logic 46.4% (95%: 43.5%–49.3%) · Math 61.3% Non-hallucination 75.6% · “I don't know” on 52.9% of questions Confidence: 53 answers kept at ≥99%, 53 right · 0 wrong · AUROC 0.768 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.079 s per question (median latency) | 46.4%43.5%–49.3% | 61.3% | 75.6% | 52.9% | 8.6 | $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.079 sp90 0.098 s |
| 8 | Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://typesafe.ai/blog/introducing-system-one-models-and-jev) Logic 46.3% (95%: 43.4%–49.2%) · Math 64.1% Non-hallucination 15.8% · “I don't know” on 9.1% of questions Confidence: 65 answers kept at ≥99%, 65 right · 0 wrong · AUROC 0.764 Runs on an API · cost $0.026$25.64 per 1,000 questionsper 1M questions · 0.167 s per question (median latency) | 46.3%43.4%–49.2% | 64.1% | 15.8% | 9.1% | 5.9 | $0.026$25.64 | – | 0.167 sp90 0.203 s |
| 8 | d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Logic 46.0% (95%: 43.2%–49.0%) · Math 65.2% Non-hallucination 90.6% · “I don't know” on 71.6% of questions Confidence: 48 answers kept at ≥99%, 48 right · 0 wrong · AUROC 0.748 Runs on an API · cost $0.006$6.02 per 1,000 questionsper 1M questions · 0.176 s per question (median latency) | 46.0%43.2%–49.0% | 65.2% | 90.6% | 71.6% | 4.4 | $0.006$6.02 | – | 0.176 sp90 0.272 s |
| 8 | StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Logic 45.9% (95%: 43.0%–48.8%) · Math 55.8% Non-hallucination 16.0% · “I don't know” on 9.2% of questions Confidence: 21 answers kept at ≥99%, 21 right · 0 wrong · AUROC 0.720 Runs on NVIDIA DGX Spark (GB10) · cost $0.005$4.62estimate · DGX Spark per 1,000 questionsper 1M questions · 0.073 s per question (median latency) | 45.9%43.0%–48.8% | 55.8% | 16.0% | 9.2% | 3.3 | $0.005$4.62estimate · DGX Spark | 78 sDGX Spark | 0.073 sp90 0.084 s |
| 8 | Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Logic 45.3% (95%: 42.4%–48.3%) · Math 58.8% Non-hallucination 12.4% · “I don't know” on 7.1% of questions Confidence: 56 answers kept at ≥99%, 56 right · 0 wrong · AUROC 0.759 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.034$33.81estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.047 s per question (median latency) | 45.3%42.4%–48.3% | 58.8% | 12.4% | 7.1% | 14.9 | $0.034$33.81estimate · RTX PRO 6000 Blackwell | 49 sRTX PRO 6000 | 0.047 sp90 0.053 s |
| 8 | Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://openrouter.ai/inception/mercury-decide) Logic 45.1% (95%: 42.2%–48.0%) · Math 53.5% Non-hallucination 20.5% · “I don't know” on 12.2% of questions Confidence: 91 answers kept at ≥99%, 84 right · 7 wrong · AUROC 0.746 Runs on an API · cost $0.008$7.52 per 1,000 questionsper 1M questions · 0.301 s per question (median latency) | 45.1%42.2%–48.0% | 53.5% | 20.5% | 12.2% | 15.4 | $0.008$7.52 | – | 0.301 sp90 0.360 s |
| 8 | Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Logic 44.7% (95%: 41.8%–47.6%) · Math 55.9% Non-hallucination 24.4% · “I don't know” on 14.9% of questions Confidence: 17 answers kept at ≥99%, 17 right · 0 wrong · AUROC 0.757 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.026$26.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.033 s per question (median latency) | 44.7%41.8%–47.6% | 55.9% | 24.4% | 14.9% | 10.1 | $0.026$26.44estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.033 sp90 0.041 s |
| 8 | Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://openrouter.ai/microsoft/microsoft-decision-1) Base model: Qwen3.5-9B Logic 44.4% (95%: 41.6%–47.4%) · Math 59.4% Non-hallucination 19.8% · “I don't know” on 12.0% of questions Confidence: 20 answers kept at ≥99%, 20 right · 0 wrong · AUROC 0.712 Runs on an API · cost $0.013$13.42 per 1,000 questionsper 1M questions · 0.315 s per question (median latency) | 44.4%41.6%–47.4% | 59.4% | 19.8% | 12.0% | 8.0 | $0.013$13.42 | – | 0.315 sp90 0.362 s |
| 8 | Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 44.4% (95%: 41.5%–47.3%) · Math 55.3% Non-hallucination 13.6% · “I don't know” on 8.2% of questions Confidence: 37 answers kept at ≥99%, 35 right · 2 wrong · AUROC 0.701 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.042$42.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.060 s per question (median latency) | 44.4%41.5%–47.3% | 55.3% | 13.6% | 8.2% | 5.8 | $0.042$42.42estimate · RTX PRO 6000 Blackwell | 61 sRTX PRO 6000 | 0.060 sp90 0.067 s |
| 8 | KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 44.3% (95%: 41.4%–47.2%) · Math 59.8% Non-hallucination 6.7% · “I don't know” on 3.9% of questions Confidence: 30 answers kept at ≥99%, 30 right · 0 wrong · AUROC 0.725 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.036$36.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.050 s per question (median latency) | 44.3%41.4%–47.2% | 59.8% | 6.7% | 3.9% | 3.5 | $0.036$36.27estimate · RTX PRO 6000 Blackwell | 52 sRTX PRO 6000 | 0.050 sp90 0.061 s |
| 8 | vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Logic 44.1% (95%: 41.2%–47.0%) · Math 56.8% Non-hallucination 13.6% · “I don't know” on 8.1% of questions Confidence: 6 answers kept at ≥99%, 6 right · 0 wrong · AUROC 0.739 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.026$25.82estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.032 s per question (median latency) | 44.1%41.2%–47.0% | 56.8% | 13.6% | 8.1% | 4.7 | $0.026$25.82estimate · RTX PRO 6000 Blackwell | 37 sRTX PRO 6000 | 0.032 sp90 0.037 s |
| 8 | KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 43.9% (95%: 41.0%–46.8%) · Math 60.1% Non-hallucination 19.5% · “I don't know” on 12.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.730 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.040$39.96estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.052 s per question (median latency) | 43.9%41.0%–46.8% | 60.1% | 19.5% | 12.1% | 3.2 | $0.040$39.96estimate · RTX PRO 6000 Blackwell | 58 sRTX PRO 6000 | 0.052 sp90 0.065 s |
| 8 | Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Logic 43.3% (95%: 40.4%–46.2%) · Math 56.0% Non-hallucination 6.0% · “I don't know” on 3.4% of questions Confidence: 27 answers kept at ≥99%, 26 right · 1 wrong · AUROC 0.752 Runs on NVIDIA DGX Spark (GB10) · cost $0.013$13.35estimate · DGX Spark per 1,000 questionsper 1M questions · 0.209 s per question (median latency) | 43.3%40.4%–46.2% | 56.0% | 6.0% | 3.4% | 17.0 | $0.013$13.35estimate · DGX Spark | 226 sDGX Spark | 0.209 sp90 0.269 s |
| 8 | Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Logic 43.1% (95%: 40.2%–46.0%) · Math 55.2% Non-hallucination 28.9% · “I don't know” on 18.3% of questions Confidence: 35 answers kept at ≥99%, 35 right · 0 wrong · AUROC 0.729 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.045 s per question (median latency) | 43.1%40.2%–46.0% | 55.2% | 28.9% | 18.3% | 6.0 | $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell | 50 sRTX PRO 6000 | 0.045 sp90 0.055 s |
| 9 | Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Logic 42.6% (95%: 39.7%–45.5%) · Math 54.6% Non-hallucination 28.2% · “I don't know” on 17.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.733 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.294$294estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.413 s per question (median latency) | 42.6%39.7%–45.5% | 54.6% | 28.2% | 17.7% | 12.9 | $0.294$294estimate · RTX PRO 6000 Blackwell | 426 sRTX PRO 6000 | 0.413 sp90 0.576 s |
| 9 | KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Logic 42.3% (95%: 39.5%–45.2%) · Math 56.0% Non-hallucination 8.3% · “I don't know” on 5.0% of questions Confidence: 18 answers kept at ≥99%, 17 right · 1 wrong · AUROC 0.708 Runs on NVIDIA DGX Spark (GB10) · cost $0.006$5.89estimate · DGX Spark per 1,000 questionsper 1M questions · 0.096 s per question (median latency) | 42.3%39.5%–45.2% | 56.0% | 8.3% | 5.0% | 13.7 | $0.006$5.89estimate · DGX Spark | 100 sDGX Spark | 0.096 sp90 0.115 s |
| 9 | Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active · Mixture of experts: 35B total, about 3B active per step. Size class uses 35B. Logic 41.9% (95%: 39.0%–44.8%) · Math 59.0% Non-hallucination 37.7% · “I don't know” on 25.1% of questions Confidence: 33 answers kept at ≥99%, 33 right · 0 wrong · AUROC 0.717 Runs on an API · cost $0.018$18.41 per 1,000 questionsper 1M questions · 0.345 s per question (median latency) | 41.9%39.0%–44.8% | 59.0% | 37.7% | 25.1% | 4.3 | $0.018$18.41 | – | 0.345 sp90 0.504 s |
| 9 | Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Logic 40.8% (95%: 38.0%–43.7%) · Math 51.1% Non-hallucination 3.6% · “I don't know” on 2.1% of questions Confidence: 6 answers kept at ≥99%, 6 right · 0 wrong · AUROC 0.697 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.196 s per question (median latency) | 40.8%38.0%–43.7% | 51.1% | 3.6% | 2.1% | 15.7 | $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell | 204 sRTX PRO 6000 | 0.196 sp90 0.222 s |
| 9 | d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Logic 40.8% (95%: 38.0%–43.7%) · Math 55.6% Non-hallucination 11.3% · “I don't know” on 7.2% of questions Confidence: 56 answers kept at ≥99%, 55 right · 1 wrong · AUROC 0.723 Runs on an API · cost $0.006$6.02 per 1,000 questionsper 1M questions · 0.188 s per question (median latency) | 40.8%38.0%–43.7% | 55.6% | 11.3% | 7.2% | 10.9 | $0.006$6.02 | – | 0.188 sp90 0.290 s |
| 9 | OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Logic 40.6% (95%: 37.8%–43.5%) · Math 53.8% Non-hallucination 1.2% · “I don't know” on 0.7% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.726 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.086$86.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.108 s per question (median latency) | 40.6%37.8%–43.5% | 53.8% | 1.2% | 0.7% | 4.6 | $0.086$86.07estimate · RTX PRO 6000 Blackwell | 124 sRTX PRO 6000 | 0.108 sp90 0.136 s |
| 9 | Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Logic 40.4% (95%: 37.5%–43.3%) · Math 50.4% Non-hallucination 14.6% · “I don't know” on 9.0% of questions Confidence: 39 answers kept at ≥99%, 39 right · 0 wrong · AUROC 0.724 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.030$29.51estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 40.4%37.5%–43.3% | 50.4% | 14.6% | 9.0% | 15.5 | $0.030$29.51estimate · RTX PRO 6000 Blackwell | 43 sRTX PRO 6000 | 0.038 sp90 0.048 s |
| 9 | Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Logic 39.7% (95%: 36.9%–42.6%) · Math 51.5% Non-hallucination 12.6% · “I don't know” on 7.9% of questions Confidence: 35 answers kept at ≥99%, 35 right · 0 wrong · AUROC 0.729 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.033$33.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.045 s per question (median latency) | 39.7%36.9%–42.6% | 51.5% | 12.6% | 7.9% | 10.8 | $0.033$33.20estimate · RTX PRO 6000 Blackwell | 48 sRTX PRO 6000 | 0.045 sp90 0.051 s |
| 9 | Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Logic 39.7% (95%: 36.9%–42.6%) · Math 46.9% Non-hallucination 29.7% · “I don't know” on 19.9% of questions Confidence: 32 answers kept at ≥99%, 32 right · 0 wrong · AUROC 0.727 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.066 s per question (median latency) | 39.7%36.9%–42.6% | 46.9% | 29.7% | 19.9% | 14.4 | $0.052$51.64estimate · RTX PRO 6000 Blackwell | 75 sRTX PRO 6000 | 0.066 sp90 0.091 s |
| 9 | Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Logic 39.3% (95%: 36.5%–42.2%) · Math 44.3% Non-hallucination 11.3% · “I don't know” on 7.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.720 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.040 s per question (median latency) | 39.3%36.5%–42.2% | 44.3% | 11.3% | 7.1% | 11.2 | $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell | 44 sRTX PRO 6000 | 0.040 sp90 0.048 s |
| 9 | Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Logic 38.9% (95%: 36.1%–41.8%) · Math 49.8% Non-hallucination 14.4% · “I don't know” on 9.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.659 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.110 s per question (median latency) | 38.9%36.1%–41.8% | 49.8% | 14.4% | 9.2% | 11.5 | $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell | 117 sRTX PRO 6000 | 0.110 sp90 0.137 s |
| 10 | GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Logic 32.3% (95%: 29.6%–35.1%) · Math 36.8% Non-hallucination 32.9% · “I don't know” on 24.6% of questions Confidence: 14 answers kept at ≥99%, 14 right · 0 wrong · AUROC 0.632 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.26estimate · DGX Spark per 1,000 questionsper 1M questions · 0.034 s per question (median latency) | 32.3%29.6%–35.1% | 36.8% | 32.9% | 24.6% | 16.2 | $0.002$2.26estimate · DGX Spark | 38 sDGX Spark | 0.034 sp90 0.042 s |
| 10 | Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Logic 32.1% (95%: 29.4%–34.9%) · Math 39.2% Non-hallucination 1.5% · “I don't know” on 1.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.660 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.091$90.99estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.123 s per question (median latency) | 32.1%29.4%–34.9% | 39.2% | 1.5% | 1.1% | 2.5 | $0.091$90.99estimate · RTX PRO 6000 Blackwell | 132 sRTX PRO 6000 | 0.123 sp90 0.152 s |
| 10 | KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Logic 31.7% (95%: 29.1%–34.5%) · Math 35.3% Non-hallucination 15.7% · “I don't know” on 11.5% of questions Confidence: 8 answers kept at ≥99%, 8 right · 0 wrong · AUROC 0.601 Runs on NVIDIA DGX Spark (GB10) · cost $0.004$3.78estimate · DGX Spark per 1,000 questionsper 1M questions · 0.059 s per question (median latency) | 31.7%29.1%–34.5% | 35.3% | 15.7% | 11.5% | 8.3 | $0.004$3.78estimate · DGX Spark | 64 sDGX Spark | 0.059 sp90 0.070 s |
| 10 | StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Logic 30.8% (95%: 28.1%–33.5%) · Math 38.7% Non-hallucination 31.4% · “I don't know” on 23.6% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.640 Runs on NVIDIA DGX Spark (GB10) · cost $0.003$2.52estimate · DGX Spark per 1,000 questionsper 1M questions · 0.040 s per question (median latency) | 30.8%28.1%–33.5% | 38.7% | 31.4% | 23.6% | 5.3 | $0.003$2.52estimate · DGX Spark | 43 sDGX Spark | 0.040 sp90 0.045 s |
| 10 | d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Logic 30.7% (95%: 28.0%–33.4%) · Math 37.1% Non-hallucination 86.5% · “I don't know” on 78.3% of questions Confidence: 3 answers kept at ≥99%, 3 right · 0 wrong · AUROC 0.620 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.015$14.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.017 s per question (median latency) | 30.7%28.0%–33.4% | 37.1% | 86.5% | 78.3% | 6.9 | $0.015$14.76estimate · RTX PRO 6000 Blackwell | 21 sRTX PRO 6000 | 0.017 sp90 0.023 s |
| 10 | GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Logic 29.2% (95%: 26.6%–31.9%) · Math 32.1% Non-hallucination 26.1% · “I don't know” on 20.0% of questions Confidence: 9 answers kept at ≥99%, 9 right · 0 wrong · AUROC 0.617 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.58estimate · DGX Spark per 1,000 questionsper 1M questions · 0.023 s per question (median latency) | 29.2%26.6%–31.9% | 32.1% | 26.1% | 20.0% | 14.0 | $0.002$1.58estimate · DGX Spark | 27 sDGX Spark | 0.023 sp90 0.029 s |
| 10 | OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Logic 27.0% (95%: 24.5%–29.7%) · Math 28.2% Non-hallucination 4.6% · “I don't know” on 3.4% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.545 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.16estimate · DGX Spark per 1,000 questionsper 1M questions · 0.016 s per question (median latency) | 27.0%24.5%–29.7% | 28.2% | 4.6% | 3.4% | 11.2 | $0.001$1.16estimate · DGX Spark | 20 sDGX Spark | 0.016 sp90 0.019 s |
| 10 | KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Logic 26.7% (95%: 24.2%–29.3%) · Math 28.2% Non-hallucination 1.2% · “I don't know” on 0.9% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.573 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.00estimate · DGX Spark per 1,000 questionsper 1M questions · 0.031 s per question (median latency) | 26.7%24.2%–29.3% | 28.2% | 1.2% | 0.9% | 8.4 | $0.002$2.00estimate · DGX Spark | 34 sDGX Spark | 0.031 sp90 0.039 s |
| 11 | StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Logic 24.7% (95%: 22.3%–27.3%) · Math 29.7% Non-hallucination 21.4% · “I don't know” on 16.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.641 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.79estimate · DGX Spark per 1,000 questionsper 1M questions · 0.027 s per question (median latency) | 24.7%22.3%–27.3% | 29.7% | 21.4% | 16.8% | 7.8 | $0.002$1.79estimate · DGX Spark | 30 sDGX Spark | 0.027 sp90 0.031 s |
| 11 | GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Logic 23.6% (95%: 21.2%–26.1%) · Math 22.7% Non-hallucination 12.4% · “I don't know” on 9.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.527 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.31estimate · DGX Spark per 1,000 questionsper 1M questions · 0.015 s per question (median latency) | 23.6%21.2%–26.1% | 22.7% | 12.4% | 9.7% | 8.7 | $0.001$1.31estimate · DGX Spark | 22 sDGX Spark | 0.015 sp90 0.028 s |
| 11 | GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Logic 23.2% (95%: 20.8%–25.8%) · Math 20.6% Non-hallucination 76.1% · “I don't know” on 69.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.556 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.95estimate · DGX Spark per 1,000 questionsper 1M questions · 0.013 s per question (median latency) | 23.2%20.8%–25.8% | 20.6% | 76.1% | 69.8% | 8.8 | $0.001$0.95estimate · DGX Spark | 16 sDGX Spark | 0.013 sp90 0.016 s |
| 11 | Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Logic 22.4% (95%: 20.1%–24.9%) · Math 28.7% Non-hallucination 8.3% · “I don't know” on 6.5% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.568 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.028$27.67estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 22.4%20.1%–24.9% | 28.7% | 8.3% | 6.5% | 12.0 | $0.028$27.67estimate · RTX PRO 6000 Blackwell | 40 sRTX PRO 6000 | 0.036 sp90 0.043 s |
| Tier | Model | Logic accuracy 1,125 questions, 95% interval; min–max for 3-seed rows | Math of 2,163 questions | Non-hallucination of answers not right, % “I don't know” | “I don't know” share declined | Calibration error, points; decision models | $ per 1,000 q$ per 1M q list price or GPU time | GPU time s per 1,000 q, our hardware | Latency median, p90 under it |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Logic 96.7% (95%: 95.5%–97.6%) · Math 99.6% Non-hallucination 56.4% · “I don't know” on 2.8% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 6.59 s per question (median latency) | 96.7%95.5%–97.6% | 99.6% | 56.4% | 2.8% | – | $0.000$0.00 | – | 6.59 sp90 14.3 s |
| 1 | Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Logic 96.7% (95%: 95.5%–97.6%) · Math 99.6% Non-hallucination 45.5% · “I don't know” on 1.8% of questions Runs on an API · 44.5 s per question (median latency) | 96.7%95.5%–97.6% | 99.6% | 45.5% | 1.8% | – | – | – | 44.5 sp90 206.7 s |
| 1 | GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Logic 96.5% (95%: 95.3%–97.5%) · Math 99.6% Non-hallucination 63.2% · “I don't know” on 3.2% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 7.07 s per question (median latency) | 96.5%95.3%–97.5% | 99.6% | 63.2% | 3.2% | – | $0.000$0.00 | – | 7.07 sp90 16.2 s |
| 1 | Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Logic 96.3% (95%: 95.0%–97.2%) · Math 99.8% Non-hallucination 0.0% · “I don't know” on 0.0% of questions Runs on an API · cost $5.690$5,690 per 1,000 questionsper 1M questions · 11.6 s per question (median latency) | 96.3%95.0%–97.2% | 99.8% | 0.0% | 0.0% | – | $5.690$5,690 | – | 11.6 sp90 38.6 s |
| 1 | Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $4.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Logic 96.2% (95%: 94.9%–97.2%) · Math 99.8% Non-hallucination 38.2% · “I don't know” on 1.9% of questions Runs on an API · cost $19.540$19,540 API list-price equivalent per 1,000 questionsper 1M questions · 5.83 s per question (median latency) | 96.2%94.9%–97.2% | 99.8% | 38.2% | 1.9% | – | $19.540$19,540 API list-price equivalent | – | 5.83 sp90 12.2 s |
| 1 | DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active · Mixture of experts: 552B total, about 16B active per step. Size class uses 552B. Logic 96.2% (95%: 94.9%–97.2%) · Math 99.6% Non-hallucination 3.8% · “I don't know” on 0.2% of questions Runs on an API · cost $1.647$1,647 per 1,000 questionsper 1M questions · 6.68 s per question (median latency) | 96.2%94.9%–97.2% | 99.6% | 3.8% | 0.2% | – | $1.647$1,647 | – | 6.68 sp90 40.0 s |
| 1 | Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Logic 96.1% (95%: 94.8%–97.1%) · Math 99.6% Non-hallucination 76.1% · “I don't know” on 4.8% of questions Runs on an API · 8.21 s per question (median latency) | 96.1%94.8%–97.1% | 99.6% | 76.1% | 4.8% | – | – | – | 8.21 sp90 24.1 s |
| 1 | Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active · Mixture of experts: 594B total, about 17B active per step. Size class uses 594B. Logic 95.6% (95%: 94.3%–96.7%) · Math 99.5% Non-hallucination 37.3% · “I don't know” on 2.0% of questions Runs on an API · cost $4.121$4,121 per 1,000 questionsper 1M questions · 13.2 s per question (median latency) | 95.6%94.3%–96.7% | 99.5% | 37.3% | 2.0% | – | $4.121$4,121 | – | 13.2 sp90 144.2 s |
| 1 | Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $2.00 per 1M input tokens: Workhorse (source: https://platform.claude.com/docs/en/about-claude/pricing) Logic 95.5% (95%: 94.1%–96.5%) · Math 99.7% Non-hallucination 41.3% · “I don't know” on 2.3% of questions Runs on an API · cost $11.480$11,480 API list-price equivalent per 1,000 questionsper 1M questions · 6.71 s per question (median latency) | 95.5%94.1%–96.5% | 99.7% | 41.3% | 2.3% | – | $11.480$11,480 API list-price equivalent | – | 6.71 sp90 14.9 s |
| 1 | Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/meta/muse-ember) Logic 95.5% (95%: 94.1%–96.5%) · Math 99.6% Non-hallucination 22.6% · “I don't know” on 1.1% of questions Runs on an API · cost $1.820$1,820 per 1,000 questionsper 1M questions · 3.11 s per question (median latency) | 95.5%94.1%–96.5% | 99.6% | 22.6% | 1.1% | – | $1.820$1,820 | – | 3.11 sp90 14.0 s |
| 1 | Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active · Mixture of experts: 276B total, about 12B active per step. Size class uses 276B. Logic 95.4% (95%: 94.0%–96.5%) · Math 99.6% Non-hallucination 50.0% · “I don't know” on 3.2% of questions Runs on an API · cost $5.489$5,489 per 1,000 questionsper 1M questions · 13.3 s per question (median latency) | 95.4%94.0%–96.5% | 99.6% | 50.0% | 3.2% | – | $5.489$5,489 | – | 13.3 sp90 78.5 s |
| 1 | Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $10.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Logic 95.2% (95%: 93.8%–96.3%) · Math 99.7% Non-hallucination 8.3% · “I don't know” on 0.4% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.82 s per question (median latency) | 95.2%93.8%–96.3% | 99.7% | 8.3% | 0.4% | – | $0.000$0.00 | – | 3.82 sp90 13.5 s |
| 1 | MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active · Mixture of experts: 309B total, about 15B active per step. Size class uses 309B. Logic 95.1% (95%: 93.7%–96.2%) · Math 99.4% Non-hallucination 23.3% · “I don't know” on 1.5% of questions Runs on an API · cost $1.390$1,390 per 1,000 questionsper 1M questions · 24.7 s per question (median latency) | 95.1%93.7%–96.2% | 99.4% | 23.3% | 1.5% | – | $1.390$1,390 | – | 24.7 sp90 179.9 s |
| 1 | Qwen3.5-397B-A17B on · chat Open · 397B MoE · 17B active Qwen3.5-397B-A17B on · chat · Alibaba · chat model · reasoning on Parameters: 397B MoE · 17B active · Mixture of experts: 397B total, about 17B active per step. Size class uses 397B. Logic 95.1% (95%: 93.7%–96.2%) · Math running Non-hallucination 31.7% · “I don't know” on 1.8% of questions Runs on an API · cost $13.426$13,426 per 1,000 questionsper 1M questions · 84.7 s per question (median latency) | 95.1%93.7%–96.2% | running | 31.7% | 1.8% | – | $13.426$13,426 | – | 84.7 sp90 167.5 s |
| 2 | Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Logic 94.0% (95%: 92.5%–95.3%) · Math 99.2% Non-hallucination 39.1% · “I don't know” on 3.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.884$4,884estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 18.7 s per question (median latency) | 94.0%92.5%–95.3% | 99.2% | 39.1% | 3.2% | – | $4.884$4,884estimate · RTX PRO 6000 Blackwell | 7,061 sRTX PRO 6000 | 18.7 sp90 101.0 s |
| 2 | GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Logic 94.0% (95%: 92.4%–95.2%) · Math 99.2% Non-hallucination 61.1% · “I don't know” on 4.9% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.64 s per question (median latency) | 94.0%92.4%–95.2% | 99.2% | 61.1% | 4.9% | – | $0.000$0.00 | – | 5.64 sp90 10.3 s |
| 2 | MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active · Mixture of experts: 428B total, about 23B active per step. Size class uses 428B. Logic 94.0% (95%: 92.4%–95.2%) · Math 98.2% Non-hallucination 38.3% · “I don't know” on 3.2% of questions Runs on an API · cost $0.803$803 per 1,000 questionsper 1M questions · 5.61 s per question (median latency) | 94.0%92.4%–95.2% | 98.2% | 38.3% | 3.2% | – | $0.803$803 | – | 5.61 sp90 37.4 s |
| 2 | Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Logic 94.0% (95%: 92.4%–95.2%) · Math 99.5% Non-hallucination 34.6% · “I don't know” on 2.5% of questions Runs on an API · cost $1.376$1,376 per 1,000 questionsper 1M questions · 17.6 s per question (median latency) | 94.0%92.4%–95.2% | 99.5% | 34.6% | 2.5% | – | $1.376$1,376 | – | 17.6 sp90 73.2 s |
| 2 | Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Logic 93.3% (95%: 91.7%–94.6%) · Math 99.4% Non-hallucination 16.3% · “I don't know” on 1.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.244$1,244estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 18.3 s per question (median latency) | 93.3%91.7%–94.6% | 99.4% | 16.3% | 1.4% | – | $1.244$1,244estimate · RTX PRO 6000 Blackwell | 1,798 sRTX PRO 6000 | 18.3 sp90 39.8 s |
| 2 | GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active · Mixture of experts: 320B total, about 18B active per step. Size class uses 320B. Logic 93.2% (95%: 91.5%–94.5%) · Math 98.8% Non-hallucination 4.2% · “I don't know” on 0.3% of questions Runs on an API · cost $1.057$1,057 per 1,000 questionsper 1M questions · 12.1 s per question (median latency) | 93.2%91.5%–94.5% | 98.8% | 4.2% | 0.3% | – | $1.057$1,057 | – | 12.1 sp90 82.8 s |
| 2 | Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Logic 92.4% (95%: 90.7%–93.8%) · Math 99.4% Non-hallucination 36.1% · “I don't know” on 3.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.525$3,525estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 142.5 s per question (median latency) | 92.4%90.7%–93.8% | 99.4% | 36.1% | 3.1% | – | $3.525$3,525estimate · RTX PRO 6000 Blackwell | 5,096 sRTX PRO 6000 | 142.5 sp90 246.3 s |
| 2 | Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Logic 91.6% (95%: 89.9%–93.1%) · Math 99.2% Non-hallucination 53.2% · “I don't know” on 5.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $10.893$10,893estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 221.3 s per question (median latency) | 91.6%89.9%–93.1% | 99.2% | 53.2% | 5.9% | – | $10.893$10,893estimate · RTX PRO 6000 Blackwell | 15,748 sRTX PRO 6000 | 221.3 sp90 1229.5 s |
| 3 | Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Logic 90.1% (95%: 88.3%–91.7%) · Math 98.5% Non-hallucination 33.6% · “I don't know” on 4.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.420$1,420estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 45.4 s per question (median latency) | 90.1%88.3%–91.7% | 98.5% | 33.6% | 4.1% | – | $1.420$1,420estimate · RTX PRO 6000 Blackwell | 2,053 sRTX PRO 6000 | 45.4 sp90 127.5 s |
| 3 | MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Logic 88.9% (mean of 3 seeds; min–max 88.3%–89.6%) · Math 99.2% Seed runs: seed 0: 88.7% (998 of 1,125, run minicpm5-2b_en) · seed 1: 88.3% (993 of 1,125, run minicpm5-2b-seed1_en) · seed 2: 89.6% (1,008 of 1,125, run minicpm5-2b-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b_en, not the mean Non-hallucination 51.5% · “I don't know” on 7.6% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.335$3,335estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 101.1 s per question (median latency) | 88.9%88.3%–89.6% min–max of 3 seeds | 99.2% | 51.5%seed 0 only | 7.6%seed 0 only | – | $3.335$3,335estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 4,821 sRTX PRO 6000mean of 3 seeds | 101.1 sp90 333.4 smean of 3 seeds |
| 3 | Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Logic 87.7% (95%: 85.7%–89.5%) · Math 98.8% Non-hallucination 15.5% · “I don't know” on 2.0% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.292$1,292estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 42.1 s per question (median latency) | 87.7%85.7%–89.5% | 98.8% | 15.5% | 2.0% | – | $1.292$1,292estimate · RTX PRO 6000 Blackwell | 1,868 sRTX PRO 6000 | 42.1 sp90 82.9 s |
| 3 | LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active · Mixture of experts: 24.0B total, about 2.45B active per step. Size class uses 24.0B. Logic 86.7% (95%: 84.6%–88.5%) · Math 99.0% Non-hallucination 37.8% · “I don't know” on 5.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $7.562$7,562estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 229.0 s per question (median latency) | 86.7%84.6%–88.5% | 99.0% | 37.8% | 5.2% | – | $7.562$7,562estimate · RTX PRO 6000 Blackwell | 10,932 sRTX PRO 6000 | 229.0 sp90 776.9 s |
| 3 | Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Logic 85.9% (mean of 3 seeds; min–max 85.5%–86.1%) · Math 98.5% Seed runs: seed 0: 86.0% (968 of 1,125, run gemma-4-e4b-it-think-32k-sampled-seed0_en) · seed 1: 86.1% (969 of 1,125, run gemma-4-e4b-it-think-32k-sampled-seed1_en) · seed 2: 85.5% (962 of 1,125, run gemma-4-e4b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e4b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 55.9% · “I don't know” on 10.5% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.993$993estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 31.6 s per question (median latency) | 85.9%85.5%–86.1% min–max of 3 seeds | 98.5% | 55.9%seed 0 only | 10.5%seed 0 only | – | $0.993$993estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,435 sRTX PRO 6000mean of 3 seeds | 31.6 sp90 83.5 smean of 3 seeds |
| 3 | LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Logic 85.5% (95%: 83.3%–87.4%) · Math 98.7% Non-hallucination 54.2% · “I don't know” on 10.4% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.487$487estimate · DGX Spark per 1,000 questionsper 1M questions · 163.9 s per question (median latency) | 85.5%83.3%–87.4% | 98.7% | 54.2% | 10.4% | – | $0.487$487estimate · DGX Spark | 8,234 sDGX Spark | 163.9 sp90 656.8 s |
| 4 | Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Logic 85.0% (95%: 82.8%–86.9%) · Math 98.5% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.503$1,503estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 57.9 s per question (median latency) | 85.0%82.8%–86.9% | 98.5% | – | – | – | $1.503$1,503estimate · RTX PRO 6000 Blackwell | 2,172 sRTX PRO 6000 | 57.9 sp90 109.4 s |
| 4 | LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Logic 84.6% (mean of 3 seeds; min–max 84.4%–85.0%) · Math 98.9% Seed runs: seed 0: 85.0% (956 of 1,125, run lfm3-1.7b-vllm-nothink-rtx-seed0_en) · seed 1: 84.4% (949 of 1,125, run lfm3-1.7b-vllm-nothink-rtx-seed1_en) · seed 2: 84.4% (949 of 1,125, run lfm3-1.7b-vllm-nothink-rtx-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm-nothink-rtx-seed0_en, not the mean Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.040$1,040estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 27.9 s per question (median latency) | 84.6%84.4%–85.0% min–max of 3 seeds | 98.9% | – | – | – | $1.040$1,040estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,503 sRTX PRO 6000mean of 3 seeds | 27.9 sp90 107.5 smean of 3 seeds |
| 4 | LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Logic 84.2% (mean of 3 seeds; min–max 83.9%–84.4%) · Math 98.9% Seed runs: seed 0: 84.4% (950 of 1,125, run lfm3-1.7b-vllm_en) · seed 1: 83.9% (944 of 1,125, run lfm3-1.7b-vllm-seed1_en) · seed 2: 84.4% (949 of 1,125, run lfm3-1.7b-vllm-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm_en, not the mean Non-hallucination 43.6% · “I don't know” on 8.8% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.017$1,017estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 29.8 s per question (median latency) | 84.2%83.9%–84.4% min–max of 3 seeds | 98.9% | 43.6%seed 0 only | 8.8%seed 0 only | – | $1.017$1,017estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,471 sRTX PRO 6000mean of 3 seeds | 29.8 sp90 113.2 smean of 3 seeds |
| – | DeepSeek-V4-Flash chat Open · 285B MoE · 13B active sample DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active · Mixture of experts: 285B total, about 13B active per step. Size class uses 285B. Not tiered: sample (317 questions) Logic 83.6% (95%: 79.1%–87.3%) · Math 99.3% Runs on NVIDIA DGX Spark (GB10) · cost $1.154$1,154estimate · DGX Spark per 1,000 questionsper 1M questions · 24.8 s per question (median latency) | 83.6%79.1%–87.3% | 99.3% | – | – | – | $1.154$1,154estimate · DGX Spark | 19,527 sDGX Spark | 24.8 sp90 91.1 s |
| 4 | MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Logic 83.6% (mean of 3 seeds; min–max 83.1%–83.9%) · Math 98.2% Seed runs: seed 0: 83.7% (942 of 1,125, run minicpm5-2b-nothink_en) · seed 1: 83.9% (944 of 1,125, run minicpm5-2b-nothink-seed1_en) · seed 2: 83.1% (935 of 1,125, run minicpm5-2b-nothink-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b-nothink_en, not the mean Non-hallucination 43.9% · “I don't know” on 11.8% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.327$1,327estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 46.5 s per question (median latency) | 83.6%83.1%–83.9% min–max of 3 seeds | 98.2% | 43.9%seed 0 only | 11.8%seed 0 only | – | $1.327$1,327estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 1,919 sRTX PRO 6000mean of 3 seeds | 46.5 sp90 111.6 smean of 3 seeds |
| 4 | Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Logic 83.2% (95%: 80.9%–85.3%) · Math 98.1% Non-hallucination 52.4% · “I don't know” on 11.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.842$842estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 32.7 s per question (median latency) | 83.2%80.9%–85.3% | 98.1% | 52.4% | 11.7% | – | $0.842$842estimate · RTX PRO 6000 Blackwell | 1,218 sRTX PRO 6000 | 32.7 sp90 64.1 s |
| 4 | Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Logic 82.6% (95%: 80.3%–84.7%) · Math 98.2% Non-hallucination 45.1% · “I don't know” on 9.3% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.650$650estimate · DGX Spark per 1,000 questionsper 1M questions · 284.1 s per question (median latency) | 82.6%80.3%–84.7% | 98.2% | 45.1% | 9.3% | – | $0.650$650estimate · DGX Spark | 10,988 sDGX Spark | 284.1 sp90 540.0 s |
| 5 | Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 76.8% (mean of 3 seeds; min–max 76.0%–77.6%) · Math 97.7% Seed runs: seed 0: 76.0% (855 of 1,125, run gemma-4-e2b-it-think-32k-sampled-seed0_en) · seed 1: 77.6% (873 of 1,125, run gemma-4-e2b-it-think-32k-sampled-seed1_en) · seed 2: 76.7% (863 of 1,125, run gemma-4-e2b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e2b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 32.6% · “I don't know” on 8.8% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.574$574estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 21.2 s per question (median latency) | 76.8%76.0%–77.6% min–max of 3 seeds | 97.7% | 32.6%seed 0 only | 8.8%seed 0 only | – | $0.574$574estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 830 sRTX PRO 6000mean of 3 seeds | 21.2 sp90 45.1 smean of 3 seeds |
| 5 | Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 75.9% (95%: 73.3%–78.3%) · Math 97.8% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.193$1,193estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 41.4 s per question (median latency) | 75.9%73.3%–78.3% | 97.8% | – | – | – | $1.193$1,193estimate · RTX PRO 6000 Blackwell | 1,724 sRTX PRO 6000 | 41.4 sp90 70.7 s |
| 5 | LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Logic 75.4% (95%: 72.8%–77.8%) · Math 94.0% Non-hallucination 44.3% · “I don't know” on 13.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.480$2,480estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 81.3 s per question (median latency) | 75.4%72.8%–77.8% | 94.0% | 44.3% | 13.6% | – | $2.480$2,480estimate · RTX PRO 6000 Blackwell | 3,585 sRTX PRO 6000 | 81.3 sp90 224.0 s |
| 5 | Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 74.6% (95%: 72.0%–77.0%) · Math 96.9% Non-hallucination 26.4% · “I don't know” on 6.9% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.363$363estimate · DGX Spark per 1,000 questionsper 1M questions · 173.2 s per question (median latency) | 74.6%72.0%–77.0% | 96.9% | 26.4% | 6.9% | – | $0.363$363estimate · DGX Spark | 6,145 sDGX Spark | 173.2 sp90 297.1 s |
| 5 | Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Logic 72.9% (95%: 70.2%–75.4%) · Math 96.8% Non-hallucination 34.0% · “I don't know” on 9.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.513$513estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 21.4 s per question (median latency) | 72.9%70.2%–75.4% | 96.8% | 34.0% | 9.8% | – | $0.513$513estimate · RTX PRO 6000 Blackwell | 742 sRTX PRO 6000 | 21.4 sp90 35.2 s |
| 5 | Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Logic 70.9% (95%: 68.2%–73.5%) · Math 97.0% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $15.330$15,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 493.2 s† per question (median latency) | 70.9%68.2%–73.5% | 97.0% | – | – | – | $15.330$15,330estimate · RTX PRO 6000 Blackwell† | 22,164 sRTX PRO 6000 | 493.2 s†p90 1396.5 s |
| 5 | Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Logic 70.3% (95%: 67.6%–72.9%) · Math 94.7% Non-hallucination 33.1% · “I don't know” on 9.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.330$4,330estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 186.3 s per question (median latency) | 70.3%67.6%–72.9% | 94.7% | 33.1% | 9.1% | – | $4.330$4,330estimate · RTX PRO 6000 Blackwell | 6,260 sRTX PRO 6000 | 186.3 sp90 272.2 s |
| 6 | Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Logic 65.3% (95%: 62.5%–68.1%) · Math 95.8% Non-hallucination 16.1% · “I don't know” on 6.3% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.772$1,772estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 16.6 s per question (median latency) | 65.3%62.5%–68.1% | 95.8% | 16.1% | 6.3% | – | $1.772$1,772estimate · RTX PRO 6000 Blackwell | 2,562 sRTX PRO 6000 | 16.6 sp90 178.7 s |
| 6 | Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Logic 65.1% (95%: 62.2%–67.8%) · Math 89.6% Non-hallucination 35.7% · “I don't know” on 12.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $9.270$9,270estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 276.7 s† per question (median latency) | 65.1%62.2%–67.8% | 89.6% | 35.7% | 12.8% | – | $9.270$9,270estimate · RTX PRO 6000 Blackwell† | 13,403 sRTX PRO 6000 | 276.7 s†p90 965.6 s |
| 6 | Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Logic 64.8% (95%: 62.0%–67.5%) · Math 94.6% Non-hallucination 35.2% · “I don't know” on 14.0% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.605$1,605estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 71.3 s per question (median latency) | 64.8%62.0%–67.5% | 94.6% | 35.2% | 14.0% | – | $1.605$1,605estimate · RTX PRO 6000 Blackwell | 2,320 sRTX PRO 6000 | 71.3 sp90 113.5 s |
| 6 | Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Logic 60.7% (95%: 57.8%–63.5%) · Math 81.6% Non-hallucination 6.6% · “I don't know” on 2.7% of questions Confidence: 172 answers kept at ≥99%, 169 right · 3 wrong · AUROC 0.816 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.128$128estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.173 s per question (median latency) | 60.7%57.8%–63.5% | 81.6% | 6.6% | 2.7% | 5.7 | $0.128$128estimate · RTX PRO 6000 Blackwell | 186 sRTX PRO 6000 | 0.173 sp90 0.211 s |
| 6 | LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active · Mixture of experts: 8.3B total, about 1.5B active per step. Size class uses 8.3B. Logic 60.2% (95%: 57.3%–63.0%) · Math 96.3% Non-hallucination 58.3% · “I don't know” on 29.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.664$4,664estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 138.1 s† per question (median latency) | 60.2%57.3%–63.0% | 96.3% | 58.3% | 29.8% | – | $4.664$4,664estimate · RTX PRO 6000 Blackwell† | 6,743 sRTX PRO 6000 | 138.1 s†p90 403.4 s |
| 6 | REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense · Two models run together, option probabilities averaged 50/50: 57.36B in all, all used on every question. Logic 59.3% (95%: 56.4%–62.1%) · Math 78.0% Non-hallucination 5.3% · “I don't know” on 2.2% of questions Confidence: 51 answers kept at ≥99%, 51 right · 0 wrong · AUROC 0.816 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.160$160estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.215 s per question (median latency) | 59.3%56.4%–62.1% | 78.0% | 5.3% | 2.2% | 3.7 | $0.160$160estimate · RTX PRO 6000 Blackwell | 232 sRTX PRO 6000 | 0.215 sp90 0.278 s |
| 6 | KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Logic 59.0% (95%: 56.1%–61.9%) · Math 77.4% Non-hallucination 0.4% · “I don't know” on 0.2% of questions Confidence: 89 answers kept at ≥99%, 89 right · 0 wrong · AUROC 0.804 Runs on NVIDIA DGX Spark (GB10) · cost $0.061$61.43estimate · DGX Spark per 1,000 questionsper 1M questions · 0.983 s per question (median latency) | 59.0%56.1%–61.9% | 77.4% | 0.4% | 0.2% | 2.6 | $0.061$61.43estimate · DGX Spark | 1,039 sDGX Spark | 0.983 sp90 1.28 s |
| 6 | KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Logic 58.9% (95%: 56.0%–61.8%) · Math 80.3% Non-hallucination 0.0% · “I don't know” on 0.0% of questions Confidence: 54 answers kept at ≥99%, 54 right · 0 wrong · AUROC 0.825 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.112$112estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.151 s per question (median latency) | 58.9%56.0%–61.8% | 80.3% | 0.0% | 0.0% | 3.7 | $0.112$112estimate · RTX PRO 6000 Blackwell | 162 sRTX PRO 6000 | 0.151 sp90 0.204 s |
| 6 | Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Logic 58.7% (95%: 55.8%–61.5%) · Math 77.3% Non-hallucination 1.9% · “I don't know” on 0.8% of questions Confidence: 66 answers kept at ≥99%, 66 right · 0 wrong · AUROC 0.801 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.136$136estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.185 s per question (median latency) | 58.7%55.8%–61.5% | 77.3% | 1.9% | 0.8% | 3.9 | $0.136$136estimate · RTX PRO 6000 Blackwell | 196 sRTX PRO 6000 | 0.185 sp90 0.255 s |
| 6 | GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Logic 58.4% (95%: 55.5%–61.2%) · Math 72.0% Non-hallucination 44.3% · “I don't know” on 21.1% of questions Confidence: 75 answers kept at ≥99%, 74 right · 1 wrong · AUROC 0.822 Runs on NVIDIA DGX Spark (GB10) · cost $0.033$33.16estimate · DGX Spark per 1,000 questionsper 1M questions · 0.544 s per question (median latency) | 58.4%55.5%–61.2% | 72.0% | 44.3% | 21.1% | 4.4 | $0.033$33.16estimate · DGX Spark | 561 sDGX Spark | 0.544 sp90 0.655 s |
| 7 | GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/openai/gpt-6-luna) Logic 58.0% (95%: 55.0%–60.8%) · Math 93.7% Non-hallucination 55.1% · “I don't know” on 28.1% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.86 s per question (median latency) | 58.0%55.0%–60.8% | 93.7% | 55.1% | 28.1% | – | $0.000$0.00 | – | 3.86 sp90 5.82 s |
| 7 | Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Logic 57.9% (95%: 55.0%–60.7%) · Math 77.1% Non-hallucination 5.5% · “I don't know” on 2.4% of questions Confidence: 81 answers kept at ≥99%, 79 right · 2 wrong · AUROC 0.797 Runs on NVIDIA DGX Spark (GB10) · cost $0.063$62.95estimate · DGX Spark per 1,000 questionsper 1M questions · 1.05 s per question (median latency) | 57.9%55.0%–60.7% | 77.1% | 5.5% | 2.4% | 2.1 | $0.063$62.95estimate · DGX Spark | 1,065 sDGX Spark | 1.05 sp90 1.33 s |
| 7 | Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Logic 57.8% (95%: 54.9%–60.6%) · Math 75.2% Non-hallucination 7.1% · “I don't know” on 3.0% of questions Confidence: 8 answers kept at ≥99%, 8 right · 0 wrong · AUROC 0.796 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.211$211estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.285 s per question (median latency) | 57.8%54.9%–60.6% | 75.2% | 7.1% | 3.0% | 5.0 | $0.211$211estimate · RTX PRO 6000 Blackwell | 306 sRTX PRO 6000 | 0.285 sp90 0.399 s |
| 7 | GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Logic 55.2% (95%: 52.3%–58.1%) · Math 69.9% Non-hallucination 50.2% · “I don't know” on 26.8% of questions Confidence: 45 answers kept at ≥99%, 45 right · 0 wrong · AUROC 0.793 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.111 s per question (median latency) | 55.2%52.3%–58.1% | 69.9% | 50.2% | 26.8% | 5.9 | $0.014$13.88 list$0.082$81.77estimate · RTX PRO 6000 Blackwell | 118 sRTX PRO 6000 | 0.111 sp90 0.140 s |
| 7 | d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://www.liquid.ai/blog/d1-ultra) Logic 54.9% (95%: 52.0%–57.8%) · Math 74.1% Non-hallucination 91.0% · “I don't know” on 64.9% of questions Confidence: 20 answers kept at ≥99%, 20 right · 0 wrong · AUROC 0.783 Runs on an API · cost $0.013$12.64 per 1,000 questionsper 1M questions · 0.182 s per question (median latency) | 54.9%52.0%–57.8% | 74.1% | 91.0% | 64.9% | 3.2 | $0.013$12.64 | – | 0.182 sp90 0.425 s |
| 7 | Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker · Closed; introductory list price (the only one published) $0.03 per 1M input tokens: Efficient (provisional) (source: https://nace.ai/drex) Logic 54.8% (95%: 51.8%–57.6%) · Math 70.1% Non-hallucination 54.1% · “I don't know” on 30.8% of questions Confidence: 90 answers kept at ≥99%, 89 right · 1 wrong · AUROC 0.793 Runs on an API · cost $0.022$22.37 per 1,000 questionsper 1M questions · 0.329 s per question (median latency) | 54.8%51.8%–57.6% | 70.1% | 54.1% | 30.8% | 6.1 | $0.022$22.37 | – | 0.329 sp90 1.54 s |
| 7 | Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Logic 54.8% (95%: 51.8%–57.6%) · Math 72.5% Non-hallucination 18.1% · “I don't know” on 9.1% of questions Confidence: 62 answers kept at ≥99%, 62 right · 0 wrong · AUROC 0.803 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.108 s per question (median latency) | 54.8%51.8%–57.6% | 72.5% | 18.1% | 9.1% | 9.0 | $0.081$81.16estimate · RTX PRO 6000 Blackwell | 117 sRTX PRO 6000 | 0.108 sp90 0.131 s |
| 7 | d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Logic 54.6% (95%: 51.7%–57.5%) · Math 71.5% Non-hallucination 10.1% · “I don't know” on 4.7% of questions Confidence: 28 answers kept at ≥99%, 28 right · 0 wrong · AUROC 0.778 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.109$109estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.156 s per question (median latency) | 54.6%51.7%–57.5% | 71.5% | 10.1% | 4.7% | 15.1 | $0.109$109estimate · RTX PRO 6000 Blackwell | 158 sRTX PRO 6000 | 0.156 sp90 0.185 s |
| 7 | GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker · Closed; list price $0.15 per 1M input tokens: Efficient (source: https://docs.fastino.ai/pricing) Logic 54.5% (95%: 51.6%–57.4%) · Math 70.8% Non-hallucination 60.0% · “I don't know” on 32.5% of questions Confidence: 52 answers kept at ≥99%, 52 right · 0 wrong · AUROC 0.814 Runs on an API · cost $0.052$52.03 per 1,000 questionsper 1M questions · 0.271 s per question (median latency) | 54.5%51.6%–57.4% | 70.8% | 60.0% | 32.5% | 6.6 | $0.052$52.03 | – | 0.271 sp90 0.291 s |
| 7 | Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Logic 54.5% (95%: 51.6%–57.4%) · Math 74.0% Non-hallucination 72.3% · “I don't know” on 42.4% of questions Confidence: 108 answers kept at ≥99%, 106 right · 2 wrong · AUROC 0.795 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.221 s per question (median latency) | 54.5%51.6%–57.4% | 74.0% | 72.3% | 42.4% | 5.3 | $0.038$38.39 list$0.151$151estimate · RTX PRO 6000 Blackwell | 218 sRTX PRO 6000 | 0.221 sp90 0.244 s |
| 7 | d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Logic 54.5% (95%: 51.6%–57.4%) · Math 70.4% Non-hallucination 14.8% · “I don't know” on 7.2% of questions Confidence: 54 answers kept at ≥99%, 54 right · 0 wrong · AUROC 0.806 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.057$57.18estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.076 s per question (median latency) | 54.5%51.6%–57.4% | 70.4% | 14.8% | 7.2% | 4.6 | $0.057$57.18estimate · RTX PRO 6000 Blackwell | 83 sRTX PRO 6000 | 0.076 sp90 0.098 s |
| 7 | Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Logic 54.2% (95%: 51.3%–57.1%) · Math 90.8% Non-hallucination 30.7% · “I don't know” on 16.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.390$390estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 10.2 s per question (median latency) | 54.2%51.3%–57.1% | 90.8% | 30.7% | 16.2% | – | $0.390$390estimate · RTX PRO 6000 Blackwell | 564 sRTX PRO 6000 | 10.2 sp90 18.4 s |
| 7 | Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Logic 53.3% (95%: 50.4%–56.2%) · Math 69.8% Non-hallucination 14.4% · “I don't know” on 7.1% of questions Confidence: 80 answers kept at ≥99%, 78 right · 2 wrong · AUROC 0.798 Runs on an API · cost $0.007$7.46 per 1,000 questionsper 1M questions · 0.238 s per question (median latency) | 53.3%50.4%–56.2% | 69.8% | 14.4% | 7.1% | 3.4 | $0.007$7.46 | – | 0.238 sp90 0.552 s |
| 7 | Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Logic 53.2% (95%: 50.3%–56.1%) · Math 70.3% Non-hallucination 13.4% · “I don't know” on 6.6% of questions Confidence: 90 answers kept at ≥99%, 88 right · 2 wrong · AUROC 0.799 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.008$7.52 list$0.105$105estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.136 s per question (median latency) Shown under Perplexity Decider 27B v1.2 (API): the same model name, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings | 53.2%50.3%–56.1% | 70.3% | 13.4% | 6.6% | 4.6 | $0.008$7.52 list$0.105$105estimate · RTX PRO 6000 Blackwell | 151 sRTX PRO 6000 | 0.136 sp90 0.175 s |
| 7 | Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Logic 53.1% (95%: 50.1%–56.0%) · Math 70.3% Non-hallucination 7.3% · “I don't know” on 3.5% of questions Confidence: 155 answers kept at ≥99%, 147 right · 8 wrong · AUROC 0.786 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.106 s per question (median latency) | 53.1%50.1%–56.0% | 70.3% | 7.3% | 3.5% | 16.2 | $0.023$22.55 list$0.081$81.16estimate · RTX PRO 6000 Blackwell | 117 sRTX PRO 6000 | 0.106 sp90 0.143 s |
| – | Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Not tiered: with web Logic 52.5% (95%: 49.6%–55.4%) · Math 69.3% Non-hallucination 37.7% · “I don't know” on 19.9% of questions Confidence: 79 answers kept at ≥99%, 78 right · 1 wrong · AUROC 0.800 Runs on an API · cost $1.030$1,030 per 1,000 questionsper 1M questions · 0.448 s per question (median latency) | 52.5%49.6%–55.4% | 69.3% | 37.7% | 19.9% | 4.2 | $1.030$1,030 | – | 0.448 sp90 0.827 s |
| 7 | SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Logic 52.4% (95%: 49.4%–55.3%) · Math 68.6% Non-hallucination 65.7% · “I don't know” on 39.0% of questions Confidence: 72 answers kept at ≥99%, 70 right · 2 wrong · AUROC 0.787 Runs on an API · cost $0.015$14.92 per 1,000 questionsper 1M questions · 0.384 s per question (median latency) Shown once with SubQ Snap Preview: the same answer to every question | 52.4%49.4%–55.3% | 68.6% | 65.7% | 39.0% | 12.9 | $0.015$14.92 | – | 0.384 sp90 0.393 s |
| 7 | SubQ Snap Preview Closed · Efficient SubQ Snap Preview · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.05 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Logic 52.4% (95%: 49.4%–55.3%) · Math 68.6% Non-hallucination 65.7% · “I don't know” on 39.0% of questions Confidence: 72 answers kept at ≥99%, 70 right · 2 wrong · AUROC 0.787 Runs on an API · cost $0.019$18.64 per 1,000 questionsper 1M questions · 0.382 s per question (median latency) Shown under SubQ Snap 1.1 (the same answers) | 52.4%49.4%–55.3% | 68.6% | 65.7% | 39.0% | 12.9 | $0.019$18.64 | – | 0.382 sp90 0.390 s |
| 7 | Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Logic 52.3% (95%: 49.3%–55.2%) · Math 69.0% Non-hallucination 1.5% · “I don't know” on 0.7% of questions Confidence: 7 answers kept at ≥99%, 7 right · 0 wrong · AUROC 0.782 Runs on NVIDIA DGX Spark (GB10) · cost $0.025$24.91estimate · DGX Spark per 1,000 questionsper 1M questions · 0.385 s per question (median latency) | 52.3%49.3%–55.2% | 69.0% | 1.5% | 0.7% | 2.9 | $0.025$24.91estimate · DGX Spark | 421 sDGX Spark | 0.385 sp90 0.533 s |
| 7 | d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Logic 51.6% (95%: 48.6%–54.5%) · Math 69.9% Non-hallucination 94.5% · “I don't know” on 71.8% of questions Confidence: 13 answers kept at ≥99%, 13 right · 0 wrong · AUROC 0.800 Runs on an API · cost $0.013$12.64 per 1,000 questionsper 1M questions · 0.220 s per question (median latency) | 51.6%48.6%–54.5% | 69.9% | 94.5% | 71.8% | 3.0 | $0.013$12.64 | – | 0.220 sp90 0.341 s |
| 7 | KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Logic 50.6% (95%: 47.7%–53.5%) · Math 66.3% Non-hallucination 8.8% · “I don't know” on 4.5% of questions Confidence: 6 answers kept at ≥99%, 6 right · 0 wrong · AUROC 0.774 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.068 s per question (median latency) | 50.6%47.7%–53.5% | 66.3% | 8.8% | 4.5% | 4.8 | $0.052$51.64estimate · RTX PRO 6000 Blackwell | 75 sRTX PRO 6000 | 0.068 sp90 0.088 s |
| 8 | Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/google/gemini-decision-1) Logic 50.0% (95%: 47.1%–53.0%) · Math 71.3% Non-hallucination 2.0% · “I don't know” on 1.0% of questions Confidence: 43 answers kept at ≥99%, 43 right · 0 wrong · AUROC 0.768 Runs on an API · cost $0.037$36.77 per 1,000 questionsper 1M questions · 0.228 s per question (median latency) | 50.0%47.1%–53.0% | 71.3% | 2.0% | 1.0% | 3.1 | $0.037$36.77 | – | 0.228 sp90 0.344 s |
| 8 | d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Logic 49.2% (95%: 46.2%–52.1%) · Math 65.5% Non-hallucination 78.8% · “I don't know” on 56.6% of questions Confidence: 28 answers kept at ≥99%, 28 right · 0 wrong · AUROC 0.742 Runs on an API · cost $0.006$6.02 per 1,000 questionsper 1M questions · 0.222 s per question (median latency) | 49.2%46.2%–52.1% | 65.5% | 78.8% | 56.6% | 2.6 | $0.006$6.02 | – | 0.222 sp90 0.390 s |
| 8 | Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 48.2% (95%: 45.3%–51.1%) · Math 59.8% Non-hallucination 23.4% · “I don't know” on 13.1% of questions Confidence: 10 answers kept at ≥99%, 10 right · 0 wrong · AUROC 0.753 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.039$39.35estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.052 s per question (median latency) | 48.2%45.3%–51.1% | 59.8% | 23.4% | 13.1% | 4.0 | $0.039$39.35estimate · RTX PRO 6000 Blackwell | 57 sRTX PRO 6000 | 0.052 sp90 0.066 s |
| 8 | Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://docs.typesafe.ai/models) Logic 48.2% (95%: 45.3%–51.1%) · Math 65.1% Non-hallucination 24.5% · “I don't know” on 14.1% of questions Confidence: 61 answers kept at ≥99%, 61 right · 0 wrong · AUROC 0.750 Runs on an API · cost $0.026$25.64 per 1,000 questionsper 1M questions · 0.166 s per question (median latency) | 48.2%45.3%–51.1% | 65.1% | 24.5% | 14.1% | 3.3 | $0.026$25.64 | – | 0.166 sp90 0.193 s |
| 8 | GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Logic 47.8% (95%: 44.9%–50.7%) · Math 57.9% Non-hallucination 30.4% · “I don't know” on 17.7% of questions Confidence: 32 answers kept at ≥99%, 32 right · 0 wrong · AUROC 0.749 Runs on NVIDIA DGX Spark (GB10) · cost $0.013$13.29estimate · DGX Spark per 1,000 questionsper 1M questions · 0.208 s per question (median latency) | 47.8%44.9%–50.7% | 57.9% | 30.4% | 17.7% | 6.7 | $0.013$13.29estimate · DGX Spark | 225 sDGX Spark | 0.208 sp90 0.251 s |
| 8 | celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Logic 47.3% (95%: 44.4%–50.2%) · Math 70.0% Non-hallucination 92.3% · “I don't know” on 72.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.735 Runs on an API · cost $0.015$15.40 per 1,000 questionsper 1M questions · 0.390 s per question (median latency) | 47.3%44.4%–50.2% | 70.0% | 92.3% | 72.2% | 10.7 | $0.015$15.40 | – | 0.390 sp90 0.394 s |
| 8 | celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.004 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Logic 47.3% (95%: 44.4%–50.2%) · Math 69.9% Non-hallucination 91.0% · “I don't know” on 71.0% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.733 Runs on an API · cost $0.002$1.54 per 1,000 questionsper 1M questions · 0.383 s per question (median latency) | 47.3%44.4%–50.2% | 69.9% | 91.0% | 71.0% | 9.8 | $0.002$1.54 | – | 0.383 sp90 0.390 s |
| 8 | StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Logic 46.6% (95%: 43.7%–49.5%) · Math 58.4% Non-hallucination 17.4% · “I don't know” on 10.0% of questions Confidence: 14 answers kept at ≥99%, 14 right · 0 wrong · AUROC 0.762 Runs on NVIDIA DGX Spark (GB10) · cost $0.010$10.04estimate · DGX Spark per 1,000 questionsper 1M questions · 0.160 s per question (median latency) | 46.6%43.7%–49.5% | 58.4% | 17.4% | 10.0% | 3.8 | $0.010$10.04estimate · DGX Spark | 170 sDGX Spark | 0.160 sp90 0.175 s |
| 8 | Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 46.4% (95%: 43.5%–49.3%) · Math 62.1% Non-hallucination 12.2% · “I don't know” on 6.8% of questions Confidence: 55 answers kept at ≥99%, 54 right · 1 wrong · AUROC 0.771 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.031$31.36estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.041 s per question (median latency) | 46.4%43.5%–49.3% | 62.1% | 12.2% | 6.8% | 7.2 | $0.031$31.36estimate · RTX PRO 6000 Blackwell | 45 sRTX PRO 6000 | 0.041 sp90 0.046 s |
| 8 | Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Logic 46.4% (95%: 43.5%–49.3%) · Math 61.3% Non-hallucination 75.6% · “I don't know” on 52.9% of questions Confidence: 53 answers kept at ≥99%, 53 right · 0 wrong · AUROC 0.768 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.079 s per question (median latency) | 46.4%43.5%–49.3% | 61.3% | 75.6% | 52.9% | 8.6 | $0.013$13.44 list$0.058$58.41estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.079 sp90 0.098 s |
| 8 | Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://typesafe.ai/blog/introducing-system-one-models-and-jev) Logic 46.3% (95%: 43.4%–49.2%) · Math 64.1% Non-hallucination 15.8% · “I don't know” on 9.1% of questions Confidence: 65 answers kept at ≥99%, 65 right · 0 wrong · AUROC 0.764 Runs on an API · cost $0.026$25.64 per 1,000 questionsper 1M questions · 0.167 s per question (median latency) | 46.3%43.4%–49.2% | 64.1% | 15.8% | 9.1% | 5.9 | $0.026$25.64 | – | 0.167 sp90 0.203 s |
| 8 | d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Logic 46.0% (95%: 43.2%–49.0%) · Math 65.2% Non-hallucination 90.6% · “I don't know” on 71.6% of questions Confidence: 48 answers kept at ≥99%, 48 right · 0 wrong · AUROC 0.748 Runs on an API · cost $0.006$6.02 per 1,000 questionsper 1M questions · 0.176 s per question (median latency) | 46.0%43.2%–49.0% | 65.2% | 90.6% | 71.6% | 4.4 | $0.006$6.02 | – | 0.176 sp90 0.272 s |
| 8 | StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Logic 45.9% (95%: 43.0%–48.8%) · Math 55.8% Non-hallucination 16.0% · “I don't know” on 9.2% of questions Confidence: 21 answers kept at ≥99%, 21 right · 0 wrong · AUROC 0.720 Runs on NVIDIA DGX Spark (GB10) · cost $0.005$4.62estimate · DGX Spark per 1,000 questionsper 1M questions · 0.073 s per question (median latency) | 45.9%43.0%–48.8% | 55.8% | 16.0% | 9.2% | 3.3 | $0.005$4.62estimate · DGX Spark | 78 sDGX Spark | 0.073 sp90 0.084 s |
| 8 | Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Logic 45.3% (95%: 42.4%–48.3%) · Math 58.8% Non-hallucination 12.4% · “I don't know” on 7.1% of questions Confidence: 56 answers kept at ≥99%, 56 right · 0 wrong · AUROC 0.759 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.034$33.81estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.047 s per question (median latency) | 45.3%42.4%–48.3% | 58.8% | 12.4% | 7.1% | 14.9 | $0.034$33.81estimate · RTX PRO 6000 Blackwell | 49 sRTX PRO 6000 | 0.047 sp90 0.053 s |
| 8 | Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://openrouter.ai/inception/mercury-decide) Logic 45.1% (95%: 42.2%–48.0%) · Math 53.5% Non-hallucination 20.5% · “I don't know” on 12.2% of questions Confidence: 91 answers kept at ≥99%, 84 right · 7 wrong · AUROC 0.746 Runs on an API · cost $0.008$7.52 per 1,000 questionsper 1M questions · 0.301 s per question (median latency) | 45.1%42.2%–48.0% | 53.5% | 20.5% | 12.2% | 15.4 | $0.008$7.52 | – | 0.301 sp90 0.360 s |
| 8 | Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Logic 44.7% (95%: 41.8%–47.6%) · Math 55.9% Non-hallucination 24.4% · “I don't know” on 14.9% of questions Confidence: 17 answers kept at ≥99%, 17 right · 0 wrong · AUROC 0.757 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.026$26.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.033 s per question (median latency) | 44.7%41.8%–47.6% | 55.9% | 24.4% | 14.9% | 10.1 | $0.026$26.44estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.033 sp90 0.041 s |
| 8 | Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://openrouter.ai/microsoft/microsoft-decision-1) Base model: Qwen3.5-9B Logic 44.4% (95%: 41.6%–47.4%) · Math 59.4% Non-hallucination 19.8% · “I don't know” on 12.0% of questions Confidence: 20 answers kept at ≥99%, 20 right · 0 wrong · AUROC 0.712 Runs on an API · cost $0.013$13.42 per 1,000 questionsper 1M questions · 0.315 s per question (median latency) | 44.4%41.6%–47.4% | 59.4% | 19.8% | 12.0% | 8.0 | $0.013$13.42 | – | 0.315 sp90 0.362 s |
| 8 | Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 44.4% (95%: 41.5%–47.3%) · Math 55.3% Non-hallucination 13.6% · “I don't know” on 8.2% of questions Confidence: 37 answers kept at ≥99%, 35 right · 2 wrong · AUROC 0.701 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.042$42.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.060 s per question (median latency) | 44.4%41.5%–47.3% | 55.3% | 13.6% | 8.2% | 5.8 | $0.042$42.42estimate · RTX PRO 6000 Blackwell | 61 sRTX PRO 6000 | 0.060 sp90 0.067 s |
| 8 | KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 44.3% (95%: 41.4%–47.2%) · Math 59.8% Non-hallucination 6.7% · “I don't know” on 3.9% of questions Confidence: 30 answers kept at ≥99%, 30 right · 0 wrong · AUROC 0.725 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.036$36.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.050 s per question (median latency) | 44.3%41.4%–47.2% | 59.8% | 6.7% | 3.9% | 3.5 | $0.036$36.27estimate · RTX PRO 6000 Blackwell | 52 sRTX PRO 6000 | 0.050 sp90 0.061 s |
| 8 | vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Logic 44.1% (95%: 41.2%–47.0%) · Math 56.8% Non-hallucination 13.6% · “I don't know” on 8.1% of questions Confidence: 6 answers kept at ≥99%, 6 right · 0 wrong · AUROC 0.739 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.026$25.82estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.032 s per question (median latency) | 44.1%41.2%–47.0% | 56.8% | 13.6% | 8.1% | 4.7 | $0.026$25.82estimate · RTX PRO 6000 Blackwell | 37 sRTX PRO 6000 | 0.032 sp90 0.037 s |
| 8 | KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Logic 43.9% (95%: 41.0%–46.8%) · Math 60.1% Non-hallucination 19.5% · “I don't know” on 12.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.730 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.040$39.96estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.052 s per question (median latency) | 43.9%41.0%–46.8% | 60.1% | 19.5% | 12.1% | 3.2 | $0.040$39.96estimate · RTX PRO 6000 Blackwell | 58 sRTX PRO 6000 | 0.052 sp90 0.065 s |
| 8 | Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Logic 43.3% (95%: 40.4%–46.2%) · Math 56.0% Non-hallucination 6.0% · “I don't know” on 3.4% of questions Confidence: 27 answers kept at ≥99%, 26 right · 1 wrong · AUROC 0.752 Runs on NVIDIA DGX Spark (GB10) · cost $0.013$13.35estimate · DGX Spark per 1,000 questionsper 1M questions · 0.209 s per question (median latency) | 43.3%40.4%–46.2% | 56.0% | 6.0% | 3.4% | 17.0 | $0.013$13.35estimate · DGX Spark | 226 sDGX Spark | 0.209 sp90 0.269 s |
| 8 | Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Logic 43.1% (95%: 40.2%–46.0%) · Math 55.2% Non-hallucination 28.9% · “I don't know” on 18.3% of questions Confidence: 35 answers kept at ≥99%, 35 right · 0 wrong · AUROC 0.729 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.045 s per question (median latency) | 43.1%40.2%–46.0% | 55.2% | 28.9% | 18.3% | 6.0 | $0.004$3.76 list$0.034$34.43estimate · RTX PRO 6000 Blackwell | 50 sRTX PRO 6000 | 0.045 sp90 0.055 s |
| 9 | Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Logic 42.6% (95%: 39.7%–45.5%) · Math 54.6% Non-hallucination 28.2% · “I don't know” on 17.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.733 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.294$294estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.413 s per question (median latency) | 42.6%39.7%–45.5% | 54.6% | 28.2% | 17.7% | 12.9 | $0.294$294estimate · RTX PRO 6000 Blackwell | 426 sRTX PRO 6000 | 0.413 sp90 0.576 s |
| 9 | KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Logic 42.3% (95%: 39.5%–45.2%) · Math 56.0% Non-hallucination 8.3% · “I don't know” on 5.0% of questions Confidence: 18 answers kept at ≥99%, 17 right · 1 wrong · AUROC 0.708 Runs on NVIDIA DGX Spark (GB10) · cost $0.006$5.89estimate · DGX Spark per 1,000 questionsper 1M questions · 0.096 s per question (median latency) | 42.3%39.5%–45.2% | 56.0% | 8.3% | 5.0% | 13.7 | $0.006$5.89estimate · DGX Spark | 100 sDGX Spark | 0.096 sp90 0.115 s |
| 9 | Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active · Mixture of experts: 35B total, about 3B active per step. Size class uses 35B. Logic 41.9% (95%: 39.0%–44.8%) · Math 59.0% Non-hallucination 37.7% · “I don't know” on 25.1% of questions Confidence: 33 answers kept at ≥99%, 33 right · 0 wrong · AUROC 0.717 Runs on an API · cost $0.018$18.41 per 1,000 questionsper 1M questions · 0.345 s per question (median latency) | 41.9%39.0%–44.8% | 59.0% | 37.7% | 25.1% | 4.3 | $0.018$18.41 | – | 0.345 sp90 0.504 s |
| 9 | Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Logic 40.8% (95%: 38.0%–43.7%) · Math 51.1% Non-hallucination 3.6% · “I don't know” on 2.1% of questions Confidence: 6 answers kept at ≥99%, 6 right · 0 wrong · AUROC 0.697 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.196 s per question (median latency) | 40.8%38.0%–43.7% | 51.1% | 3.6% | 2.1% | 15.7 | $0.106$106 list$0.141$141estimate · RTX PRO 6000 Blackwell | 204 sRTX PRO 6000 | 0.196 sp90 0.222 s |
| 9 | d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Logic 40.8% (95%: 38.0%–43.7%) · Math 55.6% Non-hallucination 11.3% · “I don't know” on 7.2% of questions Confidence: 56 answers kept at ≥99%, 55 right · 1 wrong · AUROC 0.723 Runs on an API · cost $0.006$6.02 per 1,000 questionsper 1M questions · 0.188 s per question (median latency) | 40.8%38.0%–43.7% | 55.6% | 11.3% | 7.2% | 10.9 | $0.006$6.02 | – | 0.188 sp90 0.290 s |
| 9 | OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Logic 40.6% (95%: 37.8%–43.5%) · Math 53.8% Non-hallucination 1.2% · “I don't know” on 0.7% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.726 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.086$86.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.108 s per question (median latency) | 40.6%37.8%–43.5% | 53.8% | 1.2% | 0.7% | 4.6 | $0.086$86.07estimate · RTX PRO 6000 Blackwell | 124 sRTX PRO 6000 | 0.108 sp90 0.136 s |
| 9 | Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Logic 40.4% (95%: 37.5%–43.3%) · Math 50.4% Non-hallucination 14.6% · “I don't know” on 9.0% of questions Confidence: 39 answers kept at ≥99%, 39 right · 0 wrong · AUROC 0.724 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.030$29.51estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 40.4%37.5%–43.3% | 50.4% | 14.6% | 9.0% | 15.5 | $0.030$29.51estimate · RTX PRO 6000 Blackwell | 43 sRTX PRO 6000 | 0.038 sp90 0.048 s |
| 9 | Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Logic 39.7% (95%: 36.9%–42.6%) · Math 51.5% Non-hallucination 12.6% · “I don't know” on 7.9% of questions Confidence: 35 answers kept at ≥99%, 35 right · 0 wrong · AUROC 0.729 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.033$33.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.045 s per question (median latency) | 39.7%36.9%–42.6% | 51.5% | 12.6% | 7.9% | 10.8 | $0.033$33.20estimate · RTX PRO 6000 Blackwell | 48 sRTX PRO 6000 | 0.045 sp90 0.051 s |
| 9 | Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Logic 39.7% (95%: 36.9%–42.6%) · Math 46.9% Non-hallucination 29.7% · “I don't know” on 19.9% of questions Confidence: 32 answers kept at ≥99%, 32 right · 0 wrong · AUROC 0.727 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.052$51.64estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.066 s per question (median latency) | 39.7%36.9%–42.6% | 46.9% | 29.7% | 19.9% | 14.4 | $0.052$51.64estimate · RTX PRO 6000 Blackwell | 75 sRTX PRO 6000 | 0.066 sp90 0.091 s |
| 9 | Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Logic 39.3% (95%: 36.5%–42.2%) · Math 44.3% Non-hallucination 11.3% · “I don't know” on 7.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.720 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.040 s per question (median latency) | 39.3%36.5%–42.2% | 44.3% | 11.3% | 7.1% | 11.2 | $0.040$39.71 list$0.031$30.74estimate · RTX PRO 6000 Blackwell | 44 sRTX PRO 6000 | 0.040 sp90 0.048 s |
| 9 | Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Logic 38.9% (95%: 36.1%–41.8%) · Math 49.8% Non-hallucination 14.4% · “I don't know” on 9.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.659 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.110 s per question (median latency) | 38.9%36.1%–41.8% | 49.8% | 14.4% | 9.2% | 11.5 | $0.017$16.83 list$0.081$81.16estimate · RTX PRO 6000 Blackwell | 117 sRTX PRO 6000 | 0.110 sp90 0.137 s |
| 10 | GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Logic 32.3% (95%: 29.6%–35.1%) · Math 36.8% Non-hallucination 32.9% · “I don't know” on 24.6% of questions Confidence: 14 answers kept at ≥99%, 14 right · 0 wrong · AUROC 0.632 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.26estimate · DGX Spark per 1,000 questionsper 1M questions · 0.034 s per question (median latency) | 32.3%29.6%–35.1% | 36.8% | 32.9% | 24.6% | 16.2 | $0.002$2.26estimate · DGX Spark | 38 sDGX Spark | 0.034 sp90 0.042 s |
| 10 | Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Logic 32.1% (95%: 29.4%–34.9%) · Math 39.2% Non-hallucination 1.5% · “I don't know” on 1.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.660 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.091$90.99estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.123 s per question (median latency) | 32.1%29.4%–34.9% | 39.2% | 1.5% | 1.1% | 2.5 | $0.091$90.99estimate · RTX PRO 6000 Blackwell | 132 sRTX PRO 6000 | 0.123 sp90 0.152 s |
| 10 | KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Logic 31.7% (95%: 29.1%–34.5%) · Math 35.3% Non-hallucination 15.7% · “I don't know” on 11.5% of questions Confidence: 8 answers kept at ≥99%, 8 right · 0 wrong · AUROC 0.601 Runs on NVIDIA DGX Spark (GB10) · cost $0.004$3.78estimate · DGX Spark per 1,000 questionsper 1M questions · 0.059 s per question (median latency) | 31.7%29.1%–34.5% | 35.3% | 15.7% | 11.5% | 8.3 | $0.004$3.78estimate · DGX Spark | 64 sDGX Spark | 0.059 sp90 0.070 s |
| 10 | StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Logic 30.8% (95%: 28.1%–33.5%) · Math 38.7% Non-hallucination 31.4% · “I don't know” on 23.6% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.640 Runs on NVIDIA DGX Spark (GB10) · cost $0.003$2.52estimate · DGX Spark per 1,000 questionsper 1M questions · 0.040 s per question (median latency) | 30.8%28.1%–33.5% | 38.7% | 31.4% | 23.6% | 5.3 | $0.003$2.52estimate · DGX Spark | 43 sDGX Spark | 0.040 sp90 0.045 s |
| 10 | d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Logic 30.7% (95%: 28.0%–33.4%) · Math 37.1% Non-hallucination 86.5% · “I don't know” on 78.3% of questions Confidence: 3 answers kept at ≥99%, 3 right · 0 wrong · AUROC 0.620 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.015$14.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.017 s per question (median latency) | 30.7%28.0%–33.4% | 37.1% | 86.5% | 78.3% | 6.9 | $0.015$14.76estimate · RTX PRO 6000 Blackwell | 21 sRTX PRO 6000 | 0.017 sp90 0.023 s |
| 10 | GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Logic 29.2% (95%: 26.6%–31.9%) · Math 32.1% Non-hallucination 26.1% · “I don't know” on 20.0% of questions Confidence: 9 answers kept at ≥99%, 9 right · 0 wrong · AUROC 0.617 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.58estimate · DGX Spark per 1,000 questionsper 1M questions · 0.023 s per question (median latency) | 29.2%26.6%–31.9% | 32.1% | 26.1% | 20.0% | 14.0 | $0.002$1.58estimate · DGX Spark | 27 sDGX Spark | 0.023 sp90 0.029 s |
| 10 | OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Logic 27.0% (95%: 24.5%–29.7%) · Math 28.2% Non-hallucination 4.6% · “I don't know” on 3.4% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.545 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.16estimate · DGX Spark per 1,000 questionsper 1M questions · 0.016 s per question (median latency) | 27.0%24.5%–29.7% | 28.2% | 4.6% | 3.4% | 11.2 | $0.001$1.16estimate · DGX Spark | 20 sDGX Spark | 0.016 sp90 0.019 s |
| 10 | KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Logic 26.7% (95%: 24.2%–29.3%) · Math 28.2% Non-hallucination 1.2% · “I don't know” on 0.9% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.573 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.00estimate · DGX Spark per 1,000 questionsper 1M questions · 0.031 s per question (median latency) | 26.7%24.2%–29.3% | 28.2% | 1.2% | 0.9% | 8.4 | $0.002$2.00estimate · DGX Spark | 34 sDGX Spark | 0.031 sp90 0.039 s |
| 11 | StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Logic 24.7% (95%: 22.3%–27.3%) · Math 29.7% Non-hallucination 21.4% · “I don't know” on 16.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.641 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.79estimate · DGX Spark per 1,000 questionsper 1M questions · 0.027 s per question (median latency) | 24.7%22.3%–27.3% | 29.7% | 21.4% | 16.8% | 7.8 | $0.002$1.79estimate · DGX Spark | 30 sDGX Spark | 0.027 sp90 0.031 s |
| 11 | GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Logic 23.6% (95%: 21.2%–26.1%) · Math 22.7% Non-hallucination 12.4% · “I don't know” on 9.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.527 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.31estimate · DGX Spark per 1,000 questionsper 1M questions · 0.015 s per question (median latency) | 23.6%21.2%–26.1% | 22.7% | 12.4% | 9.7% | 8.7 | $0.001$1.31estimate · DGX Spark | 22 sDGX Spark | 0.015 sp90 0.028 s |
| 11 | GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Logic 23.2% (95%: 20.8%–25.8%) · Math 20.6% Non-hallucination 76.1% · “I don't know” on 69.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.556 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.95estimate · DGX Spark per 1,000 questionsper 1M questions · 0.013 s per question (median latency) | 23.2%20.8%–25.8% | 20.6% | 76.1% | 69.8% | 8.8 | $0.001$0.95estimate · DGX Spark | 16 sDGX Spark | 0.013 sp90 0.016 s |
| 11 | Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Logic 22.4% (95%: 20.1%–24.9%) · Math 28.7% Non-hallucination 8.3% · “I don't know” on 6.5% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.568 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.028$27.67estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 22.4%20.1%–24.9% | 28.7% | 8.3% | 6.5% | 12.0 | $0.028$27.67estimate · RTX PRO 6000 Blackwell | 40 sRTX PRO 6000 | 0.036 sp90 0.043 s |
No finished run of this kind on this part yet.
updated 16 Nov 2026 22:07 UTC
Not ranked (only complete runs are): Spark-X2.6-4B (high · chat) (not running now, 20 of 1,125 stored, last answer 13 Nov 02:18 UTC); Grok 4.7 (chat) (paused for now at 576 of 1,125 (Ümit #8603, zeki #8637, 2026-11-15T11:37Z)); Claude Fable 5.2 (chat) (not running now, 147 of 1,125 stored, last answer 12 Nov 11:32 UTC); Granite-4.2-3B (off · chat) (still running, 259 of 1,125); GPT-6 Luna (max · chat) (paused for now at 55 of 1,125 (zeki #4129, 2026-11-10T14:13Z)); DeepSeek-V4-Flash (chat) (sample: a random subset by design, 317 answers stored, not the full 1,125); Claude Haiku 5.5 (max · chat) (paused for now at 47 of 1,125 (zeki #4129, 2026-11-10T14:13Z)).
Score: the share of the 1,125 Logic questions answered right, the whisker its 95% interval. # is the tier: rows in one tier are not told apart by the paired test (a line separates tiers); – : with web or sample, not tiered.
| Tier | Model | Math accuracy 2,163 questions, 95% interval; min–max for 3-seed rows | Logic of 1,125 questions | Non-hallucination of answers not right, % “I don't know” | “I don't know” share declined | Calibration error, points; decision models | $ per 1,000 q$ per 1M q list price or GPU time | GPU time s per 1,000 q, our hardware | Latency median, p90 under it |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $4.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Math 99.8% (95%: 99.5%–99.9%) · Logic 96.2% Non-hallucination 62.5% · “I don't know” on 0.2% of questions Runs on an API · cost $9.871$9,871 API list-price equivalent per 1,000 questionsper 1M questions · 3.20 s per question (median latency) | 99.8%99.5%–99.9% | 96.2% | 62.5% | 0.2% | – | $9.871$9,871 API list-price equivalent | – | 3.20 sp90 4.79 s |
| 1 | Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.8% (95%: 99.5%–99.9%) · Logic 96.3% Non-hallucination 54.5% · “I don't know” on 0.3% of questions Runs on an API · cost $1.478$1,478 per 1,000 questionsper 1M questions · 4.71 s per question (median latency) | 99.8%99.5%–99.9% | 96.3% | 54.5% | 0.3% | – | $1.478$1,478 | – | 4.71 sp90 9.13 s |
| 1 | Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $10.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Math 99.7% (95%: 99.4%–99.9%) · Logic 95.2% Non-hallucination 60.0% · “I don't know” on 0.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.03 s per question (median latency) | 99.7%99.4%–99.9% | 95.2% | 60.0% | 0.3% | – | $0.000$0.00 | – | 5.03 sp90 6.49 s |
| 1 | Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $2.00 per 1M input tokens: Workhorse (source: https://platform.claude.com/docs/en/about-claude/pricing) Math 99.7% (95%: 99.3%–99.8%) · Logic 95.5% Non-hallucination 62.5% · “I don't know” on 0.2% of questions Runs on an API · cost $4.734$4,734 API list-price equivalent per 1,000 questionsper 1M questions · 3.46 s per question (median latency) | 99.7%99.3%–99.8% | 95.5% | 62.5% | 0.2% | – | $4.734$4,734 API list-price equivalent | – | 3.46 sp90 5.97 s |
| 1 | DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active · Mixture of experts: 552B total, about 16B active per step. Size class uses 552B. Math 99.6% (95%: 99.3%–99.8%) · Logic 96.2% Non-hallucination 20.0% · “I don't know” on 0.1% of questions Runs on an API · cost $0.227$227 per 1,000 questionsper 1M questions · 1.91 s per question (median latency) | 99.6%99.3%–99.8% | 96.2% | 20.0% | 0.1% | – | $0.227$227 | – | 1.91 sp90 3.57 s |
| 1 | GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Math 99.6% (95%: 99.3%–99.8%) · Logic 96.7% Non-hallucination 78.6% · “I don't know” on 0.5% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.28 s per question (median latency) | 99.6%99.3%–99.8% | 96.7% | 78.6% | 0.5% | – | $0.000$0.00 | – | 5.28 sp90 7.26 s |
| 1 | GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Math 99.6% (95%: 99.3%–99.8%) · Logic 96.5% Non-hallucination 78.6% · “I don't know” on 0.5% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.81 s per question (median latency) | 99.6%99.3%–99.8% | 96.5% | 78.6% | 0.5% | – | $0.000$0.00 | – | 4.81 sp90 6.56 s |
| 1 | Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active · Mixture of experts: 276B total, about 12B active per step. Size class uses 276B. Math 99.6% (95%: 99.3%–99.8%) · Logic 95.4% Non-hallucination 54.5% · “I don't know” on 0.3% of questions Runs on an API · cost $0.954$954 per 1,000 questionsper 1M questions · 2.03 s per question (median latency) | 99.6%99.3%–99.8% | 95.4% | 54.5% | 0.3% | – | $0.954$954 | – | 2.03 sp90 6.33 s |
| 1 | Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.6% (95%: 99.2%–99.8%) · Logic 96.7% Non-hallucination 75.0% · “I don't know” on 0.6% of questions Runs on an API · 13.4 s per question (median latency) | 99.6%99.2%–99.8% | 96.7% | 75.0% | 0.6% | – | – | – | 13.4 sp90 27.8 s |
| 1 | Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Math 99.6% (95%: 99.2%–99.8%) · Logic 96.1% Non-hallucination 85.7% · “I don't know” on 0.8% of questions Runs on an API · 4.40 s per question (median latency) | 99.6%99.2%–99.8% | 96.1% | 85.7% | 0.8% | – | – | – | 4.40 sp90 6.68 s |
| 1 | Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/meta/muse-ember) Math 99.6% (95%: 99.2%–99.8%) · Logic 95.5% Non-hallucination 64.3% · “I don't know” on 0.4% of questions Runs on an API · cost $0.365$365 per 1,000 questionsper 1M questions · 1.26 s per question (median latency) | 99.6%99.2%–99.8% | 95.5% | 64.3% | 0.4% | – | $0.365$365 | – | 1.26 sp90 2.27 s |
| 1 | Grok 4.7 chat Closed · Undisclosed Grok 4.7 chat · xAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.5% (95%: 99.2%–99.7%) · Logic running Runs on an API · cost $12.061$12,061 API list-price equivalent per 1,000 questionsper 1M questions · 11.1 s per question (median latency) | 99.5%99.2%–99.7% | running | – | – | – | $12.061$12,061 API list-price equivalent | – | 11.1 sp90 66.5 s |
| 1 | Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active · Mixture of experts: 594B total, about 17B active per step. Size class uses 594B. Math 99.5% (95%: 99.2%–99.7%) · Logic 95.6% Non-hallucination 61.5% · “I don't know” on 0.4% of questions Runs on an API · cost $0.638$638 per 1,000 questionsper 1M questions · 2.11 s per question (median latency) | 99.5%99.2%–99.7% | 95.6% | 61.5% | 0.4% | – | $0.638$638 | – | 2.11 sp90 6.00 s |
| 1 | Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.5% (95%: 99.1%–99.7%) · Logic 94.0% Non-hallucination 16.7% · “I don't know” on 0.1% of questions Runs on an API · cost $0.397$397 per 1,000 questionsper 1M questions · 5.74 s per question (median latency) | 99.5%99.1%–99.7% | 94.0% | 16.7% | 0.1% | – | $0.397$397 | – | 5.74 sp90 11.8 s |
| 1 | MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active · Mixture of experts: 309B total, about 15B active per step. Size class uses 309B. Math 99.4% (95%: 99.0%–99.7%) · Logic 95.1% Non-hallucination 50.0% · “I don't know” on 0.3% of questions Runs on an API · cost $0.212$212 per 1,000 questionsper 1M questions · 8.20 s per question (median latency) | 99.4%99.0%–99.7% | 95.1% | 50.0% | 0.3% | – | $0.212$212 | – | 8.20 sp90 18.1 s |
| 1 | Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Math 99.4% (95%: 99.0%–99.7%) · Logic 92.4% Non-hallucination 21.4% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.271$1,271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 46.6 s per question (median latency) Other settings: Qwen3.5-4B (off · chat) | 99.4%99.0%–99.7% | 92.4% | 21.4% | 0.1% | – | $1.271$1,271estimate · RTX PRO 6000 Blackwell | 1,838 sRTX PRO 6000 | 46.6 sp90 96.5 s |
| Qwen3.5-4B off · chat Open · 4.7Bother setting of Qwen3.5-4B (on · chat) Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Math 98.5% (95%: 97.9%–98.9%) · Logic 90.1% Non-hallucination 36.0% · “I don't know” on 0.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.0 s per question (median latency) | 98.5%97.9%–98.9% | 90.1% | 36.0% | 0.4% | – | $0.300$300estimate · RTX PRO 6000 Blackwell | 434 sRTX PRO 6000 | 11.0 sp90 20.0 s | |
| 1 | Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Math 99.4% (95%: 99.0%–99.6%) · Logic 93.3% Non-hallucination 21.4% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.531$531estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.1 s per question (median latency) | 99.4%99.0%–99.6% | 93.3% | 21.4% | 0.1% | – | $0.531$531estimate · RTX PRO 6000 Blackwell | 767 sRTX PRO 6000 | 11.1 sp90 16.7 s |
| 1 | DeepSeek-V4-Flash chat Open · 285B MoE · 13B active DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active · Mixture of experts: 285B total, about 13B active per step. Size class uses 285B. Math 99.3% (95%: 98.9%–99.6%) · Logic running Runs on NVIDIA DGX Spark (GB10) · cost $0.295$295estimate · DGX Spark per 1,000 questionsper 1M questions · 8.13 s per question (median latency) | 99.3%98.9%–99.6% | running | – | – | – | $0.295$295estimate · DGX Spark | 4,987 sDGX Spark | 8.13 sp90 16.9 s |
| 1 | MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Math 99.2% (mean of 3 seeds; min–max 99.2%–99.3%) · Logic 88.9% Seed runs: seed 0: 99.3% (2,147 of 2,163, run minicpm5-2b_en) · seed 1: 99.2% (2,145 of 2,163, run minicpm5-2b-seed1_en) · seed 2: 99.3% (2,147 of 2,163, run minicpm5-2b-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b_en, not the mean Non-hallucination 53.6% · “I don't know” on 0.7% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.409$409estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 12.5 s per question (median latency) Other settings: MiniCPM5-2B (off · chat) | 99.2%99.2%–99.3% min–max of 3 seeds | 88.9% | 53.6%seed 0 only | 0.7%seed 0 only | – | $0.409$409estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 591 sRTX PRO 6000mean of 3 seeds | 12.5 sp90 32.7 smean of 3 seeds |
| MiniCPM5-2B off · chat Open · 2.5Bother setting of MiniCPM5-2B (on · chat) MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Math 98.2% (mean of 3 seeds; min–max 98.1%–98.3%) · Logic 83.6% Seed runs: seed 0: 98.3% (2,126 of 2,163, run minicpm5-2b-nothink_en) · seed 1: 98.1% (2,122 of 2,163, run minicpm5-2b-nothink-seed1_en) · seed 2: 98.1% (2,121 of 2,163, run minicpm5-2b-nothink-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b-nothink_en, not the mean Non-hallucination 31.0% · “I don't know” on 1.2% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.219$219estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 6.35 s per question (median latency) | 98.2%98.1%–98.3% min–max of 3 seeds | 83.6% | 31.0%seed 0 only | 1.2%seed 0 only | – | $0.219$219estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 316 sRTX PRO 6000mean of 3 seeds | 6.35 sp90 15.7 smean of 3 seeds | |
| 1 | GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Math 99.2% (95%: 98.7%–99.5%) · Logic 94.0% Non-hallucination 77.8% · “I don't know” on 1.0% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.08 s per question (median latency) | 99.2%98.7%–99.5% | 94.0% | 77.8% | 1.0% | – | $0.000$0.00 | – | 4.08 sp90 5.87 s |
| 1 | Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Math 99.2% (95%: 98.7%–99.5%) · Logic 91.6% Non-hallucination 61.5% · “I don't know” on 0.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.781$781estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 15.3 s per question (median latency) | 99.2%98.7%–99.5% | 91.6% | 61.5% | 0.7% | – | $0.781$781estimate · RTX PRO 6000 Blackwell | 1,129 sRTX PRO 6000 | 15.3 sp90 53.3 s |
| 1 | Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Math 99.2% (95%: 98.7%–99.5%) · Logic 94.0% Non-hallucination 13.0% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.530$530estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 10.0 s per question (median latency) | 99.2%98.7%–99.5% | 94.0% | 13.0% | 0.1% | – | $0.530$530estimate · RTX PRO 6000 Blackwell | 766 sRTX PRO 6000 | 10.0 sp90 20.6 s |
| 2 | LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active · Mixture of experts: 24.0B total, about 2.45B active per step. Size class uses 24.0B. Math 99.0% (95%: 98.5%–99.4%) · Logic 86.7% Non-hallucination 71.4% · “I don't know” on 1.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.939$939estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 27.9 s per question (median latency) | 99.0%98.5%–99.4% | 86.7% | 71.4% | 1.2% | – | $0.939$939estimate · RTX PRO 6000 Blackwell | 1,358 sRTX PRO 6000 | 27.9 sp90 60.2 s |
| 2 | LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Math 98.9% (mean of 3 seeds; min–max 98.8%–99.1%) · Logic 84.2% Seed runs: seed 0: 98.9% (2,139 of 2,163, run lfm3-1.7b-vllm_en) · seed 1: 99.1% (2,143 of 2,163, run lfm3-1.7b-vllm-seed1_en) · seed 2: 98.8% (2,138 of 2,163, run lfm3-1.7b-vllm-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm_en, not the mean Non-hallucination 67.6% · “I don't know” on 1.1% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.132$132estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 3.89 s per question (median latency) Other settings: LFM3-1.7B (off · chat), LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell) | 98.9%98.8%–99.1% min–max of 3 seeds | 84.2% | 67.6%seed 0 only | 1.1%seed 0 only | – | $0.132$132estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 191 sRTX PRO 6000mean of 3 seeds | 3.89 sp90 10.4 smean of 3 seeds |
| LFM3-1.7B off · chat Open · 1.7Bother setting of LFM3-1.7B (on · chat) LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Math 98.7% (95%: 98.1%–99.1%) · Logic 85.5% Non-hallucination 67.7% · “I don't know” on 1.0% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.060$59.94estimate · DGX Spark per 1,000 questionsper 1M questions · 19.6 s per question (median latency) | 98.7%98.1%–99.1% | 85.5% | 67.7% | 1.0% | – | $0.060$59.94estimate · DGX Spark | 1,014 sDGX Spark | 19.6 sp90 53.4 s | |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7Bother setting of LFM3-1.7B (on · chat) LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Math 98.9% (mean of 3 seeds; min–max 98.8%–99.0%) · Logic 84.6% Seed runs: seed 0: 99.0% (2,142 of 2,163, run lfm3-1.7b-vllm-nothink-rtx-seed0_en) · seed 1: 98.8% (2,136 of 2,163, run lfm3-1.7b-vllm-nothink-rtx-seed1_en) · seed 2: 99.0% (2,142 of 2,163, run lfm3-1.7b-vllm-nothink-rtx-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm-nothink-rtx-seed0_en, not the mean Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 3.99 s per question (median latency) | 98.9%98.8%–99.0% min–max of 3 seeds | 84.6% | – | – | – | $0.142$142estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 205 sRTX PRO 6000mean of 3 seeds | 3.99 sp90 10.6 smean of 3 seeds | |
| 2 | Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Math 98.8% (95%: 98.3%–99.2%) · Logic 87.7% Non-hallucination 14.3% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 19.1 s per question (median latency) | 98.8%98.3%–99.2% | 87.7% | 14.3% | 0.1% | – | $0.467$467estimate · RTX PRO 6000 Blackwell | 675 sRTX PRO 6000 | 19.1 sp90 28.1 s |
| 2 | GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active · Mixture of experts: 320B total, about 18B active per step. Size class uses 320B. Math 98.8% (95%: 98.2%–99.2%) · Logic 93.2% Non-hallucination 17.4% · “I don't know” on 0.2% of questions Runs on an API · cost $0.166$166 per 1,000 questionsper 1M questions · 3.80 s per question (median latency) | 98.8%98.2%–99.2% | 93.2% | 17.4% | 0.2% | – | $0.166$166 | – | 3.80 sp90 7.97 s |
| 2 | MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active · Mixture of experts: 428B total, about 23B active per step. Size class uses 428B. Math 98.2% (95%: 97.6%–98.7%) · Logic 94.0% Non-hallucination 40.6% · “I don't know” on 0.6% of questions Runs on an API · cost $0.103$103 per 1,000 questionsper 1M questions · 1.67 s per question (median latency) | 98.2%97.6%–98.7% | 94.0% | 40.6% | 0.6% | – | $0.103$103 | – | 1.67 sp90 7.47 s |
| 2 | Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.1% (95%: 97.4%–98.6%) · Logic 83.2% Non-hallucination 44.0% · “I don't know” on 1.0% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.271$271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 10.8 s per question (median latency) Other settings: Gemma 4 E4B (on · chat), Gemma 4 E4B (on · chat · 32k cap), Gemma 4 E4B (on · chat · 32k cap · sampled) | 98.1%97.4%–98.6% | 83.2% | 44.0% | 1.0% | – | $0.271$271estimate · RTX PRO 6000 Blackwell | 392 sRTX PRO 6000 | 10.8 sp90 17.7 s |
| Gemma 4 E4B on · chat Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.2% (95%: 97.6%–98.7%) · Logic 82.6% Non-hallucination 37.0% · “I don't know” on 0.8% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.233$233estimate · DGX Spark per 1,000 questionsper 1M questions · 108.7 s per question (median latency) | 98.2%97.6%–98.7% | 82.6% | 37.0% | 0.8% | – | $0.233$233estimate · DGX Spark | 3,939 sDGX Spark | 108.7 sp90 175.5 s | |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.5% (95%: 97.9%–99.0%) · Logic 85.0% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.518$518estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 20.6 s per question (median latency) | 98.5%97.9%–99.0% | 85.0% | – | – | – | $0.518$518estimate · RTX PRO 6000 Blackwell | 749 sRTX PRO 6000 | 20.6 sp90 34.3 s | |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.5% (mean of 3 seeds; min–max 98.4%–98.8%) · Logic 85.9% Seed runs: seed 0: 98.8% (2,136 of 2,163, run gemma-4-e4b-it-think-32k-sampled-seed0_en) · seed 1: 98.5% (2,130 of 2,163, run gemma-4-e4b-it-think-32k-sampled-seed1_en) · seed 2: 98.4% (2,128 of 2,163, run gemma-4-e4b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e4b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 45.5% · “I don't know” on 0.9% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.318$318estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.2 s per question (median latency) | 98.5%98.4%–98.8% min–max of 3 seeds | 85.9% | 45.5%seed 0 only | 0.9%seed 0 only | – | $0.318$318estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 459 sRTX PRO 6000mean of 3 seeds | 11.2 sp90 22.4 smean of 3 seeds | |
| 3 | Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 96.8% (95%: 95.9%–97.4%) · Logic 72.9% Non-hallucination 25.3% · “I don't know” on 0.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.208$208estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.12 s per question (median latency) Other settings: Gemma 4 E2B (on · chat), Gemma 4 E2B (on · chat · 32k cap), Gemma 4 E2B (on · chat · 32k cap · sampled) | 96.8%95.9%–97.4% | 72.9% | 25.3% | 0.9% | – | $0.208$208estimate · RTX PRO 6000 Blackwell | 301 sRTX PRO 6000 | 8.12 sp90 13.7 s |
| Gemma 4 E2B on · chat Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 96.9% (95%: 96.1%–97.6%) · Logic 74.6% Non-hallucination 16.5% · “I don't know” on 0.6% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.138$138estimate · DGX Spark per 1,000 questionsper 1M questions · 64.4 s per question (median latency) | 96.9%96.1%–97.6% | 74.6% | 16.5% | 0.6% | – | $0.138$138estimate · DGX Spark | 2,335 sDGX Spark | 64.4 sp90 106.3 s | |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 97.8% (95%: 97.1%–98.4%) · Logic 75.9% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.410$410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 14.9 s per question (median latency) | 97.8%97.1%–98.4% | 75.9% | – | – | – | $0.410$410estimate · RTX PRO 6000 Blackwell | 592 sRTX PRO 6000 | 14.9 sp90 26.0 s | |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 97.7% (mean of 3 seeds; min–max 97.6%–97.8%) · Logic 76.8% Seed runs: seed 0: 97.6% (2,111 of 2,163, run gemma-4-e2b-it-think-32k-sampled-seed0_en) · seed 1: 97.8% (2,116 of 2,163, run gemma-4-e2b-it-think-32k-sampled-seed1_en) · seed 2: 97.6% (2,111 of 2,163, run gemma-4-e2b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e2b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 23.0% · “I don't know” on 0.6% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.196$196estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 7.14 s per question (median latency) | 97.7%97.6%–97.8% min–max of 3 seeds | 76.8% | 23.0%seed 0 only | 0.6%seed 0 only | – | $0.196$196estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 283 sRTX PRO 6000mean of 3 seeds | 7.14 sp90 14.8 smean of 3 seeds | |
| 3 | LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active · Mixture of experts: 8.3B total, about 1.5B active per step. Size class uses 8.3B. Math 96.3% (95%: 95.4%–97.0%) · Logic 60.2% Non-hallucination 45.2% · “I don't know” on 2.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.482$482estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 14.5 s† per question (median latency) | 96.3%95.4%–97.0% | 60.2% | 45.2% | 2.6% | – | $0.482$482estimate · RTX PRO 6000 Blackwell† | 697 sRTX PRO 6000 | 14.5 s†p90 33.2 s |
| 3 | Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Math 95.8% (95%: 94.9%–96.6%) · Logic 65.3% Non-hallucination 39.3% · “I don't know” on 2.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.355$355estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.04 s per question (median latency) | 95.8%94.9%–96.6% | 65.3% | 39.3% | 2.1% | – | $0.355$355estimate · RTX PRO 6000 Blackwell | 513 sRTX PRO 6000 | 8.04 sp90 19.0 s |
| 3 | Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Math 94.6% (95%: 93.6%–95.5%) · Logic 64.8% Non-hallucination 31.3% · “I don't know” on 1.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.363$363estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.2 s per question (median latency) Other settings: Qwen3.5-2B (on · chat) | 94.6%93.6%–95.5% | 64.8% | 31.3% | 1.9% | – | $0.363$363estimate · RTX PRO 6000 Blackwell | 525 sRTX PRO 6000 | 11.2 sp90 32.1 s |
| Qwen3.5-2B on · chat Open · 2.0Bother setting of Qwen3.5-2B (off · chat) Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Math 94.7% (95%: 93.7%–95.6%) · Logic 70.3% Non-hallucination 20.3% · “I don't know” on 0.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.470$2,470estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 98.4 s per question (median latency) | 94.7%93.7%–95.6% | 70.3% | 20.3% | 0.6% | – | $2.470$2,470estimate · RTX PRO 6000 Blackwell | 3,571 sRTX PRO 6000 | 98.4 sp90 195.0 s | |
| 4 | LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Math 94.0% (95%: 92.9%–94.9%) · Logic 75.4% Non-hallucination 26.4% · “I don't know” on 1.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.86 s per question (median latency) | 94.0%92.9%–94.9% | 75.4% | 26.4% | 1.1% | – | $0.300$300estimate · RTX PRO 6000 Blackwell | 434 sRTX PRO 6000 | 9.86 sp90 21.2 s |
| 4 | GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/openai/gpt-6-luna) Math 93.7% (95%: 92.6%–94.6%) · Logic 58.0% Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.83 s per question (median latency) | 93.7%92.6%–94.6% | 58.0% | – | – | – | $0.000$0.00 | – | 3.83 sp90 5.00 s |
| 5 | Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Math 90.8% (95%: 89.5%–91.9%) · Logic 54.2% Non-hallucination 35.9% · “I don't know” on 4.3% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.266$266estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 7.89 s per question (median latency) | 90.8%89.5%–91.9% | 54.2% | 35.9% | 4.3% | – | $0.266$266estimate · RTX PRO 6000 Blackwell | 385 sRTX PRO 6000 | 7.89 sp90 14.9 s |
| 5 | Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Math 89.6% (95%: 88.2%–90.8%) · Logic 65.1% Non-hallucination 22.1% · “I don't know” on 2.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.901$1,901estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 25.6 s† per question (median latency) Other settings: Raon-1 nano (on · chat) | 89.6%88.2%–90.8% | 65.1% | 22.1% | 2.1% | – | $1.901$1,901estimate · RTX PRO 6000 Blackwell† | 2,748 sRTX PRO 6000 | 25.6 s†p90 134.8 s |
| Raon-1 nano on · chat Open · 3.1Bother setting of Raon-1 nano (off · chat) Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Math 97.0% (95%: 96.2%–97.6%) · Logic 70.9% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.670$1,670estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 34.3 s† per question (median latency) | 97.0%96.2%–97.6% | 70.9% | – | – | – | $1.670$1,670estimate · RTX PRO 6000 Blackwell† | 2,415 sRTX PRO 6000 | 34.3 s†p90 108.0 s | |
| 6 | Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Math 81.6% (95%: 79.9%–83.1%) · Logic 60.7% Non-hallucination 4.8% · “I don't know” on 0.9% of questions Confidence: 837 answers kept at ≥99%, 837 right · 0 wrong · AUROC 0.903 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.113$113estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.155 s per question (median latency) | 81.6%79.9%–83.1% | 60.7% | 4.8% | 0.9% | 1.2 | $0.113$113estimate · RTX PRO 6000 Blackwell | 163 sRTX PRO 6000 | 0.155 sp90 0.186 s |
| 6 | KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Math 80.3% (95%: 78.5%–81.9%) · Logic 58.9% Non-hallucination 0.2% · “I don't know” on 0.0% of questions Confidence: 470 answers kept at ≥99%, 469 right · 1 wrong · AUROC 0.898 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.114 s per question (median latency) | 80.3%78.5%–81.9% | 58.9% | 0.2% | 0.0% | 3.6 | $0.083$83.14estimate · RTX PRO 6000 Blackwell | 120 sRTX PRO 6000 | 0.114 sp90 0.120 s |
| 7 | REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense · Two models run together, option probabilities averaged 50/50: 57.36B in all, all used on every question. Math 78.0% (95%: 76.2%–79.7%) · Logic 59.3% Non-hallucination 2.3% · “I don't know” on 0.5% of questions Confidence: 178 answers kept at ≥99%, 178 right · 0 wrong · AUROC 0.903 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.189 s per question (median latency) | 78.0%76.2%–79.7% | 59.3% | 2.3% | 0.5% | 7.1 | $0.142$142estimate · RTX PRO 6000 Blackwell | 205 sRTX PRO 6000 | 0.189 sp90 0.232 s |
| 7 | KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Math 77.4% (95%: 75.6%–79.1%) · Logic 59.0% Non-hallucination 1.1% · “I don't know” on 0.2% of questions Confidence: 465 answers kept at ≥99%, 464 right · 1 wrong · AUROC 0.895 Runs on NVIDIA DGX Spark (GB10) · cost $0.047$46.87estimate · DGX Spark per 1,000 questionsper 1M questions · 0.764 s per question (median latency) | 77.4%75.6%–79.1% | 59.0% | 1.1% | 0.2% | 3.6 | $0.047$46.87estimate · DGX Spark | 793 sDGX Spark | 0.764 sp90 0.867 s |
| 7 | Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Math 77.3% (95%: 75.5%–79.1%) · Logic 58.7% Non-hallucination 1.7% · “I don't know” on 0.4% of questions Confidence: 318 answers kept at ≥99%, 318 right · 0 wrong · AUROC 0.899 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.084$84.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.118 s per question (median latency) | 77.3%75.5%–79.1% | 58.7% | 1.7% | 0.4% | 6.5 | $0.084$84.42estimate · RTX PRO 6000 Blackwell | 122 sRTX PRO 6000 | 0.118 sp90 0.142 s |
| 7 | Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Math 77.1% (95%: 75.3%–78.8%) · Logic 57.9% Non-hallucination 2.1% · “I don't know” on 0.5% of questions Confidence: 215 answers kept at ≥99%, 215 right · 0 wrong · AUROC 0.891 Runs on NVIDIA DGX Spark (GB10) · cost $0.039$39.14estimate · DGX Spark per 1,000 questionsper 1M questions · 0.637 s per question (median latency) | 77.1%75.3%–78.8% | 57.9% | 2.1% | 0.5% | 4.3 | $0.039$39.14estimate · DGX Spark | 662 sDGX Spark | 0.637 sp90 0.735 s |
| 8 | Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Math 75.2% (95%: 73.4%–77.0%) · Logic 57.8% Non-hallucination 2.1% · “I don't know” on 0.5% of questions Confidence: 15 answers kept at ≥99%, 15 right · 0 wrong · AUROC 0.899 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.196 s per question (median latency) | 75.2%73.4%–77.0% | 57.8% | 2.1% | 0.5% | 7.5 | $0.141$141estimate · RTX PRO 6000 Blackwell | 203 sRTX PRO 6000 | 0.196 sp90 0.221 s |
| 8 | d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://www.liquid.ai/blog/d1-ultra) Math 74.1% (95%: 72.2%–75.9%) · Logic 54.9% Non-hallucination 87.4% · “I don't know” on 38.8% of questions Confidence: 213 answers kept at ≥99%, 213 right · 0 wrong · AUROC 0.876 Runs on an API · cost $0.006$6.14 per 1,000 questionsper 1M questions · 0.179 s per question (median latency) | 74.1%72.2%–75.9% | 54.9% | 87.4% | 38.8% | 2.3 | $0.006$6.14 | – | 0.179 sp90 0.426 s |
| 8 | Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Math 74.0% (95%: 72.1%–75.8%) · Logic 54.5% Non-hallucination 70.6% · “I don't know” on 24.5% of questions Confidence: 412 answers kept at ≥99%, 412 right · 0 wrong · AUROC 0.895 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.140 s per question (median latency) | 74.0%72.1%–75.8% | 54.5% | 70.6% | 24.5% | 3.0 | $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell | 150 sRTX PRO 6000 | 0.140 sp90 0.173 s |
| 8 | Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Math 72.5% (95%: 70.6%–74.3%) · Logic 54.8% Non-hallucination 9.4% · “I don't know” on 2.8% of questions Confidence: 323 answers kept at ≥99%, 323 right · 0 wrong · AUROC 0.856 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.066$65.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.091 s per question (median latency) | 72.5%70.6%–74.3% | 54.8% | 9.4% | 2.8% | 4.5 | $0.066$65.87estimate · RTX PRO 6000 Blackwell | 95 sRTX PRO 6000 | 0.091 sp90 0.103 s |
| 8 | GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Math 72.0% (95%: 70.1%–73.8%) · Logic 58.4% Non-hallucination 21.6% · “I don't know” on 6.5% of questions Confidence: 199 answers kept at ≥99%, 198 right · 1 wrong · AUROC 0.864 Runs on NVIDIA DGX Spark (GB10) · cost $0.026$26.16estimate · DGX Spark per 1,000 questionsper 1M questions · 0.425 s per question (median latency) | 72.0%70.1%–73.8% | 58.4% | 21.6% | 6.5% | 4.1 | $0.026$26.16estimate · DGX Spark | 442 sDGX Spark | 0.425 sp90 0.461 s |
| 9 | d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Math 71.5% (95%: 69.6%–73.4%) · Logic 54.6% Non-hallucination 5.3% · “I don't know” on 1.6% of questions Confidence: 11 answers kept at ≥99%, 11 right · 0 wrong · AUROC 0.857 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.089$88.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.118 s per question (median latency) | 71.5%69.6%–73.4% | 54.6% | 5.3% | 1.6% | 8.6 | $0.089$88.58estimate · RTX PRO 6000 Blackwell | 128 sRTX PRO 6000 | 0.118 sp90 0.148 s |
| 9 | Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/google/gemini-decision-1) Math 71.3% (95%: 69.4%–73.2%) · Logic 50.0% Non-hallucination 3.9% · “I don't know” on 1.2% of questions Confidence: 214 answers kept at ≥99%, 214 right · 0 wrong · AUROC 0.859 Runs on an API · cost $0.022$22.02 per 1,000 questionsper 1M questions · 0.226 s per question (median latency) | 71.3%69.4%–73.2% | 50.0% | 3.9% | 1.2% | 2.5 | $0.022$22.02 | – | 0.226 sp90 0.351 s |
| 9 | GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker · Closed; list price $0.15 per 1M input tokens: Efficient (source: https://docs.fastino.ai/pricing) Math 70.8% (95%: 68.8%–72.7%) · Logic 54.5% Non-hallucination 40.6% · “I don't know” on 14.2% of questions Confidence: 161 answers kept at ≥99%, 161 right · 0 wrong · AUROC 0.865 Runs on an API · cost $0.028$28.30 per 1,000 questionsper 1M questions · 0.264 s per question (median latency) | 70.8%68.8%–72.7% | 54.5% | 40.6% | 14.2% | 5.0 | $0.028$28.30 | – | 0.264 sp90 0.279 s |
| 9 | d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Math 70.4% (95%: 68.5%–72.3%) · Logic 54.5% Non-hallucination 4.7% · “I don't know” on 1.4% of questions Confidence: 163 answers kept at ≥99%, 162 right · 1 wrong · AUROC 0.878 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$40.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.055 s per question (median latency) | 70.4%68.5%–72.3% | 54.5% | 4.7% | 1.4% | 4.5 | $0.041$40.61estimate · RTX PRO 6000 Blackwell | 59 sRTX PRO 6000 | 0.055 sp90 0.062 s |
| 9 | Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Math 70.3% (95%: 68.4%–72.2%) · Logic 53.1% Non-hallucination 4.5% · “I don't know” on 1.3% of questions Confidence: 729 answers kept at ≥99%, 719 right · 10 wrong · AUROC 0.861 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.076 s per question (median latency) | 70.3%68.4%–72.2% | 53.1% | 4.5% | 1.3% | 11.8 | $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell | 79 sRTX PRO 6000 | 0.076 sp90 0.089 s |
| 9 | Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker · Closed; introductory list price (the only one published) $0.03 per 1M input tokens: Efficient (provisional) (source: https://nace.ai/drex) Math 70.1% (95%: 68.1%–72.0%) · Logic 54.8% Non-hallucination 43.5% · “I don't know” on 15.6% of questions Confidence: 315 answers kept at ≥99%, 315 right · 0 wrong · AUROC 0.867 Runs on an API · cost $0.013$12.88 per 1,000 questionsper 1M questions · 0.326 s per question (median latency) | 70.1%68.1%–72.0% | 54.8% | 43.5% | 15.6% | 5.7 | $0.013$12.88 | – | 0.326 sp90 0.473 s |
| 9 | celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Math 70.0% (95%: 68.0%–71.9%) · Logic 47.3% Non-hallucination 82.7% · “I don't know” on 38.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.858 Runs on an API · cost $0.009$9.10 per 1,000 questionsper 1M questions · 0.387 s per question (median latency) | 70.0%68.0%–71.9% | 47.3% | 82.7% | 38.2% | 4.3 | $0.009$9.10 | – | 0.387 sp90 0.391 s |
| 9 | GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Math 69.9% (95%: 67.9%–71.8%) · Logic 55.2% Non-hallucination 35.8% · “I don't know” on 12.2% of questions Confidence: 131 answers kept at ≥99%, 131 right · 0 wrong · AUROC 0.862 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.079 s per question (median latency) | 69.9%67.9%–71.8% | 55.2% | 35.8% | 12.2% | 5.2 | $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.079 sp90 0.084 s |
| 9 | celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.004 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Math 69.9% (95%: 67.9%–71.8%) · Logic 47.3% Non-hallucination 82.9% · “I don't know” on 37.9% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.859 Runs on an API · cost $0.001$0.91 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) | 69.9%67.9%–71.8% | 47.3% | 82.9% | 37.9% | 4.5 | $0.001$0.91 | – | 0.381 sp90 0.386 s |
| 9 | d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Math 69.9% (95%: 67.9%–71.8%) · Logic 51.6% Non-hallucination 86.7% · “I don't know” on 41.6% of questions Confidence: 115 answers kept at ≥99%, 115 right · 0 wrong · AUROC 0.876 Runs on an API · cost $0.006$6.14 per 1,000 questionsper 1M questions · 0.215 s per question (median latency) | 69.9%67.9%–71.8% | 51.6% | 86.7% | 41.6% | 4.6 | $0.006$6.14 | – | 0.215 sp90 0.347 s |
| 9 | Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Math 69.8% (95%: 67.8%–71.7%) · Logic 53.3% Non-hallucination 7.0% · “I don't know” on 2.2% of questions Confidence: 331 answers kept at ≥99%, 331 right · 0 wrong · AUROC 0.869 Runs on an API · cost $0.004$4.29 per 1,000 questionsper 1M questions · 0.226 s per question (median latency) Other settings: Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) | 69.8%67.8%–71.7% | 53.3% | 7.0% | 2.2% | 3.2 | $0.004$4.29 | – | 0.226 sp90 0.368 s |
| Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell Open · 27Bother setting of Perplexity Decider 27B v1.2 (API) Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Math 70.3% (95%: 68.4%–72.2%) · Logic 53.2% Non-hallucination 6.2% · “I don't know” on 1.9% of questions Confidence: 377 answers kept at ≥99%, 377 right · 0 wrong · AUROC 0.866 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.004$4.35 list$0.070$70.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.095 s per question (median latency) Shown under Perplexity Decider 27B v1.2 (API): the same model name, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings | 70.3%68.4%–72.2% | 53.2% | 6.2% | 1.9% | 2.7 | $0.004$4.35 list$0.070$70.03estimate · RTX PRO 6000 Blackwell | 101 sRTX PRO 6000 | 0.095 sp90 0.125 s | |
| – | Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Not tiered: with web Math 69.3% (95%: 67.3%–71.2%) · Logic 52.5% Non-hallucination 15.4% · “I don't know” on 4.8% of questions Confidence: 326 answers kept at ≥99%, 326 right · 0 wrong · AUROC 0.873 Runs on an API · cost $1.023$1,023 per 1,000 questionsper 1M questions · 0.422 s per question (median latency) | 69.3%67.3%–71.2% | 52.5% | 15.4% | 4.8% | 3.7 | $1.023$1,023 | – | 0.422 sp90 0.641 s |
| 9 | Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Math 69.0% (95%: 67.0%–70.9%) · Logic 52.3% Non-hallucination 2.2% · “I don't know” on 0.6% of questions Confidence: 43 answers kept at ≥99%, 43 right · 0 wrong · AUROC 0.864 Runs on NVIDIA DGX Spark (GB10) · cost $0.017$17.46estimate · DGX Spark per 1,000 questionsper 1M questions · 0.278 s per question (median latency) | 69.0%67.0%–70.9% | 52.3% | 2.2% | 0.6% | 4.7 | $0.017$17.46estimate · DGX Spark | 295 sDGX Spark | 0.278 sp90 0.329 s |
| 9 | SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Math 68.6% (95%: 66.6%–70.5%) · Logic 52.4% Non-hallucination 52.5% · “I don't know” on 20.5% of questions Confidence: 361 answers kept at ≥99%, 360 right · 1 wrong · AUROC 0.839 Runs on an API · cost $0.009$8.59 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) Shown once with SubQ Snap Preview: the same answer to every question | 68.6%66.6%–70.5% | 52.4% | 52.5% | 20.5% | 9.5 | $0.009$8.59 | – | 0.381 sp90 0.390 s |
| 10 | KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Math 66.3% (95%: 64.2%–68.2%) · Logic 50.6% Non-hallucination 1.1% · “I don't know” on 0.4% of questions Confidence: 7 answers kept at ≥99%, 7 right · 0 wrong · AUROC 0.819 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.036$36.13estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.049 s per question (median latency) | 66.3%64.2%–68.2% | 50.6% | 1.1% | 0.4% | 4.0 | $0.036$36.13estimate · RTX PRO 6000 Blackwell | 52 sRTX PRO 6000 | 0.049 sp90 0.055 s |
| 10 | d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Math 65.5% (95%: 63.5%–67.5%) · Logic 49.2% Non-hallucination 56.3% · “I don't know” on 23.6% of questions Confidence: 141 answers kept at ≥99%, 141 right · 0 wrong · AUROC 0.838 Runs on an API · cost $0.003$2.92 per 1,000 questionsper 1M questions · 0.204 s per question (median latency) | 65.5%63.5%–67.5% | 49.2% | 56.3% | 23.6% | 3.2 | $0.003$2.92 | – | 0.204 sp90 0.386 s |
| 10 | d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Math 65.2% (95%: 63.2%–67.2%) · Logic 46.0% Non-hallucination 82.5% · “I don't know” on 44.1% of questions Confidence: 183 answers kept at ≥99%, 183 right · 0 wrong · AUROC 0.829 Runs on an API · cost $0.003$2.92 per 1,000 questionsper 1M questions · 0.172 s per question (median latency) | 65.2%63.2%–67.2% | 46.0% | 82.5% | 44.1% | 2.2 | $0.003$2.92 | – | 0.172 sp90 0.290 s |
| 10 | Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://docs.typesafe.ai/models) Math 65.1% (95%: 63.1%–67.1%) · Logic 48.2% Non-hallucination 7.7% · “I don't know” on 2.8% of questions Confidence: 202 answers kept at ≥99%, 202 right · 0 wrong · AUROC 0.829 Runs on an API · cost $0.019$18.99 per 1,000 questionsper 1M questions · 0.163 s per question (median latency) | 65.1%63.1%–67.1% | 48.2% | 7.7% | 2.8% | 2.8 | $0.019$18.99 | – | 0.163 sp90 0.185 s |
| 10 | Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://typesafe.ai/blog/introducing-system-one-models-and-jev) Math 64.1% (95%: 62.0%–66.1%) · Logic 46.3% Non-hallucination 5.7% · “I don't know” on 2.1% of questions Confidence: 248 answers kept at ≥99%, 248 right · 0 wrong · AUROC 0.834 Runs on an API · cost $0.019$18.99 per 1,000 questionsper 1M questions · 0.166 s per question (median latency) | 64.1%62.0%–66.1% | 46.3% | 5.7% | 2.1% | 2.6 | $0.019$18.99 | – | 0.166 sp90 0.206 s |
| 10 | Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 62.1% (95%: 60.0%–64.1%) · Logic 46.4% Non-hallucination 3.1% · “I don't know” on 1.2% of questions Confidence: 198 answers kept at ≥99%, 197 right · 1 wrong · AUROC 0.806 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 62.1%60.0%–64.1% | 46.4% | 3.1% | 1.2% | 4.8 | $0.027$26.54estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.036 sp90 0.044 s |
| 11 | Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Math 61.3% (95%: 59.3%–63.4%) · Logic 46.4% Non-hallucination 65.0% · “I don't know” on 32.9% of questions Confidence: 183 answers kept at ≥99%, 182 right · 1 wrong · AUROC 0.803 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.080 s per question (median latency) | 61.3%59.3%–63.4% | 46.4% | 65.0% | 32.9% | 4.3 | $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.080 sp90 0.092 s |
| 11 | KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 60.1% (95%: 58.0%–62.1%) · Logic 43.9% Non-hallucination 3.2% · “I don't know” on 1.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.803 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.031$31.02estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.041 s per question (median latency) | 60.1%58.0%–62.1% | 43.9% | 3.2% | 1.3% | 4.8 | $0.031$31.02estimate · RTX PRO 6000 Blackwell | 45 sRTX PRO 6000 | 0.041 sp90 0.046 s |
| 11 | Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 59.8% (95%: 57.7%–61.9%) · Logic 48.2% Non-hallucination 6.6% · “I don't know” on 2.7% of questions Confidence: 64 answers kept at ≥99%, 64 right · 0 wrong · AUROC 0.814 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.029$28.78estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.037 s per question (median latency) | 59.8%57.7%–61.9% | 48.2% | 6.6% | 2.7% | 3.8 | $0.029$28.78estimate · RTX PRO 6000 Blackwell | 42 sRTX PRO 6000 | 0.037 sp90 0.051 s |
| 11 | KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 59.8% (95%: 57.7%–61.8%) · Logic 44.3% Non-hallucination 0.9% · “I don't know” on 0.4% of questions Confidence: 123 answers kept at ≥99%, 123 right · 0 wrong · AUROC 0.799 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.037 s per question (median latency) | 59.8%57.7%–61.8% | 44.3% | 0.9% | 0.4% | 3.1 | $0.027$26.54estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.037 sp90 0.043 s |
| 11 | Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://openrouter.ai/microsoft/microsoft-decision-1) Base model: Qwen3.5-9B Math 59.4% (95%: 57.3%–61.4%) · Logic 44.4% Non-hallucination 5.1% · “I don't know” on 2.1% of questions Confidence: 27 answers kept at ≥99%, 27 right · 0 wrong · AUROC 0.787 Runs on an API · cost $0.007$6.81 per 1,000 questionsper 1M questions · 0.310 s per question (median latency) | 59.4%57.3%–61.4% | 44.4% | 5.1% | 2.1% | 4.1 | $0.007$6.81 | – | 0.310 sp90 0.359 s |
| 11 | Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active · Mixture of experts: 35B total, about 3B active per step. Size class uses 35B. Math 59.0% (95%: 56.9%–61.0%) · Logic 41.9% Non-hallucination 20.7% · “I don't know” on 9.2% of questions Confidence: 174 answers kept at ≥99%, 174 right · 0 wrong · AUROC 0.813 Runs on an API · cost $0.009$8.92 per 1,000 questionsper 1M questions · 0.334 s per question (median latency) | 59.0%56.9%–61.0% | 41.9% | 20.7% | 9.2% | 1.9 | $0.009$8.92 | – | 0.334 sp90 0.432 s |
| 11 | Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Math 58.8% (95%: 56.7%–60.9%) · Logic 45.3% Non-hallucination 1.7% · “I don't know” on 0.7% of questions Confidence: 210 answers kept at ≥99%, 207 right · 3 wrong · AUROC 0.804 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.032$31.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.045 s per question (median latency) | 58.8%56.7%–60.9% | 45.3% | 1.7% | 0.7% | 12.1 | $0.032$31.98estimate · RTX PRO 6000 Blackwell | 46 sRTX PRO 6000 | 0.045 sp90 0.050 s |
| 11 | StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Math 58.4% (95%: 56.3%–60.5%) · Logic 46.6% Non-hallucination 4.2% · “I don't know” on 1.8% of questions Confidence: 49 answers kept at ≥99%, 49 right · 0 wrong · AUROC 0.781 Runs on NVIDIA DGX Spark (GB10) · cost $0.008$8.34estimate · DGX Spark per 1,000 questionsper 1M questions · 0.140 s per question (median latency) | 58.4%56.3%–60.5% | 46.6% | 4.2% | 1.8% | 2.7 | $0.008$8.34estimate · DGX Spark | 141 sDGX Spark | 0.140 sp90 0.146 s |
| 11 | GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Math 57.9% (95%: 55.8%–60.0%) · Logic 47.8% Non-hallucination 24.6% · “I don't know” on 11.4% of questions Confidence: 73 answers kept at ≥99%, 73 right · 0 wrong · AUROC 0.798 Runs on NVIDIA DGX Spark (GB10) · cost $0.010$10.49estimate · DGX Spark per 1,000 questionsper 1M questions · 0.174 s per question (median latency) | 57.9%55.8%–60.0% | 47.8% | 24.6% | 11.4% | 7.1 | $0.010$10.49estimate · DGX Spark | 178 sDGX Spark | 0.174 sp90 0.182 s |
| 12 | vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Math 56.8% (95%: 54.7%–58.9%) · Logic 44.1% Non-hallucination 3.8% · “I don't know” on 1.7% of questions Confidence: 33 answers kept at ≥99%, 33 right · 0 wrong · AUROC 0.789 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$23.34estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.031 s per question (median latency) | 56.8%54.7%–58.9% | 44.1% | 3.8% | 1.7% | 3.0 | $0.023$23.34estimate · RTX PRO 6000 Blackwell | 34 sRTX PRO 6000 | 0.031 sp90 0.035 s |
| 12 | Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Math 56.0% (95%: 53.9%–58.1%) · Logic 43.3% Non-hallucination 1.2% · “I don't know” on 0.5% of questions Confidence: 254 answers kept at ≥99%, 244 right · 10 wrong · AUROC 0.780 Runs on NVIDIA DGX Spark (GB10) · cost $0.010$9.51estimate · DGX Spark per 1,000 questionsper 1M questions · 0.153 s per question (median latency) | 56.0%53.9%–58.1% | 43.3% | 1.2% | 0.5% | 15.7 | $0.010$9.51estimate · DGX Spark | 161 sDGX Spark | 0.153 sp90 0.175 s |
| 12 | KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Math 56.0% (95%: 53.9%–58.1%) · Logic 42.3% Non-hallucination 1.6% · “I don't know” on 0.7% of questions Confidence: 54 answers kept at ≥99%, 54 right · 0 wrong · AUROC 0.765 Runs on NVIDIA DGX Spark (GB10) · cost $0.005$4.56estimate · DGX Spark per 1,000 questionsper 1M questions · 0.074 s per question (median latency) | 56.0%53.9%–58.1% | 42.3% | 1.6% | 0.7% | 8.6 | $0.005$4.56estimate · DGX Spark | 77 sDGX Spark | 0.074 sp90 0.084 s |
| 12 | Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Math 55.9% (95%: 53.8%–58.0%) · Logic 44.7% Non-hallucination 8.4% · “I don't know” on 3.8% of questions Confidence: 114 answers kept at ≥99%, 113 right · 1 wrong · AUROC 0.787 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.022$22.38estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 55.9%53.8%–58.0% | 44.7% | 8.4% | 3.8% | 11.7 | $0.022$22.38estimate · RTX PRO 6000 Blackwell | 32 sRTX PRO 6000 | 0.028 sp90 0.031 s |
| 12 | StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Math 55.8% (95%: 53.7%–57.9%) · Logic 45.9% Non-hallucination 13.4% · “I don't know” on 6.4% of questions Confidence: 96 answers kept at ≥99%, 96 right · 0 wrong · AUROC 0.779 Runs on NVIDIA DGX Spark (GB10) · cost $0.004$4.10estimate · DGX Spark per 1,000 questionsper 1M questions · 0.067 s per question (median latency) | 55.8%53.7%–57.9% | 45.9% | 13.4% | 6.4% | 6.7 | $0.004$4.10estimate · DGX Spark | 69 sDGX Spark | 0.067 sp90 0.069 s |
| 12 | d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Math 55.6% (95%: 53.5%–57.7%) · Logic 40.8% Non-hallucination 2.9% · “I don't know” on 1.3% of questions Confidence: 152 answers kept at ≥99%, 152 right · 0 wrong · AUROC 0.767 Runs on an API · cost $0.003$2.92 per 1,000 questionsper 1M questions · 0.184 s per question (median latency) | 55.6%53.5%–57.7% | 40.8% | 2.9% | 1.3% | 3.3 | $0.003$2.92 | – | 0.184 sp90 0.297 s |
| 12 | Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 55.3% (95%: 53.2%–57.4%) · Logic 44.4% Non-hallucination 2.6% · “I don't know” on 1.2% of questions Confidence: 130 answers kept at ≥99%, 130 right · 0 wrong · AUROC 0.790 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$40.93estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.058 s per question (median latency) | 55.3%53.2%–57.4% | 44.4% | 2.6% | 1.2% | 4.1 | $0.041$40.93estimate · RTX PRO 6000 Blackwell | 59 sRTX PRO 6000 | 0.058 sp90 0.064 s |
| 12 | Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Math 55.2% (95%: 53.1%–57.3%) · Logic 43.1% Non-hallucination 12.0% · “I don't know” on 5.7% of questions Confidence: 155 answers kept at ≥99%, 150 right · 5 wrong · AUROC 0.781 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.040 s per question (median latency) | 55.2%53.1%–57.3% | 43.1% | 12.0% | 5.7% | 10.4 | $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell | 43 sRTX PRO 6000 | 0.040 sp90 0.047 s |
| 12 | Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Math 54.6% (95%: 52.4%–56.6%) · Logic 42.6% Non-hallucination 5.0% · “I don't know” on 2.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.764 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.169$169estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.233 s per question (median latency) | 54.6%52.4%–56.6% | 42.6% | 5.0% | 2.3% | 21.1 | $0.169$169estimate · RTX PRO 6000 Blackwell | 245 sRTX PRO 6000 | 0.233 sp90 0.272 s |
| 12 | OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Math 53.8% (95%: 51.7%–55.9%) · Logic 40.6% Non-hallucination 1.8% · “I don't know” on 0.8% of questions Confidence: 26 answers kept at ≥99%, 26 right · 0 wrong · AUROC 0.783 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.061$60.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.079 s per question (median latency) | 53.8%51.7%–55.9% | 40.6% | 1.8% | 0.8% | 3.6 | $0.061$60.76estimate · RTX PRO 6000 Blackwell | 88 sRTX PRO 6000 | 0.079 sp90 0.106 s |
| 12 | Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://openrouter.ai/inception/mercury-decide) Math 53.5% (95%: 51.4%–55.6%) · Logic 45.1% Non-hallucination 3.2% · “I don't know” on 1.5% of questions Confidence: 281 answers kept at ≥99%, 272 right · 9 wrong · AUROC 0.789 Runs on an API · cost $0.004$4.38 per 1,000 questionsper 1M questions · 0.288 s per question (median latency) | 53.5%51.4%–55.6% | 45.1% | 3.2% | 1.5% | 15.8 | $0.004$4.38 | – | 0.288 sp90 0.342 s |
| 13 | Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Math 51.5% (95%: 49.4%–53.6%) · Logic 39.7% Non-hallucination 6.4% · “I don't know” on 3.2% of questions Confidence: 130 answers kept at ≥99%, 126 right · 4 wrong · AUROC 0.752 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.034$33.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.047 s per question (median latency) | 51.5%49.4%–53.6% | 39.7% | 6.4% | 3.2% | 5.7 | $0.034$33.58estimate · RTX PRO 6000 Blackwell | 49 sRTX PRO 6000 | 0.047 sp90 0.052 s |
| 13 | Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Math 51.1% (95%: 49.0%–53.2%) · Logic 40.8% Non-hallucination 1.6% · “I don't know” on 0.8% of questions Confidence: 12 answers kept at ≥99%, 12 right · 0 wrong · AUROC 0.767 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.181 s per question (median latency) | 51.1%49.0%–53.2% | 40.8% | 1.6% | 0.8% | 11.0 | $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell | 190 sRTX PRO 6000 | 0.181 sp90 0.210 s |
| 13 | Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Math 50.4% (95%: 48.3%–52.5%) · Logic 40.4% Non-hallucination 3.4% · “I don't know” on 1.7% of questions Confidence: 155 answers kept at ≥99%, 149 right · 6 wrong · AUROC 0.746 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$22.70estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 50.4%48.3%–52.5% | 40.4% | 3.4% | 1.7% | 13.7 | $0.023$22.70estimate · RTX PRO 6000 Blackwell | 33 sRTX PRO 6000 | 0.028 sp90 0.032 s |
| 13 | Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Math 49.8% (95%: 47.7%–51.9%) · Logic 38.9% Non-hallucination 2.0% · “I don't know” on 1.0% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.730 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.099 s per question (median latency) | 49.8%47.7%–51.9% | 38.9% | 2.0% | 1.0% | 2.7 | $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell | 104 sRTX PRO 6000 | 0.099 sp90 0.127 s |
| 13 | Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Math 46.9% (95%: 44.8%–49.0%) · Logic 39.7% Non-hallucination 9.2% · “I don't know” on 5.1% of questions Confidence: 21 answers kept at ≥99%, 20 right · 1 wrong · AUROC 0.743 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.038$38.37estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.052 s per question (median latency) | 46.9%44.8%–49.0% | 39.7% | 9.2% | 5.1% | 8.4 | $0.038$38.37estimate · RTX PRO 6000 Blackwell | 55 sRTX PRO 6000 | 0.052 sp90 0.061 s |
| 14 | Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Math 44.3% (95%: 42.3%–46.4%) · Logic 39.3% Non-hallucination 4.9% · “I don't know” on 2.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.735 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.037 s per question (median latency) | 44.3%42.3%–46.4% | 39.3% | 4.9% | 2.8% | 5.9 | $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell | 40 sRTX PRO 6000 | 0.037 sp90 0.041 s |
| 15 | Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Math 39.2% (95%: 37.1%–41.2%) · Logic 32.1% Non-hallucination 2.2% · “I don't know” on 1.4% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.651 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.113 s per question (median latency) | 39.2%37.1%–41.2% | 32.1% | 2.2% | 1.4% | 3.3 | $0.083$83.14estimate · RTX PRO 6000 Blackwell | 120 sRTX PRO 6000 | 0.113 sp90 0.144 s |
| 15 | StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Math 38.7% (95%: 36.6%–40.7%) · Logic 30.8% Non-hallucination 7.2% · “I don't know” on 4.5% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.685 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.27estimate · DGX Spark per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 38.7%36.6%–40.7% | 30.8% | 7.2% | 4.5% | 2.6 | $0.002$2.27estimate · DGX Spark | 38 sDGX Spark | 0.036 sp90 0.037 s |
| 15 | d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Math 37.1% (95%: 35.1%–39.1%) · Logic 30.7% Non-hallucination 80.2% · “I don't know” on 66.3% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.656 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.014$14.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.016 s per question (median latency) | 37.1%35.1%–39.1% | 30.7% | 80.2% | 66.3% | 5.6 | $0.014$14.07estimate · RTX PRO 6000 Blackwell | 20 sRTX PRO 6000 | 0.016 sp90 0.021 s |
| 15 | GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Math 36.8% (95%: 34.7%–38.8%) · Logic 32.3% Non-hallucination 17.3% · “I don't know” on 11.4% of questions Confidence: 16 answers kept at ≥99%, 15 right · 1 wrong · AUROC 0.635 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.78estimate · DGX Spark per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 36.8%34.7%–38.8% | 32.3% | 17.3% | 11.4% | 17.2 | $0.002$1.78estimate · DGX Spark | 30 sDGX Spark | 0.028 sp90 0.030 s |
| 15 | KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Math 35.3% (95%: 33.3%–37.3%) · Logic 31.7% Non-hallucination 1.7% · “I don't know” on 1.1% of questions Confidence: 8 answers kept at ≥99%, 8 right · 0 wrong · AUROC 0.636 Runs on NVIDIA DGX Spark (GB10) · cost $0.003$3.20estimate · DGX Spark per 1,000 questionsper 1M questions · 0.050 s per question (median latency) | 35.3%33.3%–37.3% | 31.7% | 1.7% | 1.1% | 9.4 | $0.003$3.20estimate · DGX Spark | 54 sDGX Spark | 0.050 sp90 0.054 s |
| 16 | GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Math 32.1% (95%: 30.2%–34.1%) · Logic 29.2% Non-hallucination 20.8% · “I don't know” on 15.1% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.589 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.28estimate · DGX Spark per 1,000 questionsper 1M questions · 0.020 s per question (median latency) | 32.1%30.2%–34.1% | 29.2% | 20.8% | 15.1% | 18.8 | $0.001$1.28estimate · DGX Spark | 22 sDGX Spark | 0.020 sp90 0.021 s |
| 16 | StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Math 29.7% (95%: 27.8%–31.6%) · Logic 24.7% Non-hallucination 11.3% · “I don't know” on 8.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.583 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.53estimate · DGX Spark per 1,000 questionsper 1M questions · 0.024 s per question (median latency) | 29.7%27.8%–31.6% | 24.7% | 11.3% | 8.2% | 4.8 | $0.002$1.53estimate · DGX Spark | 26 sDGX Spark | 0.024 sp90 0.025 s |
| 16 | Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Math 28.7% (95%: 26.8%–30.6%) · Logic 22.4% Non-hallucination 3.2% · “I don't know” on 2.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.569 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.026$26.22estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.035 s per question (median latency) | 28.7%26.8%–30.6% | 22.4% | 3.2% | 2.3% | 8.8 | $0.026$26.22estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.035 sp90 0.041 s |
| 16 | OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Math 28.2% (95%: 26.4%–30.2%) · Logic 27.0% Non-hallucination 0.6% · “I don't know” on 0.5% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.592 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.09estimate · DGX Spark per 1,000 questionsper 1M questions · 0.015 s per question (median latency) | 28.2%26.4%–30.2% | 27.0% | 0.6% | 0.5% | 10.0 | $0.001$1.09estimate · DGX Spark | 18 sDGX Spark | 0.015 sp90 0.017 s |
| 16 | KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Math 28.2% (95%: 26.3%–30.1%) · Logic 26.7% Non-hallucination 0.3% · “I don't know” on 0.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.596 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.69estimate · DGX Spark per 1,000 questionsper 1M questions · 0.025 s per question (median latency) | 28.2%26.3%–30.1% | 26.7% | 0.3% | 0.2% | 10.5 | $0.002$1.69estimate · DGX Spark | 29 sDGX Spark | 0.025 sp90 0.028 s |
| 17 | GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Math 22.7% (95%: 20.9%–24.5%) · Logic 23.6% Non-hallucination 17.2% · “I don't know” on 14.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.501 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.85estimate · DGX Spark per 1,000 questionsper 1M questions · 0.012 s per question (median latency) | 22.7%20.9%–24.5% | 23.6% | 17.2% | 14.2% | 7.0 | $0.001$0.85estimate · DGX Spark | 14 sDGX Spark | 0.012 sp90 0.013 s |
| 17 | GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Math 20.6% (95%: 18.9%–22.3%) · Logic 23.2% Non-hallucination 72.8% · “I don't know” on 67.5% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.500 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.77estimate · DGX Spark per 1,000 questionsper 1M questions · 0.011 s per question (median latency) | 20.6%18.9%–22.3% | 23.2% | 72.8% | 67.5% | 12.2 | $0.001$0.77estimate · DGX Spark | 13 sDGX Spark | 0.011 sp90 0.012 s |
| Tier | Model | Math accuracy 2,163 questions, 95% interval; min–max for 3-seed rows | Logic of 1,125 questions | Non-hallucination of answers not right, % “I don't know” | “I don't know” share declined | Calibration error, points; decision models | $ per 1,000 q$ per 1M q list price or GPU time | GPU time s per 1,000 q, our hardware | Latency median, p90 under it |
|---|---|---|---|---|---|---|---|---|---|
| 1 | Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $4.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Math 99.8% (95%: 99.5%–99.9%) · Logic 96.2% Non-hallucination 62.5% · “I don't know” on 0.2% of questions Runs on an API · cost $9.871$9,871 API list-price equivalent per 1,000 questionsper 1M questions · 3.20 s per question (median latency) | 99.8%99.5%–99.9% | 96.2% | 62.5% | 0.2% | – | $9.871$9,871 API list-price equivalent | – | 3.20 sp90 4.79 s |
| 1 | Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.8% (95%: 99.5%–99.9%) · Logic 96.3% Non-hallucination 54.5% · “I don't know” on 0.3% of questions Runs on an API · cost $1.478$1,478 per 1,000 questionsper 1M questions · 4.71 s per question (median latency) | 99.8%99.5%–99.9% | 96.3% | 54.5% | 0.3% | – | $1.478$1,478 | – | 4.71 sp90 9.13 s |
| 1 | Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $10.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Math 99.7% (95%: 99.4%–99.9%) · Logic 95.2% Non-hallucination 60.0% · “I don't know” on 0.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.03 s per question (median latency) | 99.7%99.4%–99.9% | 95.2% | 60.0% | 0.3% | – | $0.000$0.00 | – | 5.03 sp90 6.49 s |
| 1 | Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $2.00 per 1M input tokens: Workhorse (source: https://platform.claude.com/docs/en/about-claude/pricing) Math 99.7% (95%: 99.3%–99.8%) · Logic 95.5% Non-hallucination 62.5% · “I don't know” on 0.2% of questions Runs on an API · cost $4.734$4,734 API list-price equivalent per 1,000 questionsper 1M questions · 3.46 s per question (median latency) | 99.7%99.3%–99.8% | 95.5% | 62.5% | 0.2% | – | $4.734$4,734 API list-price equivalent | – | 3.46 sp90 5.97 s |
| 1 | DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active · Mixture of experts: 552B total, about 16B active per step. Size class uses 552B. Math 99.6% (95%: 99.3%–99.8%) · Logic 96.2% Non-hallucination 20.0% · “I don't know” on 0.1% of questions Runs on an API · cost $0.227$227 per 1,000 questionsper 1M questions · 1.91 s per question (median latency) | 99.6%99.3%–99.8% | 96.2% | 20.0% | 0.1% | – | $0.227$227 | – | 1.91 sp90 3.57 s |
| 1 | GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Math 99.6% (95%: 99.3%–99.8%) · Logic 96.7% Non-hallucination 78.6% · “I don't know” on 0.5% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.28 s per question (median latency) | 99.6%99.3%–99.8% | 96.7% | 78.6% | 0.5% | – | $0.000$0.00 | – | 5.28 sp90 7.26 s |
| 1 | GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Math 99.6% (95%: 99.3%–99.8%) · Logic 96.5% Non-hallucination 78.6% · “I don't know” on 0.5% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.81 s per question (median latency) | 99.6%99.3%–99.8% | 96.5% | 78.6% | 0.5% | – | $0.000$0.00 | – | 4.81 sp90 6.56 s |
| 1 | Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active · Mixture of experts: 276B total, about 12B active per step. Size class uses 276B. Math 99.6% (95%: 99.3%–99.8%) · Logic 95.4% Non-hallucination 54.5% · “I don't know” on 0.3% of questions Runs on an API · cost $0.954$954 per 1,000 questionsper 1M questions · 2.03 s per question (median latency) | 99.6%99.3%–99.8% | 95.4% | 54.5% | 0.3% | – | $0.954$954 | – | 2.03 sp90 6.33 s |
| 1 | Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.6% (95%: 99.2%–99.8%) · Logic 96.7% Non-hallucination 75.0% · “I don't know” on 0.6% of questions Runs on an API · 13.4 s per question (median latency) | 99.6%99.2%–99.8% | 96.7% | 75.0% | 0.6% | – | – | – | 13.4 sp90 27.8 s |
| 1 | Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Math 99.6% (95%: 99.2%–99.8%) · Logic 96.1% Non-hallucination 85.7% · “I don't know” on 0.8% of questions Runs on an API · 4.40 s per question (median latency) | 99.6%99.2%–99.8% | 96.1% | 85.7% | 0.8% | – | – | – | 4.40 sp90 6.68 s |
| 1 | Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/meta/muse-ember) Math 99.6% (95%: 99.2%–99.8%) · Logic 95.5% Non-hallucination 64.3% · “I don't know” on 0.4% of questions Runs on an API · cost $0.365$365 per 1,000 questionsper 1M questions · 1.26 s per question (median latency) | 99.6%99.2%–99.8% | 95.5% | 64.3% | 0.4% | – | $0.365$365 | – | 1.26 sp90 2.27 s |
| 1 | Grok 4.7 chat Closed · Undisclosed Grok 4.7 chat · xAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.5% (95%: 99.2%–99.7%) · Logic running Runs on an API · cost $12.061$12,061 API list-price equivalent per 1,000 questionsper 1M questions · 11.1 s per question (median latency) | 99.5%99.2%–99.7% | running | – | – | – | $12.061$12,061 API list-price equivalent | – | 11.1 sp90 66.5 s |
| 1 | Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active · Mixture of experts: 594B total, about 17B active per step. Size class uses 594B. Math 99.5% (95%: 99.2%–99.7%) · Logic 95.6% Non-hallucination 61.5% · “I don't know” on 0.4% of questions Runs on an API · cost $0.638$638 per 1,000 questionsper 1M questions · 2.11 s per question (median latency) | 99.5%99.2%–99.7% | 95.6% | 61.5% | 0.4% | – | $0.638$638 | – | 2.11 sp90 6.00 s |
| 1 | Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Math 99.5% (95%: 99.1%–99.7%) · Logic 94.0% Non-hallucination 16.7% · “I don't know” on 0.1% of questions Runs on an API · cost $0.397$397 per 1,000 questionsper 1M questions · 5.74 s per question (median latency) | 99.5%99.1%–99.7% | 94.0% | 16.7% | 0.1% | – | $0.397$397 | – | 5.74 sp90 11.8 s |
| 1 | MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active · Mixture of experts: 309B total, about 15B active per step. Size class uses 309B. Math 99.4% (95%: 99.0%–99.7%) · Logic 95.1% Non-hallucination 50.0% · “I don't know” on 0.3% of questions Runs on an API · cost $0.212$212 per 1,000 questionsper 1M questions · 8.20 s per question (median latency) | 99.4%99.0%–99.7% | 95.1% | 50.0% | 0.3% | – | $0.212$212 | – | 8.20 sp90 18.1 s |
| 1 | Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Math 99.4% (95%: 99.0%–99.7%) · Logic 92.4% Non-hallucination 21.4% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.271$1,271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 46.6 s per question (median latency) | 99.4%99.0%–99.7% | 92.4% | 21.4% | 0.1% | – | $1.271$1,271estimate · RTX PRO 6000 Blackwell | 1,838 sRTX PRO 6000 | 46.6 sp90 96.5 s |
| 1 | Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Math 99.4% (95%: 99.0%–99.6%) · Logic 93.3% Non-hallucination 21.4% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.531$531estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.1 s per question (median latency) | 99.4%99.0%–99.6% | 93.3% | 21.4% | 0.1% | – | $0.531$531estimate · RTX PRO 6000 Blackwell | 767 sRTX PRO 6000 | 11.1 sp90 16.7 s |
| 1 | DeepSeek-V4-Flash chat Open · 285B MoE · 13B active DeepSeek-V4-Flash chat · DeepSeek · chat model Parameters: 285B MoE · 13B active · Mixture of experts: 285B total, about 13B active per step. Size class uses 285B. Math 99.3% (95%: 98.9%–99.6%) · Logic running Runs on NVIDIA DGX Spark (GB10) · cost $0.295$295estimate · DGX Spark per 1,000 questionsper 1M questions · 8.13 s per question (median latency) | 99.3%98.9%–99.6% | running | – | – | – | $0.295$295estimate · DGX Spark | 4,987 sDGX Spark | 8.13 sp90 16.9 s |
| 1 | MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Math 99.2% (mean of 3 seeds; min–max 99.2%–99.3%) · Logic 88.9% Seed runs: seed 0: 99.3% (2,147 of 2,163, run minicpm5-2b_en) · seed 1: 99.2% (2,145 of 2,163, run minicpm5-2b-seed1_en) · seed 2: 99.3% (2,147 of 2,163, run minicpm5-2b-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b_en, not the mean Non-hallucination 53.6% · “I don't know” on 0.7% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.409$409estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 12.5 s per question (median latency) | 99.2%99.2%–99.3% min–max of 3 seeds | 88.9% | 53.6%seed 0 only | 0.7%seed 0 only | – | $0.409$409estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 591 sRTX PRO 6000mean of 3 seeds | 12.5 sp90 32.7 smean of 3 seeds |
| 1 | GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Math 99.2% (95%: 98.7%–99.5%) · Logic 94.0% Non-hallucination 77.8% · “I don't know” on 1.0% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.08 s per question (median latency) | 99.2%98.7%–99.5% | 94.0% | 77.8% | 1.0% | – | $0.000$0.00 | – | 4.08 sp90 5.87 s |
| 1 | Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Math 99.2% (95%: 98.7%–99.5%) · Logic 91.6% Non-hallucination 61.5% · “I don't know” on 0.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.781$781estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 15.3 s per question (median latency) | 99.2%98.7%–99.5% | 91.6% | 61.5% | 0.7% | – | $0.781$781estimate · RTX PRO 6000 Blackwell | 1,129 sRTX PRO 6000 | 15.3 sp90 53.3 s |
| 1 | Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Math 99.2% (95%: 98.7%–99.5%) · Logic 94.0% Non-hallucination 13.0% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.530$530estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 10.0 s per question (median latency) | 99.2%98.7%–99.5% | 94.0% | 13.0% | 0.1% | – | $0.530$530estimate · RTX PRO 6000 Blackwell | 766 sRTX PRO 6000 | 10.0 sp90 20.6 s |
| 2 | LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active · Mixture of experts: 24.0B total, about 2.45B active per step. Size class uses 24.0B. Math 99.0% (95%: 98.5%–99.4%) · Logic 86.7% Non-hallucination 71.4% · “I don't know” on 1.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.939$939estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 27.9 s per question (median latency) | 99.0%98.5%–99.4% | 86.7% | 71.4% | 1.2% | – | $0.939$939estimate · RTX PRO 6000 Blackwell | 1,358 sRTX PRO 6000 | 27.9 sp90 60.2 s |
| 2 | LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Math 98.9% (mean of 3 seeds; min–max 98.8%–99.0%) · Logic 84.6% Seed runs: seed 0: 99.0% (2,142 of 2,163, run lfm3-1.7b-vllm-nothink-rtx-seed0_en) · seed 1: 98.8% (2,136 of 2,163, run lfm3-1.7b-vllm-nothink-rtx-seed1_en) · seed 2: 99.0% (2,142 of 2,163, run lfm3-1.7b-vllm-nothink-rtx-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm-nothink-rtx-seed0_en, not the mean Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 3.99 s per question (median latency) | 98.9%98.8%–99.0% min–max of 3 seeds | 84.6% | – | – | – | $0.142$142estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 205 sRTX PRO 6000mean of 3 seeds | 3.99 sp90 10.6 smean of 3 seeds |
| 2 | LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Math 98.9% (mean of 3 seeds; min–max 98.8%–99.1%) · Logic 84.2% Seed runs: seed 0: 98.9% (2,139 of 2,163, run lfm3-1.7b-vllm_en) · seed 1: 99.1% (2,143 of 2,163, run lfm3-1.7b-vllm-seed1_en) · seed 2: 98.8% (2,138 of 2,163, run lfm3-1.7b-vllm-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm_en, not the mean Non-hallucination 67.6% · “I don't know” on 1.1% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.132$132estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 3.89 s per question (median latency) | 98.9%98.8%–99.1% min–max of 3 seeds | 84.2% | 67.6%seed 0 only | 1.1%seed 0 only | – | $0.132$132estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 191 sRTX PRO 6000mean of 3 seeds | 3.89 sp90 10.4 smean of 3 seeds |
| 2 | Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Math 98.8% (95%: 98.3%–99.2%) · Logic 87.7% Non-hallucination 14.3% · “I don't know” on 0.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 19.1 s per question (median latency) | 98.8%98.3%–99.2% | 87.7% | 14.3% | 0.1% | – | $0.467$467estimate · RTX PRO 6000 Blackwell | 675 sRTX PRO 6000 | 19.1 sp90 28.1 s |
| 2 | GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active · Mixture of experts: 320B total, about 18B active per step. Size class uses 320B. Math 98.8% (95%: 98.2%–99.2%) · Logic 93.2% Non-hallucination 17.4% · “I don't know” on 0.2% of questions Runs on an API · cost $0.166$166 per 1,000 questionsper 1M questions · 3.80 s per question (median latency) | 98.8%98.2%–99.2% | 93.2% | 17.4% | 0.2% | – | $0.166$166 | – | 3.80 sp90 7.97 s |
| 2 | LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Math 98.7% (95%: 98.1%–99.1%) · Logic 85.5% Non-hallucination 67.7% · “I don't know” on 1.0% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.060$59.94estimate · DGX Spark per 1,000 questionsper 1M questions · 19.6 s per question (median latency) | 98.7%98.1%–99.1% | 85.5% | 67.7% | 1.0% | – | $0.060$59.94estimate · DGX Spark | 1,014 sDGX Spark | 19.6 sp90 53.4 s |
| 2 | Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.5% (mean of 3 seeds; min–max 98.4%–98.8%) · Logic 85.9% Seed runs: seed 0: 98.8% (2,136 of 2,163, run gemma-4-e4b-it-think-32k-sampled-seed0_en) · seed 1: 98.5% (2,130 of 2,163, run gemma-4-e4b-it-think-32k-sampled-seed1_en) · seed 2: 98.4% (2,128 of 2,163, run gemma-4-e4b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e4b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 45.5% · “I don't know” on 0.9% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.318$318estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.2 s per question (median latency) | 98.5%98.4%–98.8% min–max of 3 seeds | 85.9% | 45.5%seed 0 only | 0.9%seed 0 only | – | $0.318$318estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 459 sRTX PRO 6000mean of 3 seeds | 11.2 sp90 22.4 smean of 3 seeds |
| 2 | Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.5% (95%: 97.9%–99.0%) · Logic 85.0% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.518$518estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 20.6 s per question (median latency) | 98.5%97.9%–99.0% | 85.0% | – | – | – | $0.518$518estimate · RTX PRO 6000 Blackwell | 749 sRTX PRO 6000 | 20.6 sp90 34.3 s |
| 2 | Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Math 98.5% (95%: 97.9%–98.9%) · Logic 90.1% Non-hallucination 36.0% · “I don't know” on 0.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.0 s per question (median latency) | 98.5%97.9%–98.9% | 90.1% | 36.0% | 0.4% | – | $0.300$300estimate · RTX PRO 6000 Blackwell | 434 sRTX PRO 6000 | 11.0 sp90 20.0 s |
| 2 | Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.2% (95%: 97.6%–98.7%) · Logic 82.6% Non-hallucination 37.0% · “I don't know” on 0.8% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.233$233estimate · DGX Spark per 1,000 questionsper 1M questions · 108.7 s per question (median latency) | 98.2%97.6%–98.7% | 82.6% | 37.0% | 0.8% | – | $0.233$233estimate · DGX Spark | 3,939 sDGX Spark | 108.7 sp90 175.5 s |
| 2 | MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active · Mixture of experts: 428B total, about 23B active per step. Size class uses 428B. Math 98.2% (95%: 97.6%–98.7%) · Logic 94.0% Non-hallucination 40.6% · “I don't know” on 0.6% of questions Runs on an API · cost $0.103$103 per 1,000 questionsper 1M questions · 1.67 s per question (median latency) | 98.2%97.6%–98.7% | 94.0% | 40.6% | 0.6% | – | $0.103$103 | – | 1.67 sp90 7.47 s |
| 2 | MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Math 98.2% (mean of 3 seeds; min–max 98.1%–98.3%) · Logic 83.6% Seed runs: seed 0: 98.3% (2,126 of 2,163, run minicpm5-2b-nothink_en) · seed 1: 98.1% (2,122 of 2,163, run minicpm5-2b-nothink-seed1_en) · seed 2: 98.1% (2,121 of 2,163, run minicpm5-2b-nothink-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b-nothink_en, not the mean Non-hallucination 31.0% · “I don't know” on 1.2% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.219$219estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 6.35 s per question (median latency) | 98.2%98.1%–98.3% min–max of 3 seeds | 83.6% | 31.0%seed 0 only | 1.2%seed 0 only | – | $0.219$219estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 316 sRTX PRO 6000mean of 3 seeds | 6.35 sp90 15.7 smean of 3 seeds |
| 2 | Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Math 98.1% (95%: 97.4%–98.6%) · Logic 83.2% Non-hallucination 44.0% · “I don't know” on 1.0% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.271$271estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 10.8 s per question (median latency) | 98.1%97.4%–98.6% | 83.2% | 44.0% | 1.0% | – | $0.271$271estimate · RTX PRO 6000 Blackwell | 392 sRTX PRO 6000 | 10.8 sp90 17.7 s |
| 3 | Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 97.8% (95%: 97.1%–98.4%) · Logic 75.9% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.410$410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 14.9 s per question (median latency) | 97.8%97.1%–98.4% | 75.9% | – | – | – | $0.410$410estimate · RTX PRO 6000 Blackwell | 592 sRTX PRO 6000 | 14.9 sp90 26.0 s |
| 3 | Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 97.7% (mean of 3 seeds; min–max 97.6%–97.8%) · Logic 76.8% Seed runs: seed 0: 97.6% (2,111 of 2,163, run gemma-4-e2b-it-think-32k-sampled-seed0_en) · seed 1: 97.8% (2,116 of 2,163, run gemma-4-e2b-it-think-32k-sampled-seed1_en) · seed 2: 97.6% (2,111 of 2,163, run gemma-4-e2b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e2b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 23.0% · “I don't know” on 0.6% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.196$196estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 7.14 s per question (median latency) | 97.7%97.6%–97.8% min–max of 3 seeds | 76.8% | 23.0%seed 0 only | 0.6%seed 0 only | – | $0.196$196estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 283 sRTX PRO 6000mean of 3 seeds | 7.14 sp90 14.8 smean of 3 seeds |
| 3 | Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Math 97.0% (95%: 96.2%–97.6%) · Logic 70.9% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.670$1,670estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 34.3 s† per question (median latency) | 97.0%96.2%–97.6% | 70.9% | – | – | – | $1.670$1,670estimate · RTX PRO 6000 Blackwell† | 2,415 sRTX PRO 6000 | 34.3 s†p90 108.0 s |
| 3 | Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 96.9% (95%: 96.1%–97.6%) · Logic 74.6% Non-hallucination 16.5% · “I don't know” on 0.6% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.138$138estimate · DGX Spark per 1,000 questionsper 1M questions · 64.4 s per question (median latency) | 96.9%96.1%–97.6% | 74.6% | 16.5% | 0.6% | – | $0.138$138estimate · DGX Spark | 2,335 sDGX Spark | 64.4 sp90 106.3 s |
| 3 | Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Math 96.8% (95%: 95.9%–97.4%) · Logic 72.9% Non-hallucination 25.3% · “I don't know” on 0.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.208$208estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.12 s per question (median latency) | 96.8%95.9%–97.4% | 72.9% | 25.3% | 0.9% | – | $0.208$208estimate · RTX PRO 6000 Blackwell | 301 sRTX PRO 6000 | 8.12 sp90 13.7 s |
| 3 | LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active · Mixture of experts: 8.3B total, about 1.5B active per step. Size class uses 8.3B. Math 96.3% (95%: 95.4%–97.0%) · Logic 60.2% Non-hallucination 45.2% · “I don't know” on 2.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.482$482estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 14.5 s† per question (median latency) | 96.3%95.4%–97.0% | 60.2% | 45.2% | 2.6% | – | $0.482$482estimate · RTX PRO 6000 Blackwell† | 697 sRTX PRO 6000 | 14.5 s†p90 33.2 s |
| 4 | Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Math 95.8% (95%: 94.9%–96.6%) · Logic 65.3% Non-hallucination 39.3% · “I don't know” on 2.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.355$355estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.04 s per question (median latency) | 95.8%94.9%–96.6% | 65.3% | 39.3% | 2.1% | – | $0.355$355estimate · RTX PRO 6000 Blackwell | 513 sRTX PRO 6000 | 8.04 sp90 19.0 s |
| 4 | Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Math 94.7% (95%: 93.7%–95.6%) · Logic 70.3% Non-hallucination 20.3% · “I don't know” on 0.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.470$2,470estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 98.4 s per question (median latency) | 94.7%93.7%–95.6% | 70.3% | 20.3% | 0.6% | – | $2.470$2,470estimate · RTX PRO 6000 Blackwell | 3,571 sRTX PRO 6000 | 98.4 sp90 195.0 s |
| 4 | Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Math 94.6% (95%: 93.6%–95.5%) · Logic 64.8% Non-hallucination 31.3% · “I don't know” on 1.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.363$363estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.2 s per question (median latency) | 94.6%93.6%–95.5% | 64.8% | 31.3% | 1.9% | – | $0.363$363estimate · RTX PRO 6000 Blackwell | 525 sRTX PRO 6000 | 11.2 sp90 32.1 s |
| 4 | LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Math 94.0% (95%: 92.9%–94.9%) · Logic 75.4% Non-hallucination 26.4% · “I don't know” on 1.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.300$300estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.86 s per question (median latency) | 94.0%92.9%–94.9% | 75.4% | 26.4% | 1.1% | – | $0.300$300estimate · RTX PRO 6000 Blackwell | 434 sRTX PRO 6000 | 9.86 sp90 21.2 s |
| 4 | GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/openai/gpt-6-luna) Math 93.7% (95%: 92.6%–94.6%) · Logic 58.0% Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.83 s per question (median latency) | 93.7%92.6%–94.6% | 58.0% | – | – | – | $0.000$0.00 | – | 3.83 sp90 5.00 s |
| 5 | Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Math 90.8% (95%: 89.5%–91.9%) · Logic 54.2% Non-hallucination 35.9% · “I don't know” on 4.3% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.266$266estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 7.89 s per question (median latency) | 90.8%89.5%–91.9% | 54.2% | 35.9% | 4.3% | – | $0.266$266estimate · RTX PRO 6000 Blackwell | 385 sRTX PRO 6000 | 7.89 sp90 14.9 s |
| 5 | Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Math 89.6% (95%: 88.2%–90.8%) · Logic 65.1% Non-hallucination 22.1% · “I don't know” on 2.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.901$1,901estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 25.6 s† per question (median latency) | 89.6%88.2%–90.8% | 65.1% | 22.1% | 2.1% | – | $1.901$1,901estimate · RTX PRO 6000 Blackwell† | 2,748 sRTX PRO 6000 | 25.6 s†p90 134.8 s |
| 6 | Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Math 81.6% (95%: 79.9%–83.1%) · Logic 60.7% Non-hallucination 4.8% · “I don't know” on 0.9% of questions Confidence: 837 answers kept at ≥99%, 837 right · 0 wrong · AUROC 0.903 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.113$113estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.155 s per question (median latency) | 81.6%79.9%–83.1% | 60.7% | 4.8% | 0.9% | 1.2 | $0.113$113estimate · RTX PRO 6000 Blackwell | 163 sRTX PRO 6000 | 0.155 sp90 0.186 s |
| 6 | KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Math 80.3% (95%: 78.5%–81.9%) · Logic 58.9% Non-hallucination 0.2% · “I don't know” on 0.0% of questions Confidence: 470 answers kept at ≥99%, 469 right · 1 wrong · AUROC 0.898 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.114 s per question (median latency) | 80.3%78.5%–81.9% | 58.9% | 0.2% | 0.0% | 3.6 | $0.083$83.14estimate · RTX PRO 6000 Blackwell | 120 sRTX PRO 6000 | 0.114 sp90 0.120 s |
| 7 | REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense · Two models run together, option probabilities averaged 50/50: 57.36B in all, all used on every question. Math 78.0% (95%: 76.2%–79.7%) · Logic 59.3% Non-hallucination 2.3% · “I don't know” on 0.5% of questions Confidence: 178 answers kept at ≥99%, 178 right · 0 wrong · AUROC 0.903 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.142$142estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.189 s per question (median latency) | 78.0%76.2%–79.7% | 59.3% | 2.3% | 0.5% | 7.1 | $0.142$142estimate · RTX PRO 6000 Blackwell | 205 sRTX PRO 6000 | 0.189 sp90 0.232 s |
| 7 | KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Math 77.4% (95%: 75.6%–79.1%) · Logic 59.0% Non-hallucination 1.1% · “I don't know” on 0.2% of questions Confidence: 465 answers kept at ≥99%, 464 right · 1 wrong · AUROC 0.895 Runs on NVIDIA DGX Spark (GB10) · cost $0.047$46.87estimate · DGX Spark per 1,000 questionsper 1M questions · 0.764 s per question (median latency) | 77.4%75.6%–79.1% | 59.0% | 1.1% | 0.2% | 3.6 | $0.047$46.87estimate · DGX Spark | 793 sDGX Spark | 0.764 sp90 0.867 s |
| 7 | Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Math 77.3% (95%: 75.5%–79.1%) · Logic 58.7% Non-hallucination 1.7% · “I don't know” on 0.4% of questions Confidence: 318 answers kept at ≥99%, 318 right · 0 wrong · AUROC 0.899 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.084$84.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.118 s per question (median latency) | 77.3%75.5%–79.1% | 58.7% | 1.7% | 0.4% | 6.5 | $0.084$84.42estimate · RTX PRO 6000 Blackwell | 122 sRTX PRO 6000 | 0.118 sp90 0.142 s |
| 7 | Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Math 77.1% (95%: 75.3%–78.8%) · Logic 57.9% Non-hallucination 2.1% · “I don't know” on 0.5% of questions Confidence: 215 answers kept at ≥99%, 215 right · 0 wrong · AUROC 0.891 Runs on NVIDIA DGX Spark (GB10) · cost $0.039$39.14estimate · DGX Spark per 1,000 questionsper 1M questions · 0.637 s per question (median latency) | 77.1%75.3%–78.8% | 57.9% | 2.1% | 0.5% | 4.3 | $0.039$39.14estimate · DGX Spark | 662 sDGX Spark | 0.637 sp90 0.735 s |
| 8 | Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Math 75.2% (95%: 73.4%–77.0%) · Logic 57.8% Non-hallucination 2.1% · “I don't know” on 0.5% of questions Confidence: 15 answers kept at ≥99%, 15 right · 0 wrong · AUROC 0.899 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.141$141estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.196 s per question (median latency) | 75.2%73.4%–77.0% | 57.8% | 2.1% | 0.5% | 7.5 | $0.141$141estimate · RTX PRO 6000 Blackwell | 203 sRTX PRO 6000 | 0.196 sp90 0.221 s |
| 8 | d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://www.liquid.ai/blog/d1-ultra) Math 74.1% (95%: 72.2%–75.9%) · Logic 54.9% Non-hallucination 87.4% · “I don't know” on 38.8% of questions Confidence: 213 answers kept at ≥99%, 213 right · 0 wrong · AUROC 0.876 Runs on an API · cost $0.006$6.14 per 1,000 questionsper 1M questions · 0.179 s per question (median latency) | 74.1%72.2%–75.9% | 54.9% | 87.4% | 38.8% | 2.3 | $0.006$6.14 | – | 0.179 sp90 0.426 s |
| 8 | Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Math 74.0% (95%: 72.1%–75.8%) · Logic 54.5% Non-hallucination 70.6% · “I don't know” on 24.5% of questions Confidence: 412 answers kept at ≥99%, 412 right · 0 wrong · AUROC 0.895 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.140 s per question (median latency) | 74.0%72.1%–75.8% | 54.5% | 70.6% | 24.5% | 3.0 | $0.023$22.56 list$0.104$104estimate · RTX PRO 6000 Blackwell | 150 sRTX PRO 6000 | 0.140 sp90 0.173 s |
| 8 | Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Math 72.5% (95%: 70.6%–74.3%) · Logic 54.8% Non-hallucination 9.4% · “I don't know” on 2.8% of questions Confidence: 323 answers kept at ≥99%, 323 right · 0 wrong · AUROC 0.856 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.066$65.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.091 s per question (median latency) | 72.5%70.6%–74.3% | 54.8% | 9.4% | 2.8% | 4.5 | $0.066$65.87estimate · RTX PRO 6000 Blackwell | 95 sRTX PRO 6000 | 0.091 sp90 0.103 s |
| 8 | GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Math 72.0% (95%: 70.1%–73.8%) · Logic 58.4% Non-hallucination 21.6% · “I don't know” on 6.5% of questions Confidence: 199 answers kept at ≥99%, 198 right · 1 wrong · AUROC 0.864 Runs on NVIDIA DGX Spark (GB10) · cost $0.026$26.16estimate · DGX Spark per 1,000 questionsper 1M questions · 0.425 s per question (median latency) | 72.0%70.1%–73.8% | 58.4% | 21.6% | 6.5% | 4.1 | $0.026$26.16estimate · DGX Spark | 442 sDGX Spark | 0.425 sp90 0.461 s |
| 9 | d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Math 71.5% (95%: 69.6%–73.4%) · Logic 54.6% Non-hallucination 5.3% · “I don't know” on 1.6% of questions Confidence: 11 answers kept at ≥99%, 11 right · 0 wrong · AUROC 0.857 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.089$88.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.118 s per question (median latency) | 71.5%69.6%–73.4% | 54.6% | 5.3% | 1.6% | 8.6 | $0.089$88.58estimate · RTX PRO 6000 Blackwell | 128 sRTX PRO 6000 | 0.118 sp90 0.148 s |
| 9 | Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/google/gemini-decision-1) Math 71.3% (95%: 69.4%–73.2%) · Logic 50.0% Non-hallucination 3.9% · “I don't know” on 1.2% of questions Confidence: 214 answers kept at ≥99%, 214 right · 0 wrong · AUROC 0.859 Runs on an API · cost $0.022$22.02 per 1,000 questionsper 1M questions · 0.226 s per question (median latency) | 71.3%69.4%–73.2% | 50.0% | 3.9% | 1.2% | 2.5 | $0.022$22.02 | – | 0.226 sp90 0.351 s |
| 9 | GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker · Closed; list price $0.15 per 1M input tokens: Efficient (source: https://docs.fastino.ai/pricing) Math 70.8% (95%: 68.8%–72.7%) · Logic 54.5% Non-hallucination 40.6% · “I don't know” on 14.2% of questions Confidence: 161 answers kept at ≥99%, 161 right · 0 wrong · AUROC 0.865 Runs on an API · cost $0.028$28.30 per 1,000 questionsper 1M questions · 0.264 s per question (median latency) | 70.8%68.8%–72.7% | 54.5% | 40.6% | 14.2% | 5.0 | $0.028$28.30 | – | 0.264 sp90 0.279 s |
| 9 | d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Math 70.4% (95%: 68.5%–72.3%) · Logic 54.5% Non-hallucination 4.7% · “I don't know” on 1.4% of questions Confidence: 163 answers kept at ≥99%, 162 right · 1 wrong · AUROC 0.878 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$40.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.055 s per question (median latency) | 70.4%68.5%–72.3% | 54.5% | 4.7% | 1.4% | 4.5 | $0.041$40.61estimate · RTX PRO 6000 Blackwell | 59 sRTX PRO 6000 | 0.055 sp90 0.062 s |
| 9 | Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Math 70.3% (95%: 68.4%–72.2%) · Logic 53.1% Non-hallucination 4.5% · “I don't know” on 1.3% of questions Confidence: 729 answers kept at ≥99%, 719 right · 10 wrong · AUROC 0.861 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.076 s per question (median latency) | 70.3%68.4%–72.2% | 53.1% | 4.5% | 1.3% | 11.8 | $0.013$13.06 list$0.055$54.68estimate · RTX PRO 6000 Blackwell | 79 sRTX PRO 6000 | 0.076 sp90 0.089 s |
| 9 | Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Math 70.3% (95%: 68.4%–72.2%) · Logic 53.2% Non-hallucination 6.2% · “I don't know” on 1.9% of questions Confidence: 377 answers kept at ≥99%, 377 right · 0 wrong · AUROC 0.866 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.004$4.35 list$0.070$70.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.095 s per question (median latency) Shown under Perplexity Decider 27B v1.2 (API): the same model name, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings | 70.3%68.4%–72.2% | 53.2% | 6.2% | 1.9% | 2.7 | $0.004$4.35 list$0.070$70.03estimate · RTX PRO 6000 Blackwell | 101 sRTX PRO 6000 | 0.095 sp90 0.125 s |
| 9 | Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker · Closed; introductory list price (the only one published) $0.03 per 1M input tokens: Efficient (provisional) (source: https://nace.ai/drex) Math 70.1% (95%: 68.1%–72.0%) · Logic 54.8% Non-hallucination 43.5% · “I don't know” on 15.6% of questions Confidence: 315 answers kept at ≥99%, 315 right · 0 wrong · AUROC 0.867 Runs on an API · cost $0.013$12.88 per 1,000 questionsper 1M questions · 0.326 s per question (median latency) | 70.1%68.1%–72.0% | 54.8% | 43.5% | 15.6% | 5.7 | $0.013$12.88 | – | 0.326 sp90 0.473 s |
| 9 | celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Math 70.0% (95%: 68.0%–71.9%) · Logic 47.3% Non-hallucination 82.7% · “I don't know” on 38.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.858 Runs on an API · cost $0.009$9.10 per 1,000 questionsper 1M questions · 0.387 s per question (median latency) | 70.0%68.0%–71.9% | 47.3% | 82.7% | 38.2% | 4.3 | $0.009$9.10 | – | 0.387 sp90 0.391 s |
| 9 | GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Math 69.9% (95%: 67.9%–71.8%) · Logic 55.2% Non-hallucination 35.8% · “I don't know” on 12.2% of questions Confidence: 131 answers kept at ≥99%, 131 right · 0 wrong · AUROC 0.862 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.079 s per question (median latency) | 69.9%67.9%–71.8% | 55.2% | 35.8% | 12.2% | 5.2 | $0.008$7.55 list$0.058$57.88estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.079 sp90 0.084 s |
| 9 | celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.004 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Math 69.9% (95%: 67.9%–71.8%) · Logic 47.3% Non-hallucination 82.9% · “I don't know” on 37.9% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.859 Runs on an API · cost $0.001$0.91 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) | 69.9%67.9%–71.8% | 47.3% | 82.9% | 37.9% | 4.5 | $0.001$0.91 | – | 0.381 sp90 0.386 s |
| 9 | d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Math 69.9% (95%: 67.9%–71.8%) · Logic 51.6% Non-hallucination 86.7% · “I don't know” on 41.6% of questions Confidence: 115 answers kept at ≥99%, 115 right · 0 wrong · AUROC 0.876 Runs on an API · cost $0.006$6.14 per 1,000 questionsper 1M questions · 0.215 s per question (median latency) | 69.9%67.9%–71.8% | 51.6% | 86.7% | 41.6% | 4.6 | $0.006$6.14 | – | 0.215 sp90 0.347 s |
| 9 | Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Math 69.8% (95%: 67.8%–71.7%) · Logic 53.3% Non-hallucination 7.0% · “I don't know” on 2.2% of questions Confidence: 331 answers kept at ≥99%, 331 right · 0 wrong · AUROC 0.869 Runs on an API · cost $0.004$4.29 per 1,000 questionsper 1M questions · 0.226 s per question (median latency) | 69.8%67.8%–71.7% | 53.3% | 7.0% | 2.2% | 3.2 | $0.004$4.29 | – | 0.226 sp90 0.368 s |
| – | Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Not tiered: with web Math 69.3% (95%: 67.3%–71.2%) · Logic 52.5% Non-hallucination 15.4% · “I don't know” on 4.8% of questions Confidence: 326 answers kept at ≥99%, 326 right · 0 wrong · AUROC 0.873 Runs on an API · cost $1.023$1,023 per 1,000 questionsper 1M questions · 0.422 s per question (median latency) | 69.3%67.3%–71.2% | 52.5% | 15.4% | 4.8% | 3.7 | $1.023$1,023 | – | 0.422 sp90 0.641 s |
| 9 | Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Math 69.0% (95%: 67.0%–70.9%) · Logic 52.3% Non-hallucination 2.2% · “I don't know” on 0.6% of questions Confidence: 43 answers kept at ≥99%, 43 right · 0 wrong · AUROC 0.864 Runs on NVIDIA DGX Spark (GB10) · cost $0.017$17.46estimate · DGX Spark per 1,000 questionsper 1M questions · 0.278 s per question (median latency) | 69.0%67.0%–70.9% | 52.3% | 2.2% | 0.6% | 4.7 | $0.017$17.46estimate · DGX Spark | 295 sDGX Spark | 0.278 sp90 0.329 s |
| 9 | SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Math 68.6% (95%: 66.6%–70.5%) · Logic 52.4% Non-hallucination 52.5% · “I don't know” on 20.5% of questions Confidence: 361 answers kept at ≥99%, 360 right · 1 wrong · AUROC 0.839 Runs on an API · cost $0.009$8.59 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) Shown once with SubQ Snap Preview: the same answer to every question | 68.6%66.6%–70.5% | 52.4% | 52.5% | 20.5% | 9.5 | $0.009$8.59 | – | 0.381 sp90 0.390 s |
| 9 | SubQ Snap Preview Closed · Efficient SubQ Snap Preview · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.05 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Math 68.6% (95%: 66.6%–70.5%) · Logic 52.4% Non-hallucination 52.5% · “I don't know” on 20.5% of questions Confidence: 361 answers kept at ≥99%, 360 right · 1 wrong · AUROC 0.839 Runs on an API · cost $0.011$10.73 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) Shown under SubQ Snap 1.1 (the same answers) | 68.6%66.6%–70.5% | 52.4% | 52.5% | 20.5% | 9.5 | $0.011$10.73 | – | 0.381 sp90 0.399 s |
| 10 | KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Math 66.3% (95%: 64.2%–68.2%) · Logic 50.6% Non-hallucination 1.1% · “I don't know” on 0.4% of questions Confidence: 7 answers kept at ≥99%, 7 right · 0 wrong · AUROC 0.819 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.036$36.13estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.049 s per question (median latency) | 66.3%64.2%–68.2% | 50.6% | 1.1% | 0.4% | 4.0 | $0.036$36.13estimate · RTX PRO 6000 Blackwell | 52 sRTX PRO 6000 | 0.049 sp90 0.055 s |
| 10 | d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Math 65.5% (95%: 63.5%–67.5%) · Logic 49.2% Non-hallucination 56.3% · “I don't know” on 23.6% of questions Confidence: 141 answers kept at ≥99%, 141 right · 0 wrong · AUROC 0.838 Runs on an API · cost $0.003$2.92 per 1,000 questionsper 1M questions · 0.204 s per question (median latency) | 65.5%63.5%–67.5% | 49.2% | 56.3% | 23.6% | 3.2 | $0.003$2.92 | – | 0.204 sp90 0.386 s |
| 10 | d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Math 65.2% (95%: 63.2%–67.2%) · Logic 46.0% Non-hallucination 82.5% · “I don't know” on 44.1% of questions Confidence: 183 answers kept at ≥99%, 183 right · 0 wrong · AUROC 0.829 Runs on an API · cost $0.003$2.92 per 1,000 questionsper 1M questions · 0.172 s per question (median latency) | 65.2%63.2%–67.2% | 46.0% | 82.5% | 44.1% | 2.2 | $0.003$2.92 | – | 0.172 sp90 0.290 s |
| 10 | Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://docs.typesafe.ai/models) Math 65.1% (95%: 63.1%–67.1%) · Logic 48.2% Non-hallucination 7.7% · “I don't know” on 2.8% of questions Confidence: 202 answers kept at ≥99%, 202 right · 0 wrong · AUROC 0.829 Runs on an API · cost $0.019$18.99 per 1,000 questionsper 1M questions · 0.163 s per question (median latency) | 65.1%63.1%–67.1% | 48.2% | 7.7% | 2.8% | 2.8 | $0.019$18.99 | – | 0.163 sp90 0.185 s |
| 10 | Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://typesafe.ai/blog/introducing-system-one-models-and-jev) Math 64.1% (95%: 62.0%–66.1%) · Logic 46.3% Non-hallucination 5.7% · “I don't know” on 2.1% of questions Confidence: 248 answers kept at ≥99%, 248 right · 0 wrong · AUROC 0.834 Runs on an API · cost $0.019$18.99 per 1,000 questionsper 1M questions · 0.166 s per question (median latency) | 64.1%62.0%–66.1% | 46.3% | 5.7% | 2.1% | 2.6 | $0.019$18.99 | – | 0.166 sp90 0.206 s |
| 10 | Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 62.1% (95%: 60.0%–64.1%) · Logic 46.4% Non-hallucination 3.1% · “I don't know” on 1.2% of questions Confidence: 198 answers kept at ≥99%, 197 right · 1 wrong · AUROC 0.806 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 62.1%60.0%–64.1% | 46.4% | 3.1% | 1.2% | 4.8 | $0.027$26.54estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.036 sp90 0.044 s |
| 11 | Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Math 61.3% (95%: 59.3%–63.4%) · Logic 46.4% Non-hallucination 65.0% · “I don't know” on 32.9% of questions Confidence: 183 answers kept at ≥99%, 182 right · 1 wrong · AUROC 0.803 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.080 s per question (median latency) | 61.3%59.3%–63.4% | 46.4% | 65.0% | 32.9% | 4.3 | $0.008$7.90 list$0.058$58.20estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.080 sp90 0.092 s |
| 11 | KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 60.1% (95%: 58.0%–62.1%) · Logic 43.9% Non-hallucination 3.2% · “I don't know” on 1.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.803 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.031$31.02estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.041 s per question (median latency) | 60.1%58.0%–62.1% | 43.9% | 3.2% | 1.3% | 4.8 | $0.031$31.02estimate · RTX PRO 6000 Blackwell | 45 sRTX PRO 6000 | 0.041 sp90 0.046 s |
| 11 | Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 59.8% (95%: 57.7%–61.9%) · Logic 48.2% Non-hallucination 6.6% · “I don't know” on 2.7% of questions Confidence: 64 answers kept at ≥99%, 64 right · 0 wrong · AUROC 0.814 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.029$28.78estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.037 s per question (median latency) | 59.8%57.7%–61.9% | 48.2% | 6.6% | 2.7% | 3.8 | $0.029$28.78estimate · RTX PRO 6000 Blackwell | 42 sRTX PRO 6000 | 0.037 sp90 0.051 s |
| 11 | KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 59.8% (95%: 57.7%–61.8%) · Logic 44.3% Non-hallucination 0.9% · “I don't know” on 0.4% of questions Confidence: 123 answers kept at ≥99%, 123 right · 0 wrong · AUROC 0.799 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.027$26.54estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.037 s per question (median latency) | 59.8%57.7%–61.8% | 44.3% | 0.9% | 0.4% | 3.1 | $0.027$26.54estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.037 sp90 0.043 s |
| 11 | Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://openrouter.ai/microsoft/microsoft-decision-1) Base model: Qwen3.5-9B Math 59.4% (95%: 57.3%–61.4%) · Logic 44.4% Non-hallucination 5.1% · “I don't know” on 2.1% of questions Confidence: 27 answers kept at ≥99%, 27 right · 0 wrong · AUROC 0.787 Runs on an API · cost $0.007$6.81 per 1,000 questionsper 1M questions · 0.310 s per question (median latency) | 59.4%57.3%–61.4% | 44.4% | 5.1% | 2.1% | 4.1 | $0.007$6.81 | – | 0.310 sp90 0.359 s |
| 11 | Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active · Mixture of experts: 35B total, about 3B active per step. Size class uses 35B. Math 59.0% (95%: 56.9%–61.0%) · Logic 41.9% Non-hallucination 20.7% · “I don't know” on 9.2% of questions Confidence: 174 answers kept at ≥99%, 174 right · 0 wrong · AUROC 0.813 Runs on an API · cost $0.009$8.92 per 1,000 questionsper 1M questions · 0.334 s per question (median latency) | 59.0%56.9%–61.0% | 41.9% | 20.7% | 9.2% | 1.9 | $0.009$8.92 | – | 0.334 sp90 0.432 s |
| 11 | Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Math 58.8% (95%: 56.7%–60.9%) · Logic 45.3% Non-hallucination 1.7% · “I don't know” on 0.7% of questions Confidence: 210 answers kept at ≥99%, 207 right · 3 wrong · AUROC 0.804 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.032$31.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.045 s per question (median latency) | 58.8%56.7%–60.9% | 45.3% | 1.7% | 0.7% | 12.1 | $0.032$31.98estimate · RTX PRO 6000 Blackwell | 46 sRTX PRO 6000 | 0.045 sp90 0.050 s |
| 11 | StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Math 58.4% (95%: 56.3%–60.5%) · Logic 46.6% Non-hallucination 4.2% · “I don't know” on 1.8% of questions Confidence: 49 answers kept at ≥99%, 49 right · 0 wrong · AUROC 0.781 Runs on NVIDIA DGX Spark (GB10) · cost $0.008$8.34estimate · DGX Spark per 1,000 questionsper 1M questions · 0.140 s per question (median latency) | 58.4%56.3%–60.5% | 46.6% | 4.2% | 1.8% | 2.7 | $0.008$8.34estimate · DGX Spark | 141 sDGX Spark | 0.140 sp90 0.146 s |
| 11 | GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Math 57.9% (95%: 55.8%–60.0%) · Logic 47.8% Non-hallucination 24.6% · “I don't know” on 11.4% of questions Confidence: 73 answers kept at ≥99%, 73 right · 0 wrong · AUROC 0.798 Runs on NVIDIA DGX Spark (GB10) · cost $0.010$10.49estimate · DGX Spark per 1,000 questionsper 1M questions · 0.174 s per question (median latency) | 57.9%55.8%–60.0% | 47.8% | 24.6% | 11.4% | 7.1 | $0.010$10.49estimate · DGX Spark | 178 sDGX Spark | 0.174 sp90 0.182 s |
| 12 | vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Math 56.8% (95%: 54.7%–58.9%) · Logic 44.1% Non-hallucination 3.8% · “I don't know” on 1.7% of questions Confidence: 33 answers kept at ≥99%, 33 right · 0 wrong · AUROC 0.789 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$23.34estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.031 s per question (median latency) | 56.8%54.7%–58.9% | 44.1% | 3.8% | 1.7% | 3.0 | $0.023$23.34estimate · RTX PRO 6000 Blackwell | 34 sRTX PRO 6000 | 0.031 sp90 0.035 s |
| 12 | Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Math 56.0% (95%: 53.9%–58.1%) · Logic 43.3% Non-hallucination 1.2% · “I don't know” on 0.5% of questions Confidence: 254 answers kept at ≥99%, 244 right · 10 wrong · AUROC 0.780 Runs on NVIDIA DGX Spark (GB10) · cost $0.010$9.51estimate · DGX Spark per 1,000 questionsper 1M questions · 0.153 s per question (median latency) | 56.0%53.9%–58.1% | 43.3% | 1.2% | 0.5% | 15.7 | $0.010$9.51estimate · DGX Spark | 161 sDGX Spark | 0.153 sp90 0.175 s |
| 12 | KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Math 56.0% (95%: 53.9%–58.1%) · Logic 42.3% Non-hallucination 1.6% · “I don't know” on 0.7% of questions Confidence: 54 answers kept at ≥99%, 54 right · 0 wrong · AUROC 0.765 Runs on NVIDIA DGX Spark (GB10) · cost $0.005$4.56estimate · DGX Spark per 1,000 questionsper 1M questions · 0.074 s per question (median latency) | 56.0%53.9%–58.1% | 42.3% | 1.6% | 0.7% | 8.6 | $0.005$4.56estimate · DGX Spark | 77 sDGX Spark | 0.074 sp90 0.084 s |
| 12 | Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Math 55.9% (95%: 53.8%–58.0%) · Logic 44.7% Non-hallucination 8.4% · “I don't know” on 3.8% of questions Confidence: 114 answers kept at ≥99%, 113 right · 1 wrong · AUROC 0.787 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.022$22.38estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 55.9%53.8%–58.0% | 44.7% | 8.4% | 3.8% | 11.7 | $0.022$22.38estimate · RTX PRO 6000 Blackwell | 32 sRTX PRO 6000 | 0.028 sp90 0.031 s |
| 12 | StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Math 55.8% (95%: 53.7%–57.9%) · Logic 45.9% Non-hallucination 13.4% · “I don't know” on 6.4% of questions Confidence: 96 answers kept at ≥99%, 96 right · 0 wrong · AUROC 0.779 Runs on NVIDIA DGX Spark (GB10) · cost $0.004$4.10estimate · DGX Spark per 1,000 questionsper 1M questions · 0.067 s per question (median latency) | 55.8%53.7%–57.9% | 45.9% | 13.4% | 6.4% | 6.7 | $0.004$4.10estimate · DGX Spark | 69 sDGX Spark | 0.067 sp90 0.069 s |
| 12 | d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Math 55.6% (95%: 53.5%–57.7%) · Logic 40.8% Non-hallucination 2.9% · “I don't know” on 1.3% of questions Confidence: 152 answers kept at ≥99%, 152 right · 0 wrong · AUROC 0.767 Runs on an API · cost $0.003$2.92 per 1,000 questionsper 1M questions · 0.184 s per question (median latency) | 55.6%53.5%–57.7% | 40.8% | 2.9% | 1.3% | 3.3 | $0.003$2.92 | – | 0.184 sp90 0.297 s |
| 12 | Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Math 55.3% (95%: 53.2%–57.4%) · Logic 44.4% Non-hallucination 2.6% · “I don't know” on 1.2% of questions Confidence: 130 answers kept at ≥99%, 130 right · 0 wrong · AUROC 0.790 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$40.93estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.058 s per question (median latency) | 55.3%53.2%–57.4% | 44.4% | 2.6% | 1.2% | 4.1 | $0.041$40.93estimate · RTX PRO 6000 Blackwell | 59 sRTX PRO 6000 | 0.058 sp90 0.064 s |
| 12 | Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Math 55.2% (95%: 53.1%–57.3%) · Logic 43.1% Non-hallucination 12.0% · “I don't know” on 5.7% of questions Confidence: 155 answers kept at ≥99%, 150 right · 5 wrong · AUROC 0.781 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.040 s per question (median latency) | 55.2%53.1%–57.3% | 43.1% | 12.0% | 5.7% | 10.4 | $0.002$2.18 list$0.030$30.06estimate · RTX PRO 6000 Blackwell | 43 sRTX PRO 6000 | 0.040 sp90 0.047 s |
| 12 | Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Math 54.6% (95%: 52.4%–56.6%) · Logic 42.6% Non-hallucination 5.0% · “I don't know” on 2.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.764 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.169$169estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.233 s per question (median latency) | 54.6%52.4%–56.6% | 42.6% | 5.0% | 2.3% | 21.1 | $0.169$169estimate · RTX PRO 6000 Blackwell | 245 sRTX PRO 6000 | 0.233 sp90 0.272 s |
| 12 | OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Math 53.8% (95%: 51.7%–55.9%) · Logic 40.6% Non-hallucination 1.8% · “I don't know” on 0.8% of questions Confidence: 26 answers kept at ≥99%, 26 right · 0 wrong · AUROC 0.783 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.061$60.76estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.079 s per question (median latency) | 53.8%51.7%–55.9% | 40.6% | 1.8% | 0.8% | 3.6 | $0.061$60.76estimate · RTX PRO 6000 Blackwell | 88 sRTX PRO 6000 | 0.079 sp90 0.106 s |
| 12 | Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://openrouter.ai/inception/mercury-decide) Math 53.5% (95%: 51.4%–55.6%) · Logic 45.1% Non-hallucination 3.2% · “I don't know” on 1.5% of questions Confidence: 281 answers kept at ≥99%, 272 right · 9 wrong · AUROC 0.789 Runs on an API · cost $0.004$4.38 per 1,000 questionsper 1M questions · 0.288 s per question (median latency) | 53.5%51.4%–55.6% | 45.1% | 3.2% | 1.5% | 15.8 | $0.004$4.38 | – | 0.288 sp90 0.342 s |
| 13 | Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Math 51.5% (95%: 49.4%–53.6%) · Logic 39.7% Non-hallucination 6.4% · “I don't know” on 3.2% of questions Confidence: 130 answers kept at ≥99%, 126 right · 4 wrong · AUROC 0.752 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.034$33.58estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.047 s per question (median latency) | 51.5%49.4%–53.6% | 39.7% | 6.4% | 3.2% | 5.7 | $0.034$33.58estimate · RTX PRO 6000 Blackwell | 49 sRTX PRO 6000 | 0.047 sp90 0.052 s |
| 13 | Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Math 51.1% (95%: 49.0%–53.2%) · Logic 40.8% Non-hallucination 1.6% · “I don't know” on 0.8% of questions Confidence: 12 answers kept at ≥99%, 12 right · 0 wrong · AUROC 0.767 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.181 s per question (median latency) | 51.1%49.0%–53.2% | 40.8% | 1.6% | 0.8% | 11.0 | $0.069$68.54 list$0.131$131estimate · RTX PRO 6000 Blackwell | 190 sRTX PRO 6000 | 0.181 sp90 0.210 s |
| 13 | Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Math 50.4% (95%: 48.3%–52.5%) · Logic 40.4% Non-hallucination 3.4% · “I don't know” on 1.7% of questions Confidence: 155 answers kept at ≥99%, 149 right · 6 wrong · AUROC 0.746 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.023$22.70estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 50.4%48.3%–52.5% | 40.4% | 3.4% | 1.7% | 13.7 | $0.023$22.70estimate · RTX PRO 6000 Blackwell | 33 sRTX PRO 6000 | 0.028 sp90 0.032 s |
| 13 | Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Math 49.8% (95%: 47.7%–51.9%) · Logic 38.9% Non-hallucination 2.0% · “I don't know” on 1.0% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.730 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.099 s per question (median latency) | 49.8%47.7%–51.9% | 38.9% | 2.0% | 1.0% | 2.7 | $0.011$10.85 list$0.072$71.95estimate · RTX PRO 6000 Blackwell | 104 sRTX PRO 6000 | 0.099 sp90 0.127 s |
| 13 | Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Math 46.9% (95%: 44.8%–49.0%) · Logic 39.7% Non-hallucination 9.2% · “I don't know” on 5.1% of questions Confidence: 21 answers kept at ≥99%, 20 right · 1 wrong · AUROC 0.743 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.038$38.37estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.052 s per question (median latency) | 46.9%44.8%–49.0% | 39.7% | 9.2% | 5.1% | 8.4 | $0.038$38.37estimate · RTX PRO 6000 Blackwell | 55 sRTX PRO 6000 | 0.052 sp90 0.061 s |
| 14 | Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Math 44.3% (95%: 42.3%–46.4%) · Logic 39.3% Non-hallucination 4.9% · “I don't know” on 2.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.735 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.037 s per question (median latency) | 44.3%42.3%–46.4% | 39.3% | 4.9% | 2.8% | 5.9 | $0.024$23.94 list$0.028$27.50estimate · RTX PRO 6000 Blackwell | 40 sRTX PRO 6000 | 0.037 sp90 0.041 s |
| 15 | Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Math 39.2% (95%: 37.1%–41.2%) · Logic 32.1% Non-hallucination 2.2% · “I don't know” on 1.4% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.651 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.083$83.14estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.113 s per question (median latency) | 39.2%37.1%–41.2% | 32.1% | 2.2% | 1.4% | 3.3 | $0.083$83.14estimate · RTX PRO 6000 Blackwell | 120 sRTX PRO 6000 | 0.113 sp90 0.144 s |
| 15 | StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Math 38.7% (95%: 36.6%–40.7%) · Logic 30.8% Non-hallucination 7.2% · “I don't know” on 4.5% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.685 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.27estimate · DGX Spark per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 38.7%36.6%–40.7% | 30.8% | 7.2% | 4.5% | 2.6 | $0.002$2.27estimate · DGX Spark | 38 sDGX Spark | 0.036 sp90 0.037 s |
| 15 | d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Math 37.1% (95%: 35.1%–39.1%) · Logic 30.7% Non-hallucination 80.2% · “I don't know” on 66.3% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.656 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.014$14.07estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.016 s per question (median latency) | 37.1%35.1%–39.1% | 30.7% | 80.2% | 66.3% | 5.6 | $0.014$14.07estimate · RTX PRO 6000 Blackwell | 20 sRTX PRO 6000 | 0.016 sp90 0.021 s |
| 15 | GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Math 36.8% (95%: 34.7%–38.8%) · Logic 32.3% Non-hallucination 17.3% · “I don't know” on 11.4% of questions Confidence: 16 answers kept at ≥99%, 15 right · 1 wrong · AUROC 0.635 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.78estimate · DGX Spark per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 36.8%34.7%–38.8% | 32.3% | 17.3% | 11.4% | 17.2 | $0.002$1.78estimate · DGX Spark | 30 sDGX Spark | 0.028 sp90 0.030 s |
| 15 | KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Math 35.3% (95%: 33.3%–37.3%) · Logic 31.7% Non-hallucination 1.7% · “I don't know” on 1.1% of questions Confidence: 8 answers kept at ≥99%, 8 right · 0 wrong · AUROC 0.636 Runs on NVIDIA DGX Spark (GB10) · cost $0.003$3.20estimate · DGX Spark per 1,000 questionsper 1M questions · 0.050 s per question (median latency) | 35.3%33.3%–37.3% | 31.7% | 1.7% | 1.1% | 9.4 | $0.003$3.20estimate · DGX Spark | 54 sDGX Spark | 0.050 sp90 0.054 s |
| 16 | GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Math 32.1% (95%: 30.2%–34.1%) · Logic 29.2% Non-hallucination 20.8% · “I don't know” on 15.1% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.589 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.28estimate · DGX Spark per 1,000 questionsper 1M questions · 0.020 s per question (median latency) | 32.1%30.2%–34.1% | 29.2% | 20.8% | 15.1% | 18.8 | $0.001$1.28estimate · DGX Spark | 22 sDGX Spark | 0.020 sp90 0.021 s |
| 16 | StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Math 29.7% (95%: 27.8%–31.6%) · Logic 24.7% Non-hallucination 11.3% · “I don't know” on 8.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.583 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.53estimate · DGX Spark per 1,000 questionsper 1M questions · 0.024 s per question (median latency) | 29.7%27.8%–31.6% | 24.7% | 11.3% | 8.2% | 4.8 | $0.002$1.53estimate · DGX Spark | 26 sDGX Spark | 0.024 sp90 0.025 s |
| 16 | Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Math 28.7% (95%: 26.8%–30.6%) · Logic 22.4% Non-hallucination 3.2% · “I don't know” on 2.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.569 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.026$26.22estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.035 s per question (median latency) | 28.7%26.8%–30.6% | 22.4% | 3.2% | 2.3% | 8.8 | $0.026$26.22estimate · RTX PRO 6000 Blackwell | 38 sRTX PRO 6000 | 0.035 sp90 0.041 s |
| 16 | OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Math 28.2% (95%: 26.4%–30.2%) · Logic 27.0% Non-hallucination 0.6% · “I don't know” on 0.5% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.592 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.09estimate · DGX Spark per 1,000 questionsper 1M questions · 0.015 s per question (median latency) | 28.2%26.4%–30.2% | 27.0% | 0.6% | 0.5% | 10.0 | $0.001$1.09estimate · DGX Spark | 18 sDGX Spark | 0.015 sp90 0.017 s |
| 16 | KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Math 28.2% (95%: 26.3%–30.1%) · Logic 26.7% Non-hallucination 0.3% · “I don't know” on 0.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.596 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.69estimate · DGX Spark per 1,000 questionsper 1M questions · 0.025 s per question (median latency) | 28.2%26.3%–30.1% | 26.7% | 0.3% | 0.2% | 10.5 | $0.002$1.69estimate · DGX Spark | 29 sDGX Spark | 0.025 sp90 0.028 s |
| 17 | GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Math 22.7% (95%: 20.9%–24.5%) · Logic 23.6% Non-hallucination 17.2% · “I don't know” on 14.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.501 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.85estimate · DGX Spark per 1,000 questionsper 1M questions · 0.012 s per question (median latency) | 22.7%20.9%–24.5% | 23.6% | 17.2% | 14.2% | 7.0 | $0.001$0.85estimate · DGX Spark | 14 sDGX Spark | 0.012 sp90 0.013 s |
| 17 | GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Math 20.6% (95%: 18.9%–22.3%) · Logic 23.2% Non-hallucination 72.8% · “I don't know” on 67.5% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.500 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.77estimate · DGX Spark per 1,000 questionsper 1M questions · 0.011 s per question (median latency) | 20.6%18.9%–22.3% | 23.2% | 72.8% | 67.5% | 12.2 | $0.001$0.77estimate · DGX Spark | 13 sDGX Spark | 0.011 sp90 0.012 s |
No finished run of this kind on this part yet.
updated 16 Nov 2026 22:07 UTC
Not ranked (only complete runs are): Claude Fable 5.2 (chat) (not running now, 5 of 2,163 stored, last answer 11 Nov 12:02 UTC); Gemini 4 Carbon (high · chat) (paused for now at 54 of 2,163 (zeki #4681, 2026-11-11T11:53Z)); Granite-4.2-3B (off · chat) (not running now, 25 of 2,163 stored, last answer 16 Nov 20:02 UTC); Qwen3.5-397B-A17B (on · chat) (Logic only; on this test only its 25-question smoke test); GPT-6 Luna (max · chat) (paused for now at 200 of 2,163 (zeki #4129, 2026-11-10T14:13Z)).
Score: the share of the 2,163 Math questions answered right, the whisker its 95% interval. # is the tier: rows in one tier are not told apart by the paired test (a line separates tiers); – : with web or sample, not tiered.
| Tier | Model | Logic and Math accuracy 3,288 questions pooled, 95% interval; min–max for 3-seed rows | Logic · Math each part's accuracy | Non-hallucination of answers not right, % “I don't know” | “I don't know” share declined | Calibration error, points; decision models | $ per 1,000 q$ per 1M q list price or GPU time | GPU time s per 1,000 q, our hardware | Latency median, p90 under it |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.2%–99.0%) · Logic 96.7% · Math 99.6% Non-hallucination 60.9% · “I don't know” on 1.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.58 s per question (median latency) | 98.6%98.2%–99.0% | Logic 96.7% Math 99.6% | 60.9% | 1.3% | – | $0.000$0.00 | – | 5.58 sp90 9.58 s |
| 1 | Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.1%–98.9%) · Logic 96.7% · Math 99.6% Non-hallucination 53.3% · “I don't know” on 1.0% of questions Runs on an API · 16.4 s per question (median latency) | 98.6%98.1%–98.9% | Logic 96.7% Math 99.6% | 53.3% | 1.0% | – | – | – | 16.4 sp90 90.6 s |
| 1 | GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.1%–98.9%) · Logic 96.5% · Math 99.6% Non-hallucination 66.2% · “I don't know” on 1.4% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.24 s per question (median latency) | 98.6%98.1%–98.9% | Logic 96.5% Math 99.6% | 66.2% | 1.4% | – | $0.000$0.00 | – | 5.24 sp90 10.4 s |
| 1 | Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.1%–98.9%) · Logic 96.3% · Math 99.8% Non-hallucination 10.7% · “I don't know” on 0.2% of questions Runs on an API · cost $2.919$2,919 per 1,000 questionsper 1M questions · 5.70 s per question (median latency) | 98.6%98.1%–98.9% | Logic 96.3% Math 99.8% | 10.7% | 0.2% | – | $2.919$2,919 | – | 5.70 sp90 19.8 s |
| 1 | Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $4.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Both 98.5% (95%: 98.1%–98.9%) · Logic 96.2% · Math 99.8% Non-hallucination 41.3% · “I don't know” on 0.8% of questions Runs on an API · cost $13.179$13,179 API list-price equivalent per 1,000 questionsper 1M questions · 3.60 s per question (median latency) | 98.5%98.1%–98.9% | Logic 96.2% Math 99.8% | 41.3% | 0.8% | – | $13.179$13,179 API list-price equivalent | – | 3.60 sp90 8.38 s |
| 1 | DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active · Mixture of experts: 552B total, about 16B active per step. Size class uses 552B. Both 98.4% (95%: 98.0%–98.8%) · Logic 96.2% · Math 99.6% Non-hallucination 6.3% · “I don't know” on 0.1% of questions Runs on an API · cost $0.713$713 per 1,000 questionsper 1M questions · 2.27 s per question (median latency) | 98.4%98.0%–98.8% | Logic 96.2% Math 99.6% | 6.3% | 0.1% | – | $0.713$713 | – | 2.27 sp90 14.9 s |
| 1 | Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Both 98.4% (95%: 97.9%–98.8%) · Logic 96.1% · Math 99.6% Non-hallucination 78.3% · “I don't know” on 2.2% of questions Runs on an API · 4.92 s per question (median latency) | 98.4%97.9%–98.8% | Logic 96.1% Math 99.6% | 78.3% | 2.2% | – | – | – | 4.92 sp90 12.2 s |
| 1 | Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $2.00 per 1M input tokens: Workhorse (source: https://platform.claude.com/docs/en/about-claude/pricing) Both 98.2% (95%: 97.7%–98.6%) · Logic 95.5% · Math 99.7% Non-hallucination 43.7% · “I don't know” on 0.9% of questions Runs on an API · cost $7.042$7,042 API list-price equivalent per 1,000 questionsper 1M questions · 4.03 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.5% Math 99.7% | 43.7% | 0.9% | – | $7.042$7,042 API list-price equivalent | – | 4.03 sp90 10.1 s |
| 1 | Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active · Mixture of experts: 594B total, about 17B active per step. Size class uses 594B. Both 98.2% (95%: 97.7%–98.6%) · Logic 95.6% · Math 99.5% Non-hallucination 41.7% · “I don't know” on 0.9% of questions Runs on an API · cost $1.830$1,830 per 1,000 questionsper 1M questions · 2.87 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.6% Math 99.5% | 41.7% | 0.9% | – | $1.830$1,830 | – | 2.87 sp90 35.5 s |
| 1 | Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $10.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Both 98.2% (95%: 97.7%–98.6%) · Logic 95.2% · Math 99.7% Non-hallucination 15.7% · “I don't know” on 0.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.94 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.2% Math 99.7% | 15.7% | 0.3% | – | $0.000$0.00 | – | 4.94 sp90 8.62 s |
| 1 | Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active · Mixture of experts: 276B total, about 12B active per step. Size class uses 276B. Both 98.2% (95%: 97.7%–98.6%) · Logic 95.4% · Math 99.6% Non-hallucination 50.6% · “I don't know” on 1.3% of questions Runs on an API · cost $2.506$2,506 per 1,000 questionsper 1M questions · 2.75 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.4% Math 99.6% | 50.6% | 1.3% | – | $2.506$2,506 | – | 2.75 sp90 34.1 s |
| 1 | Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/meta/muse-ember) Both 98.2% (95%: 97.7%–98.6%) · Logic 95.5% · Math 99.6% Non-hallucination 31.3% · “I don't know” on 0.6% of questions Runs on an API · cost $0.863$863 per 1,000 questionsper 1M questions · 1.46 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.5% Math 99.6% | 31.3% | 0.6% | – | $0.863$863 | – | 1.46 sp90 6.30 s |
| 1 | MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active · Mixture of experts: 309B total, about 15B active per step. Size class uses 309B. Both 98.0% (95%: 97.4%–98.4%) · Logic 95.1% · Math 99.4% Non-hallucination 27.6% · “I don't know” on 0.7% of questions Runs on an API · cost $0.615$615 per 1,000 questionsper 1M questions · 10.6 s per question (median latency) | 98.0%97.4%–98.4% | Logic 95.1% Math 99.4% | 27.6% | 0.7% | – | $0.615$615 | – | 10.6 sp90 54.0 s |
| 2 | Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Both 97.6% (95%: 97.0%–98.1%) · Logic 94.0% · Math 99.5% Non-hallucination 31.3% · “I don't know” on 0.9% of questions Runs on an API · cost $0.732$732 per 1,000 questionsper 1M questions · 6.78 s per question (median latency) | 97.6%97.0%–98.1% | Logic 94.0% Math 99.5% | 31.3% | 0.9% | – | $0.732$732 | – | 6.78 sp90 36.3 s |
| 2 | Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Both 97.4% (95%: 96.8%–97.9%) · Logic 94.0% · Math 99.2% Non-hallucination 33.9% · “I don't know” on 1.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.020$2,020estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.2 s per question (median latency) | 97.4%96.8%–97.9% | Logic 94.0% Math 99.2% | 33.9% | 1.2% | – | $2.020$2,020estimate · RTX PRO 6000 Blackwell | 2,920 sRTX PRO 6000 | 11.2 sp90 39.6 s |
| 2 | GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Both 97.4% (95%: 96.8%–97.9%) · Logic 94.0% · Math 99.2% Non-hallucination 65.0% · “I don't know” on 2.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.44 s per question (median latency) | 97.4%96.8%–97.9% | Logic 94.0% Math 99.2% | 65.0% | 2.3% | – | $0.000$0.00 | – | 4.44 sp90 8.29 s |
| 2 | Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Both 97.3% (95%: 96.7%–97.8%) · Logic 93.3% · Math 99.4% Non-hallucination 17.0% · “I don't know” on 0.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.775$775estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 12.6 s per question (median latency) | 97.3%96.7%–97.8% | Logic 93.3% Math 99.4% | 17.0% | 0.6% | – | $0.775$775estimate · RTX PRO 6000 Blackwell | 1,120 sRTX PRO 6000 | 12.6 sp90 25.3 s |
| 2 | Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Both 97.0% (95%: 96.4%–97.5%) · Logic 92.4% · Math 99.4% Non-hallucination 34.2% · “I don't know” on 1.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.042$2,042estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 63.3 s per question (median latency) Other settings: Qwen3.5-4B (off · chat) | 97.0%96.4%–97.5% | Logic 92.4% Math 99.4% | 34.2% | 1.2% | – | $2.042$2,042estimate · RTX PRO 6000 Blackwell | 2,953 sRTX PRO 6000 | 63.3 sp90 193.9 s |
| Qwen3.5-4B off · chat Open · 4.7Bother setting of Qwen3.5-4B (on · chat) Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Both 95.6% (95%: 94.9%–96.3%) · Logic 90.1% · Math 98.5% Non-hallucination 34.0% · “I don't know” on 1.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.683$683estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 14.3 s per question (median latency) | 95.6%94.9%–96.3% | Logic 90.1% Math 98.5% | 34.0% | 1.7% | – | $0.683$683estimate · RTX PRO 6000 Blackwell | 988 sRTX PRO 6000 | 14.3 sp90 78.2 s | |
| 2 | GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active · Mixture of experts: 320B total, about 18B active per step. Size class uses 320B. Both 96.9% (95%: 96.2%–97.4%) · Logic 93.2% · Math 98.8% Non-hallucination 7.4% · “I don't know” on 0.2% of questions Runs on an API · cost $0.471$471 per 1,000 questionsper 1M questions · 4.64 s per question (median latency) | 96.9%96.2%–97.4% | Logic 93.2% Math 98.8% | 7.4% | 0.2% | – | $0.471$471 | – | 4.64 sp90 25.9 s |
| 2 | MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active · Mixture of experts: 428B total, about 23B active per step. Size class uses 428B. Both 96.8% (95%: 96.1%–97.3%) · Logic 94.0% · Math 98.2% Non-hallucination 38.9% · “I don't know” on 1.5% of questions Runs on an API · cost $0.342$342 per 1,000 questionsper 1M questions · 2.17 s per question (median latency) | 96.8%96.1%–97.3% | Logic 94.0% Math 98.2% | 38.9% | 1.5% | – | $0.342$342 | – | 2.17 sp90 15.8 s |
| 2 | Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Both 96.6% (95%: 96.0%–97.2%) · Logic 91.6% · Math 99.2% Non-hallucination 54.7% · “I don't know” on 2.5% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.241$4,241estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 25.6 s per question (median latency) | 96.6%96.0%–97.2% | Logic 91.6% Math 99.2% | 54.7% | 2.5% | – | $4.241$4,241estimate · RTX PRO 6000 Blackwell | 6,131 sRTX PRO 6000 | 25.6 sp90 523.1 s |
| 3 | MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Both 95.7% (mean of 3 seeds; min–max 95.4%–96.0%) · Logic 88.9% · Math 99.2% Seed runs: seed 0: 95.7% (3,145 of 3,288, run minicpm5-2b_en+minicpm5-2b_en) · seed 1: 95.4% (3,138 of 3,288, run minicpm5-2b-seed1_en+minicpm5-2b-seed1_en) · seed 2: 96.0% (3,155 of 3,288, run minicpm5-2b-seed2_en+minicpm5-2b-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b_en+minicpm5-2b_en, not the mean Non-hallucination 51.8% · “I don't know” on 3.0% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.410$1,410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 19.3 s per question (median latency) Other settings: MiniCPM5-2B (off · chat) | 95.7%95.4%–96.0% min–max of 3 seeds | Logic 88.9% Math 99.2% | 51.8%seed 0 only | 3.0%seed 0 only | – | $1.410$1,410estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 2,038 sRTX PRO 6000mean of 3 seeds | 19.3 sp90 194.9 smean of 3 seeds |
| MiniCPM5-2B off · chat Open · 2.5Bother setting of MiniCPM5-2B (on · chat) MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Both 93.2% (mean of 3 seeds; min–max 92.9%–93.3%) · Logic 83.6% · Math 98.2% Seed runs: seed 0: 93.3% (3,068 of 3,288, run minicpm5-2b-nothink_en+minicpm5-2b-nothink_en) · seed 1: 93.2% (3,066 of 3,288, run minicpm5-2b-nothink-seed1_en+minicpm5-2b-nothink-seed1_en) · seed 2: 92.9% (3,056 of 3,288, run minicpm5-2b-nothink-seed2_en+minicpm5-2b-nothink-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b-nothink_en+minicpm5-2b-nothink_en, not the mean Non-hallucination 41.0% · “I don't know” on 4.9% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.598$598estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.06 s per question (median latency) | 93.2%92.9%–93.3% min–max of 3 seeds | Logic 83.6% Math 98.2% | 41.0%seed 0 only | 4.9%seed 0 only | – | $0.598$598estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 865 sRTX PRO 6000mean of 3 seeds | 9.06 sp90 75.7 smean of 3 seeds | |
| 3 | Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Both 95.0% (95%: 94.2%–95.7%) · Logic 87.7% · Math 98.8% Non-hallucination 15.3% · “I don't know” on 0.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.749$749estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 22.3 s per question (median latency) | 95.0%94.2%–95.7% | Logic 87.7% Math 98.8% | 15.3% | 0.8% | – | $0.749$749estimate · RTX PRO 6000 Blackwell | 1,083 sRTX PRO 6000 | 22.3 sp90 53.4 s |
| 3 | LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active · Mixture of experts: 24.0B total, about 2.45B active per step. Size class uses 24.0B. Both 94.8% (95%: 94.0%–95.5%) · Logic 86.7% · Math 99.0% Non-hallucination 44.0% · “I don't know” on 2.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.205$3,205estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 42.4 s per question (median latency) | 94.8%94.0%–95.5% | Logic 86.7% Math 99.0% | 44.0% | 2.6% | – | $3.205$3,205estimate · RTX PRO 6000 Blackwell | 4,634 sRTX PRO 6000 | 42.4 sp90 432.5 s |
| 3 | LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Both 94.2% (95%: 93.3%–94.9%) · Logic 85.5% · Math 98.7% Non-hallucination 55.9% · “I don't know” on 4.2% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.206$206estimate · DGX Spark per 1,000 questionsper 1M questions · 29.0 s per question (median latency) Other settings: LFM3-1.7B (on · chat), LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell) | 94.2%93.3%–94.9% | Logic 85.5% Math 98.7% | 55.9% | 4.2% | – | $0.206$206estimate · DGX Spark | 3,484 sDGX Spark | 29.0 sp90 343.8 s |
| LFM3-1.7B on · chat Open · 1.7Bother setting of LFM3-1.7B (off · chat) LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Both 93.9% (mean of 3 seeds; min–max 93.89%–93.95%) · Logic 84.2% · Math 98.9% Seed runs: seed 0: 93.9% (3,089 of 3,288, run lfm3-1.7b-vllm_en+lfm3-1.7b-vllm_en) · seed 1: 93.9% (3,087 of 3,288, run lfm3-1.7b-vllm-seed1_en+lfm3-1.7b-vllm-seed1_en) · seed 2: 93.9% (3,087 of 3,288, run lfm3-1.7b-vllm-seed2_en+lfm3-1.7b-vllm-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm_en+lfm3-1.7b-vllm_en, not the mean Non-hallucination 46.7% · “I don't know” on 3.7% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.435$435estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 5.63 s per question (median latency) | 93.9%93.89%–93.95% min–max of 3 seeds | Logic 84.2% Math 98.9% | 46.7%seed 0 only | 3.7%seed 0 only | – | $0.435$435estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 629 sRTX PRO 6000mean of 3 seeds | 5.63 sp90 63.1 smean of 3 seeds | |
| LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7Bother setting of LFM3-1.7B (off · chat) LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Both 94.0% (mean of 3 seeds; min–max 93.8%–94.2%) · Logic 84.6% · Math 98.9% Seed runs: seed 0: 94.2% (3,098 of 3,288, run lfm3-1.7b-vllm-nothink-rtx-seed0_en+lfm3-1.7b-vllm-nothink-rtx-seed0_en) · seed 1: 93.8% (3,085 of 3,288, run lfm3-1.7b-vllm-nothink-rtx-seed1_en+lfm3-1.7b-vllm-nothink-rtx-seed1_en) · seed 2: 94.0% (3,091 of 3,288, run lfm3-1.7b-vllm-nothink-rtx-seed2_en+lfm3-1.7b-vllm-nothink-rtx-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm-nothink-rtx-seed0_en+lfm3-1.7b-vllm-nothink-rtx-seed0_en, not the mean Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.449$449estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 5.67 s per question (median latency) | 94.0%93.8%–94.2% min–max of 3 seeds | Logic 84.6% Math 98.9% | – | – | – | $0.449$449estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 649 sRTX PRO 6000mean of 3 seeds | 5.67 sp90 58.3 smean of 3 seeds | |
| 4 | Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Both 93.0% (95%: 92.1%–93.8%) · Logic 83.2% · Math 98.1% Non-hallucination 51.0% · “I don't know” on 4.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 13.2 s per question (median latency) Other settings: Gemma 4 E4B (on · chat), Gemma 4 E4B (on · chat · 32k cap), Gemma 4 E4B (on · chat · 32k cap · sampled) | 93.0%92.1%–93.8% | Logic 83.2% Math 98.1% | 51.0% | 4.7% | – | $0.467$467estimate · RTX PRO 6000 Blackwell | 675 sRTX PRO 6000 | 13.2 sp90 45.0 s |
| Gemma 4 E4B on · chat Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Both 92.9% (95%: 92.0%–93.7%) · Logic 82.6% · Math 98.2% Non-hallucination 43.7% · “I don't know” on 3.7% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.375$375estimate · DGX Spark per 1,000 questionsper 1M questions · 131.5 s per question (median latency) | 92.9%92.0%–93.7% | Logic 82.6% Math 98.2% | 43.7% | 3.7% | – | $0.375$375estimate · DGX Spark | 6,351 sDGX Spark | 131.5 sp90 392.6 s | |
| Gemma 4 E4B on · chat · 32k cap Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Both 93.9% (95%: 93.0%–94.7%) · Logic 85.0% · Math 98.5% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.855$855estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 25.5 s per question (median latency) | 93.9%93.0%–94.7% | Logic 85.0% Math 98.5% | – | – | – | $0.855$855estimate · RTX PRO 6000 Blackwell | 1,236 sRTX PRO 6000 | 25.5 sp90 77.4 s | |
| Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0Bother setting of Gemma 4 E4B (chat) Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Both 94.2% (mean of 3 seeds; min–max 94.0%–94.4%) · Logic 85.9% · Math 98.5% Seed runs: seed 0: 94.4% (3,104 of 3,288, run gemma-4-e4b-it-think-32k-sampled-seed0_en+gemma-4-e4b-it-think-32k-sampled-seed0_en) · seed 1: 94.3% (3,099 of 3,288, run gemma-4-e4b-it-think-32k-sampled-seed1_en+gemma-4-e4b-it-think-32k-sampled-seed1_en) · seed 2: 94.0% (3,090 of 3,288, run gemma-4-e4b-it-think-32k-sampled-seed2_en+gemma-4-e4b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e4b-it-think-32k-sampled-seed0_en+gemma-4-e4b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 54.1% · “I don't know” on 4.2% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.549$549estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 13.9 s per question (median latency) | 94.2%94.0%–94.4% min–max of 3 seeds | Logic 85.9% Math 98.5% | 54.1%seed 0 only | 4.2%seed 0 only | – | $0.549$549estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 793 sRTX PRO 6000mean of 3 seeds | 13.9 sp90 52.0 smean of 3 seeds | |
| 5 | Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 88.6% (95%: 87.5%–89.6%) · Logic 72.9% · Math 96.8% Non-hallucination 32.3% · “I don't know” on 3.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.313$313estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.93 s per question (median latency) Other settings: Gemma 4 E2B (on · chat), Gemma 4 E2B (on · chat · 32k cap), Gemma 4 E2B (on · chat · 32k cap · sampled) | 88.6%87.5%–89.6% | Logic 72.9% Math 96.8% | 32.3% | 3.9% | – | $0.313$313estimate · RTX PRO 6000 Blackwell | 452 sRTX PRO 6000 | 9.93 sp90 27.7 s |
| Gemma 4 E2B on · chat Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 89.3% (95%: 88.2%–90.3%) · Logic 74.6% · Math 96.9% Non-hallucination 24.3% · “I don't know” on 2.8% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.215$215estimate · DGX Spark per 1,000 questionsper 1M questions · 77.9 s per question (median latency) | 89.3%88.2%–90.3% | Logic 74.6% Math 96.9% | 24.3% | 2.8% | – | $0.215$215estimate · DGX Spark | 3,639 sDGX Spark | 77.9 sp90 232.8 s | |
| Gemma 4 E2B on · chat · 32k cap Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 90.3% (95%: 89.3%–91.3%) · Logic 75.9% · Math 97.8% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.678$678estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 19.1 s per question (median latency) | 90.3%89.3%–91.3% | Logic 75.9% Math 97.8% | – | – | – | $0.678$678estimate · RTX PRO 6000 Blackwell | 980 sRTX PRO 6000 | 19.1 sp90 54.6 s | |
| Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1Bother setting of Gemma 4 E2B (chat) Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 90.5% (mean of 3 seeds; min–max 90.2%–90.9%) · Logic 76.8% · Math 97.7% Seed runs: seed 0: 90.2% (2,966 of 3,288, run gemma-4-e2b-it-think-32k-sampled-seed0_en+gemma-4-e2b-it-think-32k-sampled-seed0_en) · seed 1: 90.9% (2,989 of 3,288, run gemma-4-e2b-it-think-32k-sampled-seed1_en+gemma-4-e2b-it-think-32k-sampled-seed1_en) · seed 2: 90.5% (2,974 of 3,288, run gemma-4-e2b-it-think-32k-sampled-seed2_en+gemma-4-e2b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e2b-it-think-32k-sampled-seed0_en+gemma-4-e2b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 31.0% · “I don't know” on 3.4% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.325$325estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.87 s per question (median latency) | 90.5%90.2%–90.9% min–max of 3 seeds | Logic 76.8% Math 97.7% | 31.0%seed 0 only | 3.4%seed 0 only | – | $0.325$325estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 470 sRTX PRO 6000mean of 3 seeds | 8.87 sp90 32.5 smean of 3 seeds | |
| 5 | LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Both 87.6% (95%: 86.5%–88.7%) · Logic 75.4% · Math 94.0% Non-hallucination 40.6% · “I don't know” on 5.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.046$1,046estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 14.0 s per question (median latency) | 87.6%86.5%–88.7% | Logic 75.4% Math 94.0% | 40.6% | 5.4% | – | $1.046$1,046estimate · RTX PRO 6000 Blackwell | 1,512 sRTX PRO 6000 | 14.0 sp90 143.0 s |
| 6 | Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Both 85.4% (95%: 84.1%–86.5%) · Logic 65.3% · Math 95.8% Non-hallucination 20.9% · “I don't know” on 3.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.840$840estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.38 s per question (median latency) | 85.4%84.1%–86.5% | Logic 65.3% Math 95.8% | 20.9% | 3.6% | – | $0.840$840estimate · RTX PRO 6000 Blackwell | 1,214 sRTX PRO 6000 | 9.38 sp90 35.1 s |
| 6 | Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Both 84.4% (95%: 83.1%–85.6%) · Logic 64.8% · Math 94.6% Non-hallucination 34.3% · “I don't know” on 6.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.788$788estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 17.8 s per question (median latency) Other settings: Qwen3.5-2B (on · chat) | 84.4%83.1%–85.6% | Logic 64.8% Math 94.6% | 34.3% | 6.1% | – | $0.788$788estimate · RTX PRO 6000 Blackwell | 1,139 sRTX PRO 6000 | 17.8 sp90 92.1 s |
| Qwen3.5-2B on · chat Open · 2.0Bother setting of Qwen3.5-2B (off · chat) Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Both 86.4% (95%: 85.2%–87.5%) · Logic 70.3% · Math 94.7% Non-hallucination 30.9% · “I don't know” on 3.5% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.106$3,106estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 121.8 s per question (median latency) | 86.4%85.2%–87.5% | Logic 70.3% Math 94.7% | 30.9% | 3.5% | – | $3.106$3,106estimate · RTX PRO 6000 Blackwell | 4,491 sRTX PRO 6000 | 121.8 sp90 241.6 s | |
| 6 | LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active · Mixture of experts: 8.3B total, about 1.5B active per step. Size class uses 8.3B. Both 83.9% (95%: 82.6%–85.1%) · Logic 60.2% · Math 96.3% Non-hallucination 55.9% · “I don't know” on 11.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.913$1,913estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 20.1 s† per question (median latency) | 83.9%82.6%–85.1% | Logic 60.2% Math 96.3% | 55.9% | 11.9% | – | $1.913$1,913estimate · RTX PRO 6000 Blackwell† | 2,766 sRTX PRO 6000 | 20.1 s†p90 230.6 s |
| 7 | GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/openai/gpt-6-luna) Both 81.4% (95%: 80.1%–82.7%) · Logic 58.0% · Math 93.7% Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.83 s per question (median latency) | 81.4%80.1%–82.7% | Logic 58.0% Math 93.7% | – | – | – | $0.000$0.00 | – | 3.83 sp90 5.30 s |
| 7 | Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Both 81.2% (95%: 79.8%–82.5%) · Logic 65.1% · Math 89.6% Non-hallucination 31.1% · “I don't know” on 5.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.422$4,422estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 42.6 s† per question (median latency) Other settings: Raon-1 nano (on · chat) | 81.2%79.8%–82.5% | Logic 65.1% Math 89.6% | 31.1% | 5.7% | – | $4.422$4,422estimate · RTX PRO 6000 Blackwell† | 6,394 sRTX PRO 6000 | 42.6 s†p90 594.0 s |
| Raon-1 nano on · chat Open · 3.1Bother setting of Raon-1 nano (off · chat) Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Both 88.1% (95%: 86.9%–89.1%) · Logic 70.9% · Math 97.0% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $6.344$6,344estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 55.0 s† per question (median latency) | 88.1%86.9%–89.1% | Logic 70.9% Math 97.0% | – | – | – | $6.344$6,344estimate · RTX PRO 6000 Blackwell† | 9,172 sRTX PRO 6000 | 55.0 s†p90 987.7 s | |
| 8 | Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Both 78.3% (95%: 76.8%–79.6%) · Logic 54.2% · Math 90.8% Non-hallucination 32.3% · “I don't know” on 8.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.308$308estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.74 s per question (median latency) | 78.3%76.8%–79.6% | Logic 54.2% Math 90.8% | 32.3% | 8.4% | – | $0.308$308estimate · RTX PRO 6000 Blackwell | 446 sRTX PRO 6000 | 8.74 sp90 16.4 s |
| 9 | Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Both 74.4% (95%: 72.9%–75.9%) · Logic 60.7% · Math 81.6% Non-hallucination 5.7% · “I don't know” on 1.5% of questions Confidence: 1,009 answers kept at ≥99%, 1,006 right · 3 wrong · AUROC 0.882 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.118$118estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.162 s per question (median latency) | 74.4%72.9%–75.9% | Logic 60.7% Math 81.6% | 5.7% | 1.5% | 2.7 | $0.118$118estimate · RTX PRO 6000 Blackwell | 171 sRTX PRO 6000 | 0.162 sp90 0.198 s |
| 9 | KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Both 73.0% (95%: 71.4%–74.5%) · Logic 58.9% · Math 80.3% Non-hallucination 0.1% · “I don't know” on 0.0% of questions Confidence: 524 answers kept at ≥99%, 523 right · 1 wrong · AUROC 0.882 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.093$92.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.116 s per question (median latency) | 73.0%71.4%–74.5% | Logic 58.9% Math 80.3% | 0.1% | 0.0% | 2.4 | $0.093$92.98estimate · RTX PRO 6000 Blackwell | 134 sRTX PRO 6000 | 0.116 sp90 0.177 s |
| 10 | REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense · Two models run together, option probabilities averaged 50/50: 57.36B in all, all used on every question. Both 71.6% (95%: 70.0%–73.1%) · Logic 59.3% · Math 78.0% Non-hallucination 3.8% · “I don't know” on 1.1% of questions Confidence: 229 answers kept at ≥99%, 229 right · 0 wrong · AUROC 0.878 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.148$148estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.200 s per question (median latency) | 71.6%70.0%–73.1% | Logic 59.3% Math 78.0% | 3.8% | 1.1% | 4.7 | $0.148$148estimate · RTX PRO 6000 Blackwell | 214 sRTX PRO 6000 | 0.200 sp90 0.260 s |
| 10 | KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Both 71.1% (95%: 69.5%–72.6%) · Logic 59.0% · Math 77.4% Non-hallucination 0.7% · “I don't know” on 0.2% of questions Confidence: 554 answers kept at ≥99%, 553 right · 1 wrong · AUROC 0.870 Runs on NVIDIA DGX Spark (GB10) · cost $0.052$51.85estimate · DGX Spark per 1,000 questionsper 1M questions · 0.811 s per question (median latency) | 71.1%69.5%–72.6% | Logic 59.0% Math 77.4% | 0.7% | 0.2% | 2.6 | $0.052$51.85estimate · DGX Spark | 877 sDGX Spark | 0.811 sp90 1.11 s |
| 10 | Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Both 71.0% (95%: 69.4%–72.5%) · Logic 58.7% · Math 77.3% Non-hallucination 1.8% · “I don't know” on 0.5% of questions Confidence: 384 answers kept at ≥99%, 384 right · 0 wrong · AUROC 0.873 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.102$102estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.128 s per question (median latency) | 71.0%69.4%–72.5% | Logic 58.7% Math 77.3% | 1.8% | 0.5% | 5.0 | $0.102$102estimate · RTX PRO 6000 Blackwell | 148 sRTX PRO 6000 | 0.128 sp90 0.218 s |
| 10 | Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Both 70.5% (95%: 68.9%–72.1%) · Logic 57.9% · Math 77.1% Non-hallucination 3.8% · “I don't know” on 1.1% of questions Confidence: 296 answers kept at ≥99%, 294 right · 2 wrong · AUROC 0.865 Runs on NVIDIA DGX Spark (GB10) · cost $0.047$47.29estimate · DGX Spark per 1,000 questionsper 1M questions · 0.676 s per question (median latency) | 70.5%68.9%–72.1% | Logic 57.9% Math 77.1% | 3.8% | 1.1% | 2.6 | $0.047$47.29estimate · DGX Spark | 800 sDGX Spark | 0.676 sp90 1.17 s |
| 11 | Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Both 69.3% (95%: 67.7%–70.8%) · Logic 57.8% · Math 75.2% Non-hallucination 4.4% · “I don't know” on 1.4% of questions Confidence: 23 answers kept at ≥99%, 23 right · 0 wrong · AUROC 0.868 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.165$165estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.207 s per question (median latency) | 69.3%67.7%–70.8% | Logic 57.8% Math 75.2% | 4.4% | 1.4% | 6.0 | $0.165$165estimate · RTX PRO 6000 Blackwell | 238 sRTX PRO 6000 | 0.207 sp90 0.324 s |
| 11 | d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://www.liquid.ai/blog/d1-ultra) Both 67.5% (95%: 65.9%–69.1%) · Logic 54.9% · Math 74.1% Non-hallucination 89.1% · “I don't know” on 47.7% of questions Confidence: 233 answers kept at ≥99%, 233 right · 0 wrong · AUROC 0.852 Runs on an API · cost $0.008$8.37 per 1,000 questionsper 1M questions · 0.180 s per question (median latency) | 67.5%65.9%–69.1% | Logic 54.9% Math 74.1% | 89.1% | 47.7% | 2.1 | $0.008$8.37 | – | 0.180 sp90 0.426 s |
| 11 | GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Both 67.3% (95%: 65.7%–68.9%) · Logic 58.4% · Math 72.0% Non-hallucination 31.9% · “I don't know” on 11.5% of questions Confidence: 274 answers kept at ≥99%, 272 right · 2 wrong · AUROC 0.852 Runs on NVIDIA DGX Spark (GB10) · cost $0.029$28.55estimate · DGX Spark per 1,000 questionsper 1M questions · 0.450 s per question (median latency) | 67.3%65.7%–68.9% | Logic 58.4% Math 72.0% | 31.9% | 11.5% | 4.1 | $0.029$28.55estimate · DGX Spark | 483 sDGX Spark | 0.450 sp90 0.587 s |
| 11 | Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Both 67.3% (95%: 65.7%–68.9%) · Logic 54.5% · Math 74.0% Non-hallucination 71.4% · “I don't know” on 30.6% of questions Confidence: 520 answers kept at ≥99%, 518 right · 2 wrong · AUROC 0.868 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.150 s per question (median latency) | 67.3%65.7%–68.9% | Logic 54.5% Math 74.0% | 71.4% | 30.6% | 3.2 | $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell | 173 sRTX PRO 6000 | 0.150 sp90 0.232 s |
| 12 | Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Both 66.4% (95%: 64.8%–68.0%) · Logic 54.8% · Math 72.5% Non-hallucination 13.5% · “I don't know” on 4.9% of questions Confidence: 385 answers kept at ≥99%, 385 right · 0 wrong · AUROC 0.844 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.071$71.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.097 s per question (median latency) | 66.4%64.8%–68.0% | Logic 54.8% Math 72.5% | 13.5% | 4.9% | 5.9 | $0.071$71.10estimate · RTX PRO 6000 Blackwell | 103 sRTX PRO 6000 | 0.097 sp90 0.125 s |
| 12 | d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Both 65.7% (95%: 64.1%–67.3%) · Logic 54.6% · Math 71.5% Non-hallucination 7.5% · “I don't know” on 2.6% of questions Confidence: 39 answers kept at ≥99%, 39 right · 0 wrong · AUROC 0.835 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.096$95.71estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.129 s per question (median latency) | 65.7%64.1%–67.3% | Logic 54.6% Math 71.5% | 7.5% | 2.6% | 10.1 | $0.096$95.71estimate · RTX PRO 6000 Blackwell | 138 sRTX PRO 6000 | 0.129 sp90 0.168 s |
| 12 | GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker · Closed; list price $0.15 per 1M input tokens: Efficient (source: https://docs.fastino.ai/pricing) Both 65.2% (95%: 63.6%–66.8%) · Logic 54.5% · Math 70.8% Non-hallucination 49.2% · “I don't know” on 20.5% of questions Confidence: 213 answers kept at ≥99%, 213 right · 0 wrong · AUROC 0.852 Runs on an API · cost $0.036$36.42 per 1,000 questionsper 1M questions · 0.266 s per question (median latency) | 65.2%63.6%–66.8% | Logic 54.5% Math 70.8% | 49.2% | 20.5% | 5.3 | $0.036$36.42 | – | 0.266 sp90 0.284 s |
| 12 | d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Both 65.0% (95%: 63.3%–66.6%) · Logic 54.5% · Math 70.4% Non-hallucination 9.3% · “I don't know” on 3.4% of questions Confidence: 217 answers kept at ≥99%, 216 right · 1 wrong · AUROC 0.858 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.046$46.28estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.057 s per question (median latency) | 65.0%63.3%–66.6% | Logic 54.5% Math 70.4% | 9.3% | 3.4% | 4.4 | $0.046$46.28estimate · RTX PRO 6000 Blackwell | 67 sRTX PRO 6000 | 0.057 sp90 0.086 s |
| 12 | Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker · Closed; introductory list price (the only one published) $0.03 per 1M input tokens: Efficient (provisional) (source: https://nace.ai/drex) Both 64.8% (95%: 63.2%–66.5%) · Logic 54.8% · Math 70.1% Non-hallucination 48.3% · “I don't know” on 20.8% of questions Confidence: 405 answers kept at ≥99%, 404 right · 1 wrong · AUROC 0.846 Runs on an API · cost $0.016$16.13 per 1,000 questionsper 1M questions · 0.327 s per question (median latency) | 64.8%63.2%–66.5% | Logic 54.8% Math 70.1% | 48.3% | 20.8% | 5.8 | $0.016$16.13 | – | 0.327 sp90 0.518 s |
| 12 | GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Both 64.8% (95%: 63.2%–66.5%) · Logic 55.2% · Math 69.9% Non-hallucination 42.3% · “I don't know” on 17.2% of questions Confidence: 176 answers kept at ≥99%, 176 right · 0 wrong · AUROC 0.843 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.081 s per question (median latency) | 64.8%63.2%–66.5% | Logic 55.2% Math 69.9% | 42.3% | 17.2% | 5.1 | $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell | 95 sRTX PRO 6000 | 0.081 sp90 0.136 s |
| 12 | Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Both 64.4% (95%: 62.8%–66.0%) · Logic 53.1% · Math 70.3% Non-hallucination 5.7% · “I don't know” on 2.1% of questions Confidence: 884 answers kept at ≥99%, 866 right · 18 wrong · AUROC 0.841 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.081 s per question (median latency) | 64.4%62.8%–66.0% | Logic 53.1% Math 70.3% | 5.7% | 2.1% | 13.3 | $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell | 92 sRTX PRO 6000 | 0.081 sp90 0.130 s |
| 12 | Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Both 64.1% (95%: 62.5%–65.8%) · Logic 53.3% · Math 69.8% Non-hallucination 10.3% · “I don't know” on 3.9% of questions Confidence: 411 answers kept at ≥99%, 409 right · 2 wrong · AUROC 0.849 Runs on an API · cost $0.005$5.38 per 1,000 questionsper 1M questions · 0.230 s per question (median latency) Other settings: Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) | 64.1%62.5%–65.8% | Logic 53.3% Math 69.8% | 10.3% | 3.9% | 3.1 | $0.005$5.38 | – | 0.230 sp90 0.439 s |
| Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell Open · 27Bother setting of Perplexity Decider 27B v1.2 (API) Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Both 64.5% (95%: 62.8%–66.1%) · Logic 53.2% · Math 70.3% Non-hallucination 9.5% · “I don't know” on 3.5% of questions Confidence: 467 answers kept at ≥99%, 465 right · 2 wrong · AUROC 0.848 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.005$5.44 list$0.082$81.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.098 s per question (median latency) Shown under Perplexity Decider 27B v1.2 (API): the same model name, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings | 64.5%62.8%–66.1% | Logic 53.2% Math 70.3% | 9.5% | 3.5% | 3.3 | $0.005$5.44 list$0.082$81.83estimate · RTX PRO 6000 Blackwell | 118 sRTX PRO 6000 | 0.098 sp90 0.166 s | |
| 12 | Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/google/gemini-decision-1) Both 64.1% (95%: 62.4%–65.7%) · Logic 50.0% · Math 71.3% Non-hallucination 3.0% · “I don't know” on 1.1% of questions Confidence: 257 answers kept at ≥99%, 257 right · 0 wrong · AUROC 0.838 Runs on an API · cost $0.027$27.07 per 1,000 questionsper 1M questions · 0.227 s per question (median latency) | 64.1%62.4%–65.7% | Logic 50.0% Math 71.3% | 3.0% | 1.1% | 2.0 | $0.027$27.07 | – | 0.227 sp90 0.347 s |
| 12 | d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Both 63.6% (95%: 61.9%–65.2%) · Logic 51.6% · Math 69.9% Non-hallucination 90.2% · “I don't know” on 51.9% of questions Confidence: 128 answers kept at ≥99%, 128 right · 0 wrong · AUROC 0.856 Runs on an API · cost $0.008$8.37 per 1,000 questionsper 1M questions · 0.217 s per question (median latency) | 63.6%61.9%–65.2% | Logic 51.6% Math 69.9% | 90.2% | 51.9% | 3.2 | $0.008$8.37 | – | 0.217 sp90 0.347 s |
| – | Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Not tiered: with web Both 63.6% (95%: 61.9%–65.2%) · Logic 52.5% · Math 69.3% Non-hallucination 25.8% · “I don't know” on 9.9% of questions Confidence: 405 answers kept at ≥99%, 404 right · 1 wrong · AUROC 0.854 Runs on an API · cost $1.025$1,025 per 1,000 questionsper 1M questions · 0.430 s per question (median latency) | 63.6%61.9%–65.2% | Logic 52.5% Math 69.3% | 25.8% | 9.9% | 3.7 | $1.025$1,025 | – | 0.430 sp90 0.707 s |
| 12 | Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Both 63.3% (95%: 61.6%–64.9%) · Logic 52.3% · Math 69.0% Non-hallucination 1.9% · “I don't know” on 0.7% of questions Confidence: 50 answers kept at ≥99%, 50 right · 0 wrong · AUROC 0.844 Runs on NVIDIA DGX Spark (GB10) · cost $0.020$20.01estimate · DGX Spark per 1,000 questionsper 1M questions · 0.301 s per question (median latency) | 63.3%61.6%–64.9% | Logic 52.3% Math 69.0% | 1.9% | 0.7% | 3.5 | $0.020$20.01estimate · DGX Spark | 339 sDGX Spark | 0.301 sp90 0.448 s |
| 13 | SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Both 63.0% (95%: 61.4%–64.7%) · Logic 52.4% · Math 68.6% Non-hallucination 58.3% · “I don't know” on 26.8% of questions Confidence: 433 answers kept at ≥99%, 430 right · 3 wrong · AUROC 0.827 Runs on an API · cost $0.011$10.75 per 1,000 questionsper 1M questions · 0.382 s per question (median latency) Shown once with SubQ Snap Preview: the same answer to every question | 63.0%61.4%–64.7% | Logic 52.4% Math 68.6% | 58.3% | 26.8% | 10.5 | $0.011$10.75 | – | 0.382 sp90 0.391 s |
| 13 | celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Both 62.2% (95%: 60.6%–63.9%) · Logic 47.3% · Math 70.0% Non-hallucination 87.2% · “I don't know” on 49.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.829 Runs on an API · cost $0.011$11.25 per 1,000 questionsper 1M questions · 0.388 s per question (median latency) | 62.2%60.6%–63.9% | Logic 47.3% Math 70.0% | 87.2% | 49.8% | 5.8 | $0.011$11.25 | – | 0.388 sp90 0.392 s |
| 13 | celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.004 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Both 62.1% (95%: 60.5%–63.8%) · Logic 47.3% · Math 69.9% Non-hallucination 86.7% · “I don't know” on 49.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.830 Runs on an API · cost $0.001$1.13 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) | 62.1%60.5%–63.8% | Logic 47.3% Math 69.9% | 86.7% | 49.2% | 6.0 | $0.001$1.13 | – | 0.381 sp90 0.388 s |
| 13 | KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Both 60.9% (95%: 59.2%–62.5%) · Logic 50.6% · Math 66.3% Non-hallucination 4.5% · “I don't know” on 1.8% of questions Confidence: 13 answers kept at ≥99%, 13 right · 0 wrong · AUROC 0.810 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.051 s per question (median latency) | 60.9%59.2%–62.5% | Logic 50.6% Math 66.3% | 4.5% | 1.8% | 2.9 | $0.041$41.44estimate · RTX PRO 6000 Blackwell | 60 sRTX PRO 6000 | 0.051 sp90 0.076 s |
| 13 | d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Both 59.9% (95%: 58.2%–61.6%) · Logic 49.2% · Math 65.5% Non-hallucination 66.9% · “I don't know” on 34.9% of questions Confidence: 169 answers kept at ≥99%, 169 right · 0 wrong · AUROC 0.812 Runs on an API · cost $0.004$3.98 per 1,000 questionsper 1M questions · 0.208 s per question (median latency) | 59.9%58.2%–61.6% | Logic 49.2% Math 65.5% | 66.9% | 34.9% | 2.5 | $0.004$3.98 | – | 0.208 sp90 0.390 s |
| 14 | Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://docs.typesafe.ai/models) Both 59.3% (95%: 57.6%–61.0%) · Logic 48.2% · Math 65.1% Non-hallucination 15.4% · “I don't know” on 6.7% of questions Confidence: 263 answers kept at ≥99%, 263 right · 0 wrong · AUROC 0.810 Runs on an API · cost $0.021$21.27 per 1,000 questionsper 1M questions · 0.164 s per question (median latency) | 59.3%57.6%–61.0% | Logic 48.2% Math 65.1% | 15.4% | 6.7% | 1.8 | $0.021$21.27 | – | 0.164 sp90 0.188 s |
| 14 | d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Both 58.6% (95%: 56.9%–60.3%) · Logic 46.0% · Math 65.2% Non-hallucination 86.0% · “I don't know” on 53.5% of questions Confidence: 231 answers kept at ≥99%, 231 right · 0 wrong · AUROC 0.811 Runs on an API · cost $0.004$3.98 per 1,000 questionsper 1M questions · 0.174 s per question (median latency) | 58.6%56.9%–60.3% | Logic 46.0% Math 65.2% | 86.0% | 53.5% | 0.8 | $0.004$3.98 | – | 0.174 sp90 0.285 s |
| 14 | Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://typesafe.ai/blog/introducing-system-one-models-and-jev) Both 58.0% (95%: 56.3%–59.7%) · Logic 46.3% · Math 64.1% Non-hallucination 10.2% · “I don't know” on 4.5% of questions Confidence: 313 answers kept at ≥99%, 313 right · 0 wrong · AUROC 0.817 Runs on an API · cost $0.021$21.27 per 1,000 questionsper 1M questions · 0.166 s per question (median latency) | 58.0%56.3%–59.7% | Logic 46.3% Math 64.1% | 10.2% | 4.5% | 3.2 | $0.021$21.27 | – | 0.166 sp90 0.205 s |
| 14 | Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 56.7% (95%: 55.0%–58.4%) · Logic 46.4% · Math 62.1% Non-hallucination 7.0% · “I don't know” on 3.1% of questions Confidence: 253 answers kept at ≥99%, 251 right · 2 wrong · AUROC 0.802 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.028$28.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 56.7%55.0%–58.4% | Logic 46.4% Math 62.1% | 7.0% | 3.1% | 5.6 | $0.028$28.19estimate · RTX PRO 6000 Blackwell | 41 sRTX PRO 6000 | 0.038 sp90 0.045 s |
| 14 | Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Both 56.2% (95%: 54.5%–57.9%) · Logic 46.4% · Math 61.3% Non-hallucination 69.4% · “I don't know” on 39.7% of questions Confidence: 236 answers kept at ≥99%, 235 right · 1 wrong · AUROC 0.797 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.080 s per question (median latency) | 56.2%54.5%–57.9% | Logic 46.4% Math 61.3% | 69.4% | 39.7% | 5.6 | $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.080 sp90 0.094 s |
| 14 | Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 55.8% (95%: 54.1%–57.5%) · Logic 48.2% · Math 59.8% Non-hallucination 13.5% · “I don't know” on 6.3% of questions Confidence: 74 answers kept at ≥99%, 74 right · 0 wrong · AUROC 0.798 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.032$32.40estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.039 s per question (median latency) | 55.8%54.1%–57.5% | Logic 48.2% Math 59.8% | 13.5% | 6.3% | 3.6 | $0.032$32.40estimate · RTX PRO 6000 Blackwell | 47 sRTX PRO 6000 | 0.039 sp90 0.053 s |
| 15 | KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 54.6% (95%: 52.9%–56.3%) · Logic 43.9% · Math 60.1% Non-hallucination 10.4% · “I don't know” on 5.0% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.786 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.034$34.08estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.042 s per question (median latency) | 54.6%52.9%–56.3% | Logic 43.9% Math 60.1% | 10.4% | 5.0% | 3.6 | $0.034$34.08estimate · RTX PRO 6000 Blackwell | 49 sRTX PRO 6000 | 0.042 sp90 0.057 s |
| 15 | GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Both 54.5% (95%: 52.8%–56.2%) · Logic 47.8% · Math 57.9% Non-hallucination 26.9% · “I don't know” on 13.5% of questions Confidence: 105 answers kept at ≥99%, 105 right · 0 wrong · AUROC 0.786 Runs on NVIDIA DGX Spark (GB10) · cost $0.011$11.45estimate · DGX Spark per 1,000 questionsper 1M questions · 0.177 s per question (median latency) | 54.5%52.8%–56.2% | Logic 47.8% Math 57.9% | 26.9% | 13.5% | 6.7 | $0.011$11.45estimate · DGX Spark | 194 sDGX Spark | 0.177 sp90 0.222 s |
| 15 | KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 54.5% (95%: 52.8%–56.2%) · Logic 44.3% · Math 59.8% Non-hallucination 3.4% · “I don't know” on 1.6% of questions Confidence: 153 answers kept at ≥99%, 153 right · 0 wrong · AUROC 0.782 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.030$29.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 54.5%52.8%–56.2% | Logic 44.3% Math 59.8% | 3.4% | 1.6% | 2.2 | $0.030$29.87estimate · RTX PRO 6000 Blackwell | 43 sRTX PRO 6000 | 0.038 sp90 0.055 s |
| 15 | StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Both 54.4% (95%: 52.7%–56.1%) · Logic 46.6% · Math 58.4% Non-hallucination 9.6% · “I don't know” on 4.6% of questions Confidence: 63 answers kept at ≥99%, 63 right · 0 wrong · AUROC 0.779 Runs on NVIDIA DGX Spark (GB10) · cost $0.009$8.92estimate · DGX Spark per 1,000 questionsper 1M questions · 0.142 s per question (median latency) | 54.4%52.7%–56.1% | Logic 46.6% Math 58.4% | 9.6% | 4.6% | 2.7 | $0.009$8.92estimate · DGX Spark | 151 sDGX Spark | 0.142 sp90 0.165 s |
| 15 | Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://openrouter.ai/microsoft/microsoft-decision-1) Base model: Qwen3.5-9B Both 54.3% (95%: 52.6%–56.0%) · Logic 44.4% · Math 59.4% Non-hallucination 11.4% · “I don't know” on 5.5% of questions Confidence: 47 answers kept at ≥99%, 47 right · 0 wrong · AUROC 0.769 Runs on an API · cost $0.009$9.07 per 1,000 questionsper 1M questions · 0.312 s per question (median latency) | 54.3%52.6%–56.0% | Logic 44.4% Math 59.4% | 11.4% | 5.5% | 4.9 | $0.009$9.07 | – | 0.312 sp90 0.360 s |
| 15 | Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Both 54.2% (95%: 52.5%–55.9%) · Logic 45.3% · Math 58.8% Non-hallucination 6.1% · “I don't know” on 2.9% of questions Confidence: 266 answers kept at ≥99%, 263 right · 3 wrong · AUROC 0.795 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.033$32.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.046 s per question (median latency) | 54.2%52.5%–55.9% | Logic 45.3% Math 58.8% | 6.1% | 2.9% | 13.1 | $0.033$32.61estimate · RTX PRO 6000 Blackwell | 47 sRTX PRO 6000 | 0.046 sp90 0.051 s |
| 15 | Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active · Mixture of experts: 35B total, about 3B active per step. Size class uses 35B. Both 53.1% (95%: 51.4%–54.8%) · Logic 41.9% · Math 59.0% Non-hallucination 28.1% · “I don't know” on 14.7% of questions Confidence: 207 answers kept at ≥99%, 207 right · 0 wrong · AUROC 0.789 Runs on an API · cost $0.012$12.17 per 1,000 questionsper 1M questions · 0.339 s per question (median latency) | 53.1%51.4%–54.8% | Logic 41.9% Math 59.0% | 28.1% | 14.7% | 1.1 | $0.012$12.17 | – | 0.339 sp90 0.472 s |
| 15 | vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Both 52.5% (95%: 50.8%–54.2%) · Logic 44.1% · Math 56.8% Non-hallucination 7.9% · “I don't know” on 3.9% of questions Confidence: 39 answers kept at ≥99%, 39 right · 0 wrong · AUROC 0.777 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.024$24.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.031 s per question (median latency) | 52.5%50.8%–54.2% | Logic 44.1% Math 56.8% | 7.9% | 3.9% | 3.5 | $0.024$24.19estimate · RTX PRO 6000 Blackwell | 35 sRTX PRO 6000 | 0.031 sp90 0.036 s |
| 15 | StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Both 52.4% (95%: 50.7%–54.1%) · Logic 45.9% · Math 55.8% Non-hallucination 14.4% · “I don't know” on 7.4% of questions Confidence: 117 answers kept at ≥99%, 117 right · 0 wrong · AUROC 0.763 Runs on NVIDIA DGX Spark (GB10) · cost $0.004$4.28estimate · DGX Spark per 1,000 questionsper 1M questions · 0.068 s per question (median latency) | 52.4%50.7%–54.1% | Logic 45.9% Math 55.8% | 14.4% | 7.4% | 5.4 | $0.004$4.28estimate · DGX Spark | 72 sDGX Spark | 0.068 sp90 0.080 s |
| 15 | Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Both 52.1% (95%: 50.4%–53.8%) · Logic 44.7% · Math 55.9% Non-hallucination 15.0% · “I don't know” on 7.6% of questions Confidence: 131 answers kept at ≥99%, 130 right · 1 wrong · AUROC 0.782 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.024$23.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.029 s per question (median latency) | 52.1%50.4%–53.8% | Logic 44.7% Math 55.9% | 15.0% | 7.6% | 11.1 | $0.024$23.77estimate · RTX PRO 6000 Blackwell | 34 sRTX PRO 6000 | 0.029 sp90 0.035 s |
| 15 | Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Both 51.7% (95%: 50.0%–53.4%) · Logic 43.3% · Math 56.0% Non-hallucination 3.1% · “I don't know” on 1.5% of questions Confidence: 281 answers kept at ≥99%, 270 right · 11 wrong · AUROC 0.776 Runs on NVIDIA DGX Spark (GB10) · cost $0.011$10.82estimate · DGX Spark per 1,000 questionsper 1M questions · 0.163 s per question (median latency) | 51.7%50.0%–53.4% | Logic 43.3% Math 56.0% | 3.1% | 1.5% | 16.2 | $0.011$10.82estimate · DGX Spark | 183 sDGX Spark | 0.163 sp90 0.240 s |
| 15 | Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 51.6% (95%: 49.9%–53.3%) · Logic 44.4% · Math 55.3% Non-hallucination 7.1% · “I don't know” on 3.6% of questions Confidence: 167 answers kept at ≥99%, 165 right · 2 wrong · AUROC 0.765 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.059 s per question (median latency) | 51.6%49.9%–53.3% | Logic 44.4% Math 55.3% | 7.1% | 3.6% | 4.5 | $0.041$41.44estimate · RTX PRO 6000 Blackwell | 60 sRTX PRO 6000 | 0.059 sp90 0.065 s |
| 15 | KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Both 51.3% (95%: 49.6%–53.0%) · Logic 42.3% · Math 56.0% Non-hallucination 4.4% · “I don't know” on 2.2% of questions Confidence: 72 answers kept at ≥99%, 71 right · 1 wrong · AUROC 0.752 Runs on NVIDIA DGX Spark (GB10) · cost $0.005$5.02estimate · DGX Spark per 1,000 questionsper 1M questions · 0.076 s per question (median latency) | 51.3%49.6%–53.0% | Logic 42.3% Math 56.0% | 4.4% | 2.2% | 10.4 | $0.005$5.02estimate · DGX Spark | 85 sDGX Spark | 0.076 sp90 0.104 s |
| 15 | Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Both 51.1% (95%: 49.4%–52.8%) · Logic 43.1% · Math 55.2% Non-hallucination 18.9% · “I don't know” on 10.0% of questions Confidence: 190 answers kept at ≥99%, 185 right · 5 wrong · AUROC 0.769 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.041 s per question (median latency) | 51.1%49.4%–52.8% | Logic 43.1% Math 55.2% | 18.9% | 10.0% | 8.9 | $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell | 46 sRTX PRO 6000 | 0.041 sp90 0.051 s |
| 16 | Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://openrouter.ai/inception/mercury-decide) Both 50.6% (95%: 48.9%–52.3%) · Logic 45.1% · Math 53.5% Non-hallucination 10.1% · “I don't know” on 5.1% of questions Confidence: 372 answers kept at ≥99%, 356 right · 16 wrong · AUROC 0.777 Runs on an API · cost $0.005$5.46 per 1,000 questionsper 1M questions · 0.293 s per question (median latency) | 50.6%48.9%–52.3% | Logic 45.1% Math 53.5% | 10.1% | 5.1% | 15.7 | $0.005$5.46 | – | 0.293 sp90 0.349 s |
| 16 | d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Both 50.5% (95%: 48.8%–52.3%) · Logic 40.8% · Math 55.6% Non-hallucination 6.4% · “I don't know” on 3.3% of questions Confidence: 208 answers kept at ≥99%, 207 right · 1 wrong · AUROC 0.757 Runs on an API · cost $0.004$3.98 per 1,000 questionsper 1M questions · 0.185 s per question (median latency) | 50.5%48.8%–52.3% | Logic 40.8% Math 55.6% | 6.4% | 3.3% | 5.8 | $0.004$3.98 | – | 0.185 sp90 0.292 s |
| 16 | Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Both 50.5% (95%: 48.7%–52.2%) · Logic 42.6% · Math 54.6% Non-hallucination 14.5% · “I don't know” on 7.6% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.759 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.212$212estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.250 s per question (median latency) | 50.5%48.7%–52.2% | Logic 42.6% Math 54.6% | 14.5% | 7.6% | 18.3 | $0.212$212estimate · RTX PRO 6000 Blackwell | 307 sRTX PRO 6000 | 0.250 sp90 0.482 s |
| 16 | OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Both 49.3% (95%: 47.6%–51.0%) · Logic 40.6% · Math 53.8% Non-hallucination 1.6% · “I don't know” on 0.8% of questions Confidence: 30 answers kept at ≥99%, 30 right · 0 wrong · AUROC 0.770 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.069$69.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.080 s per question (median latency) | 49.3%47.6%–51.0% | Logic 40.6% Math 53.8% | 1.6% | 0.8% | 3.1 | $0.069$69.42estimate · RTX PRO 6000 Blackwell | 100 sRTX PRO 6000 | 0.080 sp90 0.134 s |
| 16 | Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Both 47.6% (95%: 45.9%–49.3%) · Logic 40.8% · Math 51.1% Non-hallucination 2.4% · “I don't know” on 1.2% of questions Confidence: 18 answers kept at ≥99%, 18 right · 0 wrong · AUROC 0.747 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.185 s per question (median latency) | 47.6%45.9%–49.3% | Logic 40.8% Math 51.1% | 2.4% | 1.2% | 12.6 | $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell | 195 sRTX PRO 6000 | 0.185 sp90 0.216 s |
| 16 | Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Both 47.5% (95%: 45.8%–49.2%) · Logic 39.7% · Math 51.5% Non-hallucination 8.9% · “I don't know” on 4.8% of questions Confidence: 165 answers kept at ≥99%, 161 right · 4 wrong · AUROC 0.747 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.033$33.45estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.046 s per question (median latency) | 47.5%45.8%–49.2% | Logic 39.7% Math 51.5% | 8.9% | 4.8% | 7.3 | $0.033$33.45estimate · RTX PRO 6000 Blackwell | 48 sRTX PRO 6000 | 0.046 sp90 0.052 s |
| 16 | Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Both 47.0% (95%: 45.3%–48.7%) · Logic 40.4% · Math 50.4% Non-hallucination 7.8% · “I don't know” on 4.2% of questions Confidence: 194 answers kept at ≥99%, 188 right · 6 wrong · AUROC 0.743 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.025$25.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.030 s per question (median latency) | 47.0%45.3%–48.7% | Logic 40.4% Math 50.4% | 7.8% | 4.2% | 14.3 | $0.025$25.03estimate · RTX PRO 6000 Blackwell | 36 sRTX PRO 6000 | 0.030 sp90 0.042 s |
| 17 | Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Both 46.1% (95%: 44.4%–47.8%) · Logic 38.9% · Math 49.8% Non-hallucination 6.9% · “I don't know” on 3.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.707 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.103 s per question (median latency) | 46.1%44.4%–47.8% | Logic 38.9% Math 49.8% | 6.9% | 3.8% | 5.7 | $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell | 109 sRTX PRO 6000 | 0.103 sp90 0.131 s |
| 17 | Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Both 44.4% (95%: 42.7%–46.1%) · Logic 39.7% · Math 46.9% Non-hallucination 17.1% · “I don't know” on 10.2% of questions Confidence: 53 answers kept at ≥99%, 52 right · 1 wrong · AUROC 0.737 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.043$42.91estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.055 s per question (median latency) | 44.4%42.7%–46.1% | Logic 39.7% Math 46.9% | 17.1% | 10.2% | 10.5 | $0.043$42.91estimate · RTX PRO 6000 Blackwell | 62 sRTX PRO 6000 | 0.055 sp90 0.072 s |
| 17 | Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Both 42.6% (95%: 40.9%–44.3%) · Logic 39.3% · Math 44.3% Non-hallucination 7.3% · “I don't know” on 4.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.729 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 42.6%40.9%–44.3% | Logic 39.3% Math 44.3% | 7.3% | 4.3% | 7.4 | $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell | 41 sRTX PRO 6000 | 0.038 sp90 0.044 s |
| 18 | Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Both 36.7% (95%: 35.1%–38.4%) · Logic 32.1% · Math 39.2% Non-hallucination 2.0% · “I don't know” on 1.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.659 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.086$85.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.117 s per question (median latency) | 36.7%35.1%–38.4% | Logic 32.1% Math 39.2% | 2.0% | 1.3% | 2.2 | $0.086$85.83estimate · RTX PRO 6000 Blackwell | 124 sRTX PRO 6000 | 0.117 sp90 0.146 s |
| 18 | StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Both 35.9% (95%: 34.3%–37.6%) · Logic 30.8% · Math 38.7% Non-hallucination 16.4% · “I don't know” on 11.1% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.675 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.36estimate · DGX Spark per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 35.9%34.3%–37.6% | Logic 30.8% Math 38.7% | 16.4% | 11.1% | 3.3 | $0.002$2.36estimate · DGX Spark | 40 sDGX Spark | 0.036 sp90 0.042 s |
| 18 | GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Both 35.2% (95%: 33.6%–36.9%) · Logic 32.3% · Math 36.8% Non-hallucination 23.1% · “I don't know” on 15.9% of questions Confidence: 30 answers kept at ≥99%, 29 right · 1 wrong · AUROC 0.637 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.94estimate · DGX Spark per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 35.2%33.6%–36.9% | Logic 32.3% Math 36.8% | 23.1% | 15.9% | 16.8 | $0.002$1.94estimate · DGX Spark | 33 sDGX Spark | 0.028 sp90 0.037 s |
| 18 | d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Both 34.9% (95%: 33.3%–36.5%) · Logic 30.7% · Math 37.1% Non-hallucination 82.5% · “I don't know” on 70.4% of questions Confidence: 7 answers kept at ≥99%, 7 right · 0 wrong · AUROC 0.648 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.014$14.30estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.017 s per question (median latency) | 34.9%33.3%–36.5% | Logic 30.7% Math 37.1% | 82.5% | 70.4% | 5.9 | $0.014$14.30estimate · RTX PRO 6000 Blackwell | 21 sRTX PRO 6000 | 0.017 sp90 0.021 s |
| 18 | KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Both 34.1% (95%: 32.5%–35.7%) · Logic 31.7% · Math 35.3% Non-hallucination 6.8% · “I don't know” on 4.7% of questions Confidence: 16 answers kept at ≥99%, 16 right · 0 wrong · AUROC 0.626 Runs on NVIDIA DGX Spark (GB10) · cost $0.003$3.40estimate · DGX Spark per 1,000 questionsper 1M questions · 0.051 s per question (median latency) | 34.1%32.5%–35.7% | Logic 31.7% Math 35.3% | 6.8% | 4.7% | 9.0 | $0.003$3.40estimate · DGX Spark | 57 sDGX Spark | 0.051 sp90 0.063 s |
| 19 | GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Both 31.1% (95%: 29.5%–32.7%) · Logic 29.2% · Math 32.1% Non-hallucination 22.7% · “I don't know” on 16.8% of questions Confidence: 13 answers kept at ≥99%, 13 right · 0 wrong · AUROC 0.602 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.38estimate · DGX Spark per 1,000 questionsper 1M questions · 0.020 s per question (median latency) | 31.1%29.5%–32.7% | Logic 29.2% Math 32.1% | 22.7% | 16.8% | 17.1 | $0.001$1.38estimate · DGX Spark | 23 sDGX Spark | 0.020 sp90 0.025 s |
| 19 | StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Both 28.0% (95%: 26.5%–29.5%) · Logic 24.7% · Math 29.7% Non-hallucination 14.9% · “I don't know” on 11.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.605 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.62estimate · DGX Spark per 1,000 questionsper 1M questions · 0.024 s per question (median latency) | 28.0%26.5%–29.5% | Logic 24.7% Math 29.7% | 14.9% | 11.1% | 5.8 | $0.002$1.62estimate · DGX Spark | 27 sDGX Spark | 0.024 sp90 0.029 s |
| 19 | OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Both 27.8% (95%: 26.3%–29.4%) · Logic 27.0% · Math 28.2% Non-hallucination 2.0% · “I don't know” on 1.5% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.576 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.11estimate · DGX Spark per 1,000 questionsper 1M questions · 0.015 s per question (median latency) | 27.8%26.3%–29.4% | Logic 27.0% Math 28.2% | 2.0% | 1.5% | 10.4 | $0.001$1.11estimate · DGX Spark | 19 sDGX Spark | 0.015 sp90 0.017 s |
| 19 | KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Both 27.7% (95%: 26.2%–29.2%) · Logic 26.7% · Math 28.2% Non-hallucination 0.6% · “I don't know” on 0.5% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.590 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.80estimate · DGX Spark per 1,000 questionsper 1M questions · 0.026 s per question (median latency) | 27.7%26.2%–29.2% | Logic 26.7% Math 28.2% | 0.6% | 0.5% | 9.8 | $0.002$1.80estimate · DGX Spark | 30 sDGX Spark | 0.026 sp90 0.035 s |
| 20 | Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Both 26.5% (95%: 25.0%–28.1%) · Logic 22.4% · Math 28.7% Non-hallucination 5.0% · “I don't know” on 3.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.573 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.027$26.72estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.035 s per question (median latency) | 26.5%25.0%–28.1% | Logic 22.4% Math 28.7% | 5.0% | 3.7% | 9.9 | $0.027$26.72estimate · RTX PRO 6000 Blackwell | 39 sRTX PRO 6000 | 0.035 sp90 0.042 s |
| 20 | GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Both 23.0% (95%: 21.6%–24.4%) · Logic 23.6% · Math 22.7% Non-hallucination 15.6% · “I don't know” on 12.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.511 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.01estimate · DGX Spark per 1,000 questionsper 1M questions · 0.012 s per question (median latency) | 23.0%21.6%–24.4% | Logic 23.6% Math 22.7% | 15.6% | 12.7% | 7.6 | $0.001$1.01estimate · DGX Spark | 17 sDGX Spark | 0.012 sp90 0.023 s |
| 21 | GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Both 21.5% (95%: 20.1%–22.9%) · Logic 23.2% · Math 20.6% Non-hallucination 73.9% · “I don't know” on 68.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.520 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.83estimate · DGX Spark per 1,000 questionsper 1M questions · 0.011 s per question (median latency) | 21.5%20.1%–22.9% | Logic 23.2% Math 20.6% | 73.9% | 68.3% | 11.0 | $0.001$0.83estimate · DGX Spark | 14 sDGX Spark | 0.011 sp90 0.014 s |
| Tier | Model | Logic and Math accuracy 3,288 questions pooled, 95% interval; min–max for 3-seed rows | Logic · Math each part's accuracy | Non-hallucination of answers not right, % “I don't know” | “I don't know” share declined | Calibration error, points; decision models | $ per 1,000 q$ per 1M q list price or GPU time | GPU time s per 1,000 q, our hardware | Latency median, p90 under it |
|---|---|---|---|---|---|---|---|---|---|
| 1 | GPT-6 Astra chat Closed · Undisclosed GPT-6 Astra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.2%–99.0%) · Logic 96.7% · Math 99.6% Non-hallucination 60.9% · “I don't know” on 1.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.58 s per question (median latency) | 98.6%98.2%–99.0% | Logic 96.7% Math 99.6% | 60.9% | 1.3% | – | $0.000$0.00 | – | 5.58 sp90 9.58 s |
| 1 | Gemini 4 Carbon chat Closed · Undisclosed Gemini 4 Carbon chat · Google · chat model Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.1%–98.9%) · Logic 96.7% · Math 99.6% Non-hallucination 53.3% · “I don't know” on 1.0% of questions Runs on an API · 16.4 s per question (median latency) | 98.6%98.1%–98.9% | Logic 96.7% Math 99.6% | 53.3% | 1.0% | – | – | – | 16.4 sp90 90.6 s |
| 1 | GPT-6.1 Sol chat Closed · Undisclosed GPT-6.1 Sol chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.1%–98.9%) · Logic 96.5% · Math 99.6% Non-hallucination 66.2% · “I don't know” on 1.4% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 5.24 s per question (median latency) | 98.6%98.1%–98.9% | Logic 96.5% Math 99.6% | 66.2% | 1.4% | – | $0.000$0.00 | – | 5.24 sp90 10.4 s |
| 1 | Qwen4-Max high · chat Closed · Undisclosed Qwen4-Max high · chat · Alibaba · chat model · reasoning high Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Both 98.6% (95%: 98.1%–98.9%) · Logic 96.3% · Math 99.8% Non-hallucination 10.7% · “I don't know” on 0.2% of questions Runs on an API · cost $2.919$2,919 per 1,000 questionsper 1M questions · 5.70 s per question (median latency) | 98.6%98.1%–98.9% | Logic 96.3% Math 99.8% | 10.7% | 0.2% | – | $2.919$2,919 | – | 5.70 sp90 19.8 s |
| 1 | Claude Opus 5.5 chat Closed · Flagship Claude Opus 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $4.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Both 98.5% (95%: 98.1%–98.9%) · Logic 96.2% · Math 99.8% Non-hallucination 41.3% · “I don't know” on 0.8% of questions Runs on an API · cost $13.179$13,179 API list-price equivalent per 1,000 questionsper 1M questions · 3.60 s per question (median latency) | 98.5%98.1%–98.9% | Logic 96.2% Math 99.8% | 41.3% | 0.8% | – | $13.179$13,179 API list-price equivalent | – | 3.60 sp90 8.38 s |
| 1 | DeepSeek-V4.1-Flash high · chat · 1017 Open · 552B MoE · 16B active DeepSeek-V4.1-Flash high · chat · 1017 · DeepSeek · chat model · reasoning high Parameters: 552B MoE · 16B active · Mixture of experts: 552B total, about 16B active per step. Size class uses 552B. Both 98.4% (95%: 98.0%–98.8%) · Logic 96.2% · Math 99.6% Non-hallucination 6.3% · “I don't know” on 0.1% of questions Runs on an API · cost $0.713$713 per 1,000 questionsper 1M questions · 2.27 s per question (median latency) | 98.4%98.0%–98.8% | Logic 96.2% Math 99.6% | 6.3% | 0.1% | – | $0.713$713 | – | 2.27 sp90 14.9 s |
| 1 | Gemini 4 Hydrogen chat Gemini 4 Hydrogen chat · Google · chat model Parameters: not on record yet Both 98.4% (95%: 97.9%–98.8%) · Logic 96.1% · Math 99.6% Non-hallucination 78.3% · “I don't know” on 2.2% of questions Runs on an API · 4.92 s per question (median latency) | 98.4%97.9%–98.8% | Logic 96.1% Math 99.6% | 78.3% | 2.2% | – | – | – | 4.92 sp90 12.2 s |
| 1 | Claude Sonnet 5.5 chat Closed · Workhorse Claude Sonnet 5.5 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $2.00 per 1M input tokens: Workhorse (source: https://platform.claude.com/docs/en/about-claude/pricing) Both 98.2% (95%: 97.7%–98.6%) · Logic 95.5% · Math 99.7% Non-hallucination 43.7% · “I don't know” on 0.9% of questions Runs on an API · cost $7.042$7,042 API list-price equivalent per 1,000 questionsper 1M questions · 4.03 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.5% Math 99.7% | 43.7% | 0.9% | – | $7.042$7,042 API list-price equivalent | – | 4.03 sp90 10.1 s |
| 1 | Inkling 2 Small high · chat Open · 594B MoE · 17B active Inkling 2 Small high · chat · Thinking Machines · chat model · reasoning high Parameters: 594B MoE · 17B active · Mixture of experts: 594B total, about 17B active per step. Size class uses 594B. Both 98.2% (95%: 97.7%–98.6%) · Logic 95.6% · Math 99.5% Non-hallucination 41.7% · “I don't know” on 0.9% of questions Runs on an API · cost $1.830$1,830 per 1,000 questionsper 1M questions · 2.87 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.6% Math 99.5% | 41.7% | 0.9% | – | $1.830$1,830 | – | 2.87 sp90 35.5 s |
| 1 | Claude Fable 5.1 chat Closed · Flagship Claude Fable 5.1 chat · Anthropic · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $10.00 per 1M input tokens: Flagship (source: https://platform.claude.com/docs/en/about-claude/pricing) Both 98.2% (95%: 97.7%–98.6%) · Logic 95.2% · Math 99.7% Non-hallucination 15.7% · “I don't know” on 0.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.94 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.2% Math 99.7% | 15.7% | 0.3% | – | $0.000$0.00 | – | 4.94 sp90 8.62 s |
| 1 | Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model · reasoning high Parameters: 276B MoE · 12B active · Mixture of experts: 276B total, about 12B active per step. Size class uses 276B. Both 98.2% (95%: 97.7%–98.6%) · Logic 95.4% · Math 99.6% Non-hallucination 50.6% · “I don't know” on 1.3% of questions Runs on an API · cost $2.506$2,506 per 1,000 questionsper 1M questions · 2.75 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.4% Math 99.6% | 50.6% | 1.3% | – | $2.506$2,506 | – | 2.75 sp90 34.1 s |
| 1 | Muse Ember max · chat Closed · Efficient Muse Ember max · chat · Meta · chat model · reasoning max Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/meta/muse-ember) Both 98.2% (95%: 97.7%–98.6%) · Logic 95.5% · Math 99.6% Non-hallucination 31.3% · “I don't know” on 0.6% of questions Runs on an API · cost $0.863$863 per 1,000 questionsper 1M questions · 1.46 s per question (median latency) | 98.2%97.7%–98.6% | Logic 95.5% Math 99.6% | 31.3% | 0.6% | – | $0.863$863 | – | 1.46 sp90 6.30 s |
| 1 | MiMo-V2.6-Flash on · chat Open · 309B MoE · 15B active MiMo-V2.6-Flash on · chat · Xiaomi · chat model · reasoning on Parameters: 309B MoE · 15B active · Mixture of experts: 309B total, about 15B active per step. Size class uses 309B. Both 98.0% (95%: 97.4%–98.4%) · Logic 95.1% · Math 99.4% Non-hallucination 27.6% · “I don't know” on 0.7% of questions Runs on an API · cost $0.615$615 per 1,000 questionsper 1M questions · 10.6 s per question (median latency) | 98.0%97.4%–98.4% | Logic 95.1% Math 99.4% | 27.6% | 0.7% | – | $0.615$615 | – | 10.6 sp90 54.0 s |
| 2 | Qwen3.8 Flash off · chat Closed · Undisclosed Qwen3.8 Flash off · chat · Alibaba · chat model · reasoning off Parameters: not disclosed by maker (closed weights) · Closed; no list price published: Undisclosed Both 97.6% (95%: 97.0%–98.1%) · Logic 94.0% · Math 99.5% Non-hallucination 31.3% · “I don't know” on 0.9% of questions Runs on an API · cost $0.732$732 per 1,000 questionsper 1M questions · 6.78 s per question (median latency) | 97.6%97.0%–98.1% | Logic 94.0% Math 99.5% | 31.3% | 0.9% | – | $0.732$732 | – | 6.78 sp90 36.3 s |
| 2 | Qwen3.8-27B low · chat Open · 27B Qwen3.8-27B low · chat · Alibaba · chat model · reasoning low Parameters: 27B dense Both 97.4% (95%: 96.8%–97.9%) · Logic 94.0% · Math 99.2% Non-hallucination 33.9% · “I don't know” on 1.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.020$2,020estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 11.2 s per question (median latency) | 97.4%96.8%–97.9% | Logic 94.0% Math 99.2% | 33.9% | 1.2% | – | $2.020$2,020estimate · RTX PRO 6000 Blackwell | 2,920 sRTX PRO 6000 | 11.2 sp90 39.6 s |
| 2 | GPT-6.1 Terra chat Closed · Undisclosed GPT-6.1 Terra chat · OpenAI · chat model Parameters: not on record yet · Closed; no list price published: Undisclosed Both 97.4% (95%: 96.8%–97.9%) · Logic 94.0% · Math 99.2% Non-hallucination 65.0% · “I don't know” on 2.3% of questions Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 4.44 s per question (median latency) | 97.4%96.8%–97.9% | Logic 94.0% Math 99.2% | 65.0% | 2.3% | – | $0.000$0.00 | – | 4.44 sp90 8.29 s |
| 2 | Gemma 4 31B chat Open · 31B Gemma 4 31B chat · Google · chat model Parameters: 30.7B dense Both 97.3% (95%: 96.7%–97.8%) · Logic 93.3% · Math 99.4% Non-hallucination 17.0% · “I don't know” on 0.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.775$775estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 12.6 s per question (median latency) | 97.3%96.7%–97.8% | Logic 93.3% Math 99.4% | 17.0% | 0.6% | – | $0.775$775estimate · RTX PRO 6000 Blackwell | 1,120 sRTX PRO 6000 | 12.6 sp90 25.3 s |
| 2 | Qwen3.5-4B on · chat Open · 4.7B Qwen3.5-4B on · chat · Alibaba · chat model Parameters: 4.66B dense Both 97.0% (95%: 96.4%–97.5%) · Logic 92.4% · Math 99.4% Non-hallucination 34.2% · “I don't know” on 1.2% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $2.042$2,042estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 63.3 s per question (median latency) | 97.0%96.4%–97.5% | Logic 92.4% Math 99.4% | 34.2% | 1.2% | – | $2.042$2,042estimate · RTX PRO 6000 Blackwell | 2,953 sRTX PRO 6000 | 63.3 sp90 193.9 s |
| 2 | GLM-5.3-Flash high · chat Open · 320B MoE · 18B active GLM-5.3-Flash high · chat · Z.ai · chat model · reasoning high Parameters: 320B MoE · 18B active · Mixture of experts: 320B total, about 18B active per step. Size class uses 320B. Both 96.9% (95%: 96.2%–97.4%) · Logic 93.2% · Math 98.8% Non-hallucination 7.4% · “I don't know” on 0.2% of questions Runs on an API · cost $0.471$471 per 1,000 questionsper 1M questions · 4.64 s per question (median latency) | 96.9%96.2%–97.4% | Logic 93.2% Math 98.8% | 7.4% | 0.2% | – | $0.471$471 | – | 4.64 sp90 25.9 s |
| 2 | MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active MiniMax-M3.1-Flash high · chat · MiniMax · chat model · reasoning high Parameters: 428B MoE · 23B active · Mixture of experts: 428B total, about 23B active per step. Size class uses 428B. Both 96.8% (95%: 96.1%–97.3%) · Logic 94.0% · Math 98.2% Non-hallucination 38.9% · “I don't know” on 1.5% of questions Runs on an API · cost $0.342$342 per 1,000 questionsper 1M questions · 2.17 s per question (median latency) | 96.8%96.1%–97.3% | Logic 94.0% Math 98.2% | 38.9% | 1.5% | – | $0.342$342 | – | 2.17 sp90 15.8 s |
| 2 | Granite-4.2-3B high · chat Open · 3.0B Granite-4.2-3B high · chat · IBM · chat model · reasoning high Parameters: 3B dense Base model: Granite-4.1-3B-Base Both 96.6% (95%: 96.0%–97.2%) · Logic 91.6% · Math 99.2% Non-hallucination 54.7% · “I don't know” on 2.5% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.241$4,241estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 25.6 s per question (median latency) | 96.6%96.0%–97.2% | Logic 91.6% Math 99.2% | 54.7% | 2.5% | – | $4.241$4,241estimate · RTX PRO 6000 Blackwell | 6,131 sRTX PRO 6000 | 25.6 sp90 523.1 s |
| 3 | MiniCPM5-2B on · chat Open · 2.5B MiniCPM5-2B on · chat · OpenBMB · chat model Parameters: 2.52B dense Both 95.7% (mean of 3 seeds; min–max 95.4%–96.0%) · Logic 88.9% · Math 99.2% Seed runs: seed 0: 95.7% (3,145 of 3,288, run minicpm5-2b_en+minicpm5-2b_en) · seed 1: 95.4% (3,138 of 3,288, run minicpm5-2b-seed1_en+minicpm5-2b-seed1_en) · seed 2: 96.0% (3,155 of 3,288, run minicpm5-2b-seed2_en+minicpm5-2b-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b_en+minicpm5-2b_en, not the mean Non-hallucination 51.8% · “I don't know” on 3.0% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.410$1,410estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 19.3 s per question (median latency) | 95.7%95.4%–96.0% min–max of 3 seeds | Logic 88.9% Math 99.2% | 51.8%seed 0 only | 3.0%seed 0 only | – | $1.410$1,410estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 2,038 sRTX PRO 6000mean of 3 seeds | 19.3 sp90 194.9 smean of 3 seeds |
| 3 | Qwen3.5-4B off · chat Open · 4.7B Qwen3.5-4B off · chat · Alibaba · chat model · reasoning off Parameters: 4.66B dense Both 95.6% (95%: 94.9%–96.3%) · Logic 90.1% · Math 98.5% Non-hallucination 34.0% · “I don't know” on 1.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.683$683estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 14.3 s per question (median latency) | 95.6%94.9%–96.3% | Logic 90.1% Math 98.5% | 34.0% | 1.7% | – | $0.683$683estimate · RTX PRO 6000 Blackwell | 988 sRTX PRO 6000 | 14.3 sp90 78.2 s |
| 3 | Gemma 4 12B Unified chat Open · 12B Gemma 4 12B Unified chat · Google · chat model Parameters: 11.95B dense Both 95.0% (95%: 94.2%–95.7%) · Logic 87.7% · Math 98.8% Non-hallucination 15.3% · “I don't know” on 0.8% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.749$749estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 22.3 s per question (median latency) | 95.0%94.2%–95.7% | Logic 87.7% Math 98.8% | 15.3% | 0.8% | – | $0.749$749estimate · RTX PRO 6000 Blackwell | 1,083 sRTX PRO 6000 | 22.3 sp90 53.4 s |
| 3 | LFM2.5-24B-A2B on · chat Open · 24B MoE · 2.5B active LFM2.5-24B-A2B on · chat · Liquid AI · chat model Parameters: 24.0B MoE · 2.45B active · Mixture of experts: 24.0B total, about 2.45B active per step. Size class uses 24.0B. Both 94.8% (95%: 94.0%–95.5%) · Logic 86.7% · Math 99.0% Non-hallucination 44.0% · “I don't know” on 2.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.205$3,205estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 42.4 s per question (median latency) | 94.8%94.0%–95.5% | Logic 86.7% Math 99.0% | 44.0% | 2.6% | – | $3.205$3,205estimate · RTX PRO 6000 Blackwell | 4,634 sRTX PRO 6000 | 42.4 sp90 432.5 s |
| 3 | Gemma 4 E4B on · chat · 32k cap · sampled Open · 8.0B Gemma 4 E4B on · chat · 32k cap · sampled · Google · chat model Parameters: 8B dense · 4.5B effective Both 94.2% (mean of 3 seeds; min–max 94.0%–94.4%) · Logic 85.9% · Math 98.5% Seed runs: seed 0: 94.4% (3,104 of 3,288, run gemma-4-e4b-it-think-32k-sampled-seed0_en+gemma-4-e4b-it-think-32k-sampled-seed0_en) · seed 1: 94.3% (3,099 of 3,288, run gemma-4-e4b-it-think-32k-sampled-seed1_en+gemma-4-e4b-it-think-32k-sampled-seed1_en) · seed 2: 94.0% (3,090 of 3,288, run gemma-4-e4b-it-think-32k-sampled-seed2_en+gemma-4-e4b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e4b-it-think-32k-sampled-seed0_en+gemma-4-e4b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 54.1% · “I don't know” on 4.2% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.549$549estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 13.9 s per question (median latency) | 94.2%94.0%–94.4% min–max of 3 seeds | Logic 85.9% Math 98.5% | 54.1%seed 0 only | 4.2%seed 0 only | – | $0.549$549estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 793 sRTX PRO 6000mean of 3 seeds | 13.9 sp90 52.0 smean of 3 seeds |
| 3 | LFM3-1.7B off · chat Open · 1.7B LFM3-1.7B off · chat · chat model · reasoning off Parameters: 1.73B dense Both 94.2% (95%: 93.3%–94.9%) · Logic 85.5% · Math 98.7% Non-hallucination 55.9% · “I don't know” on 4.2% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.206$206estimate · DGX Spark per 1,000 questionsper 1M questions · 29.0 s per question (median latency) | 94.2%93.3%–94.9% | Logic 85.5% Math 98.7% | 55.9% | 4.2% | – | $0.206$206estimate · DGX Spark | 3,484 sDGX Spark | 29.0 sp90 343.8 s |
| 3 | LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell Open · 1.7B LFM3-1.7B off · chat · NVIDIA RTX PRO 6000 Blackwell · chat model · reasoning off Parameters: 1.73B dense Both 94.0% (mean of 3 seeds; min–max 93.8%–94.2%) · Logic 84.6% · Math 98.9% Seed runs: seed 0: 94.2% (3,098 of 3,288, run lfm3-1.7b-vllm-nothink-rtx-seed0_en+lfm3-1.7b-vllm-nothink-rtx-seed0_en) · seed 1: 93.8% (3,085 of 3,288, run lfm3-1.7b-vllm-nothink-rtx-seed1_en+lfm3-1.7b-vllm-nothink-rtx-seed1_en) · seed 2: 94.0% (3,091 of 3,288, run lfm3-1.7b-vllm-nothink-rtx-seed2_en+lfm3-1.7b-vllm-nothink-rtx-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm-nothink-rtx-seed0_en+lfm3-1.7b-vllm-nothink-rtx-seed0_en, not the mean Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.449$449estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 5.67 s per question (median latency) | 94.0%93.8%–94.2% min–max of 3 seeds | Logic 84.6% Math 98.9% | – | – | – | $0.449$449estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 649 sRTX PRO 6000mean of 3 seeds | 5.67 sp90 58.3 smean of 3 seeds |
| 4 | LFM3-1.7B on · chat Open · 1.7B LFM3-1.7B on · chat · chat model Parameters: 1.73B dense Both 93.9% (mean of 3 seeds; min–max 93.89%–93.95%) · Logic 84.2% · Math 98.9% Seed runs: seed 0: 93.9% (3,089 of 3,288, run lfm3-1.7b-vllm_en+lfm3-1.7b-vllm_en) · seed 1: 93.9% (3,087 of 3,288, run lfm3-1.7b-vllm-seed1_en+lfm3-1.7b-vllm-seed1_en) · seed 2: 93.9% (3,087 of 3,288, run lfm3-1.7b-vllm-seed2_en+lfm3-1.7b-vllm-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run lfm3-1.7b-vllm_en+lfm3-1.7b-vllm_en, not the mean Non-hallucination 46.7% · “I don't know” on 3.7% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.435$435estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 5.63 s per question (median latency) | 93.9%93.89%–93.95% min–max of 3 seeds | Logic 84.2% Math 98.9% | 46.7%seed 0 only | 3.7%seed 0 only | – | $0.435$435estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 629 sRTX PRO 6000mean of 3 seeds | 5.63 sp90 63.1 smean of 3 seeds |
| 4 | Gemma 4 E4B on · chat · 32k cap Open · 8.0B Gemma 4 E4B on · chat · 32k cap · Google · chat model Parameters: 8B dense · 4.5B effective Both 93.9% (95%: 93.0%–94.7%) · Logic 85.0% · Math 98.5% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.855$855estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 25.5 s per question (median latency) | 93.9%93.0%–94.7% | Logic 85.0% Math 98.5% | – | – | – | $0.855$855estimate · RTX PRO 6000 Blackwell | 1,236 sRTX PRO 6000 | 25.5 sp90 77.4 s |
| 4 | MiniCPM5-2B off · chat Open · 2.5B MiniCPM5-2B off · chat · OpenBMB · chat model · reasoning off Parameters: 2.52B dense Both 93.2% (mean of 3 seeds; min–max 92.9%–93.3%) · Logic 83.6% · Math 98.2% Seed runs: seed 0: 93.3% (3,068 of 3,288, run minicpm5-2b-nothink_en+minicpm5-2b-nothink_en) · seed 1: 93.2% (3,066 of 3,288, run minicpm5-2b-nothink-seed1_en+minicpm5-2b-nothink-seed1_en) · seed 2: 92.9% (3,056 of 3,288, run minicpm5-2b-nothink-seed2_en+minicpm5-2b-nothink-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run minicpm5-2b-nothink_en+minicpm5-2b-nothink_en, not the mean Non-hallucination 41.0% · “I don't know” on 4.9% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.598$598estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.06 s per question (median latency) | 93.2%92.9%–93.3% min–max of 3 seeds | Logic 83.6% Math 98.2% | 41.0%seed 0 only | 4.9%seed 0 only | – | $0.598$598estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 865 sRTX PRO 6000mean of 3 seeds | 9.06 sp90 75.7 smean of 3 seeds |
| 4 | Gemma 4 E4B chat Open · 8.0B Gemma 4 E4B chat · Google · chat model Parameters: 8B dense · 4.5B effective Both 93.0% (95%: 92.1%–93.8%) · Logic 83.2% · Math 98.1% Non-hallucination 51.0% · “I don't know” on 4.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.467$467estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 13.2 s per question (median latency) | 93.0%92.1%–93.8% | Logic 83.2% Math 98.1% | 51.0% | 4.7% | – | $0.467$467estimate · RTX PRO 6000 Blackwell | 675 sRTX PRO 6000 | 13.2 sp90 45.0 s |
| 4 | Gemma 4 E4B on · chat Open · 8.0B Gemma 4 E4B on · chat · Google · chat model Parameters: 8B dense · 4.5B effective Both 92.9% (95%: 92.0%–93.7%) · Logic 82.6% · Math 98.2% Non-hallucination 43.7% · “I don't know” on 3.7% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.375$375estimate · DGX Spark per 1,000 questionsper 1M questions · 131.5 s per question (median latency) | 92.9%92.0%–93.7% | Logic 82.6% Math 98.2% | 43.7% | 3.7% | – | $0.375$375estimate · DGX Spark | 6,351 sDGX Spark | 131.5 sp90 392.6 s |
| 5 | Gemma 4 E2B on · chat · 32k cap · sampled Open · 5.1B Gemma 4 E2B on · chat · 32k cap · sampled · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 90.5% (mean of 3 seeds; min–max 90.2%–90.9%) · Logic 76.8% · Math 97.7% Seed runs: seed 0: 90.2% (2,966 of 3,288, run gemma-4-e2b-it-think-32k-sampled-seed0_en+gemma-4-e2b-it-think-32k-sampled-seed0_en) · seed 1: 90.9% (2,989 of 3,288, run gemma-4-e2b-it-think-32k-sampled-seed1_en+gemma-4-e2b-it-think-32k-sampled-seed1_en) · seed 2: 90.5% (2,974 of 3,288, run gemma-4-e2b-it-think-32k-sampled-seed2_en+gemma-4-e2b-it-think-32k-sampled-seed2_en). Tiers, tie letters, topics, confidence, “I don't know” and the answer links are from the seed-0 run gemma-4-e2b-it-think-32k-sampled-seed0_en+gemma-4-e2b-it-think-32k-sampled-seed0_en, not the mean Non-hallucination 31.0% · “I don't know” on 3.4% of questions · seed 0's run only (one run, not a 3-seed mean) Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.325$325estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.87 s per question (median latency) | 90.5%90.2%–90.9% min–max of 3 seeds | Logic 76.8% Math 97.7% | 31.0%seed 0 only | 3.4%seed 0 only | – | $0.325$325estimate · RTX PRO 6000 Blackwellmean of 3 seeds | 470 sRTX PRO 6000mean of 3 seeds | 8.87 sp90 32.5 smean of 3 seeds |
| 5 | Gemma 4 E2B on · chat · 32k cap Open · 5.1B Gemma 4 E2B on · chat · 32k cap · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 90.3% (95%: 89.3%–91.3%) · Logic 75.9% · Math 97.8% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.678$678estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 19.1 s per question (median latency) | 90.3%89.3%–91.3% | Logic 75.9% Math 97.8% | – | – | – | $0.678$678estimate · RTX PRO 6000 Blackwell | 980 sRTX PRO 6000 | 19.1 sp90 54.6 s |
| 5 | Gemma 4 E2B on · chat Open · 5.1B Gemma 4 E2B on · chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 89.3% (95%: 88.2%–90.3%) · Logic 74.6% · Math 96.9% Non-hallucination 24.3% · “I don't know” on 2.8% of questions Runs on NVIDIA DGX Spark (GB10) · cost $0.215$215estimate · DGX Spark per 1,000 questionsper 1M questions · 77.9 s per question (median latency) | 89.3%88.2%–90.3% | Logic 74.6% Math 96.9% | 24.3% | 2.8% | – | $0.215$215estimate · DGX Spark | 3,639 sDGX Spark | 77.9 sp90 232.8 s |
| 5 | Gemma 4 E2B chat Open · 5.1B Gemma 4 E2B chat · Google · chat model Parameters: 5.1B dense · 2.3B effective Both 88.6% (95%: 87.5%–89.6%) · Logic 72.9% · Math 96.8% Non-hallucination 32.3% · “I don't know” on 3.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.313$313estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.93 s per question (median latency) | 88.6%87.5%–89.6% | Logic 72.9% Math 96.8% | 32.3% | 3.9% | – | $0.313$313estimate · RTX PRO 6000 Blackwell | 452 sRTX PRO 6000 | 9.93 sp90 27.7 s |
| 5 | Raon-1 nano on · chat Open · 3.1B Raon-1 nano on · chat · KRAFTON · chat model Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Both 88.1% (95%: 86.9%–89.1%) · Logic 70.9% · Math 97.0% Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $6.344$6,344estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 55.0 s† per question (median latency) | 88.1%86.9%–89.1% | Logic 70.9% Math 97.0% | – | – | – | $6.344$6,344estimate · RTX PRO 6000 Blackwell† | 9,172 sRTX PRO 6000 | 55.0 s†p90 987.7 s |
| 6 | LFM2.5-2.6B on · chat Open · 2.7B LFM2.5-2.6B on · chat · Liquid AI · chat model Parameters: 2.69B dense Both 87.6% (95%: 86.5%–88.7%) · Logic 75.4% · Math 94.0% Non-hallucination 40.6% · “I don't know” on 5.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.046$1,046estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 14.0 s per question (median latency) | 87.6%86.5%–88.7% | Logic 75.4% Math 94.0% | 40.6% | 5.4% | – | $1.046$1,046estimate · RTX PRO 6000 Blackwell | 1,512 sRTX PRO 6000 | 14.0 sp90 143.0 s |
| 6 | Qwen3.5-2B on · chat Open · 2.0B Qwen3.5-2B on · chat · Alibaba · chat model Parameters: 2B dense Both 86.4% (95%: 85.2%–87.5%) · Logic 70.3% · Math 94.7% Non-hallucination 30.9% · “I don't know” on 3.5% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $3.106$3,106estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 121.8 s per question (median latency) | 86.4%85.2%–87.5% | Logic 70.3% Math 94.7% | 30.9% | 3.5% | – | $3.106$3,106estimate · RTX PRO 6000 Blackwell | 4,491 sRTX PRO 6000 | 121.8 sp90 241.6 s |
| 6 | Qwen3-14B off · chat Open · 15B Qwen3-14B off · chat · Alibaba · chat model · reasoning off Parameters: 14.8B dense Both 85.4% (95%: 84.1%–86.5%) · Logic 65.3% · Math 95.8% Non-hallucination 20.9% · “I don't know” on 3.6% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.840$840estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 9.38 s per question (median latency) | 85.4%84.1%–86.5% | Logic 65.3% Math 95.8% | 20.9% | 3.6% | – | $0.840$840estimate · RTX PRO 6000 Blackwell | 1,214 sRTX PRO 6000 | 9.38 sp90 35.1 s |
| 7 | Qwen3.5-2B off · chat Open · 2.0B Qwen3.5-2B off · chat · Alibaba · chat model · reasoning off Parameters: 2B dense Both 84.4% (95%: 83.1%–85.6%) · Logic 64.8% · Math 94.6% Non-hallucination 34.3% · “I don't know” on 6.1% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.788$788estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 17.8 s per question (median latency) | 84.4%83.1%–85.6% | Logic 64.8% Math 94.6% | 34.3% | 6.1% | – | $0.788$788estimate · RTX PRO 6000 Blackwell | 1,139 sRTX PRO 6000 | 17.8 sp90 92.1 s |
| 7 | LFM2.5-8B-A1B on · chat Open · 8.3B MoE · 1.5B active LFM2.5-8B-A1B on · chat · Liquid AI · chat model Parameters: 8.3B MoE · 1.5B active · Mixture of experts: 8.3B total, about 1.5B active per step. Size class uses 8.3B. Both 83.9% (95%: 82.6%–85.1%) · Logic 60.2% · Math 96.3% Non-hallucination 55.9% · “I don't know” on 11.9% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $1.913$1,913estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 20.1 s† per question (median latency) | 83.9%82.6%–85.1% | Logic 60.2% Math 96.3% | 55.9% | 11.9% | – | $1.913$1,913estimate · RTX PRO 6000 Blackwell† | 2,766 sRTX PRO 6000 | 20.1 s†p90 230.6 s |
| 8 | GPT-6 Luna chat Closed · Efficient GPT-6 Luna chat · OpenAI · chat model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/openai/gpt-6-luna) Both 81.4% (95%: 80.1%–82.7%) · Logic 58.0% · Math 93.7% Runs on an API · cost $0.000$0.00 per 1,000 questionsper 1M questions · 3.83 s per question (median latency) | 81.4%80.1%–82.7% | Logic 58.0% Math 93.7% | – | – | – | $0.000$0.00 | – | 3.83 sp90 5.30 s |
| 8 | Raon-1 nano off · chat Open · 3.1B Raon-1 nano off · chat · KRAFTON · chat model · reasoning off Parameters: 3.1B dense · 1.5B effective Base model: Raon-1 mini preview (MoE-to-Dense pruning and distillation) Both 81.2% (95%: 79.8%–82.5%) · Logic 65.1% · Math 89.6% Non-hallucination 31.1% · “I don't know” on 5.7% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $4.422$4,422estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions† · 42.6 s† per question (median latency) | 81.2%79.8%–82.5% | Logic 65.1% Math 89.6% | 31.1% | 5.7% | – | $4.422$4,422estimate · RTX PRO 6000 Blackwell† | 6,394 sRTX PRO 6000 | 42.6 s†p90 594.0 s |
| 9 | Erk-14B off · chat Open · 15B Erk-14B off · chat · eCloud Technology · chat model · reasoning off Parameters: 14.77B dense Both 78.3% (95%: 76.8%–79.6%) · Logic 54.2% · Math 90.8% Non-hallucination 32.3% · “I don't know” on 8.4% of questions Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.308$308estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 8.74 s per question (median latency) | 78.3%76.8%–79.6% | Logic 54.2% Math 90.8% | 32.3% | 8.4% | – | $0.308$308estimate · RTX PRO 6000 Blackwell | 446 sRTX PRO 6000 | 8.74 sp90 16.4 s |
| 10 | Karpo Decision 31B Open · 31B Karpo Decision 31B · MachinePulse · decision model Parameters: 31B dense Both 74.4% (95%: 72.9%–75.9%) · Logic 60.7% · Math 81.6% Non-hallucination 5.7% · “I don't know” on 1.5% of questions Confidence: 1,009 answers kept at ≥99%, 1,006 right · 3 wrong · AUROC 0.882 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.118$118estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.162 s per question (median latency) | 74.4%72.9%–75.9% | Logic 60.7% Math 81.6% | 5.7% | 1.5% | 2.7 | $0.118$118estimate · RTX PRO 6000 Blackwell | 171 sRTX PRO 6000 | 0.162 sp90 0.198 s |
| 10 | KnowLine-31B-Gen2 Open · 31B KnowLine-31B-Gen2 · PelaAI · decision model Parameters: 31B dense Base model: Gemma 4 31B Both 73.0% (95%: 71.4%–74.5%) · Logic 58.9% · Math 80.3% Non-hallucination 0.1% · “I don't know” on 0.0% of questions Confidence: 524 answers kept at ≥99%, 523 right · 1 wrong · AUROC 0.882 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.093$92.98estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.116 s per question (median latency) | 73.0%71.4%–74.5% | Logic 58.9% Math 80.3% | 0.1% | 0.0% | 2.4 | $0.093$92.98estimate · RTX PRO 6000 Blackwell | 134 sRTX PRO 6000 | 0.116 sp90 0.177 s |
| 11 | REAX 1.3 Open · 57B REAX 1.3 · REAX · decision model Parameters: 57.36B dense · Two models run together, option probabilities averaged 50/50: 57.36B in all, all used on every question. Both 71.6% (95%: 70.0%–73.1%) · Logic 59.3% · Math 78.0% Non-hallucination 3.8% · “I don't know” on 1.1% of questions Confidence: 229 answers kept at ≥99%, 229 right · 0 wrong · AUROC 0.878 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.148$148estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.200 s per question (median latency) | 71.6%70.0%–73.1% | Logic 59.3% Math 78.0% | 3.8% | 1.1% | 4.7 | $0.148$148estimate · RTX PRO 6000 Blackwell | 214 sRTX PRO 6000 | 0.200 sp90 0.260 s |
| 11 | KnowLine-31B-Gen1.5 Open · 31B KnowLine-31B-Gen1.5 · PelaAI · decision model Parameters: 31B dense Both 71.1% (95%: 69.5%–72.6%) · Logic 59.0% · Math 77.4% Non-hallucination 0.7% · “I don't know” on 0.2% of questions Confidence: 554 answers kept at ≥99%, 553 right · 1 wrong · AUROC 0.870 Runs on NVIDIA DGX Spark (GB10) · cost $0.052$51.85estimate · DGX Spark per 1,000 questionsper 1M questions · 0.811 s per question (median latency) | 71.1%69.5%–72.6% | Logic 59.0% Math 77.4% | 0.7% | 0.2% | 2.6 | $0.052$51.85estimate · DGX Spark | 877 sDGX Spark | 0.811 sp90 1.11 s |
| 11 | Copernicus-31B Open · 31B Copernicus-31B · Poindexter Labs · decision model Parameters: 31B dense Both 71.0% (95%: 69.4%–72.5%) · Logic 58.7% · Math 77.3% Non-hallucination 1.8% · “I don't know” on 0.5% of questions Confidence: 384 answers kept at ≥99%, 384 right · 0 wrong · AUROC 0.873 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.102$102estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.128 s per question (median latency) | 71.0%69.4%–72.5% | Logic 58.7% Math 77.3% | 1.8% | 0.5% | 5.0 | $0.102$102estimate · RTX PRO 6000 Blackwell | 148 sRTX PRO 6000 | 0.128 sp90 0.218 s |
| 11 | Sieve 31B v3 Open · 31B Sieve 31B v3 · Sthānika AI · decision model Parameters: 31B dense Both 70.5% (95%: 68.9%–72.1%) · Logic 57.9% · Math 77.1% Non-hallucination 3.8% · “I don't know” on 1.1% of questions Confidence: 296 answers kept at ≥99%, 294 right · 2 wrong · AUROC 0.865 Runs on NVIDIA DGX Spark (GB10) · cost $0.047$47.29estimate · DGX Spark per 1,000 questionsper 1M questions · 0.676 s per question (median latency) | 70.5%68.9%–72.1% | Logic 57.9% Math 77.1% | 3.8% | 1.1% | 2.6 | $0.047$47.29estimate · DGX Spark | 800 sDGX Spark | 0.676 sp90 1.17 s |
| 12 | Logits Sigma-1 Duo Open · 57B Logits Sigma-1 Duo · Logits · decision model Parameters: 57.36B dense Both 69.3% (95%: 67.7%–70.8%) · Logic 57.8% · Math 75.2% Non-hallucination 4.4% · “I don't know” on 1.4% of questions Confidence: 23 answers kept at ≥99%, 23 right · 0 wrong · AUROC 0.868 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.165$165estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.207 s per question (median latency) | 69.3%67.7%–70.8% | Logic 57.8% Math 75.2% | 4.4% | 1.4% | 6.0 | $0.165$165estimate · RTX PRO 6000 Blackwell | 238 sRTX PRO 6000 | 0.207 sp90 0.324 s |
| 12 | d1-ultra Closed · Efficient d1-ultra · Liquid AI · decision model Parameters: not disclosed by maker (built on a larger open-weight model, not named with a count) · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://www.liquid.ai/blog/d1-ultra) Both 67.5% (95%: 65.9%–69.1%) · Logic 54.9% · Math 74.1% Non-hallucination 89.1% · “I don't know” on 47.7% of questions Confidence: 233 answers kept at ≥99%, 233 right · 0 wrong · AUROC 0.852 Runs on an API · cost $0.008$8.37 per 1,000 questionsper 1M questions · 0.180 s per question (median latency) | 67.5%65.9%–69.1% | Logic 54.9% Math 74.1% | 89.1% | 47.7% | 2.1 | $0.008$8.37 | – | 0.180 sp90 0.426 s |
| 12 | GLiDE 27B Open · 27B GLiDE 27B · Fastino · decision model Parameters: 27B dense Both 67.3% (95%: 65.7%–68.9%) · Logic 58.4% · Math 72.0% Non-hallucination 31.9% · “I don't know” on 11.5% of questions Confidence: 274 answers kept at ≥99%, 272 right · 2 wrong · AUROC 0.852 Runs on NVIDIA DGX Spark (GB10) · cost $0.029$28.55estimate · DGX Spark per 1,000 questionsper 1M questions · 0.450 s per question (median latency) | 67.3%65.7%–68.9% | Logic 58.4% Math 72.0% | 31.9% | 11.5% | 4.1 | $0.029$28.55estimate · DGX Spark | 483 sDGX Spark | 0.450 sp90 0.587 s |
| 12 | Mammoth-1 Open · 27B Mammoth-1 · Base Labs · decision model Parameters: 27B dense Both 67.3% (95%: 65.7%–68.9%) · Logic 54.5% · Math 74.0% Non-hallucination 71.4% · “I don't know” on 30.6% of questions Confidence: 520 answers kept at ≥99%, 518 right · 2 wrong · AUROC 0.868 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.150 s per question (median latency) | 67.3%65.7%–68.9% | Logic 54.5% Math 74.0% | 71.4% | 30.6% | 3.2 | $0.028$27.98 list$0.120$120estimate · RTX PRO 6000 Blackwell | 173 sRTX PRO 6000 | 0.150 sp90 0.232 s |
| 13 | Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.3 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Both 66.4% (95%: 64.8%–68.0%) · Logic 54.8% · Math 72.5% Non-hallucination 13.5% · “I don't know” on 4.9% of questions Confidence: 385 answers kept at ≥99%, 385 right · 0 wrong · AUROC 0.844 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.071$71.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.097 s per question (median latency) | 66.4%64.8%–68.0% | Logic 54.8% Math 72.5% | 13.5% | 4.9% | 5.9 | $0.071$71.10estimate · RTX PRO 6000 Blackwell | 103 sRTX PRO 6000 | 0.097 sp90 0.125 s |
| 13 | d3-ultra Open · 26B d3-ultra · vLLM Semantic Router · decision model Parameters: 26.09B dense Both 65.7% (95%: 64.1%–67.3%) · Logic 54.6% · Math 71.5% Non-hallucination 7.5% · “I don't know” on 2.6% of questions Confidence: 39 answers kept at ≥99%, 39 right · 0 wrong · AUROC 0.835 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.096$95.71estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.129 s per question (median latency) | 65.7%64.1%–67.3% | Logic 54.6% Math 71.5% | 7.5% | 2.6% | 10.1 | $0.096$95.71estimate · RTX PRO 6000 Blackwell | 138 sRTX PRO 6000 | 0.129 sp90 0.168 s |
| 13 | GLiDE API Closed · Efficient GLiDE API · Fastino · decision model Parameters: not disclosed by maker · Closed; list price $0.15 per 1M input tokens: Efficient (source: https://docs.fastino.ai/pricing) Both 65.2% (95%: 63.6%–66.8%) · Logic 54.5% · Math 70.8% Non-hallucination 49.2% · “I don't know” on 20.5% of questions Confidence: 213 answers kept at ≥99%, 213 right · 0 wrong · AUROC 0.852 Runs on an API · cost $0.036$36.42 per 1,000 questionsper 1M questions · 0.266 s per question (median latency) | 65.2%63.6%–66.8% | Logic 54.5% Math 70.8% | 49.2% | 20.5% | 5.3 | $0.036$36.42 | – | 0.266 sp90 0.284 s |
| 13 | d3 Open · 26B d3 · vLLM Semantic Router · decision model Parameters: 26.09B dense Both 65.0% (95%: 63.3%–66.6%) · Logic 54.5% · Math 70.4% Non-hallucination 9.3% · “I don't know” on 3.4% of questions Confidence: 217 answers kept at ≥99%, 216 right · 1 wrong · AUROC 0.858 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.046$46.28estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.057 s per question (median latency) | 65.0%63.3%–66.6% | Logic 54.5% Math 70.4% | 9.3% | 3.4% | 4.4 | $0.046$46.28estimate · RTX PRO 6000 Blackwell | 67 sRTX PRO 6000 | 0.057 sp90 0.086 s |
| 13 | Drex Max v1.1 Closed · Efficient (provisional) Drex Max v1.1 · Nace.AI · decision model Parameters: not disclosed by maker · Closed; introductory list price (the only one published) $0.03 per 1M input tokens: Efficient (provisional) (source: https://nace.ai/drex) Both 64.8% (95%: 63.2%–66.5%) · Logic 54.8% · Math 70.1% Non-hallucination 48.3% · “I don't know” on 20.8% of questions Confidence: 405 answers kept at ≥99%, 404 right · 1 wrong · AUROC 0.846 Runs on an API · cost $0.016$16.13 per 1,000 questionsper 1M questions · 0.327 s per question (median latency) | 64.8%63.2%–66.5% | Logic 54.8% Math 70.1% | 48.3% | 20.8% | 5.8 | $0.016$16.13 | – | 0.327 sp90 0.518 s |
| 13 | GLiDE v1.2 Open · 27B GLiDE v1.2 · Fastino · decision model Parameters: 27B dense Both 64.8% (95%: 63.2%–66.5%) · Logic 55.2% · Math 69.9% Non-hallucination 42.3% · “I don't know” on 17.2% of questions Confidence: 176 answers kept at ≥99%, 176 right · 0 wrong · AUROC 0.843 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.081 s per question (median latency) | 64.8%63.2%–66.5% | Logic 55.2% Math 69.9% | 42.3% | 17.2% | 5.1 | $0.010$9.71 list$0.066$66.05estimate · RTX PRO 6000 Blackwell | 95 sRTX PRO 6000 | 0.081 sp90 0.136 s |
| 13 | Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell Open · 27B Perplexity Decider 27B v1.2 NVIDIA RTX PRO 6000 Blackwell · Perplexity · decision model Parameters: 27B dense Both 64.5% (95%: 62.8%–66.1%) · Logic 53.2% · Math 70.3% Non-hallucination 9.5% · “I don't know” on 3.5% of questions Confidence: 467 answers kept at ≥99%, 465 right · 2 wrong · AUROC 0.848 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.005$5.44 list$0.082$81.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.098 s per question (median latency) Shown under Perplexity Decider 27B v1.2 (API): the same model name, one on an NVIDIA RTX PRO 6000 Blackwell (96 GB) and one through Perplexity's API; not verified to be identical weights or settings | 64.5%62.8%–66.1% | Logic 53.2% Math 70.3% | 9.5% | 3.5% | 3.3 | $0.005$5.44 list$0.082$81.83estimate · RTX PRO 6000 Blackwell | 118 sRTX PRO 6000 | 0.098 sp90 0.166 s |
| 13 | Drex Max Open · 27B Drex Max · Nace.AI · decision model Parameters: 27B dense Both 64.4% (95%: 62.8%–66.0%) · Logic 53.1% · Math 70.3% Non-hallucination 5.7% · “I don't know” on 2.1% of questions Confidence: 884 answers kept at ≥99%, 866 right · 18 wrong · AUROC 0.841 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.081 s per question (median latency) | 64.4%62.8%–66.0% | Logic 53.1% Math 70.3% | 5.7% | 2.1% | 13.3 | $0.016$16.31 list$0.064$63.74estimate · RTX PRO 6000 Blackwell | 92 sRTX PRO 6000 | 0.081 sp90 0.130 s |
| 13 | Perplexity Decider 27B v1.2 API Open · 27B Perplexity Decider 27B v1.2 API · Perplexity · decision model Parameters: 27B dense Both 64.1% (95%: 62.5%–65.8%) · Logic 53.3% · Math 69.8% Non-hallucination 10.3% · “I don't know” on 3.9% of questions Confidence: 411 answers kept at ≥99%, 409 right · 2 wrong · AUROC 0.849 Runs on an API · cost $0.005$5.38 per 1,000 questionsper 1M questions · 0.230 s per question (median latency) | 64.1%62.5%–65.8% | Logic 53.3% Math 69.8% | 10.3% | 3.9% | 3.1 | $0.005$5.38 | – | 0.230 sp90 0.439 s |
| 13 | Gemini Decision 1 20261111 Closed · Efficient Gemini Decision 1 20261111 · Google · decision model Parameters: not disclosed by maker (closed weights) · Closed; list price $0.10 per 1M input tokens: Efficient (source: https://openrouter.ai/google/gemini-decision-1) Both 64.1% (95%: 62.4%–65.7%) · Logic 50.0% · Math 71.3% Non-hallucination 3.0% · “I don't know” on 1.1% of questions Confidence: 257 answers kept at ≥99%, 257 right · 0 wrong · AUROC 0.838 Runs on an API · cost $0.027$27.07 per 1,000 questionsper 1M questions · 0.227 s per question (median latency) | 64.1%62.4%–65.7% | Logic 50.0% Math 71.3% | 3.0% | 1.1% | 2.0 | $0.027$27.07 | – | 0.227 sp90 0.347 s |
| 13 | d1.2-ultra Closed · Undisclosed d1.2-ultra · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Both 63.6% (95%: 61.9%–65.2%) · Logic 51.6% · Math 69.9% Non-hallucination 90.2% · “I don't know” on 51.9% of questions Confidence: 128 answers kept at ≥99%, 128 right · 0 wrong · AUROC 0.856 Runs on an API · cost $0.008$8.37 per 1,000 questionsper 1M questions · 0.217 s per question (median latency) | 63.6%61.9%–65.2% | Logic 51.6% Math 69.9% | 90.2% | 51.9% | 3.2 | $0.008$8.37 | – | 0.217 sp90 0.347 s |
| – | Perplexity Decider 27B v1.2 API · with web Open · 27B Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model Parameters: 27B dense Not tiered: with web Both 63.6% (95%: 61.9%–65.2%) · Logic 52.5% · Math 69.3% Non-hallucination 25.8% · “I don't know” on 9.9% of questions Confidence: 405 answers kept at ≥99%, 404 right · 1 wrong · AUROC 0.854 Runs on an API · cost $1.025$1,025 per 1,000 questionsper 1M questions · 0.430 s per question (median latency) | 63.6%61.9%–65.2% | Logic 52.5% Math 69.3% | 25.8% | 9.9% | 3.7 | $1.025$1,025 | – | 0.430 sp90 0.707 s |
| 13 | Turion Jev 31B Open · 31B Turion Jev 31B · Turion.AI · decision model Parameters: 31B dense Both 63.3% (95%: 61.6%–64.9%) · Logic 52.3% · Math 69.0% Non-hallucination 1.9% · “I don't know” on 0.7% of questions Confidence: 50 answers kept at ≥99%, 50 right · 0 wrong · AUROC 0.844 Runs on NVIDIA DGX Spark (GB10) · cost $0.020$20.01estimate · DGX Spark per 1,000 questionsper 1M questions · 0.301 s per question (median latency) | 63.3%61.6%–64.9% | Logic 52.3% Math 69.0% | 1.9% | 0.7% | 3.5 | $0.020$20.01estimate · DGX Spark | 339 sDGX Spark | 0.301 sp90 0.448 s |
| 14 | SubQ Snap 1.1 Closed · Efficient SubQ Snap 1.1 · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Both 63.0% (95%: 61.4%–64.7%) · Logic 52.4% · Math 68.6% Non-hallucination 58.3% · “I don't know” on 26.8% of questions Confidence: 433 answers kept at ≥99%, 430 right · 3 wrong · AUROC 0.827 Runs on an API · cost $0.011$10.75 per 1,000 questionsper 1M questions · 0.382 s per question (median latency) Shown once with SubQ Snap Preview: the same answer to every question | 63.0%61.4%–64.7% | Logic 52.4% Math 68.6% | 58.3% | 26.8% | 10.5 | $0.011$10.75 | – | 0.382 sp90 0.391 s |
| 14 | SubQ Snap Preview Closed · Efficient SubQ Snap Preview · Subquadratic · decision model Parameters: not disclosed by maker · Closed; list price $0.05 per 1M input tokens: Efficient (source: https://docs.subq.ai/using-the-api/models/) Both 63.0% (95%: 61.4%–64.7%) · Logic 52.4% · Math 68.6% Non-hallucination 58.3% · “I don't know” on 26.8% of questions Confidence: 433 answers kept at ≥99%, 430 right · 3 wrong · AUROC 0.827 Runs on an API · cost $0.013$13.44 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) Shown under SubQ Snap 1.1 (the same answers) | 63.0%61.4%–64.7% | Logic 52.4% Math 68.6% | 58.3% | 26.8% | 10.5 | $0.013$13.44 | – | 0.381 sp90 0.394 s |
| 14 | celeris-1-decision Closed · Efficient celeris-1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Both 62.2% (95%: 60.6%–63.9%) · Logic 47.3% · Math 70.0% Non-hallucination 87.2% · “I don't know” on 49.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.829 Runs on an API · cost $0.011$11.25 per 1,000 questionsper 1M questions · 0.388 s per question (median latency) | 62.2%60.6%–63.9% | Logic 47.3% Math 70.0% | 87.2% | 49.8% | 5.8 | $0.011$11.25 | – | 0.388 sp90 0.392 s |
| 14 | celeris-1.1-decision Closed · Efficient celeris-1.1-decision · Celeris · decision model Parameters: not disclosed by maker · Closed; list price $0.004 per 1M input tokens: Efficient (source: https://docs.celeris.ai/pricing) Both 62.1% (95%: 60.5%–63.8%) · Logic 47.3% · Math 69.9% Non-hallucination 86.7% · “I don't know” on 49.2% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.830 Runs on an API · cost $0.001$1.13 per 1,000 questionsper 1M questions · 0.381 s per question (median latency) | 62.1%60.5%–63.8% | Logic 47.3% Math 69.9% | 86.7% | 49.2% | 6.0 | $0.001$1.13 | – | 0.381 sp90 0.388 s |
| 14 | KnowLine-12B-Gen1 Open · 12B KnowLine-12B-Gen1 · PelaAI · decision model Parameters: 12B dense Base model: Gemma 4 12B Both 60.9% (95%: 59.2%–62.5%) · Logic 50.6% · Math 66.3% Non-hallucination 4.5% · “I don't know” on 1.8% of questions Confidence: 13 answers kept at ≥99%, 13 right · 0 wrong · AUROC 0.810 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.051 s per question (median latency) | 60.9%59.2%–62.5% | Logic 50.6% Math 66.3% | 4.5% | 1.8% | 2.9 | $0.041$41.44estimate · RTX PRO 6000 Blackwell | 60 sRTX PRO 6000 | 0.051 sp90 0.076 s |
| 14 | d1.2 Closed · Undisclosed d1.2 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; no list price published: Undisclosed Both 59.9% (95%: 58.2%–61.6%) · Logic 49.2% · Math 65.5% Non-hallucination 66.9% · “I don't know” on 34.9% of questions Confidence: 169 answers kept at ≥99%, 169 right · 0 wrong · AUROC 0.812 Runs on an API · cost $0.004$3.98 per 1,000 questionsper 1M questions · 0.208 s per question (median latency) | 59.9%58.2%–61.6% | Logic 49.2% Math 65.5% | 66.9% | 34.9% | 2.5 | $0.004$3.98 | – | 0.208 sp90 0.390 s |
| 15 | Jev 1.14 Closed · Efficient Jev 1.14 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://docs.typesafe.ai/models) Both 59.3% (95%: 57.6%–61.0%) · Logic 48.2% · Math 65.1% Non-hallucination 15.4% · “I don't know” on 6.7% of questions Confidence: 263 answers kept at ≥99%, 263 right · 0 wrong · AUROC 0.810 Runs on an API · cost $0.021$21.27 per 1,000 questionsper 1M questions · 0.164 s per question (median latency) | 59.3%57.6%–61.0% | Logic 48.2% Math 65.1% | 15.4% | 6.7% | 1.8 | $0.021$21.27 | – | 0.164 sp90 0.188 s |
| 15 | d1.1 Closed · Efficient d1.1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Both 58.6% (95%: 56.9%–60.3%) · Logic 46.0% · Math 65.2% Non-hallucination 86.0% · “I don't know” on 53.5% of questions Confidence: 231 answers kept at ≥99%, 231 right · 0 wrong · AUROC 0.811 Runs on an API · cost $0.004$3.98 per 1,000 questionsper 1M questions · 0.174 s per question (median latency) | 58.6%56.9%–60.3% | Logic 46.0% Math 65.2% | 86.0% | 53.5% | 0.8 | $0.004$3.98 | – | 0.174 sp90 0.285 s |
| 15 | Jev 1.13 Closed · Efficient Jev 1.13 · TypeSafe · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://typesafe.ai/blog/introducing-system-one-models-and-jev) Both 58.0% (95%: 56.3%–59.7%) · Logic 46.3% · Math 64.1% Non-hallucination 10.2% · “I don't know” on 4.5% of questions Confidence: 313 answers kept at ≥99%, 313 right · 0 wrong · AUROC 0.817 Runs on an API · cost $0.021$21.27 per 1,000 questionsper 1M questions · 0.166 s per question (median latency) | 58.0%56.3%–59.7% | Logic 46.3% Math 64.1% | 10.2% | 4.5% | 3.2 | $0.021$21.27 | – | 0.166 sp90 0.205 s |
| 15 | Adjudex 9B v5 Open · 9.0B Adjudex 9B v5 · newsbang · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 56.7% (95%: 55.0%–58.4%) · Logic 46.4% · Math 62.1% Non-hallucination 7.0% · “I don't know” on 3.1% of questions Confidence: 253 answers kept at ≥99%, 251 right · 2 wrong · AUROC 0.802 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.028$28.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 56.7%55.0%–58.4% | Logic 46.4% Math 62.1% | 7.0% | 3.1% | 5.6 | $0.028$28.19estimate · RTX PRO 6000 Blackwell | 41 sRTX PRO 6000 | 0.038 sp90 0.045 s |
| 15 | Elephant-1 Open · 9.0B Elephant-1 · Base Labs · decision model Parameters: 9B dense Both 56.2% (95%: 54.5%–57.9%) · Logic 46.4% · Math 61.3% Non-hallucination 69.4% · “I don't know” on 39.7% of questions Confidence: 236 answers kept at ≥99%, 235 right · 1 wrong · AUROC 0.797 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.080 s per question (median latency) | 56.2%54.5%–57.9% | Logic 46.4% Math 61.3% | 69.4% | 39.7% | 5.6 | $0.010$9.79 list$0.058$58.27estimate · RTX PRO 6000 Blackwell | 84 sRTX PRO 6000 | 0.080 sp90 0.094 s |
| 15 | Jeffan-9B-1211 Open · 9.0B Jeffan-9B-1211 · tattrongvu · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 55.8% (95%: 54.1%–57.5%) · Logic 48.2% · Math 59.8% Non-hallucination 13.5% · “I don't know” on 6.3% of questions Confidence: 74 answers kept at ≥99%, 74 right · 0 wrong · AUROC 0.798 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.032$32.40estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.039 s per question (median latency) | 55.8%54.1%–57.5% | Logic 48.2% Math 59.8% | 13.5% | 6.3% | 3.6 | $0.032$32.40estimate · RTX PRO 6000 Blackwell | 47 sRTX PRO 6000 | 0.039 sp90 0.053 s |
| 16 | KnowLine-9B-Gen2 Open · 9.0B KnowLine-9B-Gen2 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 54.6% (95%: 52.9%–56.3%) · Logic 43.9% · Math 60.1% Non-hallucination 10.4% · “I don't know” on 5.0% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.786 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.034$34.08estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.042 s per question (median latency) | 54.6%52.9%–56.3% | Logic 43.9% Math 60.1% | 10.4% | 5.0% | 3.6 | $0.034$34.08estimate · RTX PRO 6000 Blackwell | 49 sRTX PRO 6000 | 0.042 sp90 0.057 s |
| 16 | GLiDE 9B Open · 7.9B GLiDE 9B · Fastino · decision model Parameters: 7.94B dense Both 54.5% (95%: 52.8%–56.2%) · Logic 47.8% · Math 57.9% Non-hallucination 26.9% · “I don't know” on 13.5% of questions Confidence: 105 answers kept at ≥99%, 105 right · 0 wrong · AUROC 0.786 Runs on NVIDIA DGX Spark (GB10) · cost $0.011$11.45estimate · DGX Spark per 1,000 questionsper 1M questions · 0.177 s per question (median latency) | 54.5%52.8%–56.2% | Logic 47.8% Math 57.9% | 26.9% | 13.5% | 6.7 | $0.011$11.45estimate · DGX Spark | 194 sDGX Spark | 0.177 sp90 0.222 s |
| 16 | KnowLine-9B-Gen1 Open · 9.0B KnowLine-9B-Gen1 · PelaAI · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 54.5% (95%: 52.8%–56.2%) · Logic 44.3% · Math 59.8% Non-hallucination 3.4% · “I don't know” on 1.6% of questions Confidence: 153 answers kept at ≥99%, 153 right · 0 wrong · AUROC 0.782 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.030$29.87estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 54.5%52.8%–56.2% | Logic 44.3% Math 59.8% | 3.4% | 1.6% | 2.2 | $0.030$29.87estimate · RTX PRO 6000 Blackwell | 43 sRTX PRO 6000 | 0.038 sp90 0.055 s |
| 16 | StartLux-Decision-v1.1-9B-Instant Open · 9.0B StartLux-Decision-v1.1-9B-Instant · StartLux · decision model Parameters: 9B dense Both 54.4% (95%: 52.7%–56.1%) · Logic 46.6% · Math 58.4% Non-hallucination 9.6% · “I don't know” on 4.6% of questions Confidence: 63 answers kept at ≥99%, 63 right · 0 wrong · AUROC 0.779 Runs on NVIDIA DGX Spark (GB10) · cost $0.009$8.92estimate · DGX Spark per 1,000 questionsper 1M questions · 0.142 s per question (median latency) | 54.4%52.7%–56.1% | Logic 46.6% Math 58.4% | 9.6% | 4.6% | 2.7 | $0.009$8.92estimate · DGX Spark | 151 sDGX Spark | 0.142 sp90 0.165 s |
| 16 | Microsoft-Decision-1 Closed · Efficient Microsoft-Decision-1 · Microsoft · decision model Parameters: not disclosed by maker · Closed; list price $0.042 per 1M input tokens: Efficient (source: https://openrouter.ai/microsoft/microsoft-decision-1) Base model: Qwen3.5-9B Both 54.3% (95%: 52.6%–56.0%) · Logic 44.4% · Math 59.4% Non-hallucination 11.4% · “I don't know” on 5.5% of questions Confidence: 47 answers kept at ≥99%, 47 right · 0 wrong · AUROC 0.769 Runs on an API · cost $0.009$9.07 per 1,000 questionsper 1M questions · 0.312 s per question (median latency) | 54.3%52.6%–56.0% | Logic 44.4% Math 59.4% | 11.4% | 5.5% | 4.9 | $0.009$9.07 | – | 0.312 sp90 0.360 s |
| 16 | Dexy-Decision-9B-V2 Open · 9.0B Dexy-Decision-9B-V2 · Patrick-18 · decision model Parameters: 9B dense Base model: vllm-sr/d3-flash Both 54.2% (95%: 52.5%–55.9%) · Logic 45.3% · Math 58.8% Non-hallucination 6.1% · “I don't know” on 2.9% of questions Confidence: 266 answers kept at ≥99%, 263 right · 3 wrong · AUROC 0.795 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.033$32.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.046 s per question (median latency) | 54.2%52.5%–55.9% | Logic 45.3% Math 58.8% | 6.1% | 2.9% | 13.1 | $0.033$32.61estimate · RTX PRO 6000 Blackwell | 47 sRTX PRO 6000 | 0.046 sp90 0.051 s |
| 16 | Drex 2.0 Open · 35B MoE · 3.0B active Drex 2.0 · Nace.AI · decision model Parameters: 35B MoE · 3B active · Mixture of experts: 35B total, about 3B active per step. Size class uses 35B. Both 53.1% (95%: 51.4%–54.8%) · Logic 41.9% · Math 59.0% Non-hallucination 28.1% · “I don't know” on 14.7% of questions Confidence: 207 answers kept at ≥99%, 207 right · 0 wrong · AUROC 0.789 Runs on an API · cost $0.012$12.17 per 1,000 questionsper 1M questions · 0.339 s per question (median latency) | 53.1%51.4%–54.8% | Logic 41.9% Math 59.0% | 28.1% | 14.7% | 1.1 | $0.012$12.17 | – | 0.339 sp90 0.472 s |
| 16 | vera-medium-omni Open · 4.2B vera-medium-omni · scar-ai · decision model Parameters: 4.2B dense Base model: Qwen3.5-4B Both 52.5% (95%: 50.8%–54.2%) · Logic 44.1% · Math 56.8% Non-hallucination 7.9% · “I don't know” on 3.9% of questions Confidence: 39 answers kept at ≥99%, 39 right · 0 wrong · AUROC 0.777 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.024$24.19estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.031 s per question (median latency) | 52.5%50.8%–54.2% | Logic 44.1% Math 56.8% | 7.9% | 3.9% | 3.5 | $0.024$24.19estimate · RTX PRO 6000 Blackwell | 35 sRTX PRO 6000 | 0.031 sp90 0.036 s |
| 16 | StartLux-Decision-v1.1-4B-Instant Open · 4.0B StartLux-Decision-v1.1-4B-Instant · StartLux · decision model Parameters: 4B dense Both 52.4% (95%: 50.7%–54.1%) · Logic 45.9% · Math 55.8% Non-hallucination 14.4% · “I don't know” on 7.4% of questions Confidence: 117 answers kept at ≥99%, 117 right · 0 wrong · AUROC 0.763 Runs on NVIDIA DGX Spark (GB10) · cost $0.004$4.28estimate · DGX Spark per 1,000 questionsper 1M questions · 0.068 s per question (median latency) | 52.4%50.7%–54.1% | Logic 45.9% Math 55.8% | 14.4% | 7.4% | 5.4 | $0.004$4.28estimate · DGX Spark | 72 sDGX Spark | 0.068 sp90 0.080 s |
| 16 | Perplexity Decider Flash v1.3 Open · 4.0B Perplexity Decider Flash v1.3 · Perplexity · decision model Parameters: 4B dense Both 52.1% (95%: 50.4%–53.8%) · Logic 44.7% · Math 55.9% Non-hallucination 15.0% · “I don't know” on 7.6% of questions Confidence: 131 answers kept at ≥99%, 130 right · 1 wrong · AUROC 0.782 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.024$23.77estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.029 s per question (median latency) | 52.1%50.4%–53.8% | Logic 44.7% Math 55.9% | 15.0% | 7.6% | 11.1 | $0.024$23.77estimate · RTX PRO 6000 Blackwell | 34 sRTX PRO 6000 | 0.029 sp90 0.035 s |
| 16 | Cerno 9B v1.2 Open · 9.0B Cerno 9B v1.2 · Apertix AI · decision model Parameters: 9B dense Both 51.7% (95%: 50.0%–53.4%) · Logic 43.3% · Math 56.0% Non-hallucination 3.1% · “I don't know” on 1.5% of questions Confidence: 281 answers kept at ≥99%, 270 right · 11 wrong · AUROC 0.776 Runs on NVIDIA DGX Spark (GB10) · cost $0.011$10.82estimate · DGX Spark per 1,000 questionsper 1M questions · 0.163 s per question (median latency) | 51.7%50.0%–53.4% | Logic 43.3% Math 56.0% | 3.1% | 1.5% | 16.2 | $0.011$10.82estimate · DGX Spark | 183 sDGX Spark | 0.163 sp90 0.240 s |
| 16 | Valen-Decision-1.5-9B Open · 9.0B Valen-Decision-1.5-9B · Valen-Team · decision model Parameters: 9B dense Base model: Qwen3.5-9B Both 51.6% (95%: 49.9%–53.3%) · Logic 44.4% · Math 55.3% Non-hallucination 7.1% · “I don't know” on 3.6% of questions Confidence: 167 answers kept at ≥99%, 165 right · 2 wrong · AUROC 0.765 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.041$41.44estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.059 s per question (median latency) | 51.6%49.9%–53.3% | Logic 44.4% Math 55.3% | 7.1% | 3.6% | 4.5 | $0.041$41.44estimate · RTX PRO 6000 Blackwell | 60 sRTX PRO 6000 | 0.059 sp90 0.065 s |
| 16 | KnowLine 4B Gen5.5 Open · 4.0B KnowLine 4B Gen5.5 · PelaAI · decision model Parameters: 4B dense Both 51.3% (95%: 49.6%–53.0%) · Logic 42.3% · Math 56.0% Non-hallucination 4.4% · “I don't know” on 2.2% of questions Confidence: 72 answers kept at ≥99%, 71 right · 1 wrong · AUROC 0.752 Runs on NVIDIA DGX Spark (GB10) · cost $0.005$5.02estimate · DGX Spark per 1,000 questionsper 1M questions · 0.076 s per question (median latency) | 51.3%49.6%–53.0% | Logic 42.3% Math 56.0% | 4.4% | 2.2% | 10.4 | $0.005$5.02estimate · DGX Spark | 85 sDGX Spark | 0.076 sp90 0.104 s |
| 16 | Perplexity Decider Flash v1.2 Open · 4.0B Perplexity Decider Flash v1.2 · Perplexity · decision model Parameters: 4B dense Both 51.1% (95%: 49.4%–52.8%) · Logic 43.1% · Math 55.2% Non-hallucination 18.9% · “I don't know” on 10.0% of questions Confidence: 190 answers kept at ≥99%, 185 right · 5 wrong · AUROC 0.769 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.041 s per question (median latency) | 51.1%49.4%–52.8% | Logic 43.1% Math 55.2% | 18.9% | 10.0% | 8.9 | $0.003$2.72 list$0.032$31.55estimate · RTX PRO 6000 Blackwell | 46 sRTX PRO 6000 | 0.041 sp90 0.051 s |
| 17 | Mercury Decide Closed · Efficient Mercury Decide · Inception · decision model Parameters: not disclosed by maker · Closed; list price $0.04 per 1M input tokens: Efficient (source: https://openrouter.ai/inception/mercury-decide) Both 50.6% (95%: 48.9%–52.3%) · Logic 45.1% · Math 53.5% Non-hallucination 10.1% · “I don't know” on 5.1% of questions Confidence: 372 answers kept at ≥99%, 356 right · 16 wrong · AUROC 0.777 Runs on an API · cost $0.005$5.46 per 1,000 questionsper 1M questions · 0.293 s per question (median latency) | 50.6%48.9%–52.3% | Logic 45.1% Math 53.5% | 10.1% | 5.1% | 15.7 | $0.005$5.46 | – | 0.293 sp90 0.349 s |
| 17 | d1 Closed · Efficient d1 · Liquid AI · decision model Parameters: not disclosed by maker · Closed; list price $0.02 per 1M input tokens: Efficient (source: https://www.liquid.ai/pricing) Both 50.5% (95%: 48.8%–52.3%) · Logic 40.8% · Math 55.6% Non-hallucination 6.4% · “I don't know” on 3.3% of questions Confidence: 208 answers kept at ≥99%, 207 right · 1 wrong · AUROC 0.757 Runs on an API · cost $0.004$3.98 per 1,000 questionsper 1M questions · 0.185 s per question (median latency) | 50.5%48.8%–52.3% | Logic 40.8% Math 55.6% | 6.4% | 3.3% | 5.8 | $0.004$3.98 | – | 0.185 sp90 0.292 s |
| 17 | Open-Jev-27B-v1.1 Open · 27B Open-Jev-27B-v1.1 · Zefan Cai · decision model Parameters: 27B dense Base model: Qwen3.8-27B + adapter Both 50.5% (95%: 48.7%–52.2%) · Logic 42.6% · Math 54.6% Non-hallucination 14.5% · “I don't know” on 7.6% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.759 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.212$212estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.250 s per question (median latency) | 50.5%48.7%–52.2% | Logic 42.6% Math 54.6% | 14.5% | 7.6% | 18.3 | $0.212$212estimate · RTX PRO 6000 Blackwell | 307 sRTX PRO 6000 | 0.250 sp90 0.482 s |
| 17 | OneJev-27B Open · 27B OneJev-27B · OmniJev · decision model Parameters: 27B dense Both 49.3% (95%: 47.6%–51.0%) · Logic 40.6% · Math 53.8% Non-hallucination 1.6% · “I don't know” on 0.8% of questions Confidence: 30 answers kept at ≥99%, 30 right · 0 wrong · AUROC 0.770 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.069$69.42estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.080 s per question (median latency) | 49.3%47.6%–51.0% | Logic 40.6% Math 53.8% | 1.6% | 0.8% | 3.1 | $0.069$69.42estimate · RTX PRO 6000 Blackwell | 100 sRTX PRO 6000 | 0.080 sp90 0.134 s |
| 17 | Clef Open · 27B Clef · Cloudflare · decision model Parameters: 27B dense Both 47.6% (95%: 45.9%–49.3%) · Logic 40.8% · Math 51.1% Non-hallucination 2.4% · “I don't know” on 1.2% of questions Confidence: 18 answers kept at ≥99%, 18 right · 0 wrong · AUROC 0.747 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.185 s per question (median latency) | 47.6%45.9%–49.3% | Logic 40.8% Math 51.1% | 2.4% | 1.2% | 12.6 | $0.081$81.46 list$0.135$135estimate · RTX PRO 6000 Blackwell | 195 sRTX PRO 6000 | 0.185 sp90 0.216 s |
| 17 | Verdin-4B-v1.1 Open · 4.0B Verdin-4B-v1.1 · victorbennett · decision model Parameters: 4B dense Base model: Qwen3.5-4B Both 47.5% (95%: 45.8%–49.2%) · Logic 39.7% · Math 51.5% Non-hallucination 8.9% · “I don't know” on 4.8% of questions Confidence: 165 answers kept at ≥99%, 161 right · 4 wrong · AUROC 0.747 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.033$33.45estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.046 s per question (median latency) | 47.5%45.8%–49.2% | Logic 39.7% Math 51.5% | 8.9% | 4.8% | 7.3 | $0.033$33.45estimate · RTX PRO 6000 Blackwell | 48 sRTX PRO 6000 | 0.046 sp90 0.052 s |
| 17 | Chinese Decision Pointer 4B Open · 4.0B Chinese Decision Pointer 4B · quincyqiang · decision model Parameters: 4B dense Base model: Qwen3.5-4B Both 47.0% (95%: 45.3%–48.7%) · Logic 40.4% · Math 50.4% Non-hallucination 7.8% · “I don't know” on 4.2% of questions Confidence: 194 answers kept at ≥99%, 188 right · 6 wrong · AUROC 0.743 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.025$25.03estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.030 s per question (median latency) | 47.0%45.3%–48.7% | Logic 40.4% Math 50.4% | 7.8% | 4.2% | 14.3 | $0.025$25.03estimate · RTX PRO 6000 Blackwell | 36 sRTX PRO 6000 | 0.030 sp90 0.042 s |
| 18 | Clef-Flash Open · 9.0B Clef-Flash · Cloudflare · decision model Parameters: 9B dense Both 46.1% (95%: 44.4%–47.8%) · Logic 38.9% · Math 49.8% Non-hallucination 6.9% · “I don't know” on 3.8% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.707 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.103 s per question (median latency) | 46.1%44.4%–47.8% | Logic 38.9% Math 49.8% | 6.9% | 3.8% | 5.7 | $0.013$12.90 list$0.075$75.10estimate · RTX PRO 6000 Blackwell | 109 sRTX PRO 6000 | 0.103 sp90 0.131 s |
| 18 | Winnow-12B Open · 12B Winnow-12B · EldanRing · decision model Parameters: 12B dense Both 44.4% (95%: 42.7%–46.1%) · Logic 39.7% · Math 46.9% Non-hallucination 17.1% · “I don't know” on 10.2% of questions Confidence: 53 answers kept at ≥99%, 52 right · 1 wrong · AUROC 0.737 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.043$42.91estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.055 s per question (median latency) | 44.4%42.7%–46.1% | Logic 39.7% Math 46.9% | 17.1% | 10.2% | 10.5 | $0.043$42.91estimate · RTX PRO 6000 Blackwell | 62 sRTX PRO 6000 | 0.055 sp90 0.072 s |
| 18 | Cygnet Open · 12B Cygnet · Blockbrain Labs · decision model Parameters: 11.95B dense Base model: Gemma 4 12B IT Both 42.6% (95%: 40.9%–44.3%) · Logic 39.3% · Math 44.3% Non-hallucination 7.3% · “I don't know” on 4.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.729 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.038 s per question (median latency) | 42.6%40.9%–44.3% | Logic 39.3% Math 44.3% | 7.3% | 4.3% | 7.4 | $0.029$29.34 list$0.029$28.61estimate · RTX PRO 6000 Blackwell | 41 sRTX PRO 6000 | 0.038 sp90 0.044 s |
| 19 | Metask-Jev-4B Open · 4.5B Metask-Jev-4B · Metask Lab · decision model Parameters: 4.54B dense Both 36.7% (95%: 35.1%–38.4%) · Logic 32.1% · Math 39.2% Non-hallucination 2.0% · “I don't know” on 1.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.659 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.086$85.83estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.117 s per question (median latency) | 36.7%35.1%–38.4% | Logic 32.1% Math 39.2% | 2.0% | 1.3% | 2.2 | $0.086$85.83estimate · RTX PRO 6000 Blackwell | 124 sRTX PRO 6000 | 0.117 sp90 0.146 s |
| 19 | StartLux-Decision-v1.1-2B-Instant Open · 2.0B StartLux-Decision-v1.1-2B-Instant · StartLux · decision model Parameters: 2B dense Both 35.9% (95%: 34.3%–37.6%) · Logic 30.8% · Math 38.7% Non-hallucination 16.4% · “I don't know” on 11.1% of questions Confidence: 4 answers kept at ≥99%, 4 right · 0 wrong · AUROC 0.675 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$2.36estimate · DGX Spark per 1,000 questionsper 1M questions · 0.036 s per question (median latency) | 35.9%34.3%–37.6% | Logic 30.8% Math 38.7% | 16.4% | 11.1% | 3.3 | $0.002$2.36estimate · DGX Spark | 40 sDGX Spark | 0.036 sp90 0.042 s |
| 19 | GLiDE 2B Open · 1.9B GLiDE 2B · Fastino · decision model Parameters: 1.88B dense Both 35.2% (95%: 33.6%–36.9%) · Logic 32.3% · Math 36.8% Non-hallucination 23.1% · “I don't know” on 15.9% of questions Confidence: 30 answers kept at ≥99%, 29 right · 1 wrong · AUROC 0.637 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.94estimate · DGX Spark per 1,000 questionsper 1M questions · 0.028 s per question (median latency) | 35.2%33.6%–36.9% | Logic 32.3% Math 36.8% | 23.1% | 15.9% | 16.8 | $0.002$1.94estimate · DGX Spark | 33 sDGX Spark | 0.028 sp90 0.037 s |
| 19 | d1-2B Open · 1.9B d1-2B · Liquid AI · decision model Parameters: 1.92B dense Base model: LFM3-VL-2B Both 34.9% (95%: 33.3%–36.5%) · Logic 30.7% · Math 37.1% Non-hallucination 82.5% · “I don't know” on 70.4% of questions Confidence: 7 answers kept at ≥99%, 7 right · 0 wrong · AUROC 0.648 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.014$14.30estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.017 s per question (median latency) | 34.9%33.3%–36.5% | Logic 30.7% Math 37.1% | 82.5% | 70.4% | 5.9 | $0.014$14.30estimate · RTX PRO 6000 Blackwell | 21 sRTX PRO 6000 | 0.017 sp90 0.021 s |
| 19 | KnowLine-2B-Gen1 Open · 2.0B KnowLine-2B-Gen1 · PelaAI · decision model Parameters: 2B dense Base model: Qwen3.5-2B Both 34.1% (95%: 32.5%–35.7%) · Logic 31.7% · Math 35.3% Non-hallucination 6.8% · “I don't know” on 4.7% of questions Confidence: 16 answers kept at ≥99%, 16 right · 0 wrong · AUROC 0.626 Runs on NVIDIA DGX Spark (GB10) · cost $0.003$3.40estimate · DGX Spark per 1,000 questionsper 1M questions · 0.051 s per question (median latency) | 34.1%32.5%–35.7% | Logic 31.7% Math 35.3% | 6.8% | 4.7% | 9.0 | $0.003$3.40estimate · DGX Spark | 57 sDGX Spark | 0.051 sp90 0.063 s |
| 20 | GLiDE 0.8B Open · 0.8B GLiDE 0.8B · Fastino · decision model Parameters: 0.75B dense Both 31.1% (95%: 29.5%–32.7%) · Logic 29.2% · Math 32.1% Non-hallucination 22.7% · “I don't know” on 16.8% of questions Confidence: 13 answers kept at ≥99%, 13 right · 0 wrong · AUROC 0.602 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.38estimate · DGX Spark per 1,000 questionsper 1M questions · 0.020 s per question (median latency) | 31.1%29.5%–32.7% | Logic 29.2% Math 32.1% | 22.7% | 16.8% | 17.1 | $0.001$1.38estimate · DGX Spark | 23 sDGX Spark | 0.020 sp90 0.025 s |
| 20 | StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Both 28.0% (95%: 26.5%–29.5%) · Logic 24.7% · Math 29.7% Non-hallucination 14.9% · “I don't know” on 11.1% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.605 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.62estimate · DGX Spark per 1,000 questionsper 1M questions · 0.024 s per question (median latency) | 28.0%26.5%–29.5% | Logic 24.7% Math 29.7% | 14.9% | 11.1% | 5.8 | $0.002$1.62estimate · DGX Spark | 27 sDGX Spark | 0.024 sp90 0.029 s |
| 20 | OneJev1.6-0.8B Open · 0.8B OneJev1.6-0.8B · OmniJev · decision model Parameters: 0.8B dense Both 27.8% (95%: 26.3%–29.4%) · Logic 27.0% · Math 28.2% Non-hallucination 2.0% · “I don't know” on 1.5% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.576 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.11estimate · DGX Spark per 1,000 questionsper 1M questions · 0.015 s per question (median latency) | 27.8%26.3%–29.4% | Logic 27.0% Math 28.2% | 2.0% | 1.5% | 10.4 | $0.001$1.11estimate · DGX Spark | 19 sDGX Spark | 0.015 sp90 0.017 s |
| 20 | KnowLine-0.8B-Gen1 Open · 0.8B KnowLine-0.8B-Gen1 · PelaAI · decision model Parameters: 0.8B dense Base model: Qwen3.5-0.8B Both 27.7% (95%: 26.2%–29.2%) · Logic 26.7% · Math 28.2% Non-hallucination 0.6% · “I don't know” on 0.5% of questions Confidence: 2 answers kept at ≥99%, 2 right · 0 wrong · AUROC 0.590 Runs on NVIDIA DGX Spark (GB10) · cost $0.002$1.80estimate · DGX Spark per 1,000 questionsper 1M questions · 0.026 s per question (median latency) | 27.7%26.2%–29.2% | Logic 26.7% Math 28.2% | 0.6% | 0.5% | 9.8 | $0.002$1.80estimate · DGX Spark | 30 sDGX Spark | 0.026 sp90 0.035 s |
| 21 | Strands Decider 2B Open · 2.0B Strands Decider 2B · Strands Agents · decision model Parameters: 2B dense Both 26.5% (95%: 25.0%–28.1%) · Logic 22.4% · Math 28.7% Non-hallucination 5.0% · “I don't know” on 3.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.573 Runs on NVIDIA RTX PRO 6000 Blackwell (96 GB) · cost $0.027$26.72estimate · RTX PRO 6000 Blackwell per 1,000 questionsper 1M questions · 0.035 s per question (median latency) | 26.5%25.0%–28.1% | Logic 22.4% Math 28.7% | 5.0% | 3.7% | 9.9 | $0.027$26.72estimate · RTX PRO 6000 Blackwell | 39 sRTX PRO 6000 | 0.035 sp90 0.042 s |
| 21 | GLiClass v4 Edge Open · 0.4B GLiClass v4 Edge · decision model Parameters: 0.4B dense Base model: DeBERTa-v3-large Both 23.0% (95%: 21.6%–24.4%) · Logic 23.6% · Math 22.7% Non-hallucination 15.6% · “I don't know” on 12.7% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.511 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$1.01estimate · DGX Spark per 1,000 questionsper 1M questions · 0.012 s per question (median latency) | 23.0%21.6%–24.4% | Logic 23.6% Math 22.7% | 15.6% | 12.7% | 7.6 | $0.001$1.01estimate · DGX Spark | 17 sDGX Spark | 0.012 sp90 0.023 s |
| 22 | GLiDE 0.4B Open · 0.4B GLiDE 0.4B · Fastino · decision model Parameters: 0.40B dense Both 21.5% (95%: 20.1%–22.9%) · Logic 23.2% · Math 20.6% Non-hallucination 73.9% · “I don't know” on 68.3% of questions Confidence, kept at ≥99%: No answers at this threshold · AUROC 0.520 Runs on NVIDIA DGX Spark (GB10) · cost $0.001$0.83estimate · DGX Spark per 1,000 questionsper 1M questions · 0.011 s per question (median latency) | 21.5%20.1%–22.9% | Logic 23.2% Math 20.6% | 73.9% | 68.3% | 11.0 | $0.001$0.83estimate · DGX Spark | 14 sDGX Spark | 0.011 sp90 0.014 s |
No finished run of this kind on this part yet.
updated 16 Nov 2026 22:07 UTC
Score: the share of the 3,288 Logic and Math questions together (setups that finished both) answered right, the whisker its 95% interval. # is the tier: rows in one tier are not told apart by the paired test (a line separates tiers); – : with web or sample, not tiered.
Class: by the maker's total parameters, or by list price when the size is undisclosed *
Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.
One row per model: its highest reasoning setting (off < low < medium < high), a model with one setup that setup; a tie by the earliest run, then the name. Perplexity Decider 27B v1.2: its API setup on every part (its other setup is under other settings). The model's other settings are under its row; All setups lists every setup.
# is the tier: rows in accuracy order, the top row starts Tier 1, a row joins the tier unless the paired test behind the tie letters (exact McNemar, Holm-corrected over all pairs of tiered rows in view, p < 0.05) separates it from the tier's top row; otherwise it starts the next tier. – : with web or sample, not tiered.
Paired comparisons
| Pair | Benchmark | Questions | First | Second | Difference (pts) | p | Holm p | Result |
|---|---|---|---|---|---|---|---|---|
| LFM3-1.7B (on) vs MiniCPM5-2B (on) | Logic | 1,125 | 84.2% | 88.9% | -4.6 | < 0.0001 | < 0.0001 | MiniCPM5-2B (on) ahead |
| LFM3-1.7B (on) vs MiniCPM5-2B (on) | Math | 2,163 | 98.9% | 99.2% | -0.3 | 0.043 | 0.171 | no clear difference |
| LFM3-1.7B (on) vs MiniCPM5-2B (on) | English reading | 547 | 71.1% | 78.1% | -7.0 | < 0.0001 | < 0.0001 | MiniCPM5-2B (on) ahead |
| LFM3-1.7B (off) vs MiniCPM5-2B (off) | Logic | 1,125 | 84.6% | 83.6% | +1.0 | 0.238 | 0.238 | no clear difference |
| LFM3-1.7B (off) vs MiniCPM5-2B (off) | Math | 2,163 | 98.9% | 98.2% | +0.8 | 0.0002 | 0.0017 | LFM3-1.7B (off) ahead |
| LFM3-1.7B (off) vs MiniCPM5-2B (off) | English reading | 547 | 71.8% | 67.8% | +4.0 | 0.022 | 0.109 | no clear difference |
| LFM3-1.7B (on) vs Gemma 4 E4B (on, 32k, sampled) | Logic | 1,125 | 84.2% | 85.9% | -1.7 | 0.051 | 0.171 | no clear difference |
| LFM3-1.7B (on) vs Gemma 4 E4B (on, 32k, sampled) | Math | 2,163 | 98.9% | 98.5% | +0.4 | 0.050 | 0.171 | no clear difference |
| LFM3-1.7B (on) vs Gemma 4 E2B (on, 32k, sampled) | Logic | 1,125 | 84.2% | 76.8% | +7.5 | < 0.0001 | < 0.0001 | LFM3-1.7B (on) ahead |
| LFM3-1.7B (on) vs Gemma 4 E2B (on, 32k, sampled) | Math | 2,163 | 98.9% | 97.7% | +1.3 | < 0.0001 | < 0.0001 | LFM3-1.7B (on) ahead |
| LFM3-1.7B (on) vs Claude Opus 5.5 | Math | 2,163 | 98.9% | 99.8% | -0.8 | < 0.0001 | < 0.0001 | Claude Opus 5.5 ahead |
| LFM3-1.7B (on) vs Claude Sonnet 5.5 | Math | 2,163 | 98.9% | 99.7% | -0.7 | < 0.0001 | < 0.0001 | Claude Sonnet 5.5 ahead |
| LFM3-1.7B (on) vs Grok 4.7 | Math | 2,163 | 98.9% | 99.5% | -0.6 | 0.0001 | 0.0011 | Grok 4.7 ahead |
| LFM3-1.7B (on) vs Muse Ember | Math | 2,163 | 98.9% | 99.6% | -0.6 | 0.0001 | 0.0011 | Muse Ember ahead |
| LFM3-1.7B (on) vs Gemma 4 31B | Math | 2,163 | 98.9% | 99.4% | -0.5 | 0.0050 | 0.030 | Gemma 4 31B ahead |
Each question's score is the mean over a setup's runs (one run: 0 or 1; three seeds: 0, ⅓, ⅔ or 1); only questions every run of both setups answered count. The difference is the first setup's mean score minus the second's, in points. p: two-sided sign-flip (paired permutation) test on the sum of the per-question differences themselves (each flip keeps its size, e.g. ⅓ or 1), 200,000 random flips with a fixed seed. Holm: corrected over all 15 planned tests in this table, a pending one counted as p = 1 until its runs are in. 'Ahead' when the Holm-corrected p is below 0.05. LFM3-1.7B against the large models is its thinking-on 3-seed setup. English reading: Reading's 547 English questions (its per-question results, turkce #10695). A 3-seed row's questions per minute, cost and GPU time elsewhere on the page are the mean of its three runs' own values (each run's questions over its own wall time), not pooled.
Chat and decision models on Logic
OpenAI Google Alibaba Anthropic DeepSeek Thinking Machines Meta Xiaomi MiniMax Z.ai IBM OpenBMB Liquid AI Other KRAFTON eCloud Technology MachinePulse REAX PelaAI Poindexter Labs Chance, 20% (five options)
Each bar is a leaderboard row on Logic with its 95% interval: the chat models and the highest decision models. A chat model writes a reply that ends in a letter; a decision model scores the five options and picks one.
Logic and Math, setup by setup
GPT-6 Astra (chat): Logic 96.7% (95.5%–97.6%), Math 99.6% (99.3%–99.8%)
Gemini 4 Carbon (chat): Logic 96.7% (95.5%–97.6%), Math 99.6% (99.2%–99.8%)
GPT-6.1 Sol (chat): Logic 96.5% (95.3%–97.5%), Math 99.6% (99.3%–99.8%)
Qwen4-Max (high · chat): Logic 96.3% (95.0%–97.2%), Math 99.8% (99.5%–99.9%)
Claude Opus 5.5 (chat): Logic 96.2% (94.9%–97.2%), Math 99.8% (99.5%–99.9%)
DeepSeek-V4.1-Flash (high · chat · 1017): Logic 96.2% (94.9%–97.2%), Math 99.6% (99.3%–99.8%)
Gemini 4 Hydrogen (chat): Logic 96.1% (94.8%–97.1%), Math 99.6% (99.2%–99.8%)
Claude Sonnet 5.5 (chat): Logic 95.5% (94.1%–96.5%), Math 99.7% (99.3%–99.8%)
Inkling 2 Small (high · chat): Logic 95.6% (94.3%–96.7%), Math 99.5% (99.2%–99.7%)
Claude Fable 5.1 (chat): Logic 95.2% (93.8%–96.3%), Math 99.7% (99.4%–99.9%)
Inkling-Small (high · chat · Jul 2026): Logic 95.4% (94.0%–96.5%), Math 99.6% (99.3%–99.8%)
Muse Ember (max · chat): Logic 95.5% (94.1%–96.5%), Math 99.6% (99.2%–99.8%)
MiMo-V2.6-Flash (on · chat): Logic 95.1% (93.7%–96.2%), Math 99.4% (99.0%–99.7%)
Qwen3.8 Flash (off · chat): Logic 94.0% (92.4%–95.2%), Math 99.5% (99.1%–99.7%)
Qwen3.8-27B (low · chat): Logic 94.0% (92.5%–95.3%), Math 99.2% (98.7%–99.5%)
GPT-6.1 Terra (chat): Logic 94.0% (92.4%–95.2%), Math 99.2% (98.7%–99.5%)
Gemma 4 31B (chat): Logic 93.3% (91.7%–94.6%), Math 99.4% (99.0%–99.6%)
Qwen3.5-4B (on · chat): Logic 92.4% (90.7%–93.8%), Math 99.4% (99.0%–99.7%)
GLM-5.3-Flash (high · chat): Logic 93.2% (91.5%–94.5%), Math 98.8% (98.2%–99.2%)
MiniMax-M3.1-Flash (high · chat): Logic 94.0% (92.4%–95.2%), Math 98.2% (97.6%–98.7%)
Granite-4.2-3B (high · chat): Logic 91.6% (89.9%–93.1%), Math 99.2% (98.7%–99.5%)
MiniCPM5-2B (on · chat): Logic 88.7% (86.7%–90.4%), Math 99.3% (98.8%–99.5%)
Qwen3.5-4B (off · chat): Logic 90.1% (88.3%–91.7%), Math 98.5% (97.9%–98.9%)
Gemma 4 12B Unified (chat): Logic 87.7% (85.7%–89.5%), Math 98.8% (98.3%–99.2%)
LFM2.5-24B-A2B (on · chat): Logic 86.7% (84.6%–88.5%), Math 99.0% (98.5%–99.4%)
Gemma 4 E4B (on · chat · 32k cap · sampled): Logic 86.0% (83.9%–87.9%), Math 98.8% (98.2%–99.1%)
LFM3-1.7B (off · chat): Logic 85.5% (83.3%–87.4%), Math 98.7% (98.1%–99.1%)
LFM3-1.7B (off · chat · NVIDIA RTX PRO 6000 Blackwell): Logic 85.0% (82.8%–86.9%), Math 99.0% (98.5%–99.4%)
LFM3-1.7B (on · chat): Logic 84.4% (82.2%–86.4%), Math 98.9% (98.4%–99.3%)
Gemma 4 E4B (on · chat · 32k cap): Logic 85.0% (82.8%–86.9%), Math 98.5% (97.9%–99.0%)
MiniCPM5-2B (off · chat): Logic 83.7% (81.5%–85.8%), Math 98.3% (97.7%–98.8%)
Gemma 4 E4B (chat): Logic 83.2% (80.9%–85.3%), Math 98.1% (97.4%–98.6%)
Gemma 4 E4B (on · chat): Logic 82.6% (80.3%–84.7%), Math 98.2% (97.6%–98.7%)
Gemma 4 E2B (on · chat · 32k cap · sampled): Logic 76.0% (73.4%–78.4%), Math 97.6% (96.9%–98.2%)
Gemma 4 E2B (on · chat · 32k cap): Logic 75.9% (73.3%–78.3%), Math 97.8% (97.1%–98.4%)
Gemma 4 E2B (on · chat): Logic 74.6% (72.0%–77.0%), Math 96.9% (96.1%–97.6%)
Gemma 4 E2B (chat): Logic 72.9% (70.2%–75.4%), Math 96.8% (95.9%–97.4%)
Raon-1 nano (on · chat): Logic 70.9% (68.2%–73.5%), Math 97.0% (96.2%–97.6%)
LFM2.5-2.6B (on · chat): Logic 75.4% (72.8%–77.8%), Math 94.0% (92.9%–94.9%)
Qwen3.5-2B (on · chat): Logic 70.3% (67.6%–72.9%), Math 94.7% (93.7%–95.6%)
Qwen3-14B (off · chat): Logic 65.3% (62.5%–68.1%), Math 95.8% (94.9%–96.6%)
Qwen3.5-2B (off · chat): Logic 64.8% (62.0%–67.5%), Math 94.6% (93.6%–95.5%)
LFM2.5-8B-A1B (on · chat): Logic 60.2% (57.3%–63.0%), Math 96.3% (95.4%–97.0%)
GPT-6 Luna (chat): Logic 58.0% (55.0%–60.8%), Math 93.7% (92.6%–94.6%)
Raon-1 nano (off · chat): Logic 65.1% (62.2%–67.8%), Math 89.6% (88.2%–90.8%)
Erk-14B (off · chat): Logic 54.2% (51.3%–57.1%), Math 90.8% (89.5%–91.9%)
Karpo Decision 31B: Logic 60.7% (57.8%–63.5%), Math 81.6% (79.9%–83.1%)
KnowLine-31B-Gen2: Logic 58.9% (56.0%–61.8%), Math 80.3% (78.5%–81.9%)
REAX 1.3: Logic 59.3% (56.4%–62.1%), Math 78.0% (76.2%–79.7%)
KnowLine-31B-Gen1.5: Logic 59.0% (56.1%–61.9%), Math 77.4% (75.6%–79.1%)
Copernicus-31B: Logic 58.7% (55.8%–61.5%), Math 77.3% (75.5%–79.1%)
Sieve 31B v3: Logic 57.9% (55.0%–60.7%), Math 77.1% (75.3%–78.8%)
Logits Sigma-1 Duo: Logic 57.8% (54.9%–60.6%), Math 75.2% (73.4%–77.0%)
d1-ultra: Logic 54.9% (52.0%–57.8%), Math 74.1% (72.2%–75.9%)
GLiDE 27B: Logic 58.4% (55.5%–61.2%), Math 72.0% (70.1%–73.8%)
Mammoth-1: Logic 54.5% (51.6%–57.4%), Math 74.0% (72.1%–75.8%)
Perplexity Decider 27B v1.3 (NVIDIA RTX PRO 6000 Blackwell): Logic 54.8% (51.8%–57.6%), Math 72.5% (70.6%–74.3%)
d3-ultra: Logic 54.6% (51.7%–57.5%), Math 71.5% (69.6%–73.4%)
GLiDE (API): Logic 54.5% (51.6%–57.4%), Math 70.8% (68.8%–72.7%)
d3: Logic 54.5% (51.6%–57.4%), Math 70.4% (68.5%–72.3%)
Drex Max v1.1: Logic 54.8% (51.8%–57.6%), Math 70.1% (68.1%–72.0%)
GLiDE (v1.2): Logic 55.2% (52.3%–58.1%), Math 69.9% (67.9%–71.8%)
Drex Max: Logic 53.1% (50.1%–56.0%), Math 70.3% (68.4%–72.2%)
Perplexity Decider 27B v1.2 (API): Logic 53.3% (50.4%–56.2%), Math 69.8% (67.8%–71.7%)
Gemini Decision 1 (20261111): Logic 50.0% (47.1%–53.0%), Math 71.3% (69.4%–73.2%)
d1.2-ultra: Logic 51.6% (48.6%–54.5%), Math 69.9% (67.9%–71.8%)
Perplexity Decider 27B v1.2 (API · with web): Logic 52.5% (49.6%–55.4%), Math 69.3% (67.3%–71.2%)
Turion Jev 31B: Logic 52.3% (49.3%–55.2%), Math 69.0% (67.0%–70.9%)
SubQ Snap 1.1: Logic 52.4% (49.4%–55.3%), Math 68.6% (66.6%–70.5%)
celeris-1-decision: Logic 47.3% (44.4%–50.2%), Math 70.0% (68.0%–71.9%)
celeris-1.1-decision: Logic 47.3% (44.4%–50.2%), Math 69.9% (67.9%–71.8%)
KnowLine-12B-Gen1: Logic 50.6% (47.7%–53.5%), Math 66.3% (64.2%–68.2%)
d1.2: Logic 49.2% (46.2%–52.1%), Math 65.5% (63.5%–67.5%)
Jev 1.14: Logic 48.2% (45.3%–51.1%), Math 65.1% (63.1%–67.1%)
d1.1: Logic 46.0% (43.2%–49.0%), Math 65.2% (63.2%–67.2%)
Jev 1.13: Logic 46.3% (43.4%–49.2%), Math 64.1% (62.0%–66.1%)
Adjudex 9B v5: Logic 46.4% (43.5%–49.3%), Math 62.1% (60.0%–64.1%)
Elephant-1: Logic 46.4% (43.5%–49.3%), Math 61.3% (59.3%–63.4%)
Jeffan-9B-1211: Logic 48.2% (45.3%–51.1%), Math 59.8% (57.7%–61.9%)
KnowLine-9B-Gen2: Logic 43.9% (41.0%–46.8%), Math 60.1% (58.0%–62.1%)
GLiDE 9B: Logic 47.8% (44.9%–50.7%), Math 57.9% (55.8%–60.0%)
KnowLine-9B-Gen1: Logic 44.3% (41.4%–47.2%), Math 59.8% (57.7%–61.8%)
StartLux-Decision-v1.1-9B-Instant: Logic 46.6% (43.7%–49.5%), Math 58.4% (56.3%–60.5%)
Microsoft-Decision-1: Logic 44.4% (41.6%–47.4%), Math 59.4% (57.3%–61.4%)
Dexy-Decision-9B-V2: Logic 45.3% (42.4%–48.3%), Math 58.8% (56.7%–60.9%)
Drex 2.0: Logic 41.9% (39.0%–44.8%), Math 59.0% (56.9%–61.0%)
vera-medium-omni: Logic 44.1% (41.2%–47.0%), Math 56.8% (54.7%–58.9%)
StartLux-Decision-v1.1-4B-Instant: Logic 45.9% (43.0%–48.8%), Math 55.8% (53.7%–57.9%)
Perplexity Decider Flash v1.3: Logic 44.7% (41.8%–47.6%), Math 55.9% (53.8%–58.0%)
Cerno 9B v1.2: Logic 43.3% (40.4%–46.2%), Math 56.0% (53.9%–58.1%)
Valen-Decision-1.5-9B: Logic 44.4% (41.5%–47.3%), Math 55.3% (53.2%–57.4%)
KnowLine 4B Gen5.5: Logic 42.3% (39.5%–45.2%), Math 56.0% (53.9%–58.1%)
Perplexity Decider Flash v1.2: Logic 43.1% (40.2%–46.0%), Math 55.2% (53.1%–57.3%)
Mercury Decide: Logic 45.1% (42.2%–48.0%), Math 53.5% (51.4%–55.6%)
d1: Logic 40.8% (38.0%–43.7%), Math 55.6% (53.5%–57.7%)
Open-Jev-27B-v1.1: Logic 42.6% (39.7%–45.5%), Math 54.6% (52.4%–56.6%)
OneJev-27B: Logic 40.6% (37.8%–43.5%), Math 53.8% (51.7%–55.9%)
Clef: Logic 40.8% (38.0%–43.7%), Math 51.1% (49.0%–53.2%)
Verdin-4B-v1.1: Logic 39.7% (36.9%–42.6%), Math 51.5% (49.4%–53.6%)
Chinese Decision Pointer 4B: Logic 40.4% (37.5%–43.3%), Math 50.4% (48.3%–52.5%)
Clef-Flash: Logic 38.9% (36.1%–41.8%), Math 49.8% (47.7%–51.9%)
Winnow-12B: Logic 39.7% (36.9%–42.6%), Math 46.9% (44.8%–49.0%)
Cygnet: Logic 39.3% (36.5%–42.2%), Math 44.3% (42.3%–46.4%)
Metask-Jev-4B: Logic 32.1% (29.4%–34.9%), Math 39.2% (37.1%–41.2%)
StartLux-Decision-v1.1-2B-Instant: Logic 30.8% (28.1%–33.5%), Math 38.7% (36.6%–40.7%)
GLiDE 2B: Logic 32.3% (29.6%–35.1%), Math 36.8% (34.7%–38.8%)
d1-2B: Logic 30.7% (28.0%–33.4%), Math 37.1% (35.1%–39.1%)
KnowLine-2B-Gen1: Logic 31.7% (29.1%–34.5%), Math 35.3% (33.3%–37.3%)
GLiDE 0.8B: Logic 29.2% (26.6%–31.9%), Math 32.1% (30.2%–34.1%)
StartLux-Decision-v1.1-0.8B-Instant: Logic 24.7% (22.3%–27.3%), Math 29.7% (27.8%–31.6%)
OneJev1.6-0.8B: Logic 27.0% (24.5%–29.7%), Math 28.2% (26.4%–30.2%)
KnowLine-0.8B-Gen1: Logic 26.7% (24.2%–29.3%), Math 28.2% (26.3%–30.1%)
Strands Decider 2B: Logic 22.4% (20.1%–24.9%), Math 28.7% (26.8%–30.6%)
GLiClass v4 Edge: Logic 23.6% (21.2%–26.1%), Math 22.7% (20.9%–24.5%)
GLiDE 0.4B: Logic 23.2% (20.8%–25.8%), Math 20.6% (18.9%–22.3%)
Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell): Logic 53.2% (50.3%–56.1%), Math 70.3% (68.4%–72.2%)
OpenAI Google Alibaba Anthropic DeepSeek Thinking Machines Meta Xiaomi Z.ai MiniMax IBM OpenBMB Liquid AI Other KRAFTON eCloud Technology MachinePulse PelaAI REAX Poindexter Labs Sthānika AI Logits Fastino Base Labs Perplexity vLLM Semantic Router Nace.AI Turion.AI Subquadratic Celeris TypeSafe newsbang tattrongvu StartLux Microsoft Patrick-18 scar-ai Apertix AI Valen-Team Inception Zefan Cai OmniJev Cloudflare victorbennett quincyqiang EldanRing Blockbrain Labs Metask Lab Strands Agents
One dot per setup that has finished both benchmarks (116 setups, chat models included): across, its Logic accuracy; up, its Math accuracy; the whiskers are 95% intervals. The count is of setups, not models (Perplexity Decider 27B v1.2's two setups count twice). Above the dashed line a setup does better on Math.
Numerical reasoning is the decision models' lowest-scoring logic topic: median 34.0%, against verbal reasoning 50.5%, formal logic 79.5%
Median of the 69 decision setups Median of the 47 chat setups
The median accuracy of the finished setups on each logic topic, decision models and chat models apart; it is the lowest topic for 67 of the 69 decision setups. Medians over finished runs only.
How the scores work
- One question per request. A decision model scores the five options and picks one; a chat model writes a reply that ends in a letter.
- Accuracy is right / all questions. No partial credit; a reply without a letter counts as wrong.
- Uncertainty is shown. Intervals are 95% Wilson intervals; two setups are tied unless a paired exact test (McNemar) separates them after Holm's correction.
- Costs are what a run would cost. The provider's price × the usage its API reported (unmarked), the maker's list price × our token counts (“list”), or for our own hardware its wall-clock time from first to last answer while it was running (pauses, estimated from gaps of more than 10 minutes between answers, left out), × $2.49/h per RTX PRO 6000 or × $0.213/h per DGX Spark, shown as “On our NVIDIA RTX PRO 6000 Blackwell (96 GB)” or “On our NVIDIA DGX Spark (GB10)” (at most: runs that shared the hardware are each charged in full).
- The questions are English translations from six published Turkish question books; about a third of the logic questions and nearly all math questions are past public questions of ÖSYM, Turkey's national exam centre.