Reading bench: Text

Leaderboard

One row per model: Turkish accuracy
TierModelAccuracyof 2,198 questions, 95% intervalReadingof 656 questionsGrammarof 1,542 questionsNon-hallucination% “I don't know” of answers not rightConfidencecalibration gap, pointsCostper 1,000 questionsper 1M questionsLatencymedian s per request
1Gemini 4 Carbon high · chat‡ Closed · FlagshipLatency 217.7 sCost $31.309$31,309 per 1,000per 1M

Gemini 4 Carbon high · chat · Google · chat model

Parameters: not disclosed by maker

Class: Flagship, by its standard list price: $4.00 per 1M input tokens (https://ai.google.dev/gemini-api/docs/pricing, read 2026-11-12)

Accuracy 98.5% (95%: 97.8–98.9%)

Reading 97.6% · grammar 98.8%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 17.9% · answered 99.7% · error rate 1.5%

Confidence: calibration gap 0.4 points (written estimate)

Cost $31.309$31,309 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 217.7 s median per request, 1 in flight

Open its answers →

98.5%97.8%–98.9%97.6%98.8%17.9%answered 100%0.4written estimate$31.309$31,309subscription, priced as the API217.716 per request1 in flight
1Claude Fable 5.2 max · chat‡ Closed · FlagshipLatency 81.1 sCost $53.381$53,381 per 1,000per 1M

Claude Fable 5.2 max · chat · Anthropic · chat model

Parameters: not disclosed by maker

Class: Flagship, by its standard list price: $8.00 per 1M input tokens (https://platform.claude.com/docs/en/about-claude/pricing, read 2026-11-11)

Accuracy 98.1% (95%: 97.5–98.6%)

Reading 97.7% · grammar 98.3%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know”: Queued (its run with the option has not started yet)

Confidence: calibration gap 7.5 points (written estimate)

Cost $53.381$53,381 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 81.1 s median per request, 1 in flight

Open its answers →

98.1%97.5%–98.6%97.7%98.3%Queued7.5written estimate$53.381$53,381subscription, priced as the API81.116 per request1 in flight
1Gemini 3.8 Flash high · chat‡ Closed · WorkhorseLatency 53.7 sCost $2.419$2,419 per 1,000per 1M

Gemini 3.8 Flash high · chat · Google · chat model

Parameters: not disclosed by maker

Class: Workhorse, by its standard list price: $1.50 per 1M input tokens (https://ai.google.dev/gemini-api/docs/pricing, read 2026-10-08)

Accuracy 98.1% (95%: 97.4–98.6%)

Reading 97.1% · grammar 98.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 4.5% · answered 99.9% · error rate 1.9%

Confidence: calibration gap 1.1 points (written estimate)

Cost $2.419$2,419 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 53.7 s median per request, 4 in flight

Open its answers →

98.1%97.4%–98.6%97.1%98.5%4.5%answered 100%1.1written estimate$2.419$2,419at list prices53.716 per request4 in flight
1Gemini 4 Hydrogen high · chat‡ Closed · WorkhorseLatency 31.7 sCost $4.806$4,806 per 1,000per 1M

Gemini 4 Hydrogen high · chat · Google · chat model

Parameters: not disclosed by maker

Class: Workhorse, by its standard list price: $1.50 per 1M input tokens (https://ai.google.dev/gemini-api/docs/pricing, read 2026-11-12)

Accuracy 98.1% (95%: 97.4–98.6%)

Reading 97.3% · grammar 98.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 17.4% · answered 99.1% · error rate 1.7%

Confidence: calibration gap 1.2 points (written estimate)

Cost $4.806$4,806 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 31.7 s median per request, 1 in flight

Open its answers →

98.1%97.4%–98.6%97.3%98.4%17.4%answered 99%1.2written estimate$4.806$4,806subscription, priced as the API31.716 per request1 in flight
1GPT-6 Astra high · chat‡ Closed · FlagshipLatency 21.7 sCost $6.066$6,066 per 1,000per 1M

GPT-6 Astra high · chat · OpenAI · chat model

Parameters: not disclosed by maker

Class: Flagship, by its standard list price: $10.00 per 1M input tokens (https://developers.openai.com/api/docs/pricing, read 2026-10-08)

Accuracy 98.0% (95%: 97.3–98.5%)

Reading 97.1% · grammar 98.3%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 14.3% · answered 99.7% · error rate 1.9%

Confidence: calibration gap 0.7 points (written estimate)

Cost $6.066$6,066 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 21.7 s median per request, 1 in flight

Open its answers →

98.0%97.3%–98.5%97.1%98.3%14.3%answered 100%0.7written estimate$6.066$6,066subscription, priced as the API21.716 per request1 in flight
1GPT-6.1 Sol high · chat‡ Closed · WorkhorseLatency 24.7 sCost $1.622$1,622 per 1,000per 1M

GPT-6.1 Sol high · chat · OpenAI · chat model

Parameters: not disclosed by maker

Class: Workhorse, by its standard list price: $2.00 per 1M input tokens (https://developers.openai.com/api/docs/pricing, read 2026-10-08)

Accuracy 97.8% (95%: 97.1–98.3%)

Reading 96.5% · grammar 98.3%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 15.4% · answered 99.6% · error rate 2.0%

Confidence: calibration gap 1.0 points (written estimate)

Cost $1.622$1,622 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 24.7 s median per request, 1 in flight

Open its answers →

97.8%97.1%–98.3%96.5%98.3%15.4%answered 100%1.0written estimate$1.622$1,622subscription, priced as the API24.716 per request1 in flight
2Gemini 4 Argon low · chat‡ Closed · FlagshipLatency 5.46 sCost $1.786$1,786 per 1,000per 1M

Gemini 4 Argon low · chat · Google · chat model

Parameters: not disclosed by maker

Class: Flagship, by its standard list price: $4.00 per 1M input tokens (https://ai.google.dev/gemini-api/docs/pricing, read 2026-10-28)

Accuracy 96.4% (95%: 95.5–97.1%)

Reading 97.0% · grammar 96.1%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 7.0% · answered 99.7% · error rate 3.6%

Confidence: calibration gap 2.2 points (written estimate)

Cost $1.786$1,786 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 5.46 s median per request, 1 in flight

Open its answers →

96.4%95.5%–97.1%97.0%96.1%7.0%answered 100%2.2written estimate$1.786$1,786at list prices5.4616 per request1 in flight
2Claude Fable 5.1 high · chat‡ Closed · FlagshipLatency 30.3 sCost $17.042$17,042 per 1,000per 1M

Claude Fable 5.1 high · chat · Anthropic · chat model

Parameters: not disclosed by maker

Class: Flagship, by its standard list price: $10.00 per 1M input tokens (https://platform.claude.com/docs/en/about-claude/pricing, read 2026-10-28)

Accuracy 96.1% (95%: 95.2–96.9%)

Reading 95.6% · grammar 96.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 6.0% · answered 99.8% · error rate 3.6%

Confidence: calibration gap 12.0 points (written estimate)

Cost $17.042$17,042 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 30.3 s median per request, 1 in flight

Open its answers →

96.1%95.2%–96.9%95.6%96.4%6.0%answered 100%12.0written estimate$17.042$17,042subscription, priced as the API30.316 per request1 in flight
2Claude Opus 5.5 high · chat‡ Closed · FlagshipLatency 22.1 sCost $6.509$6,509 per 1,000per 1M

Claude Opus 5.5 high · chat · Anthropic · chat model

Parameters: not disclosed by maker

Class: Flagship, by its standard list price: $4.00 per 1M input tokens (https://platform.claude.com/docs/en/about-claude/pricing, read 2026-10-08)

Accuracy 95.0% (95%: 94.0–95.8%)

Reading 95.6% · grammar 94.7%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 2.7% · answered 99.9% · error rate 4.9%

Confidence: calibration gap 14.6 points (written estimate)

Cost $6.509$6,509 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 22.1 s median per request, 1 in flight

Open its answers →

95.0%94.0%–95.8%95.6%94.7%2.7%answered 100%14.6written estimate$6.509$6,509subscription, priced as the API22.116 per request1 in flight
3GPT-6.1 Terra high · chat‡ Closed · WorkhorseLatency 7.10 sCost $0.515$515 per 1,000per 1M

GPT-6.1 Terra high · chat · OpenAI · chat model

Parameters: not disclosed by maker

Class: Workhorse, by its standard list price: $1.00 per 1M input tokens (https://developers.openai.com/api/docs/pricing, read 2026-10-28)

Accuracy 94.3% (95%: 93.2–95.2%)

Reading 96.2% · grammar 93.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 7.3% · answered 99.6% · error rate 5.3%

Confidence: calibration gap 2.8 points (written estimate)

Cost $0.515$515 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 7.10 s median per request, 1 in flight

Open its answers →

94.3%93.2%–95.2%96.2%93.5%7.3%answered 100%2.8written estimate$0.515$515subscription, priced as the API7.1016 per request1 in flight
3Grok 4.7 high · chat‡ Closed · WorkhorseLatency 401.9 sCost $14.292$14,292 per 1,000per 1M

Grok 4.7 high · chat · xAI · chat model

Parameters: not disclosed by maker

Class: Workhorse, by its standard list price: $2.00 per 1M input tokens (https://docs.x.ai/docs/models/grok-4.7, read 2026-10-28)

Accuracy 92.7% (95%: 91.6–93.7%)

Reading 95.0% · grammar 91.8%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 29.9% · answered 97.5% · error rate 6.1%

Confidence: calibration gap 11.9 points (written estimate)

Cost $14.292$14,292 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 401.9 s median per request, 4 in flight

Open its answers →

92.7%91.6%–93.7%95.0%91.8%29.9%answered 97%11.9written estimate$14.292$14,292subscription, priced as the API401.916 per request4 in flight
3DeepSeek-V4.1-Flash reasoning on · chat · first release‡ Open · 552B MoE · 16B activeLatency 365.1 sCost $1.299$1,299 per 1,000per 1M

DeepSeek-V4.1-Flash reasoning on · chat · first release · DeepSeek · chat model

Mixture of experts: 552B total, about 16B active per step. Size class uses the total (552B).

Accuracy 92.0% (95%: 90.8–93.1%)

Reading 93.4% · grammar 91.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Confidence: calibration gap 8.0 points (answer letter probability)

Cost $1.299$1,299 per 1,000 questionsper 1M questions (at a host's list price for the same weights) · 16 questions per request

Latency: 365.1 s median per request, 8 in flight

With its dated checkpoint deepseek-v4.1-flash-1017 under its other settings

Open its answers →

92.0%90.8%–93.1%93.4%91.4%Not run8.0answer letter probability$1.299$1,299at a host's list price for the same weights365.116 per request8 in flight
4Qwen4-Max high · chat‡ Closed · WorkhorseLatency 91.5 sCost $2.533$2,533 per 1,000per 1M

Qwen4-Max high · chat · Alibaba · chat model

Parameters: not disclosed by maker

Class: Workhorse, by its standard list price: $1.50 per 1M input tokens (https://openrouter.ai/qwen/qwen4-max, read 2026-11-16)

Accuracy 91.7% (95%: 90.5–92.8%)

Reading 94.7% · grammar 90.5%

‡ Turkish not officially supported: it names no language (source, read 16 November 2026)

Non-hallucination with “I don't know” 0.7% · answered 99.2% · error rate 6.4%

Confidence: calibration gap 7.8 points (written estimate)

Cost $2.533$2,533 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 91.5 s median per request, 4 in flight

Open its answers →

91.7%90.5%–92.8%94.7%90.5%0.7%answered 99%7.8written estimate$2.533$2,533at list prices91.516 per request4 in flight
5Muse Ember max · chat‡ Closed · EfficientLatency 30.9 sCost $0.650$650 per 1,000per 1M

Muse Ember max · chat · Meta · chat model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.10 per 1M input tokens (https://openrouter.ai/meta/muse-ember, read 2026-11-10)

Accuracy 85.6% (95%: 84.1–87.0%)

Reading 90.2% · grammar 83.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 2.1% · answered 97.5% · error rate 13.1%

Confidence: calibration gap 6.2 points (written estimate)

Cost $0.650$650 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 30.9 s median per request, 4 in flight

Open its answers →

85.6%84.1%–87.0%90.2%83.7%2.1%answered 98%6.2written estimate$0.650$650at list prices30.916 per request4 in flight
5Claude Sonnet 5.5 high · chat‡ Closed · WorkhorseLatency 20.5 sCost $3.654$3,654 per 1,000per 1M

Claude Sonnet 5.5 high · chat · Anthropic · chat model

Parameters: not disclosed by maker

Class: Workhorse, by its standard list price: $2.00 per 1M input tokens (https://platform.claude.com/docs/en/about-claude/pricing, read 2026-10-08)

Accuracy 85.0% (95%: 83.5–86.5%)

Reading 85.2% · grammar 85.0%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.7% · answered 99.9% · error rate 12.8%

Confidence: calibration gap 16.0 points (written estimate)

Cost $3.654$3,654 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 20.5 s median per request, 1 in flight

Open its answers →

85.0%83.5%–86.5%85.2%85.0%0.7%answered 100%16.0written estimate$3.654$3,654subscription, priced as the API20.516 per request1 in flighton a sample
5Kimi K3.1 Pro low · chat‡ Open · 2.8T MoE · 104B activeLatency 44.9 sCost $2.048$2,048 per 1,000per 1M

Kimi K3.1 Pro low · chat · Moonshot · chat model

Mixture of experts: 2.8T total, about 104B active per step. Size class uses the total (2.8T).

Accuracy 82.8% (95%: 81.1–84.3%)

Reading 93.8% · grammar 78.1%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 2.0% · answered 99.7% · error rate 15.7%

Confidence: calibration gap 12.6 points (written estimate)

Cost $2.048$2,048 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 44.9 s median per request, 4 in flight

Open its answers →

82.8%81.1%–84.3%93.8%78.1%2.0%answered 100%12.6written estimate$2.048$2,048at list prices44.916 per request4 in flight
5Qwen3.5-397B-A17B high · chat‡ Open · 397B MoE · 17B activeLatency 251.3 sCost $2.411$2,411 per 1,000per 1M

Qwen3.5-397B-A17B high · chat · Alibaba · chat model

Mixture of experts: 397B total, about 17B active per step. Size class uses the total (397B).

Accuracy 82.7% (95%: 81.1–84.2%)

Reading 91.0% · grammar 79.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 1.5% · answered 99.7% · error rate 17.8%

Confidence: calibration gap 6.2 points (written estimate)

Cost $2.411$2,411 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 251.3 s median per request, 4 in flight

Open its answers →

82.7%81.1%–84.2%91.0%79.2%1.5%answered 100%6.2written estimate$2.411$2,411at list prices251.316 per request4 in flight
6KnowLine-31B-Gen1.5‡ Open · 31BLatency 1.05 sCost No list price

KnowLine-31B-Gen1.5 · PelaAI · decision model

Parameters: 31B dense, as the maker states it (counted from the released files: 31.27B).

Accuracy 81.8% (95%: 80.1–83.4%)

Reading 92.8% · grammar 77.1%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.2% · answered 100.0% · error rate 18.7%

Kept at 99% confidence with no observed mistakes: 6.5%

Confidence: calibration gap 3.4 points (option score)

Cost: No list price · 1 question per request

Latency: 1.05 s median per request, 1 in flight

Open its answers →

81.8%80.1%–83.4%92.8%77.1%0.2%answered 100%3.4option scoreNo list price1.051 per request1 in flight
6Copernicus-31B‡ Open · 31BLatency 0.184 sCost $0.134$134 per 1,000per 1M

Copernicus-31B · Poindexter Labs · decision model

Parameters: 31B dense, as the maker states it (counted from the released files: 31.27B).

Accuracy 81.8% (95%: 80.1–83.4%)

Reading 93.1% · grammar 77.0%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 1.2% · answered 99.8% · error rate 18.3%

Kept at 99% confidence with no observed mistakes: 4.9%

Confidence: calibration gap 9.1 points (option score)

Cost $0.134$134 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.184 s median per request, 1 in flight

Open its answers →

81.8%80.1%–83.4%93.1%77.0%1.2%answered 100%9.1option score$0.134$134from GPU time0.1841 per request1 in flight
6KnowLine-31B-Gen2‡ Open · 31BLatency 0.168 sCost $0.120$120 per 1,000per 1M

KnowLine-31B-Gen2 · PelaAI · decision model

Base model: Gemma 4 31B

Parameters: 31B dense, as the maker states it (counted from the released files: 31.27B).

Accuracy 81.2% (95%: 79.5–82.8%)

Reading 93.4% · grammar 76.0%

‡ Turkish not officially supported: it lists other languages (source, read 16 November 2026)

Non-hallucination with “I don't know” 0.0% · answered 100.0% · error rate 18.6%

Kept at 99% confidence with no observed mistakes: 3.7%

Confidence: calibration gap 5.6 points (option score)

Cost $0.120$120 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.168 s median per request, 1 in flight

Open its answers →

81.2%79.5%–82.8%93.4%76.0%0.0%answered 100%5.6option score$0.120$120from GPU time0.1681 per request1 in flight
6Turion Jev 31B‡ Open · 31BLatency 0.405 sCost No list price

Turion Jev 31B · Turion.AI · decision model

Parameters: 31B dense, as the maker states it (counted from the released files: 31.27B).

Accuracy 80.1% (95%: 78.4–81.7%)

Reading 92.7% · grammar 74.8%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 3.3% · answered 99.3% · error rate 20.0%

Kept at 99% confidence with no observed mistakes: 1.9%

Confidence: calibration gap 11.1 points (option score)

Cost: No list price · 1 question per request

Latency: 0.405 s median per request, 1 in flight

Open its answers →

80.1%78.4%–81.7%92.7%74.8%3.3%answered 99%11.1option scoreNo list price0.4051 per request1 in flight
6Inkling-Small high · chat · Jul 2026‡ Open · 276B MoE · 12B activeLatency 97.7 sCost $1.137$1,137 per 1,000per 1M

Inkling-Small high · chat · Jul 2026 · Thinking Machines · chat model

Mixture of experts: 276B total (counted from the released files: 265.96B), about 12B active per step. Size class uses the total (276B).

Accuracy 80.0% (95%: 78.3–81.6%)

Reading 87.2% · grammar 76.9%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 3.1% · answered 99.3% · error rate 21.4%

Confidence: calibration gap 7.9 points (written estimate)

Cost $1.137$1,137 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 97.7 s median per request, 4 in flight

Open its answers →

80.0%78.3%–81.6%87.2%76.9%3.1%answered 99%7.9written estimate$1.137$1,137at list prices97.716 per request4 in flight
6GLM-5.3-Flash reasoning on · chat‡ Open · 320B MoE · 18B activeLatency 102.6 sCost $0.543$543 per 1,000per 1M

GLM-5.3-Flash reasoning on · chat · Z.ai · chat model

Mixture of experts: 320B total, about 18B active per step. Size class uses the total (320B).

Accuracy 78.8% (95%: 77.0–80.5%)

Reading 89.3% · grammar 74.3%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.6% · answered 91.9% · error rate 16.6%

Confidence: calibration gap 4.5 points (written estimate)

Cost $0.543$543 per 1,000 questionsper 1M questions (at the maker's list price) · 16 questions per request

Latency: 102.6 s median per request, 8 in flight

Open its answers →

78.8%77.0%–80.5%89.3%74.3%0.6%answered 92%4.5written estimate$0.543$543at the maker's list price102.616 per request8 in flight
7REAX 1.3‡ Open · 57BLatency 0.250 sCost $0.185$185 per 1,000per 1M

REAX 1.3 · REAX · decision model

Two models run together, option probabilities averaged 50/50: 57.36B in all, all used on every question.

Accuracy 77.9% (95%: 76.2–79.6%)

Reading 92.4% · grammar 71.8%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.4% · answered 99.9% · error rate 21.6%

Kept at 99% confidence with no observed mistakes: 10.9%

Confidence: calibration gap 4.1 points (option score)

Cost $0.185$185 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.250 s median per request, 1 in flight

Open its answers →

77.9%76.2%–79.6%92.4%71.8%0.4%answered 100%4.1option score$0.185$185from GPU time0.2501 per request1 in flight
7Karpo Decision 31B‡ Open · 31BLatency 0.159 sCost $0.105$105 per 1,000per 1M

Karpo Decision 31B · MachinePulse · decision model

Parameters: 31B dense, as the maker states it (counted from the released files: 31.27B).

Accuracy 77.8% (95%: 76.0–79.5%)

Reading 93.4% · grammar 71.1%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.6% · answered 99.9% · error rate 21.6%

Kept at 99% confidence with no observed mistakes: 23.0%

Confidence: calibration gap 2.3 points (option score)

Cost $0.105$105 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.159 s median per request, 1 in flight

Open its answers →

77.8%76.0%–79.5%93.4%71.1%0.6%answered 100%2.3option score$0.105$105from GPU time0.1591 per request1 in flight
7Gemini Decision 1 20261111‡ Closed · EfficientLatency 0.227 sCost $0.037$37.19 per 1,000per 1M

Gemini Decision 1 20261111 · Google · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.10 per 1M input tokens (https://openrouter.ai/google/gemini-decision-1, read 2026-11-11)

Accuracy 77.5% (95%: 75.7–79.2%)

Reading 92.1% · grammar 71.3%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 1.9% · answered 99.5% · error rate 23.1%

Kept at 99% confidence with no observed mistakes: 14.5%

Confidence: calibration gap 2.0 points (option score)

Cost $0.037$37.19 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.227 s median per request, 1 in flight

Open its answers →

77.5%75.7%–79.2%92.1%71.3%1.9%answered 100%2.0option score$0.037$37.19at list prices0.2271 per request1 in flight
7d1-ultra‡ Closed · EfficientLatency 0.223 sCost $0.014$14.34 per 1,000per 1M

d1-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.042 per 1M input tokens (https://www.liquid.ai/blog/d1-ultra, read 2026-10-16)

Accuracy 77.1% (95%: 75.3–78.8%)

Reading 92.1% · grammar 70.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 75.1% · answered 71.9% · error rate 13.0%

Kept at 99% confidence with no observed mistakes: 4.1%

Confidence: calibration gap 3.6 points (option score)

Cost $0.014$14.34 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.223 s median per request, 1 in flight

Open its answers →

77.1%75.3%–78.8%92.1%70.7%75.1%answered 72%3.6option score$0.014$14.34at list prices0.2231 per request1 in flight
8Logits Sigma-1 Duo‡ Open · 57BLatency 0.250 sCost $0.181$181 per 1,000per 1M

Logits Sigma-1 Duo · Logits · decision model

Parameters: 57.36B dense, as the maker states it.

Accuracy 75.0% (95%: 73.2–76.8%)

Reading 91.9% · grammar 67.8%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.4% · answered 99.9% · error rate 24.7%

Kept at 99% confidence with no observed mistakes: 1.9%

Confidence: calibration gap 5.7 points (option score)

Cost $0.181$181 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.250 s median per request, 1 in flight

Open its answers →

75.0%73.2%–76.8%91.9%67.8%0.4%answered 100%5.7option score$0.181$181from GPU time0.2501 per request1 in flight
8Qwen3.8-27B low · chat‡ Open · 27BLatency 648.4 sCost $3.095$3,095 per 1,000per 1M

Qwen3.8-27B low · chat · Alibaba · chat model

Parameters: 27B dense, as the maker states it (counted from the released files: 27.78B).

Accuracy 74.7% (95%: 72.8–76.4%)

Reading 89.3% · grammar 68.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 5.9% · answered 97.9% · error rate 22.9%

Confidence: calibration gap 23.7 points (answer letter probability)

Cost $3.095$3,095 per 1,000 questionsper 1M questions (at the maker's list price) · 16 questions per request

Latency: 648.4 s median per request, 4 in flight

Open its answers →

74.7%72.8%–76.4%89.3%68.4%5.9%answered 98%23.7answer letter probability$3.095$3,095at the maker's list price648.416 per request4 in flight
8GLM-5.3 low · chat‡ Open · WorkhorseLatency 17.1 sCost $0.814$814 per 1,000per 1M

GLM-5.3 low · chat · Z.ai · chat model

Parameters: not stated by the maker (a mixture of experts); counted from the released files: 753.33B.

Class: Workhorse, by its standard list price: $1.40 per 1M input tokens (https://openrouter.ai/z-ai/glm-5.3, read 2026-10-28)

Accuracy 74.5% (95%: 72.7–76.3%)

Reading 87.7% · grammar 68.9%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 1.1% · answered 99.0% · error rate 23.9%

Confidence: calibration gap 2.6 points (written estimate)

Cost $0.814$814 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 17.1 s median per request, 4 in flight

Open its answers →

74.5%72.7%–76.3%87.7%68.9%1.1%answered 99%2.6written estimate$0.814$814at list prices17.116 per request4 in flight
8Sieve 31B v3‡ Open · 31BLatency 1.00 sCost No list price

Sieve 31B v3 · Sthānika AI · decision model

Parameters: 31B dense, as the maker states it (counted from the released files: 31.27B).

Accuracy 74.2% (95%: 72.3–75.9%)

Reading 91.8% · grammar 66.7%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.0% · answered 100.0% · error rate 25.2%

Kept at 99% confidence: 15.0%, 1 of them wrong

Confidence: calibration gap 2.9 points (option score)

Cost: No list price · 1 question per request

Latency: 1.00 s median per request, 1 in flight

Open its answers →

74.2%72.3%–75.9%91.8%66.7%0.0%answered 100%2.9option scoreNo list price1.001 per request1 in flight
8Gemma 4 31B reasoning off · chat‡ Open · 31BLatency 3.97 sCost $0.048$48.22 per 1,000per 1M

Gemma 4 31B reasoning off · chat · Google · chat model

Parameters: 30.7B dense, as the maker states it (counted from the released files: 31.27B).

Accuracy 73.1% (95%: 71.2–74.9%)

Reading 90.9% · grammar 65.6%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 4.1% · answered 98.9% · error rate 26.1%

Confidence: calibration gap 24.2 points (answer letter probability)

Cost $0.048$48.22 per 1,000 questionsper 1M questions (at a host's list price for the same weights) · 16 questions per request

Latency: 3.97 s median per request, 1 in flight

Open its answers →

73.1%71.2%–74.9%90.9%65.6%4.1%answered 99%24.2answer letter probability$0.048$48.22at a host's list price for the same weights3.9716 per request1 in flight
8MiMo-V2.6-Flash high · chat‡ Open · 309B MoE · 15B activeLatency 21.4 sCost $2.219$2,219 per 1,000per 1M

MiMo-V2.6-Flash high · chat · Xiaomi · chat model

Mixture of experts: 309B total, about 15B active per step. Size class uses the total (309B).

Accuracy 72.1% (95%: 70.2–73.9%)

Reading 86.6% · grammar 66.0%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 3.5% · answered 96.7% · error rate 26.2%

Confidence: calibration gap 7.1 points (written estimate)

Cost $2.219$2,219 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 21.4 s median per request, 32 in flight

Open its answers →

72.1%70.2%–73.9%86.6%66.0%3.5%answered 97%7.1written estimate$2.219$2,219at list prices21.41 per request32 in flight
9GLiDE 27B‡ Open · 27BLatency 0.542 sCost No list price

GLiDE 27B · Fastino · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 71.1% (95%: 69.2–73.0%)

Reading 90.9% · grammar 62.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 39.8% · answered 86.8% · error rate 23.1%

Kept at 99% confidence with no observed mistakes: 11.2%

Confidence: calibration gap 2.3 points (option score)

Cost: No list price · 1 question per request

Latency: 0.542 s median per request, 1 in flight

Open its answers →

71.1%69.2%–73.0%90.9%62.7%39.8%answered 87%2.3option scoreNo list price0.5421 per request1 in flight
–Perplexity Decider 27B v1.2 API · with web‡ Open · 27BLatency 0.482 sCost ≤ $1.034≤ $1,034 per 1,000per 1M

Perplexity Decider 27B v1.2 API · with web · Perplexity · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 68.7% (95%: 66.8–70.6%)

Reading 88.9% · grammar 60.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 3.2% · answered 99.0% · error rate 32.3%

Kept at 99% confidence: 16.0%, 1 of them wrong

Confidence: calibration gap 3.6 points (option score)

Cost ≤ $1.034≤ $1,034 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.482 s median per request, 1 in flight

Open its answers →

68.7%66.8%–70.6%88.9%60.2%3.2%answered 99%3.6option score≤ $1.034≤ $1,034at list prices0.4821 per request1 in flight
9GLiDE v1.2‡ Open · 27BLatency 0.413 sCost $0.014$14.37 per 1,000per 1M

GLiDE v1.2 · Fastino · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 68.5% (95%: 66.5–70.4%)

Reading 89.6% · grammar 59.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 39.5% · answered 85.4% · error rate 26.2%

Kept at 99% confidence with no observed mistakes: 10.9%

Confidence: calibration gap 2.9 points (option score)

Cost $0.014$14.37 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.413 s median per request, 1 in flight

Open its answers →

68.5%66.5%–70.4%89.6%59.5%39.5%answered 85%2.9option score$0.014$14.37at the maker's list price0.4131 per request1 in flight
9Perplexity Decider 27B v1.3‡ Open · 27BLatency 0.110 sCost $0.081$81.18 per 1,000per 1M

Perplexity Decider 27B v1.3 · Perplexity · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 68.4% (95%: 66.5–70.3%)

Reading 89.0% · grammar 59.7%

‡ Turkish not officially supported: it names no language (source, read 14 November 2026)

Non-hallucination with “I don't know” 0.7% · answered 99.8% · error rate 32.4%

Kept at 99% confidence with no observed mistakes: 12.6%

Confidence: calibration gap 5.0 points (option score)

Cost $0.081$81.18 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.110 s median per request, 1 in flight

Open its answers →

68.4%66.5%–70.3%89.0%59.7%0.7%answered 100%5.0option score$0.081$81.18from GPU time0.1101 per request1 in flight
9GPT-6 Luna low · chat‡ Closed · EfficientLatency 7.71 sCost $0.048$48.01 per 1,000per 1M

GPT-6 Luna low · chat · OpenAI · chat model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.10 per 1M input tokens (https://developers.openai.com/api/docs/guides/decisions#pricing-and-availability, read 2026-10-08)

Accuracy 68.3% (95%: 66.3–70.2%)

Reading 87.3% · grammar 60.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 1.0% · answered 99.7% · error rate 33.2%

Confidence: calibration gap 25.7 points (written estimate)

Cost $0.048$48.01 per 1,000 questionsper 1M questions (subscription, priced as the API) · 16 questions per request

Latency: 7.71 s median per request, 1 in flight

Open its answers →

68.3%66.3%–70.2%87.3%60.2%1.0%answered 100%25.7written estimate$0.048$48.01subscription, priced as the API7.7116 per request1 in flight
9d1.2-ultra‡ Closed · EfficientLatency 0.220 sCost $0.014$14.34 per 1,000per 1M

d1.2-ultra · Liquid AI · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.042 per 1M input tokens (https://www.liquid.ai/blog/d1-2, read 2026-11-16)

Accuracy 68.1% (95%: 66.1–70.0%)

Reading 89.8% · grammar 58.9%

‡ Turkish not officially supported: it names no language (source, read 16 November 2026)

Non-hallucination with “I don't know” 87.3% · answered 54.7% · error rate 12.1%

Kept at 99% confidence with no observed mistakes: 0.5%

Confidence: calibration gap 3.4 points (option score)

Cost $0.014$14.34 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.220 s median per request, 1 in flight

Open its answers →

68.1%66.1%–70.0%89.8%58.9%87.3%answered 55%3.4option score$0.014$14.34at list prices0.2201 per request1 in flight
9GLiDE API‡ Closed · EfficientLatency 2.09 sCost $0.160$160 per 1,000per 1M

GLiDE API · Fastino · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.04 per 1M input tokens (https://docs.fastino.ai/pricing, read 2026-10-22)

Accuracy 67.9% (95%: 65.9–69.8%)

Reading 87.8% · grammar 59.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 59.4% · answered 76.3% · error rate 21.2%

Kept at 99% confidence with no observed mistakes: 5.5%

Confidence: calibration gap 4.5 points (option score)

Cost $0.160$160 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 2.09 s median per request, 4 in flight

Open its answers →

67.9%65.9%–69.8%87.8%59.4%59.4%answered 76%4.5option score$0.160$160at list prices2.091 per request4 in flight
10d3‡ Open · 26BLatency 0.092 sCost $0.070$70.21 per 1,000per 1M

d3 · vLLM Semantic Router · decision model

Parameters: 26.09B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 67.8% (95%: 65.8–69.7%)

Reading 89.2% · grammar 58.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.0% · answered 100.0% · error rate 32.5%

Kept at 99% confidence with no observed mistakes: 11.4%

Confidence: calibration gap 2.6 points (option score)

Cost $0.070$70.21 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.092 s median per request, 1 in flight

Open its answers →

67.8%65.8%–69.7%89.2%58.7%0.0%answered 100%2.6option score$0.070$70.21from GPU time0.0921 per request1 in flight
10d3-ultra‡ Open · 26BLatency 0.333 sCost $0.216$216 per 1,000per 1M

d3-ultra · vLLM Semantic Router · decision model

Parameters: 26.09B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 67.8% (95%: 65.8–69.7%)

Reading 89.8% · grammar 58.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 1.4% · answered 99.5% · error rate 32.2%

Kept at 99% confidence: 17.5%, 1 of them wrong

Confidence: calibration gap 12.5 points (option score)

Cost $0.216$216 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.333 s median per request, 1 in flight

Open its answers →

67.8%65.8%–69.7%89.8%58.4%1.4%answered 100%12.5option score$0.216$216from GPU time0.3331 per request1 in flight
10Drex Max‡ Open · 27BLatency 0.082 sCost $0.023$23.29 per 1,000per 1M

Drex Max · Nace.AI · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 67.7% (95%: 65.8–69.7%)

Reading 89.6% · grammar 58.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.3% · answered 99.9% · error rate 32.4%

Kept at 99% confidence: 20.2%, 1 of them wrong

Confidence: calibration gap 6.3 points (option score)

Cost $0.023$23.29 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.082 s median per request, 1 in flight

Open its answers →

67.7%65.8%–69.7%89.6%58.4%0.3%answered 100%6.3option score$0.023$23.29at the maker's list price0.0821 per request1 in flight
10Drex Max v1.1‡ Closed · Efficient (provisional)Latency 0.313 sCost $0.023$23.11 per 1,000per 1M

Drex Max v1.1 · Nace.AI · decision model

Parameters: not disclosed by maker

Class: Efficient (provisional), by its introductory price, the only one published: $0.03 per 1M input tokens (https://nace.ai/developers, read 2026-11-10)

Accuracy 67.4% (95%: 65.4–69.3%)

Reading 88.6% · grammar 58.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 38.2% · answered 85.4% · error rate 27.5%

Kept at 99% confidence with no observed mistakes: 10.0%

Confidence: calibration gap 2.9 points (option score)

Cost $0.023$23.11 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.313 s median per request, 1 in flight

Open its answers →

67.4%65.4%–69.3%88.6%58.4%38.2%answered 85%2.9option score$0.023$23.11at list prices0.3131 per request1 in flight
10Mammoth-1‡ Open · 27BLatency 0.169 sCost $0.040$39.62 per 1,000per 1M

Mammoth-1 · Base Labs · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 27.36B).

Accuracy 67.1% (95%: 65.1–69.0%)

Reading 90.4% · grammar 57.2%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 68.7% · answered 68.6% · error rate 20.8%

Kept at 99% confidence with no observed mistakes: 13.0%

Confidence: calibration gap 5.6 points (option score)

Cost $0.040$39.62 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.169 s median per request, 1 in flight

Open its answers →

67.1%65.1%–69.0%90.4%57.2%68.7%answered 69%5.6option score$0.040$39.62at the maker's list price0.1691 per request1 in flight
10Perplexity Decider 27B v1.2 DGX Spark‡ Open · 27BLatency 0.580 sCost $0.008$7.76 per 1,000per 1M

Perplexity Decider 27B v1.2 DGX Spark · Perplexity · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 67.1% (95%: 65.1–69.0%)

Reading 89.2% · grammar 57.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.3% · answered 99.9% · error rate 32.7%

Kept at 99% confidence with no observed mistakes: 16.4%

Confidence: calibration gap 4.4 points (option score)

Cost $0.008$7.76 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.580 s median per request, 1 in flight

Open its answers →

67.1%65.1%–69.0%89.2%57.7%0.3%answered 100%4.4option score$0.008$7.76at the maker's list price0.5801 per request1 in flight
10Perplexity Decider 27B v1.1‡ Open · 27BLatency 0.233 sCost $0.008$7.76 per 1,000per 1M

Perplexity Decider 27B v1.1 · Perplexity · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 66.9% (95%: 64.9–68.8%)

Reading 89.0% · grammar 57.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence with no observed mistakes: 15.9%

Confidence: calibration gap 3.8 points (option score)

Cost $0.008$7.76 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.233 s median per request, 1 in flight

Open its answers →

66.9%64.9%–68.8%89.0%57.5%Not run3.8option score$0.008$7.76at list prices0.2331 per request1 in flight
10MiniMax-M3.1-Flash high · chat‡ Open · 428B MoE · 23B activeLatency 112.6 sCost $0.814$814 per 1,000per 1M

MiniMax-M3.1-Flash high · chat · MiniMax · chat model

Mixture of experts: 428B total, about 23B active per step. Size class uses the total (428B).

Accuracy 66.5% (95%: 64.5–68.4%)

Reading 82.9% · grammar 59.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 5.5% · answered 97.6% · error rate 32.8%

Confidence: calibration gap 21.1 points (written estimate)

Cost $0.814$814 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 112.6 s median per request, 4 in flight

Open its answers →

66.5%64.5%–68.4%82.9%59.5%5.5%answered 98%21.1written estimate$0.814$814at list prices112.616 per request4 in flight
10SubQ Snap Preview‡ Closed · EfficientLatency 0.389 sCost $0.019$19.41 per 1,000per 1M

SubQ Snap Preview · Subquadratic · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.05 per 1M input tokens (https://docs.subq.ai/using-the-api/models/, read 2026-10-13)

Accuracy 65.2% (95%: 63.2–67.2%)

Reading 87.7% · grammar 55.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 4.9 points (option score)

Cost $0.019$19.41 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.389 s median per request, 1 in flight

Open its answers →

65.2%63.2%–67.2%87.7%55.7%Not run4.9option score$0.019$19.41at list prices0.3891 per request1 in flight
11SubQ Snap 1.1‡ Closed · EfficientLatency 0.378 sCost $0.015$15.41 per 1,000per 1M

SubQ Snap 1.1 · Subquadratic · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.04 per 1M input tokens (https://docs.subq.ai/using-the-api/models/, read 2026-11-01)

Accuracy 63.9% (95%: 61.8–65.9%)

Reading 87.0% · grammar 54.0%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 47.1% · answered 80.1% · error rate 27.8%

Kept at 99% confidence: 13.6%, 1 of them wrong

Confidence: calibration gap 11.2 points (option score)

Cost $0.015$15.41 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.378 s median per request, 1 in flight

Open its answers →

63.9%61.8%–65.9%87.0%54.0%47.1%answered 80%11.2option score$0.015$15.41at list prices0.3781 per request1 in flight
11GPT-6 Luna Decisions API‡ Closed · EfficientLatency 0.191 sCost $0.040$40.44 per 1,000per 1M

GPT-6 Luna Decisions API · OpenAI · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.10 per 1M input tokens (https://developers.openai.com/api/docs/guides/decisions#pricing-and-availability, read 2026-10-08)

Accuracy 63.6% (95%: 61.6–65.6%)

Reading 85.5% · grammar 54.3%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: 16.7%, 3 of them wrong

Confidence: calibration gap 2.4 points (option score)

Cost $0.040$40.44 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.191 s median per request, 4 in flight

Open its answers →

63.6%61.6%–65.6%85.5%54.3%Not run2.4option score$0.040$40.44at list prices0.1911 per request4 in flight
11Qwen3.8 Flash no reasoning · chat‡ Closed · EfficientLatency 10.0 sCost $0.047$47.13 per 1,000per 1M

Qwen3.8 Flash no reasoning · chat · Alibaba · chat model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.15 per 1M input tokens (https://openrouter.ai/qwen/qwen3.8-flash, read 2026-11-11)

Accuracy 63.1% (95%: 61.0–65.1%)

Reading 87.2% · grammar 52.8%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 4.7% · answered 98.3% · error rate 35.4%

Confidence: calibration gap 28.7 points (written estimate)

Cost $0.047$47.13 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 10.0 s median per request, 4 in flight

Open its answers →

63.1%61.0%–65.1%87.2%52.8%4.7%answered 98%28.7written estimate$0.047$47.13at list prices10.016 per request4 in flight
11KnowLine-12B-Gen1‡ Open · 12BLatency 0.071 sCost $0.052$52.28 per 1,000per 1M

KnowLine-12B-Gen1 · PelaAI · decision model

Base model: Gemma 4 12B Unified

Parameters: 12B dense, as the maker states it (counted from the released files: 11.96B).

Accuracy 62.5% (95%: 60.4–64.5%)

Reading 86.0% · grammar 52.5%

‡ Turkish not officially supported: it lists other languages (source, read 16 November 2026)

Non-hallucination with “I don't know” 0.1% · answered 100.0% · error rate 36.6%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 6.2 points (option score)

Cost $0.052$52.28 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.071 s median per request, 1 in flight

Open its answers →

62.5%60.4%–64.5%86.0%52.5%0.1%answered 100%6.2option score$0.052$52.28from GPU time0.0711 per request1 in flight
11d1.2‡ Closed · EfficientLatency 0.213 sCost $0.007$6.83 per 1,000per 1M

d1.2 · Liquid AI · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.02 per 1M input tokens (https://www.liquid.ai/blog/d1-2, read 2026-11-16)

Accuracy 61.7% (95%: 59.7–63.7%)

Reading 86.0% · grammar 51.4%

‡ Turkish not officially supported: it names no language (source, read 16 November 2026)

Non-hallucination with “I don't know” 79.4% · answered 55.7% · error rate 20.7%

Kept at 99% confidence with no observed mistakes: 1.5%

Confidence: calibration gap 3.3 points (option score)

Cost $0.007$6.83 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.213 s median per request, 1 in flight

Open its answers →

61.7%59.7%–63.7%86.0%51.4%79.4%answered 56%3.3option score$0.007$6.83at list prices0.2131 per request1 in flight
11d1.1‡ Closed · EfficientLatency 0.195 sCost $0.007$6.83 per 1,000per 1M

d1.1 · Liquid AI · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.02 per 1M input tokens (https://www.liquid.ai/pricing, read 2026-10-24)

Accuracy 61.5% (95%: 59.5–63.5%)

Reading 84.6% · grammar 51.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence with no observed mistakes: 4.0%

Confidence: calibration gap 2.0 points (option score)

Cost $0.007$6.83 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.195 s median per request, 1 in flight

Open its answers →

61.5%59.5%–63.5%84.6%51.7%Not run2.0option score$0.007$6.83at list prices0.1951 per request1 in flight
11celeris-1-decision‡ Closed · EfficientLatency 0.401 sCost $0.015$15.28 per 1,000per 1M

celeris-1-decision · Celeris · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.04 per 1M input tokens (https://docs.celeris.ai/pricing, read 2026-10-10)

Accuracy 60.5% (95%: 58.4–62.5%)

Reading 85.2% · grammar 49.9%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 4.3 points (option score)

Cost $0.015$15.28 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.401 s median per request, 1 in flight

Open its answers →

60.5%58.4%–62.5%85.2%49.9%Not run4.3option score$0.015$15.28at list prices0.4011 per request1 in flight
11Perplexity Decider 27B v1‡ Open · 27BLatency 0.216 sCost $0.016$15.54 per 1,000per 1M

Perplexity Decider 27B v1 · Perplexity · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 26.09B).

Accuracy 60.4% (95%: 58.3–62.4%)

Reading 86.0% · grammar 49.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence with no observed mistakes: 1.0%

Confidence: calibration gap 5.0 points (option score)

Cost $0.016$15.54 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.216 s median per request, 1 in flight

Open its answers →

60.4%58.3%–62.4%86.0%49.5%Not run5.0option score$0.016$15.54at list prices0.2161 per request1 in flighton its repeats
12OneJev-27B‡ Open · 27BLatency 0.125 sCost $0.099$99.09 per 1,000per 1M

OneJev-27B · OmniJev · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 27.36B).

Accuracy 59.6% (95%: 57.5–61.6%)

Reading 85.2% · grammar 48.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.1% · answered 100.0% · error rate 40.6%

Kept at 99% confidence with no observed mistakes: 0.1%

Confidence: calibration gap 5.5 points (option score)

Cost $0.099$99.09 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.125 s median per request, 1 in flight

Open its answers →

59.6%57.5%–61.6%85.2%48.7%0.1%answered 100%5.5option score$0.099$99.09from GPU time0.1251 per request1 in flight
12Jev 1.14‡ Closed · EfficientLatency 0.211 sCost $0.029$28.84 per 1,000per 1M

Jev 1.14 · TypeSafe · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.042 per 1M input tokens (https://docs.typesafe.ai/models, read 2026-10-16)

Accuracy 59.2% (95%: 57.1–61.2%)

Reading 85.1% · grammar 48.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: 11.8%, 1 of them wrong

Confidence: calibration gap 2.5 points (option score)

Cost $0.029$28.84 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.211 s median per request, 1 in flight

Open its answers →

59.2%57.1%–61.2%85.1%48.2%Not run2.5option score$0.029$28.84at list prices0.2111 per request1 in flight
12Qwen3.5-4B reasoning on · chat‡ Open · 4.0BLatency 140.3 sCost $2.998$2,998 per 1,000per 1M

Qwen3.5-4B reasoning on · chat · Alibaba · chat model

Parameters: 4B dense, as the maker states it (counted from the released files: 4.66B).

Accuracy 58.9% (95%: 56.8–61.0%)

Reading 79.1% · grammar 50.3%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 3.3% · answered 98.4% · error rate 40.7%

Confidence: not measured

Cost $2.998$2,998 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 140.3 s median per request, 32 in flight

Open its answers →

58.9%56.8%–61.0%79.1%50.3%3.3%answered 98%Not measureda plain answer line, no confidence$2.998$2,998from GPU time140.31 per request32 in flight
12Mercury Decide‡ Closed · EfficientLatency 0.313 sCost $0.008$7.71 per 1,000per 1M

Mercury Decide · Inception · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.04 per 1M input tokens (https://www.inceptionlabs.ai/, read 2026-11-10)

Accuracy 58.3% (95%: 56.2–60.3%)

Reading 84.5% · grammar 47.1%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: 20.5%, 13 of them wrong

Confidence: calibration gap 15.5 points (option score)

Cost $0.008$7.71 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.313 s median per request, 1 in flight

Open its answers →

58.3%56.2%–60.3%84.5%47.1%Not run15.5option score$0.008$7.71at list prices0.3131 per request1 in flight
12Gemma 4 E4B default reasoning · chat‡ Open · 8.0BLatency 13.4 sCost $0.307$307 per 1,000per 1M

Gemma 4 E4B default reasoning · chat · Google · chat model

Parameters: 8B dense · 4.5B effective, as the maker states it (counted from the released files: 8.00B).

Accuracy 57.6% (95%: 55.5–59.6%)

Reading 77.6% · grammar 49.0%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 29.7% · answered 86.0% · error rate 38.4%

Confidence: not measured

Cost $0.307$307 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 13.4 s median per request, 32 in flight

Open its answers →

57.6%55.5%–59.6%77.6%49.0%29.7%answered 86%Not measureda plain answer line, no confidence$0.307$307from GPU time13.41 per request32 in flight
12Jev 1.13‡ Closed · EfficientLatency 0.243 sCost $0.029$28.85 per 1,000per 1M

Jev 1.13 · TypeSafe · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.042 per 1M input tokens (https://typesafe.ai/blog/introducing-system-one-models-and-jev, read 2026-10-08)

Accuracy 57.5% (95%: 55.4–59.6%)

Reading 83.7% · grammar 46.4%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: 14.2%, 1 of them wrong

Confidence: calibration gap 5.1 points (option score)

Cost $0.029$28.85 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.243 s median per request, 1 in flight

Open its answers →

57.5%55.4%–59.6%83.7%46.4%Not run5.1option score$0.029$28.85at list prices0.2431 per request1 in flighton its repeats
12Clef‡ Open · 27BLatency 0.205 sCost $0.110$110 per 1,000per 1M

Clef · Cloudflare · decision model

Parameters: 27B dense, as the maker states it (counted from the released files: 27.48B).

Accuracy 57.3% (95%: 55.2–59.3%)

Reading 85.1% · grammar 45.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 7.4% · answered 96.8% · error rate 42.0%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 10.7 points (option score)

Cost $0.110$110 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.205 s median per request, 1 in flight

Open its answers →

57.3%55.2%–59.3%85.1%45.5%7.4%answered 97%10.7option score$0.110$110at the maker's list price0.2051 per request1 in flight
12Claude Haiku 5.5 max · chat‡ Closed · EfficientLatency 417.0 sCost $2.965$2,965 per 1,000per 1M

Claude Haiku 5.5 max · chat · Anthropic · chat model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.10 per 1M input tokens (https://openrouter.ai/anthropic/claude-haiku-5.5, read 2026-11-10)

Accuracy 57.1% (95%: 55.0–59.1%)

Reading 86.9% · grammar 44.4%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 22.0% · answered 67.7% · error rate 9.3%

Confidence: calibration gap 17.5 points (written estimate)

Cost $2.965$2,965 per 1,000 questionsper 1M questions (at list prices) · 16 questions per request

Latency: 417.0 s median per request, 4 in flight

Open its answers →

57.1%55.0%–59.1%86.9%44.4%22.0%answered 68%17.5written estimate$2.965$2,965at list prices417.016 per request4 in flight
12d1‡ Closed · EfficientLatency 0.305 sCost $0.007$7.43 per 1,000per 1M

d1 · Liquid AI · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.02 per 1M input tokens (https://www.liquid.ai/pricing, read 2026-10-18)

Accuracy 57.0% (95%: 54.9–59.1%)

Reading 82.8% · grammar 46.0%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: 12.5%, 1 of them wrong

Confidence: calibration gap 10.2 points (option score)

Cost $0.007$7.43 per 1,000 questionsper 1M questions (free tier, at the list price) · 1 question per request

Latency: 0.305 s median per request, 1 in flight

Open its answers →

57.0%54.9%–59.1%82.8%46.0%Not run10.2option score$0.007$7.43free tier, at the list price0.3051 per request1 in flighton its repeats
12Jeffan-9B-1211‡ Open · 9.0BLatency 0.064 sCost $0.044$44.29 per 1,000per 1M

Jeffan-9B-1211 · tattrongvu · decision model

Base model: Qwen3.5-9B

Parameters: 9B dense, as the maker states it (counted from the released files: 8.95B).

Accuracy 57.0% (95%: 54.9–59.1%)

Reading 82.9% · grammar 46.0%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 4.3% · answered 98.1% · error rate 42.5%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 4.6 points (option score)

Cost $0.044$44.29 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.064 s median per request, 1 in flight

Open its answers →

57.0%54.9%–59.1%82.9%46.0%4.3%answered 98%4.6option score$0.044$44.29from GPU time0.0641 per request1 in flight
12Drex 2.0‡ Open · 35B MoE · 3.0B activeLatency 0.325 sCost $0.019$19.15 per 1,000per 1M

Drex 2.0 · Nace.AI · decision model

Mixture of experts: 35B total (counted from the released files: 35.11B), about 3B active per step. Size class uses the total (35B).

Accuracy 55.7% (95%: 53.6–57.8%)

Reading 81.4% · grammar 44.8%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence with no observed mistakes: 1.5%

Confidence: calibration gap 4.2 points (option score)

Cost $0.019$19.15 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.325 s median per request, 1 in flight

Open its answers →

55.7%53.6%–57.8%81.4%44.8%Not run4.2option score$0.019$19.15at list prices0.3251 per request1 in flight
12Open-Jev-27B-v1.1‡ Open · 27BLatency 0.386 sCost $0.271$271 per 1,000per 1M

Open-Jev-27B-v1.1 · Zefan Cai · decision model

Base model: Qwen3.8-27B + adapter

Parameters: 27B dense, as the maker states it (counted from the released files: 27.80B).

Accuracy 55.7% (95%: 53.6–57.8%)

Reading 82.9% · grammar 44.1%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 1.4% · answered 99.4% · error rate 44.2%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 24.5 points (option score)

Cost $0.271$271 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.386 s median per request, 1 in flight

Open its answers →

55.7%53.6%–57.8%82.9%44.1%1.4%answered 99%24.5option score$0.271$271from GPU time0.3861 per request1 in flighton its repeats
12Cygnet‡ Open · 12BLatency 0.036 sCost $0.039$39.39 per 1,000per 1M

Cygnet · Blockbrain Labs · decision model

Base model: Gemma 4 12B IT

Parameters: 11.95B dense, as the maker states it (counted from the released files: 11.96B).

Accuracy 55.7% (95%: 53.6–57.8%)

Reading 82.8% · grammar 44.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 3.2% · answered 98.5% · error rate 44.2%

Kept at 99% confidence with no observed mistakes: 0.1%

Confidence: calibration gap 13.8 points (option score)

Cost $0.039$39.39 per 1,000 questionsper 1M questions (at a host's list price for the same weights) · 1 question per request

Latency: 0.036 s median per request, 1 in flight

Open its answers →

55.7%53.6%–57.8%82.8%44.2%3.2%answered 99%13.8option score$0.039$39.39at a host's list price for the same weights0.0361 per request1 in flight
12Winnow-12B‡ Open · 12BLatency 0.059 sCost $0.041$41.40 per 1,000per 1M

Winnow-12B · EldanRing · decision model

Parameters: 11.95B dense, as the maker states it (counted from the released files: 11.91B).

Accuracy 55.5% (95%: 53.4–57.5%)

Reading 82.0% · grammar 44.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 4.3% · answered 98.0% · error rate 44.8%

Kept at 99% confidence: 7.4%, 1 of them wrong

Confidence: calibration gap 19.7 points (option score)

Cost $0.041$41.40 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.059 s median per request, 1 in flight

Open its answers →

55.5%53.4%–57.5%82.0%44.2%4.3%answered 98%19.7option score$0.041$41.40from GPU time0.0591 per request1 in flight
13KnowLine-9B-Gen1‡ Open · 9.0BLatency 0.057 sCost $0.041$40.69 per 1,000per 1M

KnowLine-9B-Gen1 · PelaAI · decision model

Base model: Qwen3.5-9B

Parameters: 9B dense, as the maker states it (counted from the released files: 9.65B).

Accuracy 54.5% (95%: 52.5–56.6%)

Reading 79.9% · grammar 43.8%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.5% · answered 99.8% · error rate 45.4%

Kept at 99% confidence with no observed mistakes: 2.8%

Confidence: calibration gap 4.6 points (option score)

Cost $0.041$40.69 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.057 s median per request, 1 in flight

Open its answers →

54.5%52.5%–56.6%79.9%43.8%0.5%answered 100%4.6option score$0.041$40.69from GPU time0.0571 per request1 in flight
13Cerno 9B v1.2‡ Open · 9.0BLatency 0.051 sCost $0.036$36.02 per 1,000per 1M

Cerno 9B v1.2 · Apertix AI · decision model

Parameters: 9B dense, as the maker states it (counted from the released files: 8.95B).

Accuracy 54.5% (95%: 52.4–56.5%)

Reading 79.7% · grammar 43.7%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.2% · answered 99.9% · error rate 45.4%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 9.8 points (option score)

Cost $0.036$36.02 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.051 s median per request, 1 in flight

Open its answers →

54.5%52.4%–56.5%79.7%43.7%0.2%answered 100%9.8option score$0.036$36.02from GPU time0.0511 per request1 in flight
13KnowLine-9B-Gen2‡ Open · 9.0BLatency 0.048 sCost $0.035$35.40 per 1,000per 1M

KnowLine-9B-Gen2 · PelaAI · decision model

Base model: Qwen3.5-9B

Parameters: 9B dense, as the maker states it (counted from the released files: 9.65B).

Accuracy 53.3% (95%: 51.2–55.4%)

Reading 79.6% · grammar 42.1%

‡ Turkish not officially supported: it lists other languages (source, read 16 November 2026)

Non-hallucination with “I don't know” 2.6% · answered 98.8% · error rate 47.4%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 5.1 points (option score)

Cost $0.035$35.40 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.048 s median per request, 1 in flight

Open its answers →

53.3%51.2%–55.4%79.6%42.1%2.6%answered 99%5.1option score$0.035$35.40from GPU time0.0481 per request1 in flight
13Adjudex 9B v5‡ Open · 9.0BLatency 0.038 sCost $0.026$26.30 per 1,000per 1M

Adjudex 9B v5 · newsbang · decision model

Base model: Qwen3.5-9B

Parameters: 9B dense, as the maker states it (counted from the released files: 8.39B).

Accuracy 53.1% (95%: 51.0–55.2%)

Reading 80.8% · grammar 41.4%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.8% · answered 99.6% · error rate 46.6%

Kept at 99% confidence: 6.0%, 2 of them wrong

Confidence: calibration gap 9.2 points (option score)

Cost $0.026$26.30 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.038 s median per request, 1 in flight

Open its answers →

53.1%51.0%–55.2%80.8%41.4%0.8%answered 100%9.2option score$0.026$26.30from GPU time0.0381 per request1 in flight
13Elephant-1‡ Open · 9.0BLatency 0.074 sCost $0.014$13.87 per 1,000per 1M

Elephant-1 · Base Labs · decision model

Parameters: 9B dense, as the maker states it (counted from the released files: 9.41B).

Accuracy 52.4% (95%: 50.3–54.4%)

Reading 80.5% · grammar 40.4%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 34.8% · answered 81.6% · error rate 42.3%

Kept at 99% confidence with no observed mistakes: 7.1%

Confidence: calibration gap 11.8 points (option score)

Cost $0.014$13.87 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.074 s median per request, 1 in flight

Open its answers →

52.4%50.3%–54.4%80.5%40.4%34.8%answered 82%11.8option score$0.014$13.87at the maker's list price0.0741 per request1 in flight
13GLiDE 9B‡ Open · 7.9BLatency 0.207 sCost No list price

GLiDE 9B · Fastino · decision model

Parameters: 7.94B dense, as the maker states it (counted from the released files: 7.94B).

Accuracy 52.0% (95%: 50.0–54.1%)

Reading 80.2% · grammar 40.1%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 24.1% · answered 87.4% · error rate 45.4%

Kept at 99% confidence with no observed mistakes: 1.2%

Confidence: calibration gap 8.5 points (option score)

Cost: No list price · 1 question per request

Latency: 0.207 s median per request, 1 in flight

Open its answers →

52.0%50.0%–54.1%80.2%40.1%24.1%answered 87%8.5option scoreNo list price0.2071 per request1 in flight
13Dexy-Decision-9B-V2‡ Open · 9.0BLatency 0.031 sCost $0.023$23.03 per 1,000per 1M

Dexy-Decision-9B-V2 · Patrick-18 · decision model

Base model: vllm-sr/d3-flash

Parameters: 9B dense, as the maker states it (counted from the released files: 8.39B).

Accuracy 51.6% (95%: 49.5–53.7%)

Reading 79.6% · grammar 39.7%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 3.4% · answered 98.3% · error rate 48.4%

Kept at 99% confidence: 6.3%, 1 of them wrong

Confidence: calibration gap 9.1 points (option score)

Cost $0.023$23.03 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.031 s median per request, 1 in flight

Open its answers →

51.6%49.5%–53.7%79.6%39.7%3.4%answered 98%9.1option score$0.023$23.03from GPU time0.0311 per request1 in flight
13StartLux-Decision-v1.1-9B-Instant‡ Open · 9.0BLatency 0.154 sCost No list price

StartLux-Decision-v1.1-9B-Instant · StartLux · decision model

Parameters: 9B dense, as the maker states it (counted from the released files: 9.65B).

Accuracy 51.2% (95%: 49.1–53.3%)

Reading 78.8% · grammar 39.5%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 2.2% · answered 98.9% · error rate 49.2%

Kept at 99% confidence with no observed mistakes: 0.9%

Confidence: calibration gap 7.6 points (option score)

Cost: No list price · 1 question per request

Latency: 0.154 s median per request, 1 in flight

Open its answers →

51.2%49.1%–53.3%78.8%39.5%2.2%answered 99%7.6option scoreNo list price0.1541 per request1 in flight
14Perplexity Decider Flash v1.3‡ Open · 4.0BLatency 0.033 sCost $0.025$24.69 per 1,000per 1M

Perplexity Decider Flash v1.3 · Perplexity · decision model

Parameters: 4B dense, as the maker states it (counted from the released files: 4.54B).

Accuracy 49.7% (95%: 47.6–51.8%)

Reading 74.4% · grammar 39.2%

‡ Turkish not officially supported: it names no language (source, read 14 November 2026)

Non-hallucination with “I don't know” 6.5% · answered 96.6% · error rate 50.4%

Kept at 99% confidence with no observed mistakes: 1.3%

Confidence: calibration gap 12.6 points (option score)

Cost $0.025$24.69 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.033 s median per request, 1 in flight

Open its answers →

49.7%47.6%–51.8%74.4%39.2%6.5%answered 97%12.6option score$0.025$24.69from GPU time0.0331 per request1 in flight
14vera-medium-omni‡ Open · 4.2BLatency 0.029 sCost $0.021$20.72 per 1,000per 1M

vera-medium-omni · scar-ai · decision model

Base model: Qwen3.5-4B

Parameters: 4.2B dense, as the maker states it (counted from the released files: 4.21B).

Accuracy 49.1% (95%: 47.0–51.2%)

Reading 75.5% · grammar 37.9%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 8.9% · answered 95.2% · error rate 51.4%

Kept at 99% confidence with no observed mistakes: 0.2%

Confidence: calibration gap 3.7 points (option score)

Cost $0.021$20.72 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.029 s median per request, 1 in flight

Open its answers →

49.1%47.0%–51.2%75.5%37.9%8.9%answered 95%3.7option score$0.021$20.72from GPU time0.0291 per request1 in flight
14KnowLine 4B Gen5.5‡ Open · 4.0BLatency 0.099 sCost No list price

KnowLine 4B Gen5.5 · PelaAI · decision model

Parameters: 4B dense, as the maker states it (counted from the released files: 4.66B).

Accuracy 48.5% (95%: 46.4–50.6%)

Reading 74.8% · grammar 37.3%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 4.7% · answered 97.5% · error rate 52.3%

Kept at 99% confidence with no observed mistakes: 1.8%

Confidence: calibration gap 13.5 points (option score)

Cost: No list price · 1 question per request

Latency: 0.099 s median per request, 1 in flight

Open its answers →

48.5%46.4%–50.6%74.8%37.3%4.7%answered 97%13.5option scoreNo list price0.0991 per request1 in flight
14StartLux-Decision-v1.1-4B-Instant‡ Open · 4.0BLatency 0.074 sCost No list price

StartLux-Decision-v1.1-4B-Instant · StartLux · decision model

Parameters: 4B dense, as the maker states it (counted from the released files: 4.66B).

Accuracy 47.9% (95%: 45.8–50.0%)

Reading 74.1% · grammar 36.7%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 1.8% · answered 99.0% · error rate 52.7%

Kept at 99% confidence with no observed mistakes: 0.4%

Confidence: calibration gap 7.1 points (option score)

Cost: No list price · 1 question per request

Latency: 0.074 s median per request, 1 in flight

Open its answers →

47.9%45.8%–50.0%74.1%36.7%1.8%answered 99%7.1option scoreNo list price0.0741 per request1 in flight
14Perplexity Decider Flash v1.2‡ Open · 4.0BLatency 0.132 sCost $0.004$3.88 per 1,000per 1M

Perplexity Decider Flash v1.2 · Perplexity · decision model

Parameters: 4B dense, as the maker states it (counted from the released files: 4.54B).

Accuracy 47.8% (95%: 45.7–49.9%)

Reading 72.7% · grammar 37.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 14.7% · answered 91.8% · error rate 51.7%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 8.3 points (option score)

Cost $0.004$3.88 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.132 s median per request, 1 in flight

Open its answers →

47.8%45.7%–49.9%72.7%37.2%14.7%answered 92%8.3option score$0.004$3.88at the maker's list price0.1321 per request1 in flight
14Valen-Decision-1.5-9B‡ Open · 9.0BLatency 0.059 sCost $0.040$40.49 per 1,000per 1M

Valen-Decision-1.5-9B · Valen-Team · decision model

Base model: Qwen3.5-9B

Parameters: 9B dense, as the maker states it (counted from the released files: 8.40B).

Accuracy 47.0% (95%: 44.9–49.0%)

Reading 75.0% · grammar 35.0%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.1% · answered 100.0% · error rate 52.7%

Kept at 99% confidence with no observed mistakes: 0.3%

Confidence: calibration gap 3.8 points (option score)

Cost $0.040$40.49 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.059 s median per request, 1 in flight

Open its answers →

47.0%44.9%–49.0%75.0%35.0%0.1%answered 100%3.8option score$0.040$40.49from GPU time0.0591 per request1 in flight
14Verdin-4B-v1.1‡ Open · 4.0BLatency 0.049 sCost $0.034$34.21 per 1,000per 1M

Verdin-4B-v1.1 · victorbennett · decision model

Base model: Qwen3.5-4B

Parameters: 4B dense, as the maker states it (counted from the released files: 4.21B).

Accuracy 46.8% (95%: 44.7–48.9%)

Reading 71.2% · grammar 36.4%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 0.4% · answered 99.8% · error rate 53.3%

Kept at 99% confidence with no observed mistakes: 1.2%

Confidence: calibration gap 9.8 points (option score)

Cost $0.034$34.21 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.049 s median per request, 1 in flight

Open its answers →

46.8%44.7%–48.9%71.2%36.4%0.4%answered 100%9.8option score$0.034$34.21from GPU time0.0491 per request1 in flight
14Chinese Decision Pointer 4B‡ Open · 4.0BLatency 0.035 sCost $0.025$24.58 per 1,000per 1M

Chinese Decision Pointer 4B · quincyqiang · decision model

Base model: Qwen3.5-4B

Parameters: 4B dense, as the maker states it (counted from the released files: 4.21B).

Accuracy 46.7% (95%: 44.6–48.8%)

Reading 70.9% · grammar 36.4%

‡ Turkish not officially supported: it lists other languages (source, read 15 November 2026)

Non-hallucination with “I don't know” 11.2% · answered 93.8% · error rate 52.5%

Kept at 99% confidence: 2.3%, 1 of them wrong

Confidence: calibration gap 11.7 points (option score)

Cost $0.025$24.58 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.035 s median per request, 1 in flight

Open its answers →

46.7%44.6%–48.8%70.9%36.4%11.2%answered 94%11.7option score$0.025$24.58from GPU time0.0351 per request1 in flight
14Microsoft-Decision-1‡ Closed · EfficientLatency 0.286 sCost $0.014$13.82 per 1,000per 1M

Microsoft-Decision-1 · Microsoft · decision model

Base model: Qwen3.5-9B

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.042 per 1M input tokens (https://openrouter.ai/microsoft/microsoft-decision-1, read 2026-10-17)

Accuracy 46.5% (95%: 44.4–48.5%)

Reading 73.5% · grammar 35.0%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: 2.8%, 3 of them wrong

Confidence: calibration gap 19.8 points (option score)

Cost $0.014$13.82 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.286 s median per request, 1 in flight

Open its answers →

46.5%44.4%–48.5%73.5%35.0%Not run19.8option score$0.014$13.82at list prices0.2861 per request1 in flight
14Qwen3.5-4B reasoning off · chat‡ Open · 4.0BLatency 32.2 sCost $0.894$894 per 1,000per 1M

Qwen3.5-4B reasoning off · chat · Alibaba · chat model

Parameters: 4B dense, as the maker states it (counted from the released files: 4.66B).

Accuracy 46.2% (95%: 44.1–48.3%)

Reading 68.4% · grammar 36.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 6.9% · answered 95.4% · error rate 54.0%

Confidence: not measured

Cost $0.894$894 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 32.2 s median per request, 32 in flight

Open its answers →

46.2%44.1%–48.3%68.4%36.7%6.9%answered 95%Not measureda plain answer line, no confidence$0.894$894from GPU time32.21 per request32 in flight
14Qwen3-14B reasoning off · chat‡ Open · 15BLatency 2.85 sCost $0.032$32.31 per 1,000per 1M

Qwen3-14B reasoning off · chat · Alibaba · chat model

Parameters: 14.8B dense, as the maker states it (counted from the released files: 14.77B).

Accuracy 45.7% (95%: 43.6–47.8%)

Reading 72.1% · grammar 34.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 1.9% · answered 99.0% · error rate 53.8%

Confidence: calibration gap 41.0 points (answer letter probability)

Cost $0.032$32.31 per 1,000 questionsper 1M questions (at a host's list price for the same weights) · 16 questions per request

Latency: 2.85 s median per request, 1 in flight

Open its answers →

45.7%43.6%–47.8%72.1%34.5%1.9%answered 99%41.0answer letter probability$0.032$32.31at a host's list price for the same weights2.8516 per request1 in flight
14Clef-Flash‡ Open · 9.0BLatency 0.075 sCost $0.017$17.37 per 1,000per 1M

Clef-Flash · Cloudflare · decision model

Parameters: 9B dense, as the maker states it (counted from the released files: 9.53B).

Accuracy 45.7% (95%: 43.6–47.8%)

Reading 73.0% · grammar 34.1%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 9.0% · answered 95.0% · error rate 53.1%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 15.5 points (option score)

Cost $0.017$17.37 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.075 s median per request, 1 in flight

Open its answers →

45.7%43.6%–47.8%73.0%34.1%9.0%answered 95%15.5option score$0.017$17.37at the maker's list price0.0751 per request1 in flight
14Erk-14B reasoning off · chat Open · 14BLatency 2.39 sCost $0.106$106 per 1,000per 1M

Erk-14B reasoning off · chat · eCloud Technology · chat model

Parameters: 14B dense, as the maker states it (counted from the released files: 14.77B).

Accuracy 45.0% (95%: 42.9–47.1%)

Reading 72.3% · grammar 33.4%

Turkish: listed in its card (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.3% · answered 99.8% · error rate 55.3%

Confidence: calibration gap 13.5 points (answer letter probability)

Cost $0.106$106 per 1,000 questionsper 1M questions (from GPU time) · 16 questions per request

Latency: 2.39 s median per request, 1 in flight

Open its answers →

45.0%42.9%–47.1%72.3%33.4%0.3%answered 100%13.5answer letter probability$0.106$106from GPU time2.3916 per request1 in flight
15Metask-Jev-4B Open · 4.5BLatency 0.090 sCost $0.063$62.82 per 1,000per 1M

Metask-Jev-4B · Metask Lab · decision model

Parameters: 4.54B dense, as the maker states it (counted from the released files: 4.54B).

Accuracy 40.8% (95%: 38.7–42.8%)

Reading 62.8% · grammar 31.4%

Turkish: listed in its card (source, read 12 November 2026)

Non-hallucination with “I don't know” 0.8% · answered 99.5% · error rate 60.1%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 5.8 points (option score)

Cost $0.063$62.82 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.090 s median per request, 1 in flight

Open its answers →

40.8%38.7%–42.8%62.8%31.4%0.8%answered 100%5.8option score$0.063$62.82from GPU time0.0901 per request1 in flight
15KnowLine-2B-Gen1‡ Open · 2.0BLatency 0.062 sCost No list price

KnowLine-2B-Gen1 · PelaAI · decision model

Base model: Qwen3.5-2B

Parameters: 2B dense, as the maker states it (counted from the released files: 2.27B).

Accuracy 36.8% (95%: 34.8–38.8%)

Reading 54.3% · grammar 29.4%

‡ Turkish not officially supported: it lists other languages (source, read 13 November 2026)

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 8.4 points (option score)

Cost: No list price · 1 question per request

Latency: 0.062 s median per request, 1 in flight

Open its answers →

36.8%34.8%–38.8%54.3%29.4%Not run8.4option scoreNo list price0.0621 per request1 in flight
16GLiDE 2B‡ Open · 1.9BLatency 0.034 sCost No list price

GLiDE 2B · Fastino · decision model

Parameters: 1.88B dense, as the maker states it (counted from the released files: 1.88B).

Accuracy 35.3% (95%: 33.3–37.3%)

Reading 51.2% · grammar 28.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 38.5% · answered 72.2% · error rate 61.7%

Kept at 99% confidence with no observed mistakes: 0.9%

Confidence: calibration gap 20.7 points (option score)

Cost: No list price · 1 question per request

Latency: 0.034 s median per request, 1 in flight

Open its answers →

35.3%33.3%–37.3%51.2%28.5%38.5%answered 72%20.7option scoreNo list price0.0341 per request1 in flight
16StartLux-Decision-v1.1-2B-Instant‡ Open · 2.0BLatency 0.040 sCost No list price

StartLux-Decision-v1.1-2B-Instant · StartLux · decision model

Parameters: 2B dense, as the maker states it (counted from the released files: 2.27B).

Accuracy 34.3% (95%: 32.3–36.3%)

Reading 52.9% · grammar 26.3%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 21.5% · answered 84.8% · error rate 65.0%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 6.4 points (option score)

Cost: No list price · 1 question per request

Latency: 0.040 s median per request, 1 in flight

Open its answers →

34.3%32.3%–36.3%52.9%26.3%21.5%answered 85%6.4option scoreNo list price0.0401 per request1 in flight
16LFM2.5-2.6B default reasoning · chat‡ Open · 2.7BLatency 51.2 sCost $2.761$2,761 per 1,000per 1M

LFM2.5-2.6B default reasoning · chat · Liquid AI · chat model

Parameters: 2.69B dense, as the maker states it (counted from the released files: 2.70B).

Accuracy 33.9% (95%: 31.9–35.9%)

Reading 45.4% · grammar 29.0%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 24.7% · answered 81.6% · error rate 63.2%

Confidence: not measured

Cost $2.761$2,761 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 51.2 s median per request, 32 in flight

Open its answers →

33.9%31.9%–35.9%45.4%29.0%24.7%answered 82%Not measureda plain answer line, no confidence$2.761$2,761at the maker's list price51.21 per request32 in flight
16LFM3-1.7B reasoning on · chat‡ Open · 1.7BLatency 24.2 sCost $1.277$1,277 per 1,000per 1M

LFM3-1.7B reasoning on · chat · Liquid AI · chat model

Parameters: 1.73B dense, as the maker states it (counted from the released files: 1.73B).

Accuracy 32.4% (95%: 30.5–34.4%)

Reading 44.2% · grammar 27.4%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 67.0% · answered 45.5% · error rate 58.8%

Confidence: not measured

Cost $1.277$1,277 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 24.2 s median per request, 32 in flight

Open its answers →

32.4%30.5%–34.4%44.2%27.4%67.0%answered 46%Not measureda plain answer line, no confidence$1.277$1,277at the maker's list price24.21 per request32 in flight
17d1-2B‡ Open · 1.9BLatency 0.019 sCost $0.013$13.17 per 1,000per 1M

d1-2B · Liquid AI · decision model

Base model: LFM3-VL-2B

Parameters: 1.92B dense, as the maker states it (counted from the released files: 1.92B).

Accuracy 29.4% (95%: 27.5–31.3%)

Reading 40.9% · grammar 24.5%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 65.5% · answered 45.0% · error rate 64.4%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 5.8 points (option score)

Cost $0.013$13.17 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.019 s median per request, 1 in flight

Open its answers →

29.4%27.5%–31.3%40.9%24.5%65.5%answered 45%5.8option score$0.013$13.17from GPU time0.0191 per request1 in flight
17GLiDE 0.8B‡ Open · 0.8BLatency 0.022 sCost No list price

GLiDE 0.8B · Fastino · decision model

Parameters: 750M dense, as the maker states it (counted from the released files: 0.75B).

Accuracy 29.3% (95%: 27.4–31.2%)

Reading 42.4% · grammar 23.7%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 29.2% · answered 78.2% · error rate 67.7%

Kept at 99% confidence: 0.8%, 1 of them wrong

Confidence: calibration gap 24.8 points (option score)

Cost: No list price · 1 question per request

Latency: 0.022 s median per request, 1 in flight

Open its answers →

29.3%27.4%–31.2%42.4%23.7%29.2%answered 78%24.8option scoreNo list price0.0221 per request1 in flight
17OneJev1.6-0.8B‡ Open · 0.8BLatency 0.017 sCost No list price

OneJev1.6-0.8B · OmniJev · decision model

Parameters: 800M dense, as the maker states it (counted from the released files: 0.85B).

Accuracy 29.1% (95%: 27.3–31.1%)

Reading 39.6% · grammar 24.6%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 43.4% · answered 65.7% · error rate 68.2%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 9.3 points (option score)

Cost: No list price · 1 question per request

Latency: 0.017 s median per request, 1 in flight

Open its answers →

29.1%27.3%–31.1%39.6%24.6%43.4%answered 66%9.3option scoreNo list price0.0171 per request1 in flight
17StartLux-Decision-v1.1-0.8B-Instant‡ Open · 0.8BLatency 0.028 sCost No list price

StartLux-Decision-v1.1-0.8B-Instant · StartLux · decision model

Base model: Qwen3.5-0.8B

Parameters: 800M dense, as the maker states it (counted from the released files: 0.87B).

Accuracy 28.6% (95%: 26.7–30.5%)

Reading 35.1% · grammar 25.8%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 12.4% · answered 90.9% · error rate 70.5%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 4.4 points (option score)

Cost: No list price · 1 question per request

Latency: 0.028 s median per request, 1 in flight

Open its answers →

28.6%26.7%–30.5%35.1%25.8%12.4%answered 91%4.4option scoreNo list price0.0281 per request1 in flight
17LFM3-1.7B reasoning off · chat‡ Open · 1.7BLatency 4.90 sCost $0.646$646 per 1,000per 1M

LFM3-1.7B reasoning off · chat · Liquid AI · chat model

Parameters: 1.73B dense, as the maker states it (counted from the released files: 1.73B).

Accuracy 28.3% (95%: 26.5–30.3%)

Reading 37.2% · grammar 24.6%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 76.2% · answered 32.7% · error rate 62.4%

Confidence: not measured

Cost $0.646$646 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 4.90 s median per request, 32 in flight

Open its answers →

28.3%26.5%–30.3%37.2%24.6%76.2%answered 33%Not measureda plain answer line, no confidence$0.646$646at the maker's list price4.901 per request32 in flight
17Strands Decider 2B‡ Open · 2.0BLatency 0.026 sCost $0.020$20.12 per 1,000per 1M

Strands Decider 2B · Strands Agents · decision model

Parameters: 2B dense, as the maker states it (counted from the released files: 2.29B).

Accuracy 27.5% (95%: 25.7–29.4%)

Reading 36.9% · grammar 23.5%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 9.2% · answered 93.2% · error rate 72.4%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 12.2 points (option score)

Cost $0.020$20.12 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.026 s median per request, 1 in flight

Open its answers →

27.5%25.7%–29.4%36.9%23.5%9.2%answered 93%12.2option score$0.020$20.12from GPU time0.0261 per request1 in flight
17Decider 1‡ Closed · EfficientLatency 0.382 sCost $0.015$14.76 per 1,000per 1M

Decider 1 · meraGPT · decision model

Parameters: not disclosed by maker

Class: Efficient, by its standard list price: $0.04 per 1M input tokens (https://meragpt.com/docs/models, read 2026-10-19)

Accuracy 27.1% (95%: 25.3–29.0%)

Reading 34.6% · grammar 23.9%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 14.3 points (option score)

Cost $0.015$14.76 per 1,000 questionsper 1M questions (at list prices) · 1 question per request

Latency: 0.382 s median per request, 1 in flight

Open its answers →

27.1%25.3%–29.0%34.6%23.9%Not run14.3option score$0.015$14.76at list prices0.3821 per request1 in flight
17KnowLine-0.8B-Gen1‡ Open · 0.8BLatency 0.033 sCost No list price

KnowLine-0.8B-Gen1 · PelaAI · decision model

Base model: Qwen3.5-0.8B

Parameters: 800M dense, as the maker states it (counted from the released files: 0.87B).

Accuracy 27.1% (95%: 25.3–29.0%)

Reading 38.3% · grammar 22.4%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 6.4% · answered 95.3% · error rate 72.3%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 8.3 points (option score)

Cost: No list price · 1 question per request

Latency: 0.033 s median per request, 1 in flight

Open its answers →

27.1%25.3%–29.0%38.3%22.4%6.4%answered 95%8.3option scoreNo list price0.0331 per request1 in flight
17Qwen3.5-2B reasoning off · chat‡ Open · 2.0BLatency 8.84 sCost $0.268$268 per 1,000per 1M

Qwen3.5-2B reasoning off · chat · Alibaba · chat model

Parameters: 2B dense, as the maker states it (counted from the released files: 2.27B).

Accuracy 25.4% (95%: 23.6–27.2%)

Reading 34.3% · grammar 21.6%

‡ Turkish not officially supported: it names no language (source, read 15 November 2026)

Non-hallucination with “I don't know” 20.8% · answered 75.6% · error rate 72.3%

Confidence: not measured

Cost $0.268$268 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 8.84 s median per request, 32 in flight

Open its answers →

25.4%23.6%–27.2%34.3%21.6%20.8%answered 76%Not measureda plain answer line, no confidence$0.268$268from GPU time8.841 per request32 in flight
18GLiClass v4 Edge‡ Open · 0.4BLatency 0.025 sCost No list price

GLiClass v4 Edge · Knowledgator · decision model

Base model: DeBERTa-v3-large

Parameters: 400M dense, as the maker states it (counted from the released files: 0.44B).

Accuracy 23.3% (95%: 21.6–25.2%)

Reading 25.9% · grammar 22.2%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 22.3% · answered 81.9% · error rate 77.0%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 10.5 points (option score)

Cost: No list price · 1 question per request

Latency: 0.025 s median per request, 1 in flight

Open its answers →

23.3%21.6%–25.2%25.9%22.2%22.3%answered 82%10.5option scoreNo list price0.0251 per request1 in flight
18GLiDE 0.4B‡ Open · 0.4BLatency 0.016 sCost No list price

GLiDE 0.4B · Fastino · decision model

Parameters: 400M dense, as the maker states it (counted from the released files: 0.40B).

Accuracy 22.2% (95%: 20.6–24.0%)

Reading 24.5% · grammar 21.3%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 95.3% · answered 5.8% · error rate 78.9%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 8.0 points (option score)

Cost: No list price · 1 question per request

Latency: 0.016 s median per request, 1 in flight

Open its answers →

22.2%20.6%–24.0%24.5%21.3%95.3%answered 6%8.0option scoreNo list price0.0161 per request1 in flight
18LFM2.5-8B-A1B default reasoning · chat‡ Open · 8.3B MoE · 1.5B activeLatency 39.2 sCost $1.050$1,050 per 1,000per 1M

LFM2.5-8B-A1B default reasoning · chat · Liquid AI · chat model

Mixture of experts: 8.3B total (counted from the released files: 8.47B), about 1.5B active per step. Size class uses the total (8.3B).

Accuracy 21.0% (95%: 19.3–22.7%)

Reading 25.6% · grammar 19.0%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 56.0% · answered 42.4% · error rate 74.0%

Confidence: not measured

Cost $1.050$1,050 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 39.2 s median per request, 32 in flight

Open its answers →

21.0%19.3%–22.7%25.6%19.0%56.0%answered 42%Not measureda plain answer line, no confidence$1.050$1,050from GPU time39.21 per request32 in flight
18GLiNER2.5-multi-Decide‡ Open · 0.3BLatency 0.009 sCost $0.006$6.27 per 1,000per 1M

GLiNER2.5-multi-Decide · Fastino · decision model

Parameters: 287M dense, as the maker states it (counted from the released files: 0.29B).

Accuracy 20.2% (95%: 18.6–21.9%)

Reading 21.0% · grammar 19.8%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 6.4% · answered 94.8% · error rate 79.9%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 23.1 points (option score)

Cost $0.006$6.27 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.009 s median per request, 1 in flight

Open its answers →

20.2%18.6%–21.9%21.0%19.8%6.4%answered 95%23.1option score$0.006$6.27from GPU time0.0091 per request1 in flight
18Laya‡ Open · 0.4BLatency 0.011 sCost $0.009$9.00 per 1,000per 1M

Laya · Convai Innovations · decision model

Parameters: 421M dense, as the maker states it (counted from the released files: 0.42B).

Accuracy 19.0% (95%: 17.4–20.7%)

Reading 20.9% · grammar 18.2%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 28.3% · answered 75.6% · error rate 81.5%

Kept at 99% confidence: No answers at this threshold

Confidence: calibration gap 13.2 points (option score)

Cost $0.009$9.00 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.011 s median per request, 1 in flight

Open its answers →

19.0%17.4%–20.7%20.9%18.2%28.3%answered 76%13.2option score$0.009$9.00from GPU time0.0111 per request1 in flighton its repeats
19Laya Multilingual‡ Open · 0.3BLatency 0.009 sCost $0.008$7.74 per 1,000per 1M

Laya Multilingual · Convai Innovations · decision model

Parameters: 322M dense, as the maker states it (counted from the released files: 0.32B).

Accuracy 17.9% (95%: 16.3–19.5%)

Reading 17.1% · grammar 18.2%

‡ Turkish not officially supported: it names no language (source, read 12 November 2026)

Non-hallucination with “I don't know” 14.8% · answered 87.3% · error rate 83.8%

Kept at 99% confidence: 0.0%, 1 of them wrong

Confidence: calibration gap 27.7 points (option score)

Cost $0.008$7.74 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 0.009 s median per request, 1 in flight

Open its answers →

17.9%16.3%–19.5%17.1%18.2%14.8%answered 87%27.7option score$0.008$7.74from GPU time0.0091 per request1 in flighton its repeats
19GLiNER2.5 Multi‡ Open · 0.3BLatency 0.032 sCost $0.006$5.88 per 1,000per 1M

GLiNER2.5 Multi · Fastino · chat model

Parameters: 287M dense, as the maker states it (counted from the released files: 0.29B).

Accuracy 17.5% (95%: 15.9–19.1%)

Reading 17.2% · grammar 17.6%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 6.3% · answered 94.7% · error rate 82.6%

Confidence: calibration gap 35.6 points (option score)

Cost $0.006$5.88 per 1,000 questionsper 1M questions (at the maker's list price) · 1 question per request

Latency: 0.032 s median per request, 1 in flight

Open its answers →

17.5%15.9%–19.1%17.2%17.6%6.3%answered 95%35.6option score$0.006$5.88at the maker's list price0.0321 per request1 in flight
20MiniCPM5-2B reasoning off · chat‡ Open · 2.5BLatency 172.0 sCost $5.464$5,464 per 1,000per 1M

MiniCPM5-2B reasoning off · chat · OpenBMB · chat model

Parameters: 2.52B dense, as the maker states it.

Accuracy 5.6% (95%: 4.7–6.6%)

Reading 6.6% · grammar 5.2%

‡ Turkish not officially supported: it lists other languages (source, read 12 November 2026)

Non-hallucination with “I don't know” 25.4% · answered 20.1% · error rate 75.3%

Confidence: not measured

Cost $5.464$5,464 per 1,000 questionsper 1M questions (from GPU time) · 1 question per request

Latency: 172.0 s median per request, 32 in flight

Open its answers →

5.6%4.7%–6.6%6.6%5.2%25.4%answered 20%Not measureda plain answer line, no confidence$5.464$5,464from GPU time172.01 per request32 in flight

Accuracy: the share of the 2,198 Turkish questions answered right, with its 95% interval as the whisker; higher is better.

Tier: ranked by accuracy among the rows in view, a row joins the tier of the tier's top row when the all-pairs test behind the Accuracy section's letters (9,316 pairs, exact McNemar, Holm-corrected, 5%) found no difference between them, else it starts the next tier; a line separates tiers. Rows in one tier need not be equal. A setup with web search is in no tier (Tier: –).

‡ Turkish not officially supported: the model's own card or maker docs do not list Turkish (read 16 November 2026; Ümit #5067); of the models here only Erk-14B and Metask-Jev-4B list it.

One row per model at its highest reasoning setting, never its best score; tap or point at a row for its details. How rows are chosen

In progress

Models joining Reading (the project lead via Zeki #5064, #5071; Ümit #5067). Each side shows its state until its run is published; nothing here is a score. As of 15 November 2026, 14:05 UTC.

ModelEnglishTurkishPage images
Gemma 4 E4BGoogle · chatDoneDoneQueuedpicture only, text plus picture and two OCR routes done; the other two in the Turkish phase (Zeki #5213)
MiniCPM5-2BOpenBMB · chatDoneSkippedÜmit chose not to run it after its Turkish smoke failed (run 1120: 15 of 20 replies without a final answer line; Zeki #7178, #8830)–
Inkling 2 SmallThinking Machines · chatRunningthrough OpenRouter (Nebius), started 16 November 2026 (Ümit #9817, #10509)Runningthrough OpenRouter (Nebius)–
Spark-X2.6-4BXHToken · chatQueuedQueued–
Qwen3.5-4BAlibaba · chatDoneDoneQueuedin the Turkish phase (Zeki #5213)
Gemma 4 E2BGoogle · chatDoneQueuedin logic's GPU 1 queue on a load of its own (thinking off, 8,192 tokens), after Granite (Ümit #8343, logic #8391)Queuedin the Turkish phase (Zeki #5213)
Gemma 4 12B UnifiedGoogle · chatDoneQueuedin logic's GPU 1 queue, on its 32,768-token setup's server, which also answers to this setup's name (Ümit #6527, logic #6955)Queuedin the Turkish phase (Zeki #5213)
Granite-4.2-3BIBM · chatDoneQueuedText stopped at 741 of 2,198 so that other setups go first; it resumes on logic's next Granite load, then "Bilmiyorum" (logic #7481, #7599)–
Qwen3.5-9BAlibaba · chatQueuedQueuedQueued
Granite-4.2-8BIBM · chatQueuedQueued–
LFM2.5-24B-A2BLiquid AI · chatDoneQueuednot scheduled yet: needs the model served again on logic's GPU 1 (Ümit #6527)–
Granite-4.2-30BIBM · chatQueuedQueued–
Granite-4.2-3B (thinking off)IBM · chatQueuedin logic's GPU 1 queue (Ümit #6924, logic #6955)Queuedwith its English runs, same server–
Qwen3.5-2BAlibaba · chatDoneQueuednot asked yet: Ümit's grid asked for the English runs (#10884)–
Gemma 4 E2B (thinking on, 32,768 tokens)Google · chatDoneQueuednot asked yet: Ümit's grid asked for the English runs (#10884)–
Gemma 4 E4B (thinking on, 32,768 tokens)Google · chatQueuedin logic's GPU 1 queue after the thinking-off setups (Ümit #6523, logic #6955)Queuedwith its English runs, same server–
Gemma 4 12B Unified (thinking on, 32,768 tokens)Google · chatQueuedlast in logic's GPU 1 queue (Ümit #6924, logic #6955)Queuedwith its English runs, same server–
Gemma 4 31B (thinking on)Google · chatQueuednot set up yet: another server, after logic's GPU 1 queue (Ümit #6924, logic #6929)Queuedwith its English runs–
Qwen3-14B (thinking on)Alibaba · chatQueuednot set up yet: another server, after logic's GPU 1 queue (Ümit #6924, logic #6929)Queuedwith its English runs–
Erk-14B (thinking on)eCloud Technology · chatQueuednot set up yet: another server, after logic's GPU 1 queue (Ümit #6924, logic #6929)Queuedwith its English runs–
Raon-1 nanoKRAFTON · chatRunningthinking on, 32,768 tokens, on KRAFTON's own vLLM engine on logic's GPU 1 since 19:01 UTC (Ümit #10884)Queuednot asked yet: Ümit's grid asked for the English runs (#10884)–
Raon-1 nano (thinking off)KRAFTON · chatDoneQueuedasked on its next load in logic's GPU 1 queue, whichever way its smoke came out (run 1154: 3 of 20 replies at the 32,768-token limit, 5 without a letter; Ümit #6515 via Zeki #8211)–
d1-Omni-2BLiquid AI · decisionQueuedQueuedQueued
d1-3BLiquid AI · decisionQueuedQueuedQueued
KnowLine-2B-Gen1PelaAI · decisionDoneDoneQueuedin the Turkish phase (Zeki #5213)
Sieve 31B v3Sthānika AI · decisionDoneDoneQueuedthe four OCR routes only (its server reads text), in the Turkish phase (Zeki #5213)
Turion Jev 31BTurion.AI · decisionDoneDoneQueuedthe four OCR routes only (its server reads text), in the Turkish phase (Zeki #5213)
StartLux-Decision-v1.1-4B-InstantStartLux · decisionDoneDoneQueuedin the Turkish phase (Zeki #5213)
Cerno 9B v1.2Apertix AI · decisionDoneDoneQueuedthe four OCR routes only (its server reads text), in the Turkish phase (Zeki #5213)
Muse Glimmer 30BMeta · chatQueuedQueuedQueued
Muse EmberMeta · chatDoneDoneQueuedfive cells done; the last OCR route paused at 562 of 763, the rest in the Turkish phase (Zeki #5213, #5228)

Chat models run on logic's server on bariton GPU 1 after its Logic & Math runs, one model at a time (logic #5080, #5091, #5214); since 14 November logic's queue runs Reading's arms on the same server right after its own (logic #6929). English came first for every model (the project lead via Zeki #5213); for the small chat models Ümit asked for the Turkish runs too (#6224, #6527) and for a thinking-off and a thinking-on setup of each model that has the switch (#6924), each setup a row of its own.

Page images: chat models that read images take the same conditions through the chat route on their own server, one question per request with the plain prompt (accepted by Zeki #5157); – means the model reads no images.

Muse Glimmer 30B, d1-Omni-2B and d1-3B are being set up (logic #5089); their runs follow the brief's phases (Ümit #5067, Zeki #5090).

All setups: the full ranking

2,198 questions

Swipe for more columns →

All models

Accuracy on all 2,198 questions, with its 95% interval
ModelAccuracy (%)Reading (%)Grammar (%)Median latency per request (s)$ per 1,000 questions
Gemini 4 Carbon high · chat98.597.8 to 98.997.698.8217.731.309
Claude Fable 5.2 max · chat98.197.5 to 98.697.798.381.153.381
Gemini 3.8 Flash high · chat98.197.4 to 98.697.198.553.72.419
Gemini 4 Hydrogen high · chat98.197.4 to 98.697.398.431.74.806
GPT-6 Astra high · chat98.097.3 to 98.597.198.321.76.066
GPT-6.1 Sol high · chat97.897.1 to 98.396.598.324.71.622
GPT-6 Astra low · chat97.696.9 to 98.297.097.913.0*5.378
GPT-6 Astra medium · chat97.596.8 to 98.197.097.715.35.447
GPT-6 Astra low · chat · 1 per request97.496.7 to 98.097.397.56.4126.738
Gemini 3.8 Flash low · chat · 1 per request97.196.3 to 97.896.297.52.720.963
GPT-6.1 Sol low · chat · 1 per request96.795.9 to 97.496.097.06.615.569
GPT-6.1 Sol low · chat96.795.8 to 97.496.296.913.31.130
Gemini 4 Argon low · chat96.495.5 to 97.197.096.15.461.786
Claude Fable 5.1 high · chat96.195.2 to 96.995.696.430.317.042
Claude Opus 5.5 high · chat95.094.0 to 95.895.694.722.16.509
GPT-6.1 Terra high · chat94.393.2 to 95.296.293.57.100.515
Claude Opus 5.5 low · chat · 1 per request93.392.2 to 94.395.792.33.7111.610
Gemini 3.8 Flash low · chat93.392.1 to 94.295.192.52.190.269
Grok 4.7 high · chat92.791.6 to 93.795.091.8401.914.292
DeepSeek-V4.1-Flash reasoning on · chat · first release92.090.8 to 93.193.491.4365.11.299
DeepSeek-V4.1-Flash reasoning on · chat · 101791.890.5 to 92.894.190.8486.91.278
Qwen4-Max high · chat91.790.5 to 92.894.790.591.52.533
GPT-6.1 Terra low · chat90.989.6 to 92.095.988.74.240.426
Claude Opus 5.5 low · chat90.489.1 to 91.694.888.613.4*4.931
Claude Sonnet 5.5 high · chat · 1 per request89.287.8 to 90.494.287.04.647.748
Muse Ember max · chat85.684.1 to 87.090.283.730.90.650
Claude Sonnet 5.5 high · chat85.083.5 to 86.585.285.020.5*3.654
Kimi K3.1 Pro low · chat82.881.1 to 84.393.878.144.92.048
Qwen3.5-397B-A17B high · chat82.781.1 to 84.291.079.2251.32.411
Copernicus-31B81.880.1 to 83.493.177.00.1840.134
KnowLine-31B-Gen1.581.880.1 to 83.492.877.11.05No list price
KnowLine-31B-Gen281.279.5 to 82.893.476.00.1680.120
Turion Jev 31B80.178.4 to 81.792.774.80.405No list price
Inkling-Small high · chat · Jul 202680.078.3 to 81.687.276.997.71.137
GLM-5.3-Flash reasoning on · chat78.877.0 to 80.589.374.3102.60.543
REAX 1.377.976.2 to 79.692.471.80.2500.185
Karpo Decision 31B77.876.0 to 79.593.471.10.1590.105
Gemini Decision 1 2026111177.575.7 to 79.292.171.30.2270.037
d1-ultra77.175.3 to 78.892.170.70.2230.014
Logits Sigma-1 Duo75.073.2 to 76.891.967.80.2500.181
Qwen3.8-27B low · chat74.772.8 to 76.489.368.4648.43.095
GLM-5.3 low · chat74.572.7 to 76.387.768.917.10.814
Sieve 31B v374.272.3 to 75.991.866.71.00No list price
Gemma 4 31B reasoning off · chat73.171.2 to 74.990.965.63.970.048
Kimi K3.1 Pro no reasoning · chat72.770.8 to 74.592.564.35.750.853
MiMo-V2.6-Flash high · chat72.170.2 to 73.986.666.021.42.219
GLiDE 27B71.169.2 to 73.090.962.70.542No list price
Perplexity Decider 27B v1.2 API · with web68.766.8 to 70.688.960.20.482≤ 1.034
GLiDE v1.268.566.5 to 70.489.659.50.4130.014
Perplexity Decider 27B v1.368.466.5 to 70.389.059.70.1100.081
GPT-6 Luna low · chat68.366.3 to 70.287.360.27.710.048
d1.2-ultra68.166.1 to 70.089.858.90.2200.014
GLiDE API67.965.9 to 69.887.859.42.090.160
d367.865.8 to 69.789.258.70.0920.070
d3-ultra67.865.8 to 69.789.858.40.3330.216
Drex Max67.765.8 to 69.789.658.40.0820.023
Drex Max v1.167.465.4 to 69.388.658.40.3130.023
Mammoth-167.165.1 to 69.090.457.20.1690.040
Perplexity Decider 27B v1.2 DGX Spark67.165.1 to 69.089.257.70.5800.008
Perplexity Decider 27B v1.166.964.9 to 68.889.057.50.2330.008
Gemma 4 31B reasoning off · chat · 1 per request66.864.8 to 68.789.057.30.2980.068
MiniMax-M3.1-Flash high · chat66.564.5 to 68.482.959.5112.60.814
Perplexity Decider 27B v1.2 API66.364.3 to 68.389.056.70.2540.008
DeepSeek-V4.1-Flash reasoning off · chat · 101766.064.0 to 68.088.356.55.080.098
DeepSeek-V4.1-Flash reasoning off · chat · first release65.963.9 to 67.987.856.64.360.098
SubQ Snap Preview65.263.2 to 67.287.755.70.3890.019
SubQ Snap 1.163.961.8 to 65.987.054.00.3780.015
GPT-6 Luna Decisions API63.661.6 to 65.685.554.30.1910.040
Qwen3.8 Flash no reasoning · chat63.161.0 to 65.187.252.810.00.047
KnowLine-12B-Gen162.560.4 to 64.586.052.50.0710.052
d1.261.759.7 to 63.786.051.40.2130.007
d1.161.559.5 to 63.584.651.70.1950.007
Claude Sonnet 5.5 low · chat61.459.3 to 63.482.952.23.87*1.896
celeris-1-decision60.558.4 to 62.585.249.90.4010.015
Perplexity Decider 27B v160.458.3 to 62.486.049.50.2160.016
OneJev-27B59.657.5 to 61.685.248.70.1250.099
Jev 1.1459.257.1 to 61.285.148.20.2110.029
Qwen3.5-4B reasoning on · chat58.956.8 to 61.079.150.3140.32.998
Mercury Decide58.356.2 to 60.384.547.10.3130.008
GLM-5.3-Flash reasoning off · chat57.855.7 to 59.882.247.41.070.047
Gemma 4 E4B default reasoning · chat57.655.5 to 59.677.649.013.40.307
Jev 1.1357.555.4 to 59.683.746.40.2430.029
Clef57.355.2 to 59.385.145.50.2050.110
Claude Haiku 5.5 max · chat57.155.0 to 59.186.944.4417.02.965
Jeffan-9B-121157.054.9 to 59.182.946.00.0640.044
d157.054.9 to 59.182.846.00.3050.007
Drex 2.055.753.6 to 57.881.444.80.3250.019
Cygnet55.753.6 to 57.882.844.20.0360.039
Open-Jev-27B-v1.155.753.6 to 57.882.944.10.3860.271
Winnow-12B55.553.4 to 57.582.044.20.0590.041
KnowLine-9B-Gen154.552.5 to 56.679.943.80.0570.041
Cerno 9B v1.254.552.4 to 56.579.743.70.0510.036
Qwen3.8-27B vote share · reasoning off · chat54.352.2 to 56.484.041.71.620.825
Qwen3.8-27B stated confidence · reasoning off · chat54.252.1 to 56.383.441.80.4330.234
KnowLine-9B-Gen253.351.2 to 55.479.642.10.0480.035
Adjudex 9B v553.151.0 to 55.280.841.40.0380.026
Elephant-152.450.3 to 54.480.540.40.0740.014
GLiDE 9B52.050.0 to 54.180.240.10.207No list price
Qwen3.8-27B yes/no scoring51.949.8 to 54.080.539.80.2800.704
Dexy-Decision-9B-V251.649.5 to 53.779.639.70.0310.023
StartLux-Decision-v1.1-9B-Instant51.249.1 to 53.378.839.50.154No list price
Perplexity Decider Flash v1.349.747.6 to 51.874.439.20.0330.025
vera-medium-omni49.147.0 to 51.275.537.90.0290.021
KnowLine 4B Gen5.548.546.4 to 50.674.837.30.099No list price
StartLux-Decision-v1.1-4B-Instant47.945.8 to 50.074.136.70.074No list price
Perplexity Decider Flash v1.247.845.7 to 49.972.737.20.1320.004
Valen-Decision-1.5-9B47.044.9 to 49.075.035.00.0590.040
Verdin-4B-v1.146.844.7 to 48.971.236.40.0490.034
Chinese Decision Pointer 4B46.744.6 to 48.870.936.40.0350.025
Microsoft-Decision-146.544.4 to 48.573.535.00.2860.014
Qwen3.5-4B reasoning off · chat46.244.1 to 48.368.436.732.20.894
Clef-Flash45.743.6 to 47.873.034.10.0750.017
Qwen3-14B reasoning off · chat45.743.6 to 47.872.134.52.850.032
Erk-14B reasoning off · chat45.042.9 to 47.172.333.42.390.106
Metask-Jev-4B40.838.7 to 42.862.831.40.0900.063
KnowLine-2B-Gen136.834.8 to 38.854.329.40.062No list price
GLiDE 2B35.333.3 to 37.351.228.50.034No list price
StartLux-Decision-v1.1-2B-Instant34.332.3 to 36.352.926.30.040No list price
LFM2.5-2.6B default reasoning · chat33.931.9 to 35.945.429.051.22.761
LFM3-1.7B reasoning on · chat32.430.5 to 34.444.227.424.21.277
d1-2B29.427.5 to 31.340.924.50.0190.013
GLiDE 0.8B29.327.4 to 31.242.423.70.022No list price
OneJev1.6-0.8B29.127.3 to 31.139.624.60.017No list price
StartLux-Decision-v1.1-0.8B-Instant28.626.7 to 30.535.125.80.028No list price
LFM3-1.7B reasoning off · chat28.326.5 to 30.337.224.64.900.646
Strands Decider 2B27.525.7 to 29.436.923.50.0260.020
KnowLine-0.8B-Gen127.125.3 to 29.038.322.40.033No list price
Decider 127.125.3 to 29.034.623.90.3820.015
Qwen3.5-2B reasoning off · chat25.423.6 to 27.234.321.68.840.268
GLiClass v4 Edge23.321.6 to 25.225.922.20.025No list price
GLiDE 0.4B22.220.6 to 24.024.521.30.016No list price
LFM2.5-8B-A1B default reasoning · chat21.019.3 to 22.725.619.039.21.050
GLiNER2.5-multi-Decide20.218.6 to 21.921.019.80.0090.006
Laya19.017.4 to 20.720.918.20.0110.009
Laya Multilingual17.916.3 to 19.517.118.20.0090.008
GLiNER2.5 Multi17.515.9 to 19.117.217.60.0320.006
MiniCPM5-2B reasoning off · chat5.64.7 to 6.66.65.2172.05.464

Results 94aef0c1, updated 16 November 2026; answer key version 2.

The cost of every setup

Every setup, decision and chat models together · cheapest first, with accuracy, latency, the confidence cutoff and hallucination beside the cost · lower cost is better

Loading the table…

Accuracy against cost or latency

Up: accuracy, higher is better · across: cost or latency, lower is better

Loading charts…

2,198 questions · 115 models · 137 setups · updated 16 November 2026 · More about this page

Accuracy

The best chat setup gets 9.8 in 10 questions right; the best decision model 8.2 in 10

Share of the questions answered right · higher is better; the line is chance (20%)

Loading charts…

Non-hallucination

The decision model that chooses “I don't know” most does so on 95 in 100 of its answers that were not right; the chat model that does so most, on 76

Holding back by confidence

The answers each setup gave at 99% confidence or more, in its Text tab run, the bar their share of the 2,198 questions; how many of them were wrong; and its ranking score (AUROC: how often a right answer has higher confidence than a wrong one, 0.5 chance, 1 perfect). No wrong answer at 99% first, by answers kept, then by the share wrong among the kept answers; a setup with no answers at 99% last · more kept and fewer wrong is better

Decision models

Confidence: option probabilities.

  1. Karpo Decision 31B50500.879
  2. Perplexity Decider 27B v1.2 DGX Spark36000.837
  3. Perplexity Decider 27B v1.2 API32700.843
  4. Gemini Decision 1 2026111131800.862
  5. Mammoth-128500.833
  6. Perplexity Decider 27B v1.327600.825
  7. d325000.835
  8. GLiDE 27B24700.843
  9. GLiDE v1.224000.839
  10. REAX 1.324000.868
  11. Drex Max v1.122000.834
  12. Elephant-115500.788
  13. KnowLine-31B-Gen1.514300.867
  14. GLiDE API12100.810
  15. Copernicus-31B10700.876
  16. d1-ultra9000.850
  17. KnowLine-31B-Gen28100.867
  18. KnowLine-9B-Gen16200.785
  19. Logits Sigma-1 Duo4200.855
  20. Turion Jev 31B4200.872
  21. KnowLine 4B Gen5.53900.751
  22. d1.23300.807
  23. Perplexity Decider Flash v1.32900.742
  24. GLiDE 9B2700.789
  25. Verdin-4B-v1.12700.753
  26. StartLux-Decision-v1.1-9B-Instant2000.783
  27. GLiDE 2B1900.640
  28. d1.2-ultra1100.825
  29. StartLux-Decision-v1.1-4B-Instant900.763
  30. Valen-Decision-1.5-9B700.767
  31. vera-medium-omni400.746
  32. Cygnet200.802
  33. OneJev-27B200.818
  34. Drex Max44310.826
  35. d3-ultra38410.833
  36. Perplexity Decider 27B v1.2 API · with web35110.826
  37. Sieve 31B v332910.873
  38. SubQ Snap 1.129910.836
  39. Winnow-12B16310.789
  40. Dexy-Decision-9B-V213810.773
  41. Adjudex 9B v513220.785
  42. Chinese Decision Pointer 4B5110.729
  43. GLiDE 0.8B1810.620
  44. Laya Multilingual110.454
  45. Cerno 9B v1.2No answers at this threshold––0.777
  46. ClefNo answers at this threshold––0.785
  47. Clef-FlashNo answers at this threshold––0.740
  48. d1-2BNo answers at this threshold––0.637
  49. GLiClass v4 EdgeNo answers at this threshold––0.558
  50. GLiDE 0.4BNo answers at this threshold––0.516
  51. GLiNER2.5-multi-DecideNo answers at this threshold––0.537
  52. Jeffan-9B-1211No answers at this threshold––0.798
  53. KnowLine-0.8B-Gen1No answers at this threshold––0.634
  54. KnowLine-12B-Gen1No answers at this threshold––0.810
  55. KnowLine-9B-Gen2No answers at this threshold––0.784
  56. LayaNo answers at this threshold––0.479
  57. Metask-Jev-4BNo answers at this threshold––0.725
  58. OneJev1.6-0.8BNo answers at this threshold––0.605
  59. Open-Jev-27B-v1.1No answers at this threshold––0.823
  60. Perplexity Decider Flash v1.2No answers at this threshold––0.757
  61. StartLux-Decision-v1.1-0.8B-InstantNo answers at this threshold––0.606
  62. StartLux-Decision-v1.1-2B-InstantNo answers at this threshold––0.663
  63. Strands Decider 2BNo answers at this threshold––0.647

Karpo Perplexity Gemini Mammoth vLLM-SR Fastino REAX Drex Elephant KnowLine Copernicus Liquid Logits Turion Jev Verdin StartLux Valen vera Cygnet OneJev Sieve SubQ Winnow Dexy Adjudex Chinese Decision Pointer Laya Cerno Clef GLiClass Jeffan Metask Open-Jev Strands

Chat models

Confidence: the confidence the model states for 38; option probabilities for 8; agreement across ten answers for 1.

  1. Inkling-Small high · chat · Jul 20263400.778
  2. GLM-5.3-Flash reasoning on · chat900.760
  3. GLM-5.3-Flash reasoning off · chat400.733
  4. Qwen4-Max high · chat300.857
  5. Claude Opus 5.5 low · chat · 1 per request100.882
  6. GLM-5.3 low · chat100.739
  7. Muse Ember max · chat100.800
  8. Gemini 4 Carbon high · chat1,79750.892
  9. GPT-6 Astra low · chat1,72550.918
  10. Gemini 4 Hydrogen high · chat1,43750.890
  11. GPT-6 Astra high · chat1,79370.890
  12. Gemini 3.8 Flash low · chat · 1 per request1,36560.842
  13. GPT-6 Astra medium · chat1,74980.897
  14. Gemini 3.8 Flash high · chat1,845100.833
  15. GPT-6.1 Sol high · chat1,826100.879
  16. GPT-6.1 Sol low · chat · 1 per request1,741140.859
  17. GPT-6.1 Sol low · chat1,610130.859
  18. GPT-6 Astra low · chat · 1 per request1,839150.861
  19. Gemini 4 Argon low · chat1,842230.855
  20. GPT-6.1 Terra high · chat1,363190.812
  21. Gemini 3.8 Flash low · chat976150.766
  22. GPT-6.1 Terra low · chat1,306270.820
  23. DeepSeek-V4.1-Flash reasoning off · chat · 1017option probabilities696160.828
  24. Erk-14B reasoning off · chatoption probabilities10150.754
  25. MiMo-V2.6-Flash high · chat1810.709
  26. DeepSeek-V4.1-Flash reasoning on · chat · 1017option probabilities2,1641490.614
  27. Kimi K3.1 Pro low · chat1,4361140.750
  28. Kimi K3.1 Pro no reasoning · chat442360.728
  29. Qwen3.5-397B-A17B high · chat7880.744
  30. Qwen3.8-27B stated confidence · reasoning off · chat810.515
  31. MiniMax-M3.1-Flash high · chat375540.692
  32. Qwen3.8-27B vote share · reasoning off · chatagreement across ten answers8751290.777
  33. Qwen3.8 Flash no reasoning · chat5590.660
  34. GPT-6 Luna low · chat8301370.685
  35. Gemma 4 31B reasoning off · chatoption probabilities1,7823320.833
  36. Qwen3.8-27B low · chatoption probabilities2,1455060.636
  37. Gemma 4 31B reasoning off · chat · 1 per requestoption probabilities1,8264870.820
  38. Qwen3-14B reasoning off · chatoption probabilities9753420.734
  39. Claude Fable 5.1 high · chatNo answers at this threshold––0.858
  40. Claude Haiku 5.5 max · chatNo answers at this threshold––0.819
  41. Claude Opus 5.5 high · chatNo answers at this threshold––0.837
  42. Claude Opus 5.5 low · chatNo answers at this threshold––0.840
  43. Claude Sonnet 5.5 high · chatNo answers at this threshold––0.781
  44. Claude Sonnet 5.5 high · chat · 1 per requestNo answers at this threshold––0.838
  45. Claude Sonnet 5.5 low · chatNo answers at this threshold––0.786
  46. GLiNER2.5 Multioption probabilities · No answers at this threshold––0.451
  47. Grok 4.7 high · chatNo answers at this threshold––0.849

Inkling GLM Qwen Claude Muse Gemini GPT DeepSeek Erk MiMo Kimi MiniMax Gemma Fastino Grok Confidence not measured (a plain answer line, no confidence): LFM3-1.7B · reasoning off · chat, LFM3-1.7B · reasoning on · chat, LFM2.5-8B-A1B · default reasoning · chat, Gemma 4 E4B · default reasoning · chat, MiniCPM5-2B · reasoning off · chat, LFM2.5-2.6B · default reasoning · chat, Qwen3.5-2B · reasoning off · chat, Qwen3.5-4B · reasoning off · chat and Qwen3.5-4B · reasoning on · chat.

Holding back with an “I don't know” option

“I don't know” among the answers that were not right, with it offered as a sixth option on the 2,198 questions; beside it the share answered (right + wrong of 2,198) and the error rate among the answers (wrong of right + wrong) · higher non-hallucination is better, read with the share answered; a model that always chose the option would reach 100% with no right answers

Decision models

Each of the 63 decision models' first run with it.

  1. GLiDE 0.4B95.3%5.8%78.9%
  2. d1.2-ultra87.3%54.7%12.1%
  3. d1.279.4%55.7%20.7%
  4. d1-ultra75.1%71.9%13.0%
  5. Mammoth-168.7%68.6%20.8%
  6. d1-2B65.5%45.0%64.4%
  7. GLiDE API59.4%76.3%21.2%
  8. SubQ Snap 1.147.1%80.1%27.8%
  9. OneJev1.6-0.8B43.4%65.7%68.2%
  10. GLiDE 27B39.8%86.8%23.1%
  11. GLiDE v1.239.5%85.4%26.2%
  12. GLiDE 2B38.5%72.2%61.7%
  13. Drex Max v1.138.2%85.4%27.5%
  14. Elephant-134.8%81.6%42.3%
  15. GLiDE 0.8B29.2%78.2%67.7%
  16. Laya28.3%75.6%81.5%
  17. GLiDE 9B24.1%87.4%45.4%
  18. GLiClass v4 Edge22.3%81.9%77.0%
  19. StartLux-Decision-v1.1-2B-Instant21.5%84.8%65.0%
  20. Laya Multilingual14.8%87.3%83.8%
  21. Perplexity Decider Flash v1.214.7%91.8%51.7%
  22. StartLux-Decision-v1.1-0.8B-Instant12.4%90.9%70.5%
  23. Chinese Decision Pointer 4B11.2%93.8%52.5%
  24. Strands Decider 2B9.2%93.2%72.4%
  25. Clef-Flash9.0%95.0%53.1%
  26. vera-medium-omni8.9%95.2%51.4%
  27. Clef7.4%96.8%42.0%
  28. Perplexity Decider Flash v1.36.5%96.6%50.4%
  29. KnowLine-0.8B-Gen16.4%95.3%72.3%
  30. GLiNER2.5-multi-Decide6.4%94.8%79.9%
  31. KnowLine 4B Gen5.54.7%97.5%52.3%
  32. Jeffan-9B-12114.3%98.1%42.5%
  33. Winnow-12B4.3%98.0%44.8%
  34. Dexy-Decision-9B-V23.4%98.3%48.4%
  35. Turion Jev 31B3.3%99.3%20.0%
  36. Cygnet3.2%98.5%44.2%
  37. Perplexity Decider 27B v1.2 API · with web3.2%99.0%32.3%
  38. KnowLine-9B-Gen22.6%98.8%47.4%
  39. StartLux-Decision-v1.1-9B-Instant2.2%98.9%49.2%
  40. Gemini Decision 1 202611111.9%99.5%23.1%
  41. StartLux-Decision-v1.1-4B-Instant1.8%99.0%52.7%
  42. Open-Jev-27B-v1.11.4%99.4%44.2%
  43. d3-ultra1.4%99.5%32.2%
  44. Copernicus-31B1.2%99.8%18.3%
  45. Adjudex 9B v50.8%99.6%46.6%
  46. Metask-Jev-4B0.8%99.5%60.1%
  47. Perplexity Decider 27B v1.30.7%99.8%32.4%
  48. Karpo Decision 31B0.6%99.9%21.6%
  49. KnowLine-9B-Gen10.5%99.8%45.4%
  50. Verdin-4B-v1.10.4%99.8%53.3%
  51. REAX 1.30.4%99.9%21.6%
  52. Perplexity Decider 27B v1.2 API0.4%99.9%33.1%
  53. Logits Sigma-1 Duo0.4%99.9%24.7%
  54. Drex Max0.3%99.9%32.4%
  55. Perplexity Decider 27B v1.2 DGX Spark0.3%99.9%32.7%
  56. KnowLine-31B-Gen1.50.2%100.0%18.7%
  57. Cerno 9B v1.20.2%99.9%45.4%
  58. KnowLine-12B-Gen10.1%100.0%36.6%
  59. OneJev-27B0.1%100.0%40.6%
  60. Valen-Decision-1.5-9B0.1%100.0%52.7%
  61. KnowLine-31B-Gen20.0%100.0%18.6%
  62. Sieve 31B v30.0%100.0%25.2%
  63. d30.0%100.0%32.5%

Chose “I don't know” on fewer than 1% of the 2,198 questions: Turion Jev 31B, 15 times; Gemini Decision 1 · 20261111, 10 times; StartLux-Decision-v1.1-4B-Instant, 21 times; Open-Jev-27B-v1.1, 14 times; d3-ultra, 10 times; Copernicus-31B, 5 times; Adjudex 9B v5, 8 times; Metask-Jev-4B, 10 times; Perplexity Decider 27B v1.3, 5 times; Karpo Decision 31B, 3 times; KnowLine-9B-Gen1, 5 times; Verdin-4B-v1.1, 5 times; REAX 1.3, 2 times; Perplexity Decider 27B v1.2 · API, 3 times; Logits Sigma-1 Duo, 2 times; Drex Max, 2 times; Perplexity Decider 27B v1.2 · DGX Spark, 2 times; KnowLine-31B-Gen1.5, 1 time; Cerno 9B v1.2, 2 times; KnowLine-12B-Gen1, 1 time; OneJev-27B, 1 time; Valen-Decision-1.5-9B, 1 time; KnowLine-31B-Gen2, 0 times; Sieve 31B v3, 0 times; d3, 0 times. Their rate here is low (3.3% or less) because they almost never chose the option; the part Holding back by confidence shows how their confidence holds back wrong answers.

Fastino Liquid Mammoth SubQ OneJev Drex Elephant Laya GLiClass StartLux Perplexity Chinese Decision Pointer Strands Clef vera KnowLine Jeffan Winnow Dexy Turion Jev Cygnet Gemini Open-Jev vLLM-SR Copernicus Adjudex Metask Karpo Verdin REAX Logits Cerno Valen Sieve

Chat models

Each of the 56 chat model setups' first run with it.

  1. LFM3-1.7B reasoning off · chat76.2%32.7%62.4%
  2. LFM3-1.7B reasoning on · chat67.0%45.5%58.8%
  3. LFM2.5-8B-A1B default reasoning · chat56.0%42.4%74.0%
  4. Grok 4.7 high · chat29.9%97.5%6.1%
  5. Gemma 4 E4B default reasoning · chat29.7%86.0%38.4%
  6. MiniCPM5-2B reasoning off · chat25.4%20.1%75.3%
  7. LFM2.5-2.6B default reasoning · chat24.7%81.6%63.2%
  8. Claude Haiku 5.5 max · chat22.0%67.7%9.3%
  9. Qwen3.5-2B reasoning off · chat20.8%75.6%72.3%
  10. Gemini 4 Carbon high · chat17.9%99.7%1.5%
  11. Gemini 4 Hydrogen high · chat17.4%99.1%1.7%
  12. GPT-6.1 Sol high · chat15.4%99.6%2.0%
  13. GPT-6 Astra high · chat14.3%99.7%1.9%
  14. GPT-6 Astra medium · chat12.7%99.7%2.2%
  15. GPT-6 Astra low · chat · 1 per request11.9%99.7%2.4%
  16. GPT-6.1 Sol low · chat · 1 per request11.6%99.6%2.8%
  17. GPT-6 Astra low · chat10.5%99.7%2.3%
  18. GPT-6.1 Sol low · chat9.2%99.7%2.7%
  19. GPT-6.1 Terra high · chat7.3%99.6%5.3%
  20. Gemini 4 Argon low · chat7.0%99.7%3.6%
  21. Qwen3.5-4B reasoning off · chat6.9%95.4%54.0%
  22. GLiNER2.5 Multi6.3%94.7%82.6%
  23. Gemini 3.8 Flash low · chat · 1 per request6.2%99.8%2.7%
  24. Claude Fable 5.1 high · chat6.0%99.8%3.6%
  25. Qwen3.8-27B low · chat5.9%97.9%22.9%
  26. MiniMax-M3.1-Flash high · chat5.5%97.6%32.8%
  27. Qwen3.8 Flash no reasoning · chat4.7%98.3%35.4%
  28. Gemini 3.8 Flash high · chat4.5%99.9%1.9%
  29. Gemma 4 31B reasoning off · chat4.1%98.9%26.1%
  30. MiMo-V2.6-Flash high · chat3.5%96.7%26.2%
  31. Qwen3.5-4B reasoning on · chat3.3%98.4%40.7%
  32. Inkling-Small high · chat · Jul 20263.1%99.3%21.4%
  33. GPT-6.1 Terra low · chat2.9%99.7%9.1%
  34. Claude Opus 5.5 high · chat2.7%99.9%4.9%
  35. Muse Ember max · chat2.1%97.5%13.1%
  36. Kimi K3.1 Pro low · chat2.0%99.7%15.7%
  37. Qwen3-14B reasoning off · chat1.9%99.0%53.8%
  38. Gemma 4 31B reasoning off · chat · 1 per request1.9%99.4%33.5%
  39. Gemini 3.8 Flash low · chat1.9%99.9%7.2%
  40. Claude Opus 5.5 low · chat1.8%99.9%7.7%
  41. Qwen3.5-397B-A17B high · chat1.5%99.7%17.8%
  42. Qwen3.8-27B vote share · reasoning off · chat1.3%99.4%45.6%
  43. DeepSeek-V4.1-Flash reasoning on · chat · 10171.3%99.9%7.0%
  44. GLM-5.3 low · chat1.1%99.0%23.9%
  45. GPT-6 Luna low · chat1.0%99.7%33.2%
  46. Claude Sonnet 5.5 high · chat · 1 per request0.8%99.9%11.1%
  47. Claude Opus 5.5 low · chat · 1 per request0.7%100.0%6.2%
  48. Qwen4-Max high · chat0.7%99.2%6.4%
  49. Claude Sonnet 5.5 high · chat0.7%99.9%12.8%
  50. Claude Sonnet 5.5 low · chat0.6%99.8%35.1%
  51. Qwen3.8-27B stated confidence · reasoning off · chat0.6%99.7%45.6%
  52. GLM-5.3-Flash reasoning on · chat0.6%91.9%16.6%
  53. Kimi K3.1 Pro no reasoning · chat0.5%99.9%28.3%
  54. GLM-5.3-Flash reasoning off · chat0.3%99.9%41.0%
  55. Erk-14B reasoning off · chat0.3%99.8%55.3%
  56. DeepSeek-V4.1-Flash reasoning off · chat · 10170.0%100.0%33.2%

Liquid Grok Gemma MiniCPM Claude Qwen Gemini GPT Fastino MiniMax MiMo Inkling Muse Kimi DeepSeek GLM Erk Running: a chat model's runs with the option go English first, then Turkish, and each is added as it finishes; still running in Turkish: Claude Fable 5.2 · max · chat.

Read non-hallucination with the share answered; the Hallucination tab has the full table. More about this finding

Cost

The median decision setup costs $0.025$24.63 per 1,000 questionsper 1M questions, the median chat setup $1.277$1,277

Estimated dollars per 1,000 questionsper 1M questions · lower is better

Loading charts…

Latency

At 1 question per request, the median decision setup answers in 0.15 s, the median chat setup in 6.5 s

Median time per request · lower is better

Loading charts…

Reading vs grammar

122 of 137 setups score higher on reading than on grammar

Loading charts…

Calibration

d1.1 has the smallest calibration gap of the 77 decision models, as shipped: 2.0 points

How far a model’s confidence is from how often it is right, as shipped · lower is better

Loading charts…