Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Provisional: Drex Max v1.1, whose maker publishes only a promotional price. Left out here: the setups only this site ran (Erk-14B, GLiNER2.5 Multi, GLiNER2.5-multi-Decide, Laya, Laya Multilingual and Qwen3-14B); they show under Class: All. On this page a class narrows the rows of the tables and bar charts; the sentences, counts, tests and line plots still cover every model of the type in view.
English reading
The reading questions in English
Every setup in English
| Model | English accuracyfirst English run, or the mean of a setup's three seed runs; higher is better | Decision Score (AA formula)first English run; higher is better | Turkish accuracyText tab run, same 547; for context | English non-hallucinationof answers not right, % “I don't know” · higher is better; answered and error rate under it | English cost$ per 1,000 questionsper 1M questions, first English run; lower is better | English latencymedian seconds per request, first English run; lower is better |
|---|---|---|---|---|---|---|
| Claude Fable 5.2 max · chat Closed · Flagship | 98.0%536 of 547 right | not applicable | 97.8%535 of 547 right | 28.6%4 “I don't know” of 14 not rightanswered 99.3%error rate 1.8% | 52.94252,942subscription, priced as the API | 74.016 questions per request, 1 in flight |
| Gemini 4 Hydrogen high · chat Closed · Workhorse | 96.7%529 of 547 right | not applicable | 97.1%531 of 547 right | 20.0%4 “I don't know” of 20 not rightanswered 99.3%error rate 2.9% | 4.9274,927subscription, priced as the API | 28.416 questions per request, 1 in flight |
| GPT-6 Astra low · chat Closed · Flagship | 96.3%527 of 547 right | not applicable | 96.7%529 of 547 right | 18.2%4 “I don't know” of 22 not rightanswered 99.3%error rate 3.3% | 5.9185,918subscription, priced as the API | 11.516 questions per request, 1 in flight |
| GPT-6 Astra high · chat Closed · Flagship | 96.2%526 of 547 right | not applicable | 96.9%530 of 547 right | 19.0%4 “I don't know” of 21 not rightanswered 99.3%error rate 3.1% | 6.3346,334subscription, priced as the API | 15.616 questions per request, 1 in flight |
| GPT-6.1 Terra high · chat Closed · Workhorse | 96.2%526 of 547 right | not applicable | 96.3%527 of 547 right | 16.7%4 “I don't know” of 24 not rightanswered 99.3%error rate 3.7% | 0.509509subscription, priced as the API | 7.9416 questions per request, 1 in flight |
| GPT-6.1 Terra low · chat Closed · Workhorse | 96.2%526 of 547 right | not applicable | 95.6%523 of 547 right | 15.4%4 “I don't know” of 26 not rightanswered 99.3%error rate 4.1% | 0.491491subscription, priced as the API | 7.2916 questions per request, 1 in flight |
| Claude Fable 5.1 high · chat Closed · Flagship | 96.0%525 of 547 right | not applicable | 95.4%522 of 547 right | 7.4%2 “I don't know” of 27 not rightanswered 99.6%error rate 4.6% | 11.53111,531subscription, priced as the API | 15.616 questions per request, 1 in flight |
| Gemini 3.8 Flash high · chat Closed · Workhorse | 96.0%525 of 547 right | not applicable | 96.9%530 of 547 right | 10.0%2 “I don't know” of 20 not rightanswered 99.6%error rate 3.3% | 3.6953,695at list prices | 104.616 questions per request, 4 in flight |
| Gemini 4 Argon low · chat Closed · Flagship | 96.0%525 of 547 right | not applicable | 96.9%530 of 547 right | 18.5%5 “I don't know” of 27 not rightanswered 99.1%error rate 4.1% | 1.9001,900at list prices | 6.0316 questions per request, 1 in flight |
| GPT-6 Astra medium · chat Closed · Flagship | 96.0%525 of 547 right | not applicable | 96.7%529 of 547 right | 16.7%4 “I don't know” of 24 not rightanswered 99.3%error rate 3.7% | 4.6964,696subscription, priced as the API | 10.716 questions per request, 1 in flight |
| GPT-6.1 Sol low · chat Closed · Workhorse | 95.8%524 of 547 right | not applicable | 95.8%524 of 547 right | 14.8%4 “I don't know” of 27 not rightanswered 99.3%error rate 4.2% | 1.2191,219subscription, priced as the API | 10.316 questions per request, 1 in flight |
| GPT-6 Astra low · chat · 1 per request Closed · Flagship | 95.6%523 of 547 right | not applicable | 97.1%531 of 547 right | 14.8%4 “I don't know” of 27 not rightanswered 99.3%error rate 4.2% | 28.65328,653subscription, priced as the API | 6.431 question per request, 1 in flight |
| GPT-6.1 Sol high · chat Closed · Workhorse | 95.6%523 of 547 right | not applicable | 96.2%526 of 547 right | 16.7%4 “I don't know” of 24 not rightanswered 99.3%error rate 3.7% | 1.4271,427subscription, priced as the API | 16.916 questions per request, 1 in flight |
| Gemini 3.8 Flash low · chat · 1 per request Closed · Workhorse | 95.2%521 of 547 right | not applicable | 96.2%526 of 547 right | 7.7%2 “I don't know” of 26 not rightanswered 99.6%error rate 4.4% | 0.858858at list prices | 2.851 question per request, 1 in flight |
| GPT-6.1 Sol low · chat · 1 per request Closed · Workhorse | 95.2%521 of 547 right | not applicable | 95.6%523 of 547 right | 12.9%4 “I don't know” of 31 not rightanswered 99.3%error rate 5.0% | 6.1136,113subscription, priced as the API | 5.591 question per request, 1 in flight |
| Claude Opus 5.5 low · chat · 1 per request Closed · Flagship | 94.7%518 of 547 right | not applicable | 95.4%522 of 547 right | 0.0%0 “I don't know” of 31 not rightanswered 100.0%error rate 5.7% | 10.09410,094subscription, priced as the API | 3.501 question per request, 1 in flight |
| Claude Opus 5.5 high · chat Closed · Flagship | 94.5%517 of 547 right | not applicable | 95.6%523 of 547 right | 6.5%2 “I don't know” of 31 not rightanswered 99.6%error rate 5.3% | 4.5194,519subscription, priced as the API | 11.316 questions per request, 1 in flight |
| Qwen4-Max high · chat Closed · Workhorse | 94.5%517 of 547 right | not applicable | 94.7%518 of 547 right | 0.0%0 “I don't know” of 29 not rightanswered 100.0%error rate 5.3% | 1.1191,119at list prices | 29.516 questions per request, 4 in flight |
| Gemini 3.8 Flash low · chat Closed · Workhorse | 94.3%516 of 547 right | not applicable | 95.2%521 of 547 right | 3.4%1 “I don't know” of 29 not rightanswered 99.8%error rate 5.1% | 0.288288at list prices | 4.1916 questions per request, 1 in flight |
| Gemini 4 Carbon high · chat Closed · Flagship | 94.3%516 of 547 right | not applicable | 97.6%534 of 547 right | 25.0%4 “I don't know” of 16 not rightanswered 99.3%error rate 2.2% | 38.05338,053subscription, priced as the API | 287.816 questions per request, 1 in flight |
| KnowLine-31B-Gen1.5 Open · 31B | 93.6%512 of 547 right | 94.33chosen-answer Brier 0.0494 | 94.3%516 of 547 right | 0.0%0 “I don't know” of 37 not rightanswered 100.0%error rate 6.8% | No list price | 1.121 question per request, 1 in flight |
| Grok 4.7 high · chat Closed · Workhorse | 93.4%511 of 547 right | not applicable | 95.1%520 of 547 right | 27.9%12 “I don't know” of 43 not rightanswered 97.8%error rate 5.8% | 6.8516,851subscription, priced as the API | 197.216 questions per request, 4 in flight |
| Claude Opus 5.5 low · chat Closed · Flagship | 93.1%509 of 547 right | not applicable | 95.1%520 of 547 right | 4.7%2 “I don't know” of 43 not rightanswered 99.6%error rate 7.5% | 1.0871,087subscription, priced as the API | 7.6416 questions per request, 1 in flight |
| Copernicus-31B Open · 31B | 92.9%508 of 547 right | 93.66chosen-answer Brier 0.0556 | 94.5%517 of 547 right | 0.0%0 “I don't know” of 39 not rightanswered 100.0%error rate 7.1% | 0.142142from GPU time | 0.1881 question per request, 1 in flight |
| Claude Haiku 5.5 max · chat Closed · Efficient | 92.7%507 of 547 right | not applicable | 89.4%489 of 547 right | 7.8%4 “I don't know” of 51 not rightanswered 96.3%error rate 8.9% | 1.2611,261at list prices | 148.916 questions per request, 4 in flight |
| DeepSeek-V4.1-Flash reasoning on · chat · 1017 Open · 552B MoE · 16B active | 92.5%506 of 547 right | not applicable | 94.5%517 of 547 right | 7.1%3 “I don't know” of 42 not rightanswered 99.5%error rate 7.2% | 0.421421at the maker's list price | 189.416 questions per request, 8 in flight |
| REAX 1.3 Open · 57B | 92.5%506 of 547 right | 93.75chosen-answer Brier 0.0500 | 94.0%514 of 547 right | 6.7%3 “I don't know” of 45 not rightanswered 99.5%error rate 7.7% | 0.169169from GPU time | 0.2291 question per request, 1 in flight |
| Kimi K3.1 Pro low · chat Open · 2.8T MoE · 104B active | 92.3%505 of 547 right | not applicable | 95.2%521 of 547 right | 7.3%3 “I don't know” of 41 not rightanswered 99.5%error rate 7.0% | 0.984984at list prices | 16.616 questions per request, 4 in flight |
| KnowLine-31B-Gen2 Open · 31B | 92.3%505 of 547 right | 93.37chosen-answer Brier 0.0558 | 94.5%517 of 547 right | 0.0%0 “I don't know” of 40 not rightanswered 100.0%error rate 7.3% | 0.127127from GPU time | 0.1711 question per request, 1 in flight |
| Karpo Decision 31B Open · 31B | 92.1%504 of 547 right | 93.23chosen-answer Brier 0.0568 | 94.9%519 of 547 right | 0.0%0 “I don't know” of 42 not rightanswered 100.0%error rate 7.7% | 0.145145from GPU time | 0.2071 question per request, 1 in flight |
| Claude Sonnet 5.5 high · chat · 1 per request Closed · Workhorse | 91.6%501 of 547 right | not applicable | 94.3%516 of 547 right | 0.0%0 “I don't know” of 38 not rightanswered 100.0%error rate 6.9% | 5.5135,513subscription, priced as the API | 3.021 question per request, 1 in flight |
| Kimi K3.1 Pro no reasoning · chat Open · 2.8T MoE · 104B active | 91.4%500 of 547 right | not applicable | 94.0%514 of 547 right | 7.5%4 “I don't know” of 53 not rightanswered 99.3%error rate 9.0% | 0.735735at list prices | 6.3116 questions per request, 4 in flight |
| Qwen3.5-397B-A17B high · chat Open · 397B MoE · 17B active | 91.4%500 of 547 right | not applicable | 91.4%500 of 547 right | 6.2%3 “I don't know” of 48 not rightanswered 99.5%error rate 8.3% | 1.7421,742at list prices | 162.916 questions per request, 4 in flight |
| Gemini Decision 1 20261111 Closed · Efficient | 91.0%498 of 547 right | 92.31chosen-answer Brier 0.0642 | 92.9%508 of 547 right | 9.1%5 “I don't know” of 55 not rightanswered 99.1%error rate 9.2% | 0.04039.51at list prices | 0.2351 question per request, 1 in flight |
| Logits Sigma-1 Duo Open · 57B | 91.0%498 of 547 right | 92.55chosen-answer Brier 0.0595 | 93.4%511 of 547 right | 4.3%2 “I don't know” of 46 not rightanswered 99.6%error rate 8.1% | 0.191191from GPU time | 0.2631 question per request, 1 in flight |
| GLiDE 27B Open · 27B | 90.9%497 of 547 right | 92.36chosen-answer Brier 0.0613 | 91.2%499 of 547 right | 37.9%22 “I don't know” of 58 not rightanswered 96.0%error rate 6.9% | No list price | 0.5451 question per request, 1 in flight |
| d1-ultra Closed · Efficient | 90.7%496 of 547 right | 91.92chosen-answer Brier 0.0684 | 93.6%512 of 547 right | 56.7%51 “I don't know” of 90 not rightanswered 90.7%error rate 7.9% | 0.01413.98at list prices | 0.2201 question per request, 1 in flight |
| Sieve 31B v3 Open · 31B | 90.7%496 of 547 right | 92.51chosen-answer Brier 0.0565 | 93.4%511 of 547 right | 2.2%1 “I don't know” of 46 not rightanswered 99.8%error rate 8.2% | No list price | 1.061 question per request, 1 in flight |
| Turion Jev 31B Open · 31B | 90.7%496 of 547 right | 91.69chosen-answer Brier 0.0729 | 93.6%512 of 547 right | 8.5%4 “I don't know” of 47 not rightanswered 99.3%error rate 7.9% | No list price | 0.4451 question per request, 1 in flight |
| Gemma 4 31B reasoning off · chat Open · 31B | 90.5%495 of 547 right | not applicable | 92.5%506 of 547 right | 3.6%2 “I don't know” of 55 not rightanswered 99.6%error rate 9.7% | 0.05352.81at a host's list price for the same weights | 3.1916 questions per request, 1 in flight |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | 90.5%495 of 547 right | 91.39chosen-answer Brier 0.0771 | 90.7%496 of 547 right | 15.2%10 “I don't know” of 66 not rightanswered 98.2%error rate 10.4% | ≤ 1.036≤ 1,036at list prices | 0.4841 question per request, 1 in flight |
| GLiDE v1.2 Open · 27B | 90.3%494 of 547 right | 91.96chosen-answer Brier 0.0639 | 90.5%495 of 547 right | 45.5%30 “I don't know” of 66 not rightanswered 94.5%error rate 7.0% | 0.01514.55at the maker's list price | 0.1041 question per request, 1 in flight |
| d1.2-ultra Closed · Efficient | 90.3%494 of 547 right | 91.84chosen-answer Brier 0.0664 | 90.9%497 of 547 right | 89.3%109 “I don't know” of 122 not rightanswered 80.1%error rate 3.0% | 0.01413.98at list prices | 0.2211 question per request, 1 in flight |
| Mammoth-1 Open · 27B | 90.1%493 of 547 right | 91.75chosen-answer Brier 0.0663 | 92.9%508 of 547 right | 50.7%37 “I don't know” of 73 not rightanswered 93.2%error rate 7.1% | 0.04040.07at the maker's list price | 0.1891 question per request, 1 in flight |
| Gemma 4 31B reasoning off · chat · 1 per request Open · 31B | 89.9%492 of 547 right | not applicable | 90.3%494 of 547 right | 0.0%0 “I don't know” of 59 not rightanswered 100.0%error rate 10.8% | 0.07070.17at a host's list price for the same weights | 0.2991 question per request, 1 in flight |
| d3 Open · 26B | 89.8%491 of 547 right | 91.65chosen-answer Brier 0.0646 | 91.0%498 of 547 right | 1.7%1 “I don't know” of 59 not rightanswered 99.8%error rate 10.6% | 0.05555.35from GPU time | 0.0741 question per request, 1 in flight |
| Drex Max Open · 27B | 89.6%490 of 547 right | 91.50chosen-answer Brier 0.0658 | 91.4%500 of 547 right | 3.2%2 “I don't know” of 62 not rightanswered 99.6%error rate 11.0% | 0.02423.56at the maker's list price | 0.0811 question per request, 1 in flight |
| GLM-5.3-Flash reasoning on · chat Open · 320B MoE · 18B active | 89.6%490 of 547 right | not applicable | 90.1%493 of 547 right | 1.6%1 “I don't know” of 62 not rightanswered 99.8%error rate 11.2% | 0.08181.38at the maker's list price | 33.116 questions per request, 8 in flight |
| Muse Ember max · chat Closed · Efficient | 89.6%490 of 547 right | not applicable | 90.9%497 of 547 right | 5.3%3 “I don't know” of 57 not rightanswered 99.5%error rate 9.9% | 0.245245at list prices | 11.616 questions per request, 4 in flight |
| Perplexity Decider 27B v1.3 Open · 27B | 89.6%490 of 547 right | 91.22chosen-answer Brier 0.0714 | 90.7%496 of 547 right | 8.8%5 “I don't know” of 57 not rightanswered 99.1%error rate 9.6% | 0.08382.63from GPU time | 0.1151 question per request, 1 in flight |
| d3-ultra Open · 26B | 89.4%489 of 547 right | 91.18chosen-answer Brier 0.0704 | 91.4%500 of 547 right | 8.2%5 “I don't know” of 61 not rightanswered 99.1%error rate 10.3% | 0.106106from GPU time | 0.1491 question per request, 1 in flight |
| d1.2 Closed · Efficient | 89.0%487 of 547 right | 90.51chosen-answer Brier 0.0801 | 88.8%486 of 547 right | 61.5%59 “I don't know” of 96 not rightanswered 89.2%error rate 7.6% | 0.0076.66at list prices | 0.2191 question per request, 1 in flight |
| MiMo-V2.6-Flash high · chat Open · 309B MoE · 15B active | 89.0%487 of 547 right | not applicable | 86.8%475 of 547 right | 1.7%1 “I don't know” of 59 not rightanswered 99.8%error rate 10.6% | 0.298298at list prices | 6.671 question per request, 8 in flight |
| Qwen3.8-27B low · chat Open · 27B | 89.0%487 of 547 right | not applicable | 90.7%496 of 547 right | 3.6%2 “I don't know” of 55 not rightanswered 99.6%error rate 9.7% | 0.853853at the maker's list price | 71.716 questions per request, 4 in flight |
| Perplexity Decider 27B v1.2 DGX Spark Open · 27B | 88.7%485 of 547 right | 90.75chosen-answer Brier 0.0717 | 91.4%500 of 547 right | 3.0%2 “I don't know” of 66 not rightanswered 99.6%error rate 11.7% | 0.0087.85at the maker's list price | 0.1261 question per request, 1 in flight |
| Drex Max v1.1 Closed · Efficient (provisional) | 88.5%484 of 547 right | 90.81chosen-answer Brier 0.0685 | 90.1%493 of 547 right | 51.2%43 “I don't know” of 84 not rightanswered 92.1%error rate 8.1% | 0.02323.38at list prices | 0.3351 question per request, 1 in flight |
| Perplexity Decider 27B v1 Open · 27B | 88.5%484 of 547 right | 90.64chosen-answer Brier 0.0720 | 89.0%487 of 547 right | Not run | 0.01615.59at list prices | 0.2471 question per request, 1 in flight |
| Perplexity Decider 27B v1.2 API Open · 27B | 88.5%484 of 547 right | 90.64chosen-answer Brier 0.0721 | 91.0%498 of 547 right | 4.5%3 “I don't know” of 66 not rightanswered 99.5%error rate 11.6% | 0.0087.79at list prices | 0.2561 question per request, 1 in flight |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | 88.3%483 of 547 right | not applicable | 89.4%489 of 547 right | 0.0%0 “I don't know” of 60 not rightanswered 100.0%error rate 11.0% | 0.425425at list prices | 25.316 questions per request, 4 in flight |
| d1.1 Closed · Efficient | 88.3%483 of 547 right | 90.14chosen-answer Brier 0.0802 | 88.3%483 of 547 right | Not run | 0.0076.66at list prices | 0.1731 question per request, 1 in flight |
| GLM-5.3 low · chat Open · Workhorse | 87.8%480 of 547 right | not applicable | 89.2%488 of 547 right | 3.4%2 “I don't know” of 58 not rightanswered 99.6%error rate 10.3% | 0.408408at list prices | 6.8416 questions per request, 4 in flight |
| Jeffan-9B-1211 Open · 9.0B | 87.6%479 of 547 right | 89.05chosen-answer Brier 0.0946 | 86.5%473 of 547 right | 0.0%0 “I don't know” of 71 not rightanswered 100.0%error rate 13.0% | 0.04645.85from GPU time | 0.0671 question per request, 1 in flight |
| GLM-5.3-Flash reasoning off · chat Open · 320B MoE · 18B active | 87.2%477 of 547 right | not applicable | 84.3%461 of 547 right | 5.6%4 “I don't know” of 71 not rightanswered 99.3%error rate 12.3% | 0.04646.07at the maker's list price | 3.3716 questions per request, 1 in flight |
| DeepSeek-V4.1-Flash reasoning off · chat · 1017 Open · 552B MoE · 16B active | 87.0%476 of 547 right | not applicable | 89.9%492 of 547 right | 5.4%4 “I don't know” of 74 not rightanswered 99.3%error rate 12.9% | 0.07979.31at the maker's list price | 6.5916 questions per request, 1 in flight |
| Qwen3.5-4B reasoning on · chat Open · 4.0B | 87.0%476 of 547 right | not applicable | 82.8%453 of 547 right | 2.4%2 “I don't know” of 85 not rightanswered 99.6%error rate 15.2% | 1.3901,390from GPU time | 58.31 question per request, 32 in flight |
| Qwen3.8 Flash no reasoning · chat Closed · Efficient | 87.0%476 of 547 right | not applicable | 88.8%486 of 547 right | 5.6%4 “I don't know” of 71 not rightanswered 99.3%error rate 12.3% | 0.05049.69at list prices | 10.116 questions per request, 4 in flight |
| d1 Closed · Efficient | 86.8%475 of 547 right | 89.26chosen-answer Brier 0.0832 | 85.0%465 of 547 right | 2.9%2 “I don't know” of 69 not rightanswered 99.6%error rate 12.3% | 0.0076.66at list prices | 0.1801 question per request, 1 in flight |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | 86.8%475 of 547 right | 88.59chosen-answer Brier 0.0965 | 83.0%454 of 547 right | 2.6%2 “I don't know” of 76 not rightanswered 99.6%error rate 13.6% | No list price | 0.1561 question per request, 1 in flight |
| Gemma 4 12B Unified default reasoning · chat Open · 12B | 86.8%475 of 547 right | not applicable | Queuedits Text tab run in logic's GPU 1 queue (Ümit #6527, logic #6955) | 5.5%4 “I don't know” of 73 not rightanswered 98.9%error rate 12.8% | 0.154154at a host's list price for the same weights | 15.81 question per request, 32 in flight |
| Clef Open · 27B | 86.7%474 of 547 right | 89.19chosen-answer Brier 0.0828 | 88.1%482 of 547 right | 1.3%1 “I don't know” of 76 not rightanswered 99.8%error rate 13.7% | 0.111111at the maker's list price | 0.2201 question per request, 1 in flight |
| KnowLine-12B-Gen1 Open · 12B | 86.7%474 of 547 right | 88.66chosen-answer Brier 0.0933 | 87.6%479 of 547 right | 1.3%1 “I don't know” of 77 not rightanswered 99.8%error rate 13.9% | 0.05756.86from GPU time | 0.0781 question per request, 1 in flight |
| GLiDE 9B Open · 7.9B | 86.3%472 of 547 right | 88.60chosen-answer Brier 0.0909 | 84.3%461 of 547 right | 16.7%14 “I don't know” of 84 not rightanswered 97.4%error rate 13.1% | No list price | 0.2071 question per request, 1 in flight |
| GPT-6 Luna low · chat Closed · Efficient | 86.3%472 of 547 right | not applicable | 88.7%485 of 547 right | 6.8%5 “I don't know” of 74 not rightanswered 99.1%error rate 12.7% | 0.05857.70subscription, priced as the API | 7.7216 questions per request, 1 in flight |
| OneJev-27B Open · 27B | 86.3%472 of 547 right | 88.51chosen-answer Brier 0.0927 | 87.8%480 of 547 right | 4.0%3 “I don't know” of 75 not rightanswered 99.5%error rate 13.2% | 0.106106from GPU time | 0.1301 question per request, 1 in flight |
| Claude Sonnet 5.5 high · chat Closed · Workhorse | 86.1%471 of 547 right | not applicable | 87.6%479 of 547 right | 4.8%3 “I don't know” of 62 not rightanswered 99.5%error rate 10.8% | 1.9981,998subscription, priced as the API | 3.8516 questions per request, 1 in flight |
| Jev 1.14 Closed · Efficient | 86.1%471 of 547 right | 89.16chosen-answer Brier 0.0778 | 87.2%477 of 547 right | 4.0%3 “I don't know” of 75 not rightanswered 99.5%error rate 13.2% | 0.02726.64at list prices | 0.1811 question per request, 1 in flight |
| Cerno 9B v1.2 Open · 9.0B | 85.9%470 of 547 right | 88.42chosen-answer Brier 0.0907 | 83.5%457 of 547 right | 2.5%2 “I don't know” of 79 not rightanswered 99.6%error rate 14.1% | No list price | 0.2111 question per request, 1 in flight |
| Drex 2.0 Open · 35B MoE · 3.0B active | 85.9%470 of 547 right | 88.15chosen-answer Brier 0.0963 | 84.8%464 of 547 right | Not run | 0.01919.42at list prices | 0.2831 question per request, 1 in flight |
| Jev 1.13 Closed · Efficient | 85.9%470 of 547 right | 88.86chosen-answer Brier 0.0821 | 86.1%471 of 547 right | Not run | 0.02726.64at list prices | 0.1731 question per request, 1 in flight |
| KnowLine-9B-Gen1 Open · 9.0B | 85.9%470 of 547 right | 88.32chosen-answer Brier 0.0928 | 82.4%451 of 547 right | 1.3%1 “I don't know” of 77 not rightanswered 99.8%error rate 13.9% | 0.04241.55from GPU time | 0.0591 question per request, 1 in flight |
| SubQ Snap 1.1 Closed · Efficient | 85.9%470 of 547 right | 88.70chosen-answer Brier 0.0852 | 89.2%488 of 547 right | 25.0%21 “I don't know” of 84 not rightanswered 96.2%error rate 12.0% | 0.01615.59at list prices | 0.3811 question per request, 1 in flight |
| SubQ Snap Preview Closed · Efficient | 85.9%470 of 547 right | 88.70chosen-answer Brier 0.0852 | 89.9%492 of 547 right | Not run | 0.01919.48at list prices | 0.3811 question per request, 1 in flight |
| Adjudex 9B v5 Open · 9.0B | 85.7%469 of 547 right | 88.35chosen-answer Brier 0.0905 | 83.9%459 of 547 right | 0.0%0 “I don't know” of 77 not rightanswered 100.0%error rate 14.1% | 0.02524.93from GPU time | 0.0341 question per request, 1 in flight |
| Mercury Decide Closed · Efficient | 85.7%469 of 547 right | 88.15chosen-answer Brier 0.0943 | 87.0%476 of 547 right | 13.4%11 “I don't know” of 82 not rightanswered 98.0%error rate 13.2% | 0.0087.79at list prices | 0.3101 question per request, 1 in flight |
| Microsoft-Decision-1 Closed · Efficient | 85.7%469 of 547 right | 88.10chosen-answer Brier 0.0954 | 76.4%418 of 547 right | 3.8%3 “I don't know” of 80 not rightanswered 99.5%error rate 14.2% | 0.01414.01at list prices | 0.3151 question per request, 1 in flight |
| Elephant-1 Open · 9.0B | 85.4%467 of 547 right | 87.99chosen-answer Brier 0.0939 | 83.5%457 of 547 right | 25.6%23 “I don't know” of 90 not rightanswered 95.8%error rate 12.8% | 0.01414.02at the maker's list price | 0.0701 question per request, 1 in flight |
| Open-Jev-27B-v1.1 Open · 27B | 85.4%467 of 547 right | 77.18chosen-answer Brier 0.3102 | 86.5%473 of 547 right | 17.4%16 “I don't know” of 92 not rightanswered 97.1%error rate 14.3% | 0.351351from GPU time | 0.4951 question per request, 1 in flight |
| KnowLine-9B-Gen2 Open · 9.0B | 85.2%466 of 547 right | 87.38chosen-answer Brier 0.1044 | 82.8%453 of 547 right | 1.3%1 “I don't know” of 79 not rightanswered 99.8%error rate 14.3% | 0.03939.48from GPU time | 0.0561 question per request, 1 in flight |
| Qwen3.8-27B stated confidence · reasoning off · chat Open · 27B | 84.8%464 of 547 right | not applicable | 86.3%472 of 547 right | 3.7%3 “I don't know” of 81 not rightanswered 99.5%error rate 14.3% | 0.212212at the maker's list price | 0.3901 question per request, 1 in flight |
| Qwen3.8-27B yes/no scoring Open · 27B | 84.8%464 of 547 right | not applicable | 84.6%463 of 547 right | Not runyes/no scoring cannot offer it | 0.741741at the maker's list price | 0.4131 question per request, 1 in flight |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active | 84.5%462 of 547 right | not applicable | 86.3%472 of 547 right | 3.4%3 “I don't know” of 88 not rightanswered 99.5%error rate 15.6% | 0.193193at list prices | 26.716 questions per request, 4 in flight |
| celeris-1-decision Closed · Efficient | 84.1%460 of 547 right | 87.29chosen-answer Brier 0.0952 | 87.4%478 of 547 right | 71.3%92 “I don't know” of 129 not rightanswered 83.2%error rate 8.1% | 0.01616.09at list prices | 0.3901 question per request, 1 in flight |
| GPT-6 Luna Decisions API Closed · Efficient | 84.1%460 of 547 right | unavailable | 86.7%474 of 547 right | 9.4%10 “I don't know” of 106 not rightanswered 98.2%error rate 17.9% | 0.04141.09at list prices | 0.3231 question per request, 4 in flight |
| Claude Sonnet 5.5 low · chat Closed · Workhorse | 83.9%459 of 547 right | not applicable | 84.8%464 of 547 right | 4.4%4 “I don't know” of 90 not rightanswered 99.3%error rate 15.8% | 0.354354subscription, priced as the API | 3.6816 questions per request, 1 in flight |
| Cygnet Open · 12B | 83.9%459 of 547 right | 87.03chosen-answer Brier 0.0984 | 85.6%468 of 547 right | 11.2%10 “I don't know” of 89 not rightanswered 98.2%error rate 14.7% | 0.04141.42at a host's list price for the same weights | 0.0431 question per request, 1 in flight |
| Qwen3.8-27B vote share · reasoning off · chat Open · 27B | 83.9%459 of 547 right | not applicable | 85.9%470 of 547 right | 3.4%3 “I don't know” of 87 not rightanswered 99.5%error rate 15.4% | 0.645645at the maker's list price | 1.701 question per request, 1 in flight |
| Winnow-12B Open · 12B | 83.9%459 of 547 right | 86.18chosen-answer Brier 0.1156 | 85.0%465 of 547 right | 15.0%15 “I don't know” of 100 not rightanswered 97.3%error rate 16.0% | 0.05049.59from GPU time | 0.0701 question per request, 1 in flight |
| Perplexity Decider Flash v1.3 Open · 4.0B | 83.7%458 of 547 right | 86.96chosen-answer Brier 0.0980 | 78.8%431 of 547 right | 7.2%7 “I don't know” of 97 not rightanswered 98.7%error rate 16.7% | 0.02323.17from GPU time | 0.0331 question per request, 1 in flight |
| Valen-Decision-1.5-9B Open · 9.0B | 83.7%458 of 547 right | 86.43chosen-answer Brier 0.1087 | 79.0%432 of 547 right | 1.2%1 “I don't know” of 85 not rightanswered 99.8%error rate 15.4% | 0.04140.55from GPU time | 0.0591 question per request, 1 in flight |
| celeris-1.1-decision Closed · Efficient | 83.7%458 of 547 right | 87.14chosen-answer Brier 0.0946 | Not runEnglish only, no Text tab run | 74.8%98 “I don't know” of 131 not rightanswered 82.1%error rate 7.3% | 0.0021.61at list prices | 0.3851 question per request, 1 in flight |
| Dexy-Decision-9B-V2 Open · 9.0B | 83.5%457 of 547 right | 87.01chosen-answer Brier 0.0953 | 83.4%456 of 547 right | 1.1%1 “I don't know” of 91 not rightanswered 99.8%error rate 16.5% | 0.02525.41from GPU time | 0.0361 question per request, 1 in flight |
| vera-medium-omni Open · 4.2B | 83.5%457 of 547 right | 86.34chosen-answer Brier 0.1086 | 79.5%435 of 547 right | 7.4%7 “I don't know” of 94 not rightanswered 98.7%error rate 16.1% | 0.03736.63from GPU time | 0.0541 question per request, 1 in flight |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | 82.8%453 of 547 right | 86.35chosen-answer Brier 0.1011 | 78.4%429 of 547 right | 0.0%0 “I don't know” of 94 not rightanswered 100.0%error rate 17.2% | No list price | 0.0751 question per request, 1 in flight |
| LFM2.5-24B-A2B default reasoning · chat Open · 24B MoE · 2.5B active | 82.6%452 of 547 right | not applicable | Queuedits Text tab run not scheduled yet: needs the model served again (Ümit #6527) | 13.2%14 “I don't know” of 106 not rightanswered 97.4%error rate 17.3% | 1.7641,764at the maker's list price | 74.21 question per request, 32 in flight |
| Gemma 4 E4B default reasoning · chat Open · 8.0B | 82.4%451 of 547 right | not applicable | 80.6%441 of 547 right | 9.8%10 “I don't know” of 102 not rightanswered 97.8%error rate 17.2% | 0.199199from GPU time | 8.391 question per request, 32 in flight |
| Clef-Flash Open · 9.0B | 82.3%450 of 547 right | 85.43chosen-answer Brier 0.1141 | 77.7%425 of 547 right | 5.3%5 “I don't know” of 95 not rightanswered 99.1%error rate 16.6% | 0.01817.53at the maker's list price | 0.1401 question per request, 1 in flight |
| Verdin-4B-v1.1 Open · 4.0B | 82.3%450 of 547 right | 85.97chosen-answer Brier 0.1032 | 77.5%424 of 547 right | 2.1%2 “I don't know” of 96 not rightanswered 99.6%error rate 17.2% | 0.03434.28from GPU time | 0.0521 question per request, 1 in flight |
| Perplexity Decider Flash v1.2 Open · 4.0B | 81.9%448 of 547 right | 86.04chosen-answer Brier 0.0982 | 76.6%419 of 547 right | 4.8%5 “I don't know” of 105 not rightanswered 99.1%error rate 18.5% | 0.0043.93at the maker's list price | 0.0371 question per request, 1 in flight |
| KnowLine 4B Gen5.5 Open · 4.0B | 80.8%442 of 547 right | 84.57chosen-answer Brier 0.1167 | 78.6%430 of 547 right | 0.9%1 “I don't know” of 106 not rightanswered 99.8%error rate 19.2% | No list price | 0.0991 question per request, 1 in flight |
| Chinese Decision Pointer 4B Open · 4.0B | 80.4%440 of 547 right | 84.66chosen-answer Brier 0.1111 | 75.7%414 of 547 right | 3.7%4 “I don't know” of 108 not rightanswered 99.3%error rate 19.2% | 0.02727.21from GPU time | 0.0381 question per request, 1 in flight |
| Qwen3.5-4B reasoning off · chat Open · 4.0B | 80.4%440 of 547 right | not applicable | 72.0%394 of 547 right | 3.6%4 “I don't know” of 110 not rightanswered 99.3%error rate 19.5% | 0.309309from GPU time | 9.741 question per request, 32 in flight |
| Qwen3-14B reasoning off · chat Open · 15B | 79.7%436 of 547 right | not applicable | 76.8%420 of 547 right | 1.8%2 “I don't know” of 110 not rightanswered 99.6%error rate 19.8% | 0.03131.17at a host's list price for the same weights | 2.8616 questions per request, 1 in flight |
| MiniCPM5-2B reasoning on · chat Open · 2.5B | 78.1%76.6%–79.0% min–max of 3 seedsseed 0: run 1051, 419 of 547 right; seed 1: run 1118, 431 of 547 right; seed 2: run 1119, 432 of 547 right. Baseline server capacity not retained; repeats use max-num-seqs 16. | not applicable | Not runEnglish only, no Text tab run | 19.7%28 “I don't know” of 142 not rightanswered 94.1%error rate 22.1%from run 1052 | 0.714714from GPU time | 17.51 question per request, 32 in flight |
| Metask-Jev-4B Open · 4.5B | 77.1%422 of 547 right | 82.21chosen-answer Brier 0.1273 | 67.1%367 of 547 right | 1.6%2 “I don't know” of 125 not rightanswered 99.6%error rate 22.6% | 0.05959.11from GPU time | 0.0851 question per request, 1 in flight |
| Gemma 4 E2B reasoning on · chat Open · 5.1B | 76.2%417 of 547 right | not applicable | Not runEnglish only, no Text tab run | 11.3%17 “I don't know” of 150 not rightanswered 96.3%error rate 25.2% | 0.187187from GPU time | 7.611 question per request, 32 in flight |
| Qwen3.5-2B reasoning on · chat Open · 2.0B | 75.9%415 of 547 right | not applicable | Not runEnglish only, no Text tab run | 9.9%14 “I don't know” of 142 not rightanswered 97.1%error rate 24.1% | 1.3611,361from GPU time | 54.41 question per request, 32 in flight |
| Gemma 4 E2B default reasoning · chat Open · 5.1B | 74.6%408 of 547 right | not applicable | Queuedits Text tab run in logic's GPU 1 queue (Ümit #6527, logic #6955) | 11.5%17 “I don't know” of 148 not rightanswered 96.5%error rate 24.8% | 0.115115from GPU time | 4.841 question per request, 32 in flight |
| Granite-4.2-3B reasoning on · chat Open · 3.0B | 74.4%407 of 547 right | not applicable | Pausedits Text tab run its Text run stopped at 741 of 2,198 so that other setups go first on logic's GPU 1; it resumes when logic loads Granite again (logic #7481, #7599) | 16.1%23 “I don't know” of 143 not rightanswered 95.8%error rate 22.9% | 2.7172,717from GPU time | 40.51 question per request, 32 in flight |
| LFM2.5-2.6B default reasoning · chat Open · 2.7B | 73.9%404 of 547 right | not applicable | 47.9%262 of 547 right | 15.7%21 “I don't know” of 134 not rightanswered 95.6%error rate 21.6% | 0.964964at the maker's list price | 15.01 question per request, 32 in flight |
| Erk-14B reasoning off · chat Open · 14B | 72.0%394 of 547 right | not applicable | 75.3%412 of 547 right | 2.1%3 “I don't know” of 144 not rightanswered 99.5%error rate 25.9% | 0.123123from GPU time | 2.8616 questions per request, 1 in flight |
| LFM3-1.7B reasoning off · chat Open · 1.7B | 71.8%70.9%–72.9% min–max of 3 seedsseed 0: run 1223, 399 of 547 right; seed 1: run 1302, 388 of 547 right; seed 2: run 1303, 391 of 547 right | not applicable | 40.2%220 of 547 right | 11.6%18 “I don't know” of 155 not rightanswered 96.7%error rate 25.9%from run 1225 | 0.355355at the maker's list price | 4.691 question per request, 32 in flight |
| KnowLine-2B-Gen1 Open · 2.0B | 71.3%390 of 547 right | 77.82chosen-answer Brier 0.1567 | 58.0%317 of 547 right | 22.0%38 “I don't know” of 173 not rightanswered 93.1%error rate 26.5% | No list price | 0.0621 question per request, 1 in flight |
| LFM3-1.7B reasoning on · chat Open · 1.7B | 71.1%70.4%–71.7% min–max of 3 seedsseed 0: run 1091, 392 of 547 right; seed 1: run 1115, 390 of 547 right; seed 2: run 1116, 385 of 547 right | not applicable | 48.1%263 of 547 right | 15.6%26 “I don't know” of 167 not rightanswered 95.2%error rate 27.1%from run 1062 | 0.387387at the maker's list price | 5.241 question per request, 32 in flight |
| LFM2.5-8B-A1B default reasoning · chat Open · 8.3B MoE · 1.5B active | 70.4%385 of 547 right | not applicable | 27.1%148 of 547 right | 26.4%51 “I don't know” of 193 not rightanswered 90.5%error rate 28.7% | 0.517517from GPU time | 19.41 question per request, 32 in flight |
| GLiDE 2B Open · 1.9B | 70.0%383 of 547 right | 76.88chosen-answer Brier 0.1626 | 54.3%297 of 547 right | 26.3%47 “I don't know” of 179 not rightanswered 91.4%error rate 26.4% | No list price | 0.0351 question per request, 1 in flight |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | 69.7%381 of 547 right | 76.66chosen-answer Brier 0.1633 | 55.8%305 of 547 right | 16.9%32 “I don't know” of 189 not rightanswered 94.1%error rate 30.5% | No list price | 0.0401 question per request, 1 in flight |
| MiniCPM5-2B reasoning off · chat Open · 2.5B | 67.8%66.7%–68.9% min–max of 3 seedsseed 0: run 1094, 370 of 547 right; seed 1: run 1300, 377 of 547 right; seed 2: run 1301, 365 of 547 right. Seeds 1 and 2 on logic's load of 15 November (vLLM v0.30.0, max-num-seqs 16), which logic gives as run 1094's setup (#9335); the server's settings are not recorded with a run. | not applicable | 7.1%39 of 547 right | 16.8%33 “I don't know” of 197 not rightanswered 89.2%error rate 33.6%from run 1095 | 0.429429from GPU time | 7.291 question per request, 32 in flight |
| Qwen3.5-2B reasoning off · chat Open · 2.0B | 67.3%368 of 547 right | not applicable | 36.7%201 of 547 right | 10.3%18 “I don't know” of 174 not rightanswered 96.3%error rate 29.6% | 0.280280from GPU time | 9.271 question per request, 32 in flight |
| d1-2B Open · 1.9B | 65.6%359 of 547 right | 74.16chosen-answer Brier 0.1731 | 41.9%229 of 547 right | 60.9%151 “I don't know” of 248 not rightanswered 72.4%error rate 24.5% | 0.0088.08from GPU time | 0.0111 question per request, 1 in flight |
| GLiDE 0.8B Open · 0.8B | 60.7%332 of 547 right | 70.59chosen-answer Brier 0.1951 | 46.1%252 of 547 right | 21.7%49 “I don't know” of 226 not rightanswered 91.0%error rate 35.5% | No list price | 0.0231 question per request, 1 in flight |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | 59.6%326 of 547 right | 70.09chosen-answer Brier 0.1942 | 37.3%204 of 547 right | 5.2%12 “I don't know” of 229 not rightanswered 97.8%error rate 40.6% | No list price | 0.0281 question per request, 1 in flight |
| GLiDE 0.4B Open · 0.4B | 59.4%325 of 547 right | 69.84chosen-answer Brier 0.1973 | 25.8%141 of 547 right | 53.1%153 “I don't know” of 288 not rightanswered 72.0%error rate 34.3% | No list price | 0.0141 question per request, 1 in flight |
| OneJev1.6-0.8B Open · 0.8B | 59.4%325 of 547 right | 69.82chosen-answer Brier 0.1978 | 42.8%234 of 547 right | 28.4%74 “I don't know” of 261 not rightanswered 86.5%error rate 39.5% | No list price | 0.0181 question per request, 1 in flight |
| KnowLine-0.8B-Gen1 Open · 0.8B | 58.9%322 of 547 right | 69.65chosen-answer Brier 0.1956 | 40.2%220 of 547 right | 10.7%26 “I don't know” of 243 not rightanswered 95.2%error rate 41.7% | No list price | 0.0331 question per request, 1 in flight |
| GLiClass v4 Edge Open · 0.4B | 58.3%319 of 547 right | 69.05chosen-answer Brier 0.2021 | 26.9%147 of 547 right | 21.0%52 “I don't know” of 248 not rightanswered 90.5%error rate 39.6% | No list price | 0.0171 question per request, 1 in flight |
| Decider 1 Closed · Efficient | 55.8%305 of 547 right | 67.47chosen-answer Brier 0.2082 | 36.6%200 of 547 right | Not run | 0.01514.95at list prices | 0.6991 question per request, 1 in flight |
| Raon-1 nano reasoning off · chat Open · 3.1B | 53.7%294 of 547 right | not applicable | Queuedits Text tab run asked on its next load in logic's GPU 1 queue, whichever way its smoke came out (run 1154: 3 of 20 replies at the 32,768-token limit, 5 without a letter; Ümit #6515 via Zeki #8211) | 26.2%71 “I don't know” of 271 not rightanswered 86.8%error rate 42.1% | 0.235235from GPU time | 3.741 question per request, 32 in flight |
| Strands Decider 2B Open · 2.0B | 45.5%249 of 547 right | 64.08chosen-answer Brier 0.1736 | 39.1%214 of 547 right | 0.7%2 “I don't know” of 303 not rightanswered 99.6%error rate 55.2% | 0.01818.15from GPU time | 0.0261 question per request, 1 in flight |
| Laya Open · 0.4B | 22.9%125 of 547 right | 51.57chosen-answer Brier 0.1971 | 20.5%112 of 547 right | 18.4%83 “I don't know” of 450 not rightanswered 84.8%error rate 79.1% | 0.0087.84from GPU time | 0.0111 question per request, 1 in flight |
| GLiNER2.5-multi-Decide Open · 0.3B | 21.2%116 of 547 right | 47.58chosen-answer Brier 0.2604 | 20.7%113 of 547 right | 16.2%70 “I don't know” of 433 not rightanswered 87.2%error rate 76.1% | 0.0076.73from GPU time | 0.0091 question per request, 1 in flight |
| GLiNER2.5 Multi Open · 0.3B | 16.3%89 of 547 right | not applicable | 16.8%92 of 547 right | 19.6%92 “I don't know” of 470 not rightanswered 83.2%error rate 83.1% | 0.0088.39at the maker's list price | 0.0091 question per request, 1 in flight |
| Laya Multilingual Open · 0.3B | 15.4%84 of 547 right | 39.32chosen-answer Brier 0.3671 | 17.0%93 of 547 right | 42.6%208 “I don't know” of 488 not rightanswered 62.0%error rate 82.6% | 0.01312.62from GPU time | 0.0181 question per request, 1 in flight |
Ranked by English accuracy, highest first: right answers of the 547 kept reading questions in each setup's first English run (an invalid answer counts as not right). Turkish accuracy: the setup's Text tab run on the same 547 questions, for context (Not run: an English-only setup, with no Text tab run; Paused: its Text tab run is stopped for now, as its row says; Queued: its Text tab run has not started yet, as its row says); the paired comparison is on English and Turkish. English non-hallucination: “I don't know” among the answers that were not right, in the setup's first English run with it offered as a sixth option; Running: a chat model's run still running; Not run: a decision model not run with it yet, or an English-only setup. English cost: $ per 1,000 questionsper 1M questions, with its basis under it; English latency: the median time per request, with the request size and the requests in flight under it, each from the first English run (Cost and Latency below). No list price: a setup on our own hardware whose maker lists no price for the same model. MiniCPM5-2B, LFM3-1.7B, LFM3-1.7B and MiniCPM5-2B are run with 3 sampling seeds: until all are in, the number is the first English run's, named with the seeds in so far; then it is the mean of the runs' accuracies, ranked by it, with their lowest and highest (min–max, a range, not a confidence interval) under it. The paired comparison with Turkish and the “I don't know” runs stay on their own runs. Each name opens its model page's English section (an English-only setup has no model page: its name opens its row here); its tooltip gives its settings and English run.
Decision Score (AA formula): 50 × (1 + accuracy − chosen-answer Brier) on the first English run, the chosen-answer Brier being the mean of (p − c)², p the probability given to the chosen answer and c 1 if it is right, else 0. GPT-6 Luna · Decisions API: unavailable, 2 of 547 answers without a probability for the chosen answer.
Artificial Analysis's formula applied to our questions. Not their Index. Decision models only: chat models and the I don't know runs are excluded.
Costs: API list prices and recorded token use. Setups on our own GPUs: the maker's list price for the same model (each value's tooltip names it); our hardware cost is not measured. On our GPUs, estimated at a public host's list price for the same weights (the host, the date read and any precision difference are on each setup's page): an API-equivalent estimate, not measured spending. On our GPUs with no public price for the same weights, estimated from GPU time: the run's request time x the GPUs its server held x Runpod's on-demand price for an RTX PRO 6000, $2.49 per GPU-hour (read 5 November 2026): a cloud-equivalent estimate, not measured spending. No list price: on our own GPUs, with no estimate. ≤ Upper-bound estimate: one $0.001 web search counted per request; Perplexity bills only completed searches.
Every chat model setup in English
| Model | English accuracyfirst English run, or the mean of a setup's three seed runs; higher is better | Decision Score (AA formula)first English run; higher is better | Turkish accuracyText tab run, same 547; for context | English non-hallucinationof answers not right, % “I don't know” · higher is better; answered and error rate under it | English cost$ per 1,000 questionsper 1M questions, first English run; lower is better | English latencymedian seconds per request, first English run; lower is better |
|---|---|---|---|---|---|---|
| Claude Fable 5.2 max · chat Closed · Flagship | 98.0%536 of 547 right | not applicable | 97.8%535 of 547 right | 28.6%4 “I don't know” of 14 not rightanswered 99.3%error rate 1.8% | 52.94252,942subscription, priced as the API | 74.016 questions per request, 1 in flight |
| Gemini 4 Hydrogen high · chat Closed · Workhorse | 96.7%529 of 547 right | not applicable | 97.1%531 of 547 right | 20.0%4 “I don't know” of 20 not rightanswered 99.3%error rate 2.9% | 4.9274,927subscription, priced as the API | 28.416 questions per request, 1 in flight |
| GPT-6 Astra low · chat Closed · Flagship | 96.3%527 of 547 right | not applicable | 96.7%529 of 547 right | 18.2%4 “I don't know” of 22 not rightanswered 99.3%error rate 3.3% | 5.9185,918subscription, priced as the API | 11.516 questions per request, 1 in flight |
| GPT-6 Astra high · chat Closed · Flagship | 96.2%526 of 547 right | not applicable | 96.9%530 of 547 right | 19.0%4 “I don't know” of 21 not rightanswered 99.3%error rate 3.1% | 6.3346,334subscription, priced as the API | 15.616 questions per request, 1 in flight |
| GPT-6.1 Terra high · chat Closed · Workhorse | 96.2%526 of 547 right | not applicable | 96.3%527 of 547 right | 16.7%4 “I don't know” of 24 not rightanswered 99.3%error rate 3.7% | 0.509509subscription, priced as the API | 7.9416 questions per request, 1 in flight |
| GPT-6.1 Terra low · chat Closed · Workhorse | 96.2%526 of 547 right | not applicable | 95.6%523 of 547 right | 15.4%4 “I don't know” of 26 not rightanswered 99.3%error rate 4.1% | 0.491491subscription, priced as the API | 7.2916 questions per request, 1 in flight |
| Claude Fable 5.1 high · chat Closed · Flagship | 96.0%525 of 547 right | not applicable | 95.4%522 of 547 right | 7.4%2 “I don't know” of 27 not rightanswered 99.6%error rate 4.6% | 11.53111,531subscription, priced as the API | 15.616 questions per request, 1 in flight |
| Gemini 3.8 Flash high · chat Closed · Workhorse | 96.0%525 of 547 right | not applicable | 96.9%530 of 547 right | 10.0%2 “I don't know” of 20 not rightanswered 99.6%error rate 3.3% | 3.6953,695at list prices | 104.616 questions per request, 4 in flight |
| Gemini 4 Argon low · chat Closed · Flagship | 96.0%525 of 547 right | not applicable | 96.9%530 of 547 right | 18.5%5 “I don't know” of 27 not rightanswered 99.1%error rate 4.1% | 1.9001,900at list prices | 6.0316 questions per request, 1 in flight |
| GPT-6 Astra medium · chat Closed · Flagship | 96.0%525 of 547 right | not applicable | 96.7%529 of 547 right | 16.7%4 “I don't know” of 24 not rightanswered 99.3%error rate 3.7% | 4.6964,696subscription, priced as the API | 10.716 questions per request, 1 in flight |
| GPT-6.1 Sol low · chat Closed · Workhorse | 95.8%524 of 547 right | not applicable | 95.8%524 of 547 right | 14.8%4 “I don't know” of 27 not rightanswered 99.3%error rate 4.2% | 1.2191,219subscription, priced as the API | 10.316 questions per request, 1 in flight |
| GPT-6 Astra low · chat · 1 per request Closed · Flagship | 95.6%523 of 547 right | not applicable | 97.1%531 of 547 right | 14.8%4 “I don't know” of 27 not rightanswered 99.3%error rate 4.2% | 28.65328,653subscription, priced as the API | 6.431 question per request, 1 in flight |
| GPT-6.1 Sol high · chat Closed · Workhorse | 95.6%523 of 547 right | not applicable | 96.2%526 of 547 right | 16.7%4 “I don't know” of 24 not rightanswered 99.3%error rate 3.7% | 1.4271,427subscription, priced as the API | 16.916 questions per request, 1 in flight |
| Gemini 3.8 Flash low · chat · 1 per request Closed · Workhorse | 95.2%521 of 547 right | not applicable | 96.2%526 of 547 right | 7.7%2 “I don't know” of 26 not rightanswered 99.6%error rate 4.4% | 0.858858at list prices | 2.851 question per request, 1 in flight |
| GPT-6.1 Sol low · chat · 1 per request Closed · Workhorse | 95.2%521 of 547 right | not applicable | 95.6%523 of 547 right | 12.9%4 “I don't know” of 31 not rightanswered 99.3%error rate 5.0% | 6.1136,113subscription, priced as the API | 5.591 question per request, 1 in flight |
| Claude Opus 5.5 low · chat · 1 per request Closed · Flagship | 94.7%518 of 547 right | not applicable | 95.4%522 of 547 right | 0.0%0 “I don't know” of 31 not rightanswered 100.0%error rate 5.7% | 10.09410,094subscription, priced as the API | 3.501 question per request, 1 in flight |
| Claude Opus 5.5 high · chat Closed · Flagship | 94.5%517 of 547 right | not applicable | 95.6%523 of 547 right | 6.5%2 “I don't know” of 31 not rightanswered 99.6%error rate 5.3% | 4.5194,519subscription, priced as the API | 11.316 questions per request, 1 in flight |
| Qwen4-Max high · chat Closed · Workhorse | 94.5%517 of 547 right | not applicable | 94.7%518 of 547 right | 0.0%0 “I don't know” of 29 not rightanswered 100.0%error rate 5.3% | 1.1191,119at list prices | 29.516 questions per request, 4 in flight |
| Gemini 3.8 Flash low · chat Closed · Workhorse | 94.3%516 of 547 right | not applicable | 95.2%521 of 547 right | 3.4%1 “I don't know” of 29 not rightanswered 99.8%error rate 5.1% | 0.288288at list prices | 4.1916 questions per request, 1 in flight |
| Gemini 4 Carbon high · chat Closed · Flagship | 94.3%516 of 547 right | not applicable | 97.6%534 of 547 right | 25.0%4 “I don't know” of 16 not rightanswered 99.3%error rate 2.2% | 38.05338,053subscription, priced as the API | 287.816 questions per request, 1 in flight |
| Grok 4.7 high · chat Closed · Workhorse | 93.4%511 of 547 right | not applicable | 95.1%520 of 547 right | 27.9%12 “I don't know” of 43 not rightanswered 97.8%error rate 5.8% | 6.8516,851subscription, priced as the API | 197.216 questions per request, 4 in flight |
| Claude Opus 5.5 low · chat Closed · Flagship | 93.1%509 of 547 right | not applicable | 95.1%520 of 547 right | 4.7%2 “I don't know” of 43 not rightanswered 99.6%error rate 7.5% | 1.0871,087subscription, priced as the API | 7.6416 questions per request, 1 in flight |
| Claude Haiku 5.5 max · chat Closed · Efficient | 92.7%507 of 547 right | not applicable | 89.4%489 of 547 right | 7.8%4 “I don't know” of 51 not rightanswered 96.3%error rate 8.9% | 1.2611,261at list prices | 148.916 questions per request, 4 in flight |
| DeepSeek-V4.1-Flash reasoning on · chat · 1017 Open · 552B MoE · 16B active | 92.5%506 of 547 right | not applicable | 94.5%517 of 547 right | 7.1%3 “I don't know” of 42 not rightanswered 99.5%error rate 7.2% | 0.421421at the maker's list price | 189.416 questions per request, 8 in flight |
| Kimi K3.1 Pro low · chat Open · 2.8T MoE · 104B active | 92.3%505 of 547 right | not applicable | 95.2%521 of 547 right | 7.3%3 “I don't know” of 41 not rightanswered 99.5%error rate 7.0% | 0.984984at list prices | 16.616 questions per request, 4 in flight |
| Claude Sonnet 5.5 high · chat · 1 per request Closed · Workhorse | 91.6%501 of 547 right | not applicable | 94.3%516 of 547 right | 0.0%0 “I don't know” of 38 not rightanswered 100.0%error rate 6.9% | 5.5135,513subscription, priced as the API | 3.021 question per request, 1 in flight |
| Kimi K3.1 Pro no reasoning · chat Open · 2.8T MoE · 104B active | 91.4%500 of 547 right | not applicable | 94.0%514 of 547 right | 7.5%4 “I don't know” of 53 not rightanswered 99.3%error rate 9.0% | 0.735735at list prices | 6.3116 questions per request, 4 in flight |
| Qwen3.5-397B-A17B high · chat Open · 397B MoE · 17B active | 91.4%500 of 547 right | not applicable | 91.4%500 of 547 right | 6.2%3 “I don't know” of 48 not rightanswered 99.5%error rate 8.3% | 1.7421,742at list prices | 162.916 questions per request, 4 in flight |
| Gemma 4 31B reasoning off · chat Open · 31B | 90.5%495 of 547 right | not applicable | 92.5%506 of 547 right | 3.6%2 “I don't know” of 55 not rightanswered 99.6%error rate 9.7% | 0.05352.81at a host's list price for the same weights | 3.1916 questions per request, 1 in flight |
| Gemma 4 31B reasoning off · chat · 1 per request Open · 31B | 89.9%492 of 547 right | not applicable | 90.3%494 of 547 right | 0.0%0 “I don't know” of 59 not rightanswered 100.0%error rate 10.8% | 0.07070.17at a host's list price for the same weights | 0.2991 question per request, 1 in flight |
| GLM-5.3-Flash reasoning on · chat Open · 320B MoE · 18B active | 89.6%490 of 547 right | not applicable | 90.1%493 of 547 right | 1.6%1 “I don't know” of 62 not rightanswered 99.8%error rate 11.2% | 0.08181.38at the maker's list price | 33.116 questions per request, 8 in flight |
| Muse Ember max · chat Closed · Efficient | 89.6%490 of 547 right | not applicable | 90.9%497 of 547 right | 5.3%3 “I don't know” of 57 not rightanswered 99.5%error rate 9.9% | 0.245245at list prices | 11.616 questions per request, 4 in flight |
| MiMo-V2.6-Flash high · chat Open · 309B MoE · 15B active | 89.0%487 of 547 right | not applicable | 86.8%475 of 547 right | 1.7%1 “I don't know” of 59 not rightanswered 99.8%error rate 10.6% | 0.298298at list prices | 6.671 question per request, 8 in flight |
| Qwen3.8-27B low · chat Open · 27B | 89.0%487 of 547 right | not applicable | 90.7%496 of 547 right | 3.6%2 “I don't know” of 55 not rightanswered 99.6%error rate 9.7% | 0.853853at the maker's list price | 71.716 questions per request, 4 in flight |
| Inkling-Small high · chat · Jul 2026 Open · 276B MoE · 12B active | 88.3%483 of 547 right | not applicable | 89.4%489 of 547 right | 0.0%0 “I don't know” of 60 not rightanswered 100.0%error rate 11.0% | 0.425425at list prices | 25.316 questions per request, 4 in flight |
| GLM-5.3 low · chat Open · Workhorse | 87.8%480 of 547 right | not applicable | 89.2%488 of 547 right | 3.4%2 “I don't know” of 58 not rightanswered 99.6%error rate 10.3% | 0.408408at list prices | 6.8416 questions per request, 4 in flight |
| GLM-5.3-Flash reasoning off · chat Open · 320B MoE · 18B active | 87.2%477 of 547 right | not applicable | 84.3%461 of 547 right | 5.6%4 “I don't know” of 71 not rightanswered 99.3%error rate 12.3% | 0.04646.07at the maker's list price | 3.3716 questions per request, 1 in flight |
| DeepSeek-V4.1-Flash reasoning off · chat · 1017 Open · 552B MoE · 16B active | 87.0%476 of 547 right | not applicable | 89.9%492 of 547 right | 5.4%4 “I don't know” of 74 not rightanswered 99.3%error rate 12.9% | 0.07979.31at the maker's list price | 6.5916 questions per request, 1 in flight |
| Qwen3.5-4B reasoning on · chat Open · 4.0B | 87.0%476 of 547 right | not applicable | 82.8%453 of 547 right | 2.4%2 “I don't know” of 85 not rightanswered 99.6%error rate 15.2% | 1.3901,390from GPU time | 58.31 question per request, 32 in flight |
| Qwen3.8 Flash no reasoning · chat Closed · Efficient | 87.0%476 of 547 right | not applicable | 88.8%486 of 547 right | 5.6%4 “I don't know” of 71 not rightanswered 99.3%error rate 12.3% | 0.05049.69at list prices | 10.116 questions per request, 4 in flight |
| Gemma 4 12B Unified default reasoning · chat Open · 12B | 86.8%475 of 547 right | not applicable | Queuedits Text tab run in logic's GPU 1 queue (Ümit #6527, logic #6955) | 5.5%4 “I don't know” of 73 not rightanswered 98.9%error rate 12.8% | 0.154154at a host's list price for the same weights | 15.81 question per request, 32 in flight |
| GPT-6 Luna low · chat Closed · Efficient | 86.3%472 of 547 right | not applicable | 88.7%485 of 547 right | 6.8%5 “I don't know” of 74 not rightanswered 99.1%error rate 12.7% | 0.05857.70subscription, priced as the API | 7.7216 questions per request, 1 in flight |
| Claude Sonnet 5.5 high · chat Closed · Workhorse | 86.1%471 of 547 right | not applicable | 87.6%479 of 547 right | 4.8%3 “I don't know” of 62 not rightanswered 99.5%error rate 10.8% | 1.9981,998subscription, priced as the API | 3.8516 questions per request, 1 in flight |
| Qwen3.8-27B stated confidence · reasoning off · chat Open · 27B | 84.8%464 of 547 right | not applicable | 86.3%472 of 547 right | 3.7%3 “I don't know” of 81 not rightanswered 99.5%error rate 14.3% | 0.212212at the maker's list price | 0.3901 question per request, 1 in flight |
| Qwen3.8-27B yes/no scoring Open · 27B | 84.8%464 of 547 right | not applicable | 84.6%463 of 547 right | Not runyes/no scoring cannot offer it | 0.741741at the maker's list price | 0.4131 question per request, 1 in flight |
| MiniMax-M3.1-Flash high · chat Open · 428B MoE · 23B active | 84.5%462 of 547 right | not applicable | 86.3%472 of 547 right | 3.4%3 “I don't know” of 88 not rightanswered 99.5%error rate 15.6% | 0.193193at list prices | 26.716 questions per request, 4 in flight |
| Claude Sonnet 5.5 low · chat Closed · Workhorse | 83.9%459 of 547 right | not applicable | 84.8%464 of 547 right | 4.4%4 “I don't know” of 90 not rightanswered 99.3%error rate 15.8% | 0.354354subscription, priced as the API | 3.6816 questions per request, 1 in flight |
| Qwen3.8-27B vote share · reasoning off · chat Open · 27B | 83.9%459 of 547 right | not applicable | 85.9%470 of 547 right | 3.4%3 “I don't know” of 87 not rightanswered 99.5%error rate 15.4% | 0.645645at the maker's list price | 1.701 question per request, 1 in flight |
| LFM2.5-24B-A2B default reasoning · chat Open · 24B MoE · 2.5B active | 82.6%452 of 547 right | not applicable | Queuedits Text tab run not scheduled yet: needs the model served again (Ümit #6527) | 13.2%14 “I don't know” of 106 not rightanswered 97.4%error rate 17.3% | 1.7641,764at the maker's list price | 74.21 question per request, 32 in flight |
| Gemma 4 E4B default reasoning · chat Open · 8.0B | 82.4%451 of 547 right | not applicable | 80.6%441 of 547 right | 9.8%10 “I don't know” of 102 not rightanswered 97.8%error rate 17.2% | 0.199199from GPU time | 8.391 question per request, 32 in flight |
| Qwen3.5-4B reasoning off · chat Open · 4.0B | 80.4%440 of 547 right | not applicable | 72.0%394 of 547 right | 3.6%4 “I don't know” of 110 not rightanswered 99.3%error rate 19.5% | 0.309309from GPU time | 9.741 question per request, 32 in flight |
| Qwen3-14B reasoning off · chat Open · 15B | 79.7%436 of 547 right | not applicable | 76.8%420 of 547 right | 1.8%2 “I don't know” of 110 not rightanswered 99.6%error rate 19.8% | 0.03131.17at a host's list price for the same weights | 2.8616 questions per request, 1 in flight |
| MiniCPM5-2B reasoning on · chat Open · 2.5B | 78.1%76.6%–79.0% min–max of 3 seedsseed 0: run 1051, 419 of 547 right; seed 1: run 1118, 431 of 547 right; seed 2: run 1119, 432 of 547 right. Baseline server capacity not retained; repeats use max-num-seqs 16. | not applicable | Not runEnglish only, no Text tab run | 19.7%28 “I don't know” of 142 not rightanswered 94.1%error rate 22.1%from run 1052 | 0.714714from GPU time | 17.51 question per request, 32 in flight |
| Gemma 4 E2B reasoning on · chat Open · 5.1B | 76.2%417 of 547 right | not applicable | Not runEnglish only, no Text tab run | 11.3%17 “I don't know” of 150 not rightanswered 96.3%error rate 25.2% | 0.187187from GPU time | 7.611 question per request, 32 in flight |
| Qwen3.5-2B reasoning on · chat Open · 2.0B | 75.9%415 of 547 right | not applicable | Not runEnglish only, no Text tab run | 9.9%14 “I don't know” of 142 not rightanswered 97.1%error rate 24.1% | 1.3611,361from GPU time | 54.41 question per request, 32 in flight |
| Gemma 4 E2B default reasoning · chat Open · 5.1B | 74.6%408 of 547 right | not applicable | Queuedits Text tab run in logic's GPU 1 queue (Ümit #6527, logic #6955) | 11.5%17 “I don't know” of 148 not rightanswered 96.5%error rate 24.8% | 0.115115from GPU time | 4.841 question per request, 32 in flight |
| Granite-4.2-3B reasoning on · chat Open · 3.0B | 74.4%407 of 547 right | not applicable | Pausedits Text tab run its Text run stopped at 741 of 2,198 so that other setups go first on logic's GPU 1; it resumes when logic loads Granite again (logic #7481, #7599) | 16.1%23 “I don't know” of 143 not rightanswered 95.8%error rate 22.9% | 2.7172,717from GPU time | 40.51 question per request, 32 in flight |
| LFM2.5-2.6B default reasoning · chat Open · 2.7B | 73.9%404 of 547 right | not applicable | 47.9%262 of 547 right | 15.7%21 “I don't know” of 134 not rightanswered 95.6%error rate 21.6% | 0.964964at the maker's list price | 15.01 question per request, 32 in flight |
| Erk-14B reasoning off · chat Open · 14B | 72.0%394 of 547 right | not applicable | 75.3%412 of 547 right | 2.1%3 “I don't know” of 144 not rightanswered 99.5%error rate 25.9% | 0.123123from GPU time | 2.8616 questions per request, 1 in flight |
| LFM3-1.7B reasoning off · chat Open · 1.7B | 71.8%70.9%–72.9% min–max of 3 seedsseed 0: run 1223, 399 of 547 right; seed 1: run 1302, 388 of 547 right; seed 2: run 1303, 391 of 547 right | not applicable | 40.2%220 of 547 right | 11.6%18 “I don't know” of 155 not rightanswered 96.7%error rate 25.9%from run 1225 | 0.355355at the maker's list price | 4.691 question per request, 32 in flight |
| LFM3-1.7B reasoning on · chat Open · 1.7B | 71.1%70.4%–71.7% min–max of 3 seedsseed 0: run 1091, 392 of 547 right; seed 1: run 1115, 390 of 547 right; seed 2: run 1116, 385 of 547 right | not applicable | 48.1%263 of 547 right | 15.6%26 “I don't know” of 167 not rightanswered 95.2%error rate 27.1%from run 1062 | 0.387387at the maker's list price | 5.241 question per request, 32 in flight |
| LFM2.5-8B-A1B default reasoning · chat Open · 8.3B MoE · 1.5B active | 70.4%385 of 547 right | not applicable | 27.1%148 of 547 right | 26.4%51 “I don't know” of 193 not rightanswered 90.5%error rate 28.7% | 0.517517from GPU time | 19.41 question per request, 32 in flight |
| MiniCPM5-2B reasoning off · chat Open · 2.5B | 67.8%66.7%–68.9% min–max of 3 seedsseed 0: run 1094, 370 of 547 right; seed 1: run 1300, 377 of 547 right; seed 2: run 1301, 365 of 547 right. Seeds 1 and 2 on logic's load of 15 November (vLLM v0.30.0, max-num-seqs 16), which logic gives as run 1094's setup (#9335); the server's settings are not recorded with a run. | not applicable | 7.1%39 of 547 right | 16.8%33 “I don't know” of 197 not rightanswered 89.2%error rate 33.6%from run 1095 | 0.429429from GPU time | 7.291 question per request, 32 in flight |
| Qwen3.5-2B reasoning off · chat Open · 2.0B | 67.3%368 of 547 right | not applicable | 36.7%201 of 547 right | 10.3%18 “I don't know” of 174 not rightanswered 96.3%error rate 29.6% | 0.280280from GPU time | 9.271 question per request, 32 in flight |
| Raon-1 nano reasoning off · chat Open · 3.1B | 53.7%294 of 547 right | not applicable | Queuedits Text tab run asked on its next load in logic's GPU 1 queue, whichever way its smoke came out (run 1154: 3 of 20 replies at the 32,768-token limit, 5 without a letter; Ümit #6515 via Zeki #8211) | 26.2%71 “I don't know” of 271 not rightanswered 86.8%error rate 42.1% | 0.235235from GPU time | 3.741 question per request, 32 in flight |
| GLiNER2.5 Multi Open · 0.3B | 16.3%89 of 547 right | not applicable | 16.8%92 of 547 right | 19.6%92 “I don't know” of 470 not rightanswered 83.2%error rate 83.1% | 0.0088.39at the maker's list price | 0.0091 question per request, 1 in flight |
Ranked by English accuracy, highest first: right answers of the 547 kept reading questions in each setup's first English run (an invalid answer counts as not right). Turkish accuracy: the setup's Text tab run on the same 547 questions, for context (Not run: an English-only setup, with no Text tab run; Paused: its Text tab run is stopped for now, as its row says; Queued: its Text tab run has not started yet, as its row says); the paired comparison is on English and Turkish. English non-hallucination: “I don't know” among the answers that were not right, in the setup's first English run with it offered as a sixth option; Running: a chat model's run still running; Not run: a decision model not run with it yet, or an English-only setup. English cost: $ per 1,000 questionsper 1M questions, with its basis under it; English latency: the median time per request, with the request size and the requests in flight under it, each from the first English run (Cost and Latency below). MiniCPM5-2B, LFM3-1.7B, LFM3-1.7B and MiniCPM5-2B are run with 3 sampling seeds: until all are in, the number is the first English run's, named with the seeds in so far; then it is the mean of the runs' accuracies, ranked by it, with their lowest and highest (min–max, a range, not a confidence interval) under it. The paired comparison with Turkish and the “I don't know” runs stay on their own runs. Each name opens its model page's English section (an English-only setup has no model page: its name opens its row here); its tooltip gives its settings and English run.
Decision Score (AA formula): 50 × (1 + accuracy − chosen-answer Brier) on the first English run, the chosen-answer Brier being the mean of (p − c)², p the probability given to the chosen answer and c 1 if it is right, else 0.
Artificial Analysis's formula applied to our questions. Not their Index. Decision models only: chat models and the I don't know runs are excluded.
Costs: API list prices and recorded token use. Setups on our own GPUs: the maker's list price for the same model (each value's tooltip names it); our hardware cost is not measured. On our GPUs, estimated at a public host's list price for the same weights (the host, the date read and any precision difference are on each setup's page): an API-equivalent estimate, not measured spending. On our GPUs with no public price for the same weights, estimated from GPU time: the run's request time x the GPUs its server held x Runpod's on-demand price for an RTX PRO 6000, $2.49 per GPU-hour (read 5 November 2026): a cloud-equivalent estimate, not measured spending.
Every decision model in English
| Model | English accuracyfirst English run; higher is better | Decision Score (AA formula)first English run; higher is better | Turkish accuracyText tab run, same 547; for context | English non-hallucinationof answers not right, % “I don't know” · higher is better; answered and error rate under it | English cost$ per 1,000 questionsper 1M questions, first English run; lower is better | English latencymedian seconds per request, first English run; lower is better |
|---|---|---|---|---|---|---|
| KnowLine-31B-Gen1.5 Open · 31B | 93.6%512 of 547 right | 94.33chosen-answer Brier 0.0494 | 94.3%516 of 547 right | 0.0%0 “I don't know” of 37 not rightanswered 100.0%error rate 6.8% | No list price | 1.121 question per request, 1 in flight |
| Copernicus-31B Open · 31B | 92.9%508 of 547 right | 93.66chosen-answer Brier 0.0556 | 94.5%517 of 547 right | 0.0%0 “I don't know” of 39 not rightanswered 100.0%error rate 7.1% | 0.142142from GPU time | 0.1881 question per request, 1 in flight |
| REAX 1.3 Open · 57B | 92.5%506 of 547 right | 93.75chosen-answer Brier 0.0500 | 94.0%514 of 547 right | 6.7%3 “I don't know” of 45 not rightanswered 99.5%error rate 7.7% | 0.169169from GPU time | 0.2291 question per request, 1 in flight |
| KnowLine-31B-Gen2 Open · 31B | 92.3%505 of 547 right | 93.37chosen-answer Brier 0.0558 | 94.5%517 of 547 right | 0.0%0 “I don't know” of 40 not rightanswered 100.0%error rate 7.3% | 0.127127from GPU time | 0.1711 question per request, 1 in flight |
| Karpo Decision 31B Open · 31B | 92.1%504 of 547 right | 93.23chosen-answer Brier 0.0568 | 94.9%519 of 547 right | 0.0%0 “I don't know” of 42 not rightanswered 100.0%error rate 7.7% | 0.145145from GPU time | 0.2071 question per request, 1 in flight |
| Gemini Decision 1 20261111 Closed · Efficient | 91.0%498 of 547 right | 92.31chosen-answer Brier 0.0642 | 92.9%508 of 547 right | 9.1%5 “I don't know” of 55 not rightanswered 99.1%error rate 9.2% | 0.04039.51at list prices | 0.2351 question per request, 1 in flight |
| Logits Sigma-1 Duo Open · 57B | 91.0%498 of 547 right | 92.55chosen-answer Brier 0.0595 | 93.4%511 of 547 right | 4.3%2 “I don't know” of 46 not rightanswered 99.6%error rate 8.1% | 0.191191from GPU time | 0.2631 question per request, 1 in flight |
| GLiDE 27B Open · 27B | 90.9%497 of 547 right | 92.36chosen-answer Brier 0.0613 | 91.2%499 of 547 right | 37.9%22 “I don't know” of 58 not rightanswered 96.0%error rate 6.9% | No list price | 0.5451 question per request, 1 in flight |
| d1-ultra Closed · Efficient | 90.7%496 of 547 right | 91.92chosen-answer Brier 0.0684 | 93.6%512 of 547 right | 56.7%51 “I don't know” of 90 not rightanswered 90.7%error rate 7.9% | 0.01413.98at list prices | 0.2201 question per request, 1 in flight |
| Sieve 31B v3 Open · 31B | 90.7%496 of 547 right | 92.51chosen-answer Brier 0.0565 | 93.4%511 of 547 right | 2.2%1 “I don't know” of 46 not rightanswered 99.8%error rate 8.2% | No list price | 1.061 question per request, 1 in flight |
| Turion Jev 31B Open · 31B | 90.7%496 of 547 right | 91.69chosen-answer Brier 0.0729 | 93.6%512 of 547 right | 8.5%4 “I don't know” of 47 not rightanswered 99.3%error rate 7.9% | No list price | 0.4451 question per request, 1 in flight |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | 90.5%495 of 547 right | 91.39chosen-answer Brier 0.0771 | 90.7%496 of 547 right | 15.2%10 “I don't know” of 66 not rightanswered 98.2%error rate 10.4% | ≤ 1.036≤ 1,036at list prices | 0.4841 question per request, 1 in flight |
| GLiDE v1.2 Open · 27B | 90.3%494 of 547 right | 91.96chosen-answer Brier 0.0639 | 90.5%495 of 547 right | 45.5%30 “I don't know” of 66 not rightanswered 94.5%error rate 7.0% | 0.01514.55at the maker's list price | 0.1041 question per request, 1 in flight |
| d1.2-ultra Closed · Efficient | 90.3%494 of 547 right | 91.84chosen-answer Brier 0.0664 | 90.9%497 of 547 right | 89.3%109 “I don't know” of 122 not rightanswered 80.1%error rate 3.0% | 0.01413.98at list prices | 0.2211 question per request, 1 in flight |
| Mammoth-1 Open · 27B | 90.1%493 of 547 right | 91.75chosen-answer Brier 0.0663 | 92.9%508 of 547 right | 50.7%37 “I don't know” of 73 not rightanswered 93.2%error rate 7.1% | 0.04040.07at the maker's list price | 0.1891 question per request, 1 in flight |
| d3 Open · 26B | 89.8%491 of 547 right | 91.65chosen-answer Brier 0.0646 | 91.0%498 of 547 right | 1.7%1 “I don't know” of 59 not rightanswered 99.8%error rate 10.6% | 0.05555.35from GPU time | 0.0741 question per request, 1 in flight |
| Drex Max Open · 27B | 89.6%490 of 547 right | 91.50chosen-answer Brier 0.0658 | 91.4%500 of 547 right | 3.2%2 “I don't know” of 62 not rightanswered 99.6%error rate 11.0% | 0.02423.56at the maker's list price | 0.0811 question per request, 1 in flight |
| Perplexity Decider 27B v1.3 Open · 27B | 89.6%490 of 547 right | 91.22chosen-answer Brier 0.0714 | 90.7%496 of 547 right | 8.8%5 “I don't know” of 57 not rightanswered 99.1%error rate 9.6% | 0.08382.63from GPU time | 0.1151 question per request, 1 in flight |
| d3-ultra Open · 26B | 89.4%489 of 547 right | 91.18chosen-answer Brier 0.0704 | 91.4%500 of 547 right | 8.2%5 “I don't know” of 61 not rightanswered 99.1%error rate 10.3% | 0.106106from GPU time | 0.1491 question per request, 1 in flight |
| d1.2 Closed · Efficient | 89.0%487 of 547 right | 90.51chosen-answer Brier 0.0801 | 88.8%486 of 547 right | 61.5%59 “I don't know” of 96 not rightanswered 89.2%error rate 7.6% | 0.0076.66at list prices | 0.2191 question per request, 1 in flight |
| Perplexity Decider 27B v1.2 DGX Spark Open · 27B | 88.7%485 of 547 right | 90.75chosen-answer Brier 0.0717 | 91.4%500 of 547 right | 3.0%2 “I don't know” of 66 not rightanswered 99.6%error rate 11.7% | 0.0087.85at the maker's list price | 0.1261 question per request, 1 in flight |
| Drex Max v1.1 Closed · Efficient (provisional) | 88.5%484 of 547 right | 90.81chosen-answer Brier 0.0685 | 90.1%493 of 547 right | 51.2%43 “I don't know” of 84 not rightanswered 92.1%error rate 8.1% | 0.02323.38at list prices | 0.3351 question per request, 1 in flight |
| Perplexity Decider 27B v1 Open · 27B | 88.5%484 of 547 right | 90.64chosen-answer Brier 0.0720 | 89.0%487 of 547 right | Not run | 0.01615.59at list prices | 0.2471 question per request, 1 in flight |
| Perplexity Decider 27B v1.2 API Open · 27B | 88.5%484 of 547 right | 90.64chosen-answer Brier 0.0721 | 91.0%498 of 547 right | 4.5%3 “I don't know” of 66 not rightanswered 99.5%error rate 11.6% | 0.0087.79at list prices | 0.2561 question per request, 1 in flight |
| d1.1 Closed · Efficient | 88.3%483 of 547 right | 90.14chosen-answer Brier 0.0802 | 88.3%483 of 547 right | Not run | 0.0076.66at list prices | 0.1731 question per request, 1 in flight |
| Jeffan-9B-1211 Open · 9.0B | 87.6%479 of 547 right | 89.05chosen-answer Brier 0.0946 | 86.5%473 of 547 right | 0.0%0 “I don't know” of 71 not rightanswered 100.0%error rate 13.0% | 0.04645.85from GPU time | 0.0671 question per request, 1 in flight |
| d1 Closed · Efficient | 86.8%475 of 547 right | 89.26chosen-answer Brier 0.0832 | 85.0%465 of 547 right | 2.9%2 “I don't know” of 69 not rightanswered 99.6%error rate 12.3% | 0.0076.66at list prices | 0.1801 question per request, 1 in flight |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | 86.8%475 of 547 right | 88.59chosen-answer Brier 0.0965 | 83.0%454 of 547 right | 2.6%2 “I don't know” of 76 not rightanswered 99.6%error rate 13.6% | No list price | 0.1561 question per request, 1 in flight |
| Clef Open · 27B | 86.7%474 of 547 right | 89.19chosen-answer Brier 0.0828 | 88.1%482 of 547 right | 1.3%1 “I don't know” of 76 not rightanswered 99.8%error rate 13.7% | 0.111111at the maker's list price | 0.2201 question per request, 1 in flight |
| KnowLine-12B-Gen1 Open · 12B | 86.7%474 of 547 right | 88.66chosen-answer Brier 0.0933 | 87.6%479 of 547 right | 1.3%1 “I don't know” of 77 not rightanswered 99.8%error rate 13.9% | 0.05756.86from GPU time | 0.0781 question per request, 1 in flight |
| GLiDE 9B Open · 7.9B | 86.3%472 of 547 right | 88.60chosen-answer Brier 0.0909 | 84.3%461 of 547 right | 16.7%14 “I don't know” of 84 not rightanswered 97.4%error rate 13.1% | No list price | 0.2071 question per request, 1 in flight |
| OneJev-27B Open · 27B | 86.3%472 of 547 right | 88.51chosen-answer Brier 0.0927 | 87.8%480 of 547 right | 4.0%3 “I don't know” of 75 not rightanswered 99.5%error rate 13.2% | 0.106106from GPU time | 0.1301 question per request, 1 in flight |
| Jev 1.14 Closed · Efficient | 86.1%471 of 547 right | 89.16chosen-answer Brier 0.0778 | 87.2%477 of 547 right | 4.0%3 “I don't know” of 75 not rightanswered 99.5%error rate 13.2% | 0.02726.64at list prices | 0.1811 question per request, 1 in flight |
| Cerno 9B v1.2 Open · 9.0B | 85.9%470 of 547 right | 88.42chosen-answer Brier 0.0907 | 83.5%457 of 547 right | 2.5%2 “I don't know” of 79 not rightanswered 99.6%error rate 14.1% | No list price | 0.2111 question per request, 1 in flight |
| Drex 2.0 Open · 35B MoE · 3.0B active | 85.9%470 of 547 right | 88.15chosen-answer Brier 0.0963 | 84.8%464 of 547 right | Not run | 0.01919.42at list prices | 0.2831 question per request, 1 in flight |
| Jev 1.13 Closed · Efficient | 85.9%470 of 547 right | 88.86chosen-answer Brier 0.0821 | 86.1%471 of 547 right | Not run | 0.02726.64at list prices | 0.1731 question per request, 1 in flight |
| KnowLine-9B-Gen1 Open · 9.0B | 85.9%470 of 547 right | 88.32chosen-answer Brier 0.0928 | 82.4%451 of 547 right | 1.3%1 “I don't know” of 77 not rightanswered 99.8%error rate 13.9% | 0.04241.55from GPU time | 0.0591 question per request, 1 in flight |
| SubQ Snap 1.1 Closed · Efficient | 85.9%470 of 547 right | 88.70chosen-answer Brier 0.0852 | 89.2%488 of 547 right | 25.0%21 “I don't know” of 84 not rightanswered 96.2%error rate 12.0% | 0.01615.59at list prices | 0.3811 question per request, 1 in flight |
| SubQ Snap Preview Closed · Efficient | 85.9%470 of 547 right | 88.70chosen-answer Brier 0.0852 | 89.9%492 of 547 right | Not run | 0.01919.48at list prices | 0.3811 question per request, 1 in flight |
| Adjudex 9B v5 Open · 9.0B | 85.7%469 of 547 right | 88.35chosen-answer Brier 0.0905 | 83.9%459 of 547 right | 0.0%0 “I don't know” of 77 not rightanswered 100.0%error rate 14.1% | 0.02524.93from GPU time | 0.0341 question per request, 1 in flight |
| Mercury Decide Closed · Efficient | 85.7%469 of 547 right | 88.15chosen-answer Brier 0.0943 | 87.0%476 of 547 right | 13.4%11 “I don't know” of 82 not rightanswered 98.0%error rate 13.2% | 0.0087.79at list prices | 0.3101 question per request, 1 in flight |
| Microsoft-Decision-1 Closed · Efficient | 85.7%469 of 547 right | 88.10chosen-answer Brier 0.0954 | 76.4%418 of 547 right | 3.8%3 “I don't know” of 80 not rightanswered 99.5%error rate 14.2% | 0.01414.01at list prices | 0.3151 question per request, 1 in flight |
| Elephant-1 Open · 9.0B | 85.4%467 of 547 right | 87.99chosen-answer Brier 0.0939 | 83.5%457 of 547 right | 25.6%23 “I don't know” of 90 not rightanswered 95.8%error rate 12.8% | 0.01414.02at the maker's list price | 0.0701 question per request, 1 in flight |
| Open-Jev-27B-v1.1 Open · 27B | 85.4%467 of 547 right | 77.18chosen-answer Brier 0.3102 | 86.5%473 of 547 right | 17.4%16 “I don't know” of 92 not rightanswered 97.1%error rate 14.3% | 0.351351from GPU time | 0.4951 question per request, 1 in flight |
| KnowLine-9B-Gen2 Open · 9.0B | 85.2%466 of 547 right | 87.38chosen-answer Brier 0.1044 | 82.8%453 of 547 right | 1.3%1 “I don't know” of 79 not rightanswered 99.8%error rate 14.3% | 0.03939.48from GPU time | 0.0561 question per request, 1 in flight |
| celeris-1-decision Closed · Efficient | 84.1%460 of 547 right | 87.29chosen-answer Brier 0.0952 | 87.4%478 of 547 right | 71.3%92 “I don't know” of 129 not rightanswered 83.2%error rate 8.1% | 0.01616.09at list prices | 0.3901 question per request, 1 in flight |
| GPT-6 Luna Decisions API Closed · Efficient | 84.1%460 of 547 right | unavailable | 86.7%474 of 547 right | 9.4%10 “I don't know” of 106 not rightanswered 98.2%error rate 17.9% | 0.04141.09at list prices | 0.3231 question per request, 4 in flight |
| Cygnet Open · 12B | 83.9%459 of 547 right | 87.03chosen-answer Brier 0.0984 | 85.6%468 of 547 right | 11.2%10 “I don't know” of 89 not rightanswered 98.2%error rate 14.7% | 0.04141.42at a host's list price for the same weights | 0.0431 question per request, 1 in flight |
| Winnow-12B Open · 12B | 83.9%459 of 547 right | 86.18chosen-answer Brier 0.1156 | 85.0%465 of 547 right | 15.0%15 “I don't know” of 100 not rightanswered 97.3%error rate 16.0% | 0.05049.59from GPU time | 0.0701 question per request, 1 in flight |
| Perplexity Decider Flash v1.3 Open · 4.0B | 83.7%458 of 547 right | 86.96chosen-answer Brier 0.0980 | 78.8%431 of 547 right | 7.2%7 “I don't know” of 97 not rightanswered 98.7%error rate 16.7% | 0.02323.17from GPU time | 0.0331 question per request, 1 in flight |
| Valen-Decision-1.5-9B Open · 9.0B | 83.7%458 of 547 right | 86.43chosen-answer Brier 0.1087 | 79.0%432 of 547 right | 1.2%1 “I don't know” of 85 not rightanswered 99.8%error rate 15.4% | 0.04140.55from GPU time | 0.0591 question per request, 1 in flight |
| celeris-1.1-decision Closed · Efficient | 83.7%458 of 547 right | 87.14chosen-answer Brier 0.0946 | Not runEnglish only, no Text tab run | 74.8%98 “I don't know” of 131 not rightanswered 82.1%error rate 7.3% | 0.0021.61at list prices | 0.3851 question per request, 1 in flight |
| Dexy-Decision-9B-V2 Open · 9.0B | 83.5%457 of 547 right | 87.01chosen-answer Brier 0.0953 | 83.4%456 of 547 right | 1.1%1 “I don't know” of 91 not rightanswered 99.8%error rate 16.5% | 0.02525.41from GPU time | 0.0361 question per request, 1 in flight |
| vera-medium-omni Open · 4.2B | 83.5%457 of 547 right | 86.34chosen-answer Brier 0.1086 | 79.5%435 of 547 right | 7.4%7 “I don't know” of 94 not rightanswered 98.7%error rate 16.1% | 0.03736.63from GPU time | 0.0541 question per request, 1 in flight |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | 82.8%453 of 547 right | 86.35chosen-answer Brier 0.1011 | 78.4%429 of 547 right | 0.0%0 “I don't know” of 94 not rightanswered 100.0%error rate 17.2% | No list price | 0.0751 question per request, 1 in flight |
| Clef-Flash Open · 9.0B | 82.3%450 of 547 right | 85.43chosen-answer Brier 0.1141 | 77.7%425 of 547 right | 5.3%5 “I don't know” of 95 not rightanswered 99.1%error rate 16.6% | 0.01817.53at the maker's list price | 0.1401 question per request, 1 in flight |
| Verdin-4B-v1.1 Open · 4.0B | 82.3%450 of 547 right | 85.97chosen-answer Brier 0.1032 | 77.5%424 of 547 right | 2.1%2 “I don't know” of 96 not rightanswered 99.6%error rate 17.2% | 0.03434.28from GPU time | 0.0521 question per request, 1 in flight |
| Perplexity Decider Flash v1.2 Open · 4.0B | 81.9%448 of 547 right | 86.04chosen-answer Brier 0.0982 | 76.6%419 of 547 right | 4.8%5 “I don't know” of 105 not rightanswered 99.1%error rate 18.5% | 0.0043.93at the maker's list price | 0.0371 question per request, 1 in flight |
| KnowLine 4B Gen5.5 Open · 4.0B | 80.8%442 of 547 right | 84.57chosen-answer Brier 0.1167 | 78.6%430 of 547 right | 0.9%1 “I don't know” of 106 not rightanswered 99.8%error rate 19.2% | No list price | 0.0991 question per request, 1 in flight |
| Chinese Decision Pointer 4B Open · 4.0B | 80.4%440 of 547 right | 84.66chosen-answer Brier 0.1111 | 75.7%414 of 547 right | 3.7%4 “I don't know” of 108 not rightanswered 99.3%error rate 19.2% | 0.02727.21from GPU time | 0.0381 question per request, 1 in flight |
| Metask-Jev-4B Open · 4.5B | 77.1%422 of 547 right | 82.21chosen-answer Brier 0.1273 | 67.1%367 of 547 right | 1.6%2 “I don't know” of 125 not rightanswered 99.6%error rate 22.6% | 0.05959.11from GPU time | 0.0851 question per request, 1 in flight |
| KnowLine-2B-Gen1 Open · 2.0B | 71.3%390 of 547 right | 77.82chosen-answer Brier 0.1567 | 58.0%317 of 547 right | 22.0%38 “I don't know” of 173 not rightanswered 93.1%error rate 26.5% | No list price | 0.0621 question per request, 1 in flight |
| GLiDE 2B Open · 1.9B | 70.0%383 of 547 right | 76.88chosen-answer Brier 0.1626 | 54.3%297 of 547 right | 26.3%47 “I don't know” of 179 not rightanswered 91.4%error rate 26.4% | No list price | 0.0351 question per request, 1 in flight |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | 69.7%381 of 547 right | 76.66chosen-answer Brier 0.1633 | 55.8%305 of 547 right | 16.9%32 “I don't know” of 189 not rightanswered 94.1%error rate 30.5% | No list price | 0.0401 question per request, 1 in flight |
| d1-2B Open · 1.9B | 65.6%359 of 547 right | 74.16chosen-answer Brier 0.1731 | 41.9%229 of 547 right | 60.9%151 “I don't know” of 248 not rightanswered 72.4%error rate 24.5% | 0.0088.08from GPU time | 0.0111 question per request, 1 in flight |
| GLiDE 0.8B Open · 0.8B | 60.7%332 of 547 right | 70.59chosen-answer Brier 0.1951 | 46.1%252 of 547 right | 21.7%49 “I don't know” of 226 not rightanswered 91.0%error rate 35.5% | No list price | 0.0231 question per request, 1 in flight |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | 59.6%326 of 547 right | 70.09chosen-answer Brier 0.1942 | 37.3%204 of 547 right | 5.2%12 “I don't know” of 229 not rightanswered 97.8%error rate 40.6% | No list price | 0.0281 question per request, 1 in flight |
| GLiDE 0.4B Open · 0.4B | 59.4%325 of 547 right | 69.84chosen-answer Brier 0.1973 | 25.8%141 of 547 right | 53.1%153 “I don't know” of 288 not rightanswered 72.0%error rate 34.3% | No list price | 0.0141 question per request, 1 in flight |
| OneJev1.6-0.8B Open · 0.8B | 59.4%325 of 547 right | 69.82chosen-answer Brier 0.1978 | 42.8%234 of 547 right | 28.4%74 “I don't know” of 261 not rightanswered 86.5%error rate 39.5% | No list price | 0.0181 question per request, 1 in flight |
| KnowLine-0.8B-Gen1 Open · 0.8B | 58.9%322 of 547 right | 69.65chosen-answer Brier 0.1956 | 40.2%220 of 547 right | 10.7%26 “I don't know” of 243 not rightanswered 95.2%error rate 41.7% | No list price | 0.0331 question per request, 1 in flight |
| GLiClass v4 Edge Open · 0.4B | 58.3%319 of 547 right | 69.05chosen-answer Brier 0.2021 | 26.9%147 of 547 right | 21.0%52 “I don't know” of 248 not rightanswered 90.5%error rate 39.6% | No list price | 0.0171 question per request, 1 in flight |
| Decider 1 Closed · Efficient | 55.8%305 of 547 right | 67.47chosen-answer Brier 0.2082 | 36.6%200 of 547 right | Not run | 0.01514.95at list prices | 0.6991 question per request, 1 in flight |
| Strands Decider 2B Open · 2.0B | 45.5%249 of 547 right | 64.08chosen-answer Brier 0.1736 | 39.1%214 of 547 right | 0.7%2 “I don't know” of 303 not rightanswered 99.6%error rate 55.2% | 0.01818.15from GPU time | 0.0261 question per request, 1 in flight |
| Laya Open · 0.4B | 22.9%125 of 547 right | 51.57chosen-answer Brier 0.1971 | 20.5%112 of 547 right | 18.4%83 “I don't know” of 450 not rightanswered 84.8%error rate 79.1% | 0.0087.84from GPU time | 0.0111 question per request, 1 in flight |
| GLiNER2.5-multi-Decide Open · 0.3B | 21.2%116 of 547 right | 47.58chosen-answer Brier 0.2604 | 20.7%113 of 547 right | 16.2%70 “I don't know” of 433 not rightanswered 87.2%error rate 76.1% | 0.0076.73from GPU time | 0.0091 question per request, 1 in flight |
| Laya Multilingual Open · 0.3B | 15.4%84 of 547 right | 39.32chosen-answer Brier 0.3671 | 17.0%93 of 547 right | 42.6%208 “I don't know” of 488 not rightanswered 62.0%error rate 82.6% | 0.01312.62from GPU time | 0.0181 question per request, 1 in flight |
Ranked by English accuracy, highest first: right answers of the 547 kept reading questions in each setup's first English run (an invalid answer counts as not right). Turkish accuracy: the setup's Text tab run on the same 547 questions, for context (Not run: an English-only setup, with no Text tab run); the paired comparison is on English and Turkish. English non-hallucination: “I don't know” among the answers that were not right, in the setup's first English run with it offered as a sixth option; Running: a chat model's run still running; Not run: a decision model not run with it yet, or an English-only setup. English cost: $ per 1,000 questionsper 1M questions, with its basis under it; English latency: the median time per request, with the request size and the requests in flight under it, each from the first English run (Cost and Latency below). No list price: a setup on our own hardware whose maker lists no price for the same model. Each name opens its model page's English section (an English-only setup has no model page: its name opens its row here); its tooltip gives its settings and English run.
Decision Score (AA formula): 50 × (1 + accuracy − chosen-answer Brier) on the first English run, the chosen-answer Brier being the mean of (p − c)², p the probability given to the chosen answer and c 1 if it is right, else 0. GPT-6 Luna · Decisions API: unavailable, 2 of 547 answers without a probability for the chosen answer.
Artificial Analysis's formula applied to our questions. Not their Index. Decision models only: chat models and the I don't know runs are excluded.
Costs: API list prices and recorded token use. Setups on our own GPUs: the maker's list price for the same model (each value's tooltip names it); our hardware cost is not measured. On our GPUs, estimated at a public host's list price for the same weights (the host, the date read and any precision difference are on each setup's page): an API-equivalent estimate, not measured spending. On our GPUs with no public price for the same weights, estimated from GPU time: the run's request time x the GPUs its server held x Runpod's on-demand price for an RTX PRO 6000, $2.49 per GPU-hour (read 5 November 2026): a cloud-equivalent estimate, not measured spending. No list price: on our own GPUs, with no estimate. ≤ Upper-bound estimate: one $0.001 web search counted per request; Perplexity bills only completed searches.
Accuracy in English · 547 machine-translated questions
Share of the 547 kept reading questions answered right in English, then the right answers of 547: each of the 142 setups' first English run, or the mean of a setup's three seed runs · higher is better
Claude Gemini GPT Qwen KnowLine Grok Copernicus DeepSeek REAX Kimi Karpo Logits Fastino Liquid Sieve Turion Jev Gemma Perplexity Mammoth vLLM-SR Drex GLM Muse MiMo Inkling Jeffan StartLux Clef OneJev Jev Cerno SubQ Adjudex Inception Microsoft Elephant Open-Jev MiniMax Celeris Cygnet Winnow Valen Dexy vera Verdin Chinese Decision Pointer MiniCPM Metask Granite Erk GLiClass meraGPT Raon Strands Laya
Accuracy in English · 547 machine-translated questions
Share of the 547 kept reading questions answered right in English, then the right answers of 547: each of the 66 chat model setups' first English run, or the mean of a setup's three seed runs · higher is better
Claude Gemini GPT Qwen Grok DeepSeek Kimi Gemma GLM Muse MiMo Inkling MiniMax Liquid MiniCPM Granite Erk Raon Fastino
Accuracy in English · 547 machine-translated questions
Share of the 547 kept reading questions answered right in English, then the right answers of 547: each of the 76 decision models' first English run · higher is better
KnowLine Copernicus REAX Karpo Gemini Logits Fastino Liquid Sieve Turion Jev Perplexity Mammoth vLLM-SR Drex Jeffan StartLux Clef OneJev Jev Cerno SubQ Adjudex Inception Microsoft Elephant Open-Jev Celeris GPT Cygnet Winnow Valen Dexy vera Verdin Chinese Decision Pointer Metask GLiClass meraGPT Strands Laya
Non-hallucination
Holding back by confidence
Confidence at the threshold is not measured in English.
Decision models
Not measured in English.
Chat models
Not measured in English.
Holding back with an “I don't know” option in English
“I don't know” among the answers that were not right, with it offered as a sixth option in English, on the 547 kept questions; beside it the share answered (right + wrong of 547) and the error rate among the answers (wrong of right + wrong) · higher non-hallucination is better, read with the share answered; a model that always chose the option would reach 100% with no right answers
Decision models
Each of the 70 decision models' first English run with it.
Chose “I don't know” on fewer than 1% of the 547 questions: KnowLine-31B-Gen1.5, 0 times; Copernicus-31B, 0 times; REAX 1.3, 3 times; KnowLine-31B-Gen2, 0 times; Karpo Decision 31B, 0 times; Gemini Decision 1 · 20261111, 5 times; Logits Sigma-1 Duo, 2 times; Sieve 31B v3, 1 time; Turion Jev 31B, 4 times; d3, 1 time; Drex Max, 2 times; Perplexity Decider 27B v1.3, 5 times; d3-ultra, 5 times; Perplexity Decider 27B v1.2 · DGX Spark, 2 times; Perplexity Decider 27B v1.2 · API, 3 times; Jeffan-9B-1211, 0 times; d1, 2 times; StartLux-Decision-v1.1-9B-Instant, 2 times; Clef, 1 time; KnowLine-12B-Gen1, 1 time; OneJev-27B, 3 times; Jev 1.14, 3 times; Cerno 9B v1.2, 2 times; KnowLine-9B-Gen1, 1 time; Adjudex 9B v5, 0 times; Microsoft-Decision-1, 3 times; KnowLine-9B-Gen2, 1 time; Valen-Decision-1.5-9B, 1 time; Dexy-Decision-9B-V2, 1 time; StartLux-Decision-v1.1-4B-Instant, 0 times; Clef-Flash, 5 times; Verdin-4B-v1.1, 2 times; Perplexity Decider Flash v1.2, 5 times; KnowLine 4B Gen5.5, 1 time; Chinese Decision Pointer 4B, 4 times; Metask-Jev-4B, 2 times; Strands Decider 2B, 2 times. Their rate here is low (9.1% or less) because they almost never chose the option; confidence is not measured in English; the Turkish reference below shows their Text tab runs at the threshold.
Liquid Celeris Fastino Drex Mammoth Laya OneJev Elephant SubQ KnowLine GLiClass Open-Jev StartLux Perplexity Winnow Inception Cygnet GPT Gemini Turion Jev vLLM-SR vera REAX Clef Logits Jev Microsoft Chinese Decision Pointer Cerno Sieve Verdin Metask Valen Dexy Strands Adjudex Copernicus Jeffan Karpo Decision models not run with “I don't know” in English yet: Perplexity Decider 27B v1, d1.1, Drex 2.0, Jev 1.13, SubQ Snap Preview, Decider 1.
Chat models
Each of the 65 chat model setups' first English run with it.
Claude Grok Liquid Raon Gemini MiniCPM Fastino GPT Granite Gemma Qwen Kimi DeepSeek GLM Muse MiniMax Erk MiMo Inkling Not run, as yes/no scoring cannot offer it: Qwen3.8-27B · yes/no scoring.
Decision Score (AA formula), decision models
| Model | Decision Score (AA formula) | Accuracy | Chosen-answer Brier |
|---|---|---|---|
| KnowLine-31B-Gen1.5 Open · 31B | 94.33 | 93.60% | 0.0494 |
| Copernicus-31B Open · 31B | 93.66 | 92.87% | 0.0556 |
| REAX 1.3 Open · 57B | 93.75 | 92.50% | 0.0500 |
| KnowLine-31B-Gen2 Open · 31B | 93.37 | 92.32% | 0.0558 |
| Karpo Decision 31B Open · 31B | 93.23 | 92.14% | 0.0568 |
| Gemini Decision 1 20261111 Closed · Efficient | 92.31 | 91.04% | 0.0642 |
| Logits Sigma-1 Duo Open · 57B | 92.55 | 91.04% | 0.0595 |
| GLiDE 27B Open · 27B | 92.36 | 90.86% | 0.0613 |
| d1-ultra Closed · Efficient | 91.92 | 90.68% | 0.0684 |
| Sieve 31B v3 Open · 31B | 92.51 | 90.68% | 0.0565 |
| Turion Jev 31B Open · 31B | 91.69 | 90.68% | 0.0729 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | 91.39 | 90.49% | 0.0771 |
| GLiDE v1.2 Open · 27B | 91.96 | 90.31% | 0.0639 |
| d1.2-ultra Closed · Efficient | 91.84 | 90.31% | 0.0664 |
| Mammoth-1 Open · 27B | 91.75 | 90.13% | 0.0663 |
| d3 Open · 26B | 91.65 | 89.76% | 0.0646 |
| Drex Max Open · 27B | 91.50 | 89.58% | 0.0658 |
| Perplexity Decider 27B v1.3 Open · 27B | 91.22 | 89.58% | 0.0714 |
| d3-ultra Open · 26B | 91.18 | 89.40% | 0.0704 |
| d1.2 Closed · Efficient | 90.51 | 89.03% | 0.0801 |
| Perplexity Decider 27B v1.2 DGX Spark Open · 27B | 90.75 | 88.67% | 0.0717 |
| Drex Max v1.1 Closed · Efficient (provisional) | 90.81 | 88.48% | 0.0685 |
| Perplexity Decider 27B v1 Open · 27B | 90.64 | 88.48% | 0.0720 |
| Perplexity Decider 27B v1.2 API Open · 27B | 90.64 | 88.48% | 0.0721 |
| d1.1 Closed · Efficient | 90.14 | 88.30% | 0.0802 |
| Jeffan-9B-1211 Open · 9.0B | 89.05 | 87.57% | 0.0946 |
| d1 Closed · Efficient | 89.26 | 86.84% | 0.0832 |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | 88.59 | 86.84% | 0.0965 |
| Clef Open · 27B | 89.19 | 86.65% | 0.0828 |
| KnowLine-12B-Gen1 Open · 12B | 88.66 | 86.65% | 0.0933 |
| GLiDE 9B Open · 7.9B | 88.60 | 86.29% | 0.0909 |
| OneJev-27B Open · 27B | 88.51 | 86.29% | 0.0927 |
| Jev 1.14 Closed · Efficient | 89.16 | 86.11% | 0.0778 |
| Cerno 9B v1.2 Open · 9.0B | 88.42 | 85.92% | 0.0907 |
| Drex 2.0 Open · 35B MoE · 3.0B active | 88.15 | 85.92% | 0.0963 |
| Jev 1.13 Closed · Efficient | 88.86 | 85.92% | 0.0821 |
| KnowLine-9B-Gen1 Open · 9.0B | 88.32 | 85.92% | 0.0928 |
| SubQ Snap 1.1 Closed · Efficient | 88.70 | 85.92% | 0.0852 |
| SubQ Snap Preview Closed · Efficient | 88.70 | 85.92% | 0.0852 |
| Adjudex 9B v5 Open · 9.0B | 88.35 | 85.74% | 0.0905 |
| Mercury Decide Closed · Efficient | 88.15 | 85.74% | 0.0943 |
| Microsoft-Decision-1 Closed · Efficient | 88.10 | 85.74% | 0.0954 |
| Elephant-1 Open · 9.0B | 87.99 | 85.37% | 0.0939 |
| Open-Jev-27B-v1.1 Open · 27B | 77.18 | 85.37% | 0.3102 |
| KnowLine-9B-Gen2 Open · 9.0B | 87.38 | 85.19% | 0.1044 |
| celeris-1-decision Closed · Efficient | 87.29 | 84.10% | 0.0952 |
| GPT-6 Luna Decisions API Closed · Efficient | unavailable | unavailable | unavailable |
| Cygnet Open · 12B | 87.03 | 83.91% | 0.0984 |
| Winnow-12B Open · 12B | 86.18 | 83.91% | 0.1156 |
| Perplexity Decider Flash v1.3 Open · 4.0B | 86.96 | 83.73% | 0.0980 |
| Valen-Decision-1.5-9B Open · 9.0B | 86.43 | 83.73% | 0.1087 |
| celeris-1.1-decision Closed · Efficient | 87.14 | 83.73% | 0.0946 |
| Dexy-Decision-9B-V2 Open · 9.0B | 87.01 | 83.55% | 0.0953 |
| vera-medium-omni Open · 4.2B | 86.34 | 83.55% | 0.1086 |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | 86.35 | 82.82% | 0.1011 |
| Clef-Flash Open · 9.0B | 85.43 | 82.27% | 0.1141 |
| Verdin-4B-v1.1 Open · 4.0B | 85.97 | 82.27% | 0.1032 |
| Perplexity Decider Flash v1.2 Open · 4.0B | 86.04 | 81.90% | 0.0982 |
| KnowLine 4B Gen5.5 Open · 4.0B | 84.57 | 80.80% | 0.1167 |
| Chinese Decision Pointer 4B Open · 4.0B | 84.66 | 80.44% | 0.1111 |
| Metask-Jev-4B Open · 4.5B | 82.21 | 77.15% | 0.1273 |
| KnowLine-2B-Gen1 Open · 2.0B | 77.82 | 71.30% | 0.1567 |
| GLiDE 2B Open · 1.9B | 76.88 | 70.02% | 0.1626 |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | 76.66 | 69.65% | 0.1633 |
| d1-2B Open · 1.9B | 74.16 | 65.63% | 0.1731 |
| GLiDE 0.8B Open · 0.8B | 70.59 | 60.69% | 0.1951 |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | 70.09 | 59.60% | 0.1942 |
| GLiDE 0.4B Open · 0.4B | 69.84 | 59.41% | 0.1973 |
| OneJev1.6-0.8B Open · 0.8B | 69.82 | 59.41% | 0.1978 |
| KnowLine-0.8B-Gen1 Open · 0.8B | 69.65 | 58.87% | 0.1956 |
| GLiClass v4 Edge Open · 0.4B | 69.05 | 58.32% | 0.2021 |
| Decider 1 Closed · Efficient | 67.47 | 55.76% | 0.2082 |
| Strands Decider 2B Open · 2.0B | 64.08 | 45.52% | 0.1736 |
| Laya Open · 0.4B | 51.57 | 22.85% | 0.1971 |
| GLiNER2.5-multi-Decide Open · 0.3B | 47.58 | 21.21% | 0.2604 |
| Laya Multilingual Open · 0.3B | 39.32 | 15.36% | 0.3671 |
Decision Score (AA formula): 50 × (1 + accuracy − chosen-answer Brier) on the first English run, the chosen-answer Brier being the mean of (p − c)², p the probability given to the chosen answer and c 1 if it is right, else 0. GPT-6 Luna · Decisions API: unavailable, 2 of 547 answers without a probability for the chosen answer.
Artificial Analysis's formula applied to our questions. Not their Index. Decision models only: chat models and the I don't know runs are excluded.
Turkish reference (the Text tab runs)
The answers each setup gave at 99% confidence or more, in its Text tab run on the Turkish questions, for reference, the bar their share of the 2,198 questions; how many of them were wrong; and its ranking score (AUROC: how often a right answer has higher confidence than a wrong one, 0.5 chance, 1 perfect). No wrong answer at 99% first, by answers kept, then by the share wrong among the kept answers; a setup with no answers at 99% last · more kept and fewer wrong is better
Decision models
Confidence: option probabilities.
Karpo Perplexity Gemini Mammoth vLLM-SR Fastino REAX Drex Elephant KnowLine Copernicus Liquid Logits Turion Jev Verdin StartLux Valen vera Cygnet OneJev Sieve Jev SubQ Winnow Dexy GPT Adjudex Chinese Decision Pointer Inception Microsoft Laya Celeris Cerno Clef meraGPT GLiClass Jeffan Metask Open-Jev Strands
Chat models
Confidence: the confidence the model states for 39; option probabilities for 9; agreement across ten answers for 1.
Inkling GLM Claude Qwen Muse Gemini GPT DeepSeek Erk MiMo Kimi MiniMax Gemma Fastino Grok Confidence not measured (a plain answer line, no confidence): Qwen3.5-4B · reasoning on · chat, Gemma 4 E4B · default reasoning · chat, Qwen3.5-4B · reasoning off · chat, LFM2.5-2.6B · default reasoning · chat, LFM3-1.7B · reasoning off · chat, LFM3-1.7B · reasoning on · chat, LFM2.5-8B-A1B · default reasoning · chat, MiniCPM5-2B · reasoning off · chat and Qwen3.5-2B · reasoning off · chat.
Non-hallucination
Holding back by confidence
Confidence at the threshold is not measured in English.
Chat models
Not measured in English.
Holding back with an “I don't know” option in English
“I don't know” among the answers that were not right, with it offered as a sixth option in English, on the 547 kept questions; beside it the share answered (right + wrong of 547) and the error rate among the answers (wrong of right + wrong) · higher non-hallucination is better, read with the share answered; a model that always chose the option would reach 100% with no right answers
Chat models
Each of the 65 chat model setups' first English run with it.
Claude Grok Liquid Raon Gemini MiniCPM Fastino GPT Granite Gemma Qwen Kimi DeepSeek GLM Muse MiniMax Erk MiMo Inkling Not run, as yes/no scoring cannot offer it: Qwen3.8-27B · yes/no scoring.
Turkish reference (the Text tab runs)
The answers each setup gave at 99% confidence or more, in its Text tab run on the Turkish questions, for reference, the bar their share of the 2,198 questions; how many of them were wrong; and its ranking score (AUROC: how often a right answer has higher confidence than a wrong one, 0.5 chance, 1 perfect). No wrong answer at 99% first, by answers kept, then by the share wrong among the kept answers; a setup with no answers at 99% last · more kept and fewer wrong is better
Chat models
Confidence: the confidence the model states for 39; option probabilities for 9; agreement across ten answers for 1.
Inkling GLM Claude Qwen Muse Gemini GPT DeepSeek Erk MiMo Kimi MiniMax Gemma Fastino Grok Confidence not measured (a plain answer line, no confidence): Qwen3.5-4B · reasoning on · chat, Gemma 4 E4B · default reasoning · chat, Qwen3.5-4B · reasoning off · chat, LFM2.5-2.6B · default reasoning · chat, LFM3-1.7B · reasoning off · chat, LFM3-1.7B · reasoning on · chat, LFM2.5-8B-A1B · default reasoning · chat, MiniCPM5-2B · reasoning off · chat and Qwen3.5-2B · reasoning off · chat.
Non-hallucination
Holding back by confidence
Confidence at the threshold is not measured in English.
Decision models
Not measured in English.
Holding back with an “I don't know” option in English
“I don't know” among the answers that were not right, with it offered as a sixth option in English, on the 547 kept questions; beside it the share answered (right + wrong of 547) and the error rate among the answers (wrong of right + wrong) · higher non-hallucination is better, read with the share answered; a model that always chose the option would reach 100% with no right answers
Decision models
Each of the 70 decision models' first English run with it.
Chose “I don't know” on fewer than 1% of the 547 questions: KnowLine-31B-Gen1.5, 0 times; Copernicus-31B, 0 times; REAX 1.3, 3 times; KnowLine-31B-Gen2, 0 times; Karpo Decision 31B, 0 times; Gemini Decision 1 · 20261111, 5 times; Logits Sigma-1 Duo, 2 times; Sieve 31B v3, 1 time; Turion Jev 31B, 4 times; d3, 1 time; Drex Max, 2 times; Perplexity Decider 27B v1.3, 5 times; d3-ultra, 5 times; Perplexity Decider 27B v1.2 · DGX Spark, 2 times; Perplexity Decider 27B v1.2 · API, 3 times; Jeffan-9B-1211, 0 times; d1, 2 times; StartLux-Decision-v1.1-9B-Instant, 2 times; Clef, 1 time; KnowLine-12B-Gen1, 1 time; OneJev-27B, 3 times; Jev 1.14, 3 times; Cerno 9B v1.2, 2 times; KnowLine-9B-Gen1, 1 time; Adjudex 9B v5, 0 times; Microsoft-Decision-1, 3 times; KnowLine-9B-Gen2, 1 time; Valen-Decision-1.5-9B, 1 time; Dexy-Decision-9B-V2, 1 time; StartLux-Decision-v1.1-4B-Instant, 0 times; Clef-Flash, 5 times; Verdin-4B-v1.1, 2 times; Perplexity Decider Flash v1.2, 5 times; KnowLine 4B Gen5.5, 1 time; Chinese Decision Pointer 4B, 4 times; Metask-Jev-4B, 2 times; Strands Decider 2B, 2 times. Their rate here is low (9.1% or less) because they almost never chose the option; confidence is not measured in English; the Turkish reference below shows their Text tab runs at the threshold.
Liquid Celeris Fastino Drex Mammoth Laya OneJev Elephant SubQ KnowLine GLiClass Open-Jev StartLux Perplexity Winnow Inception Cygnet GPT Gemini Turion Jev vLLM-SR vera REAX Clef Logits Jev Microsoft Chinese Decision Pointer Cerno Sieve Verdin Metask Valen Dexy Strands Adjudex Copernicus Jeffan Karpo Decision models not run with “I don't know” in English yet: Perplexity Decider 27B v1, d1.1, Drex 2.0, Jev 1.13, SubQ Snap Preview, Decider 1.
Decision Score (AA formula), decision models
| Model | Decision Score (AA formula) | Accuracy | Chosen-answer Brier |
|---|---|---|---|
| KnowLine-31B-Gen1.5 Open · 31B | 94.33 | 93.60% | 0.0494 |
| Copernicus-31B Open · 31B | 93.66 | 92.87% | 0.0556 |
| REAX 1.3 Open · 57B | 93.75 | 92.50% | 0.0500 |
| KnowLine-31B-Gen2 Open · 31B | 93.37 | 92.32% | 0.0558 |
| Karpo Decision 31B Open · 31B | 93.23 | 92.14% | 0.0568 |
| Gemini Decision 1 20261111 Closed · Efficient | 92.31 | 91.04% | 0.0642 |
| Logits Sigma-1 Duo Open · 57B | 92.55 | 91.04% | 0.0595 |
| GLiDE 27B Open · 27B | 92.36 | 90.86% | 0.0613 |
| d1-ultra Closed · Efficient | 91.92 | 90.68% | 0.0684 |
| Sieve 31B v3 Open · 31B | 92.51 | 90.68% | 0.0565 |
| Turion Jev 31B Open · 31B | 91.69 | 90.68% | 0.0729 |
| Perplexity Decider 27B v1.2 API · with web Open · 27B | 91.39 | 90.49% | 0.0771 |
| GLiDE v1.2 Open · 27B | 91.96 | 90.31% | 0.0639 |
| d1.2-ultra Closed · Efficient | 91.84 | 90.31% | 0.0664 |
| Mammoth-1 Open · 27B | 91.75 | 90.13% | 0.0663 |
| d3 Open · 26B | 91.65 | 89.76% | 0.0646 |
| Drex Max Open · 27B | 91.50 | 89.58% | 0.0658 |
| Perplexity Decider 27B v1.3 Open · 27B | 91.22 | 89.58% | 0.0714 |
| d3-ultra Open · 26B | 91.18 | 89.40% | 0.0704 |
| d1.2 Closed · Efficient | 90.51 | 89.03% | 0.0801 |
| Perplexity Decider 27B v1.2 DGX Spark Open · 27B | 90.75 | 88.67% | 0.0717 |
| Drex Max v1.1 Closed · Efficient (provisional) | 90.81 | 88.48% | 0.0685 |
| Perplexity Decider 27B v1 Open · 27B | 90.64 | 88.48% | 0.0720 |
| Perplexity Decider 27B v1.2 API Open · 27B | 90.64 | 88.48% | 0.0721 |
| d1.1 Closed · Efficient | 90.14 | 88.30% | 0.0802 |
| Jeffan-9B-1211 Open · 9.0B | 89.05 | 87.57% | 0.0946 |
| d1 Closed · Efficient | 89.26 | 86.84% | 0.0832 |
| StartLux-Decision-v1.1-9B-Instant Open · 9.0B | 88.59 | 86.84% | 0.0965 |
| Clef Open · 27B | 89.19 | 86.65% | 0.0828 |
| KnowLine-12B-Gen1 Open · 12B | 88.66 | 86.65% | 0.0933 |
| GLiDE 9B Open · 7.9B | 88.60 | 86.29% | 0.0909 |
| OneJev-27B Open · 27B | 88.51 | 86.29% | 0.0927 |
| Jev 1.14 Closed · Efficient | 89.16 | 86.11% | 0.0778 |
| Cerno 9B v1.2 Open · 9.0B | 88.42 | 85.92% | 0.0907 |
| Drex 2.0 Open · 35B MoE · 3.0B active | 88.15 | 85.92% | 0.0963 |
| Jev 1.13 Closed · Efficient | 88.86 | 85.92% | 0.0821 |
| KnowLine-9B-Gen1 Open · 9.0B | 88.32 | 85.92% | 0.0928 |
| SubQ Snap 1.1 Closed · Efficient | 88.70 | 85.92% | 0.0852 |
| SubQ Snap Preview Closed · Efficient | 88.70 | 85.92% | 0.0852 |
| Adjudex 9B v5 Open · 9.0B | 88.35 | 85.74% | 0.0905 |
| Mercury Decide Closed · Efficient | 88.15 | 85.74% | 0.0943 |
| Microsoft-Decision-1 Closed · Efficient | 88.10 | 85.74% | 0.0954 |
| Elephant-1 Open · 9.0B | 87.99 | 85.37% | 0.0939 |
| Open-Jev-27B-v1.1 Open · 27B | 77.18 | 85.37% | 0.3102 |
| KnowLine-9B-Gen2 Open · 9.0B | 87.38 | 85.19% | 0.1044 |
| celeris-1-decision Closed · Efficient | 87.29 | 84.10% | 0.0952 |
| GPT-6 Luna Decisions API Closed · Efficient | unavailable | unavailable | unavailable |
| Cygnet Open · 12B | 87.03 | 83.91% | 0.0984 |
| Winnow-12B Open · 12B | 86.18 | 83.91% | 0.1156 |
| Perplexity Decider Flash v1.3 Open · 4.0B | 86.96 | 83.73% | 0.0980 |
| Valen-Decision-1.5-9B Open · 9.0B | 86.43 | 83.73% | 0.1087 |
| celeris-1.1-decision Closed · Efficient | 87.14 | 83.73% | 0.0946 |
| Dexy-Decision-9B-V2 Open · 9.0B | 87.01 | 83.55% | 0.0953 |
| vera-medium-omni Open · 4.2B | 86.34 | 83.55% | 0.1086 |
| StartLux-Decision-v1.1-4B-Instant Open · 4.0B | 86.35 | 82.82% | 0.1011 |
| Clef-Flash Open · 9.0B | 85.43 | 82.27% | 0.1141 |
| Verdin-4B-v1.1 Open · 4.0B | 85.97 | 82.27% | 0.1032 |
| Perplexity Decider Flash v1.2 Open · 4.0B | 86.04 | 81.90% | 0.0982 |
| KnowLine 4B Gen5.5 Open · 4.0B | 84.57 | 80.80% | 0.1167 |
| Chinese Decision Pointer 4B Open · 4.0B | 84.66 | 80.44% | 0.1111 |
| Metask-Jev-4B Open · 4.5B | 82.21 | 77.15% | 0.1273 |
| KnowLine-2B-Gen1 Open · 2.0B | 77.82 | 71.30% | 0.1567 |
| GLiDE 2B Open · 1.9B | 76.88 | 70.02% | 0.1626 |
| StartLux-Decision-v1.1-2B-Instant Open · 2.0B | 76.66 | 69.65% | 0.1633 |
| d1-2B Open · 1.9B | 74.16 | 65.63% | 0.1731 |
| GLiDE 0.8B Open · 0.8B | 70.59 | 60.69% | 0.1951 |
| StartLux-Decision-v1.1-0.8B-Instant Open · 0.8B | 70.09 | 59.60% | 0.1942 |
| GLiDE 0.4B Open · 0.4B | 69.84 | 59.41% | 0.1973 |
| OneJev1.6-0.8B Open · 0.8B | 69.82 | 59.41% | 0.1978 |
| KnowLine-0.8B-Gen1 Open · 0.8B | 69.65 | 58.87% | 0.1956 |
| GLiClass v4 Edge Open · 0.4B | 69.05 | 58.32% | 0.2021 |
| Decider 1 Closed · Efficient | 67.47 | 55.76% | 0.2082 |
| Strands Decider 2B Open · 2.0B | 64.08 | 45.52% | 0.1736 |
| Laya Open · 0.4B | 51.57 | 22.85% | 0.1971 |
| GLiNER2.5-multi-Decide Open · 0.3B | 47.58 | 21.21% | 0.2604 |
| Laya Multilingual Open · 0.3B | 39.32 | 15.36% | 0.3671 |
Decision Score (AA formula): 50 × (1 + accuracy − chosen-answer Brier) on the first English run, the chosen-answer Brier being the mean of (p − c)², p the probability given to the chosen answer and c 1 if it is right, else 0. GPT-6 Luna · Decisions API: unavailable, 2 of 547 answers without a probability for the chosen answer.
Artificial Analysis's formula applied to our questions. Not their Index. Decision models only: chat models and the I don't know runs are excluded.
Turkish reference (the Text tab runs)
The answers each setup gave at 99% confidence or more, in its Text tab run on the Turkish questions, for reference, the bar their share of the 2,198 questions; how many of them were wrong; and its ranking score (AUROC: how often a right answer has higher confidence than a wrong one, 0.5 chance, 1 perfect). No wrong answer at 99% first, by answers kept, then by the share wrong among the kept answers; a setup with no answers at 99% last · more kept and fewer wrong is better
Decision models
Confidence: option probabilities.
Karpo Perplexity Gemini Mammoth vLLM-SR Fastino REAX Drex Elephant KnowLine Copernicus Liquid Logits Turion Jev Verdin StartLux Valen vera Cygnet OneJev Sieve Jev SubQ Winnow Dexy GPT Adjudex Chinese Decision Pointer Inception Microsoft Laya Celeris Cerno Clef meraGPT GLiClass Jeffan Metask Open-Jev Strands
Cost per 1,000 English questionsper 1M English questions
Estimated dollars per 1,000 questionsper 1M questions: each of the 142 setups' first English run, ordered by English cost · lower is better
Celeris Perplexity Liquid Fastino Inception Laya Microsoft Elephant meraGPT SubQ Clef Strands Drex Adjudex Dexy Jev Chinese Decision Pointer Qwen Verdin vera KnowLine Gemini Mammoth Valen GPT Cygnet Jeffan GLM Winnow Gemma vLLM-SR Metask DeepSeek OneJev Erk Copernicus Karpo REAX Logits MiniMax Raon Muse MiMo Open-Jev Claude Inkling MiniCPM Kimi Granite Grok Cerno GLiClass Sieve StartLux Turion Jev Costs: API list prices and recorded token use. Setups on our own GPUs: the maker's list price for the same model (each value's tooltip names it); our hardware cost is not measured. On our GPUs, estimated at a public host's list price for the same weights (the host, the date read and any precision difference are on each setup's page): an API-equivalent estimate, not measured spending. On our GPUs with no public price for the same weights, estimated from GPU time: the run's request time x the GPUs its server held x Runpod's on-demand price for an RTX PRO 6000, $2.49 per GPU-hour (read 5 November 2026): a cloud-equivalent estimate, not measured spending. No list price: on our own GPUs, with no estimate. ≤ Upper-bound estimate: one $0.001 web search counted per request; Perplexity bills only completed searches. Log scale. No list price: Cerno 9B v1.2, GLiClass v4 Edge, GLiDE 0.4B, GLiDE 0.8B, GLiDE 27B, GLiDE 2B, GLiDE 9B, KnowLine 4B Gen5.5, KnowLine-0.8B-Gen1, KnowLine-2B-Gen1, KnowLine-31B-Gen1.5, OneJev1.6-0.8B, Sieve 31B v3, StartLux-Decision-v1.1-0.8B-Instant, StartLux-Decision-v1.1-2B-Instant, StartLux-Decision-v1.1-4B-Instant, StartLux-Decision-v1.1-9B-Instant and Turion Jev 31B, on our own hardware; the maker lists no price for the same model.
Cost per 1,000 English questionsper 1M English questions
Estimated dollars per 1,000 questionsper 1M questions: each of the 66 chat model setups' first English run, ordered by English cost · lower is better
Fastino Qwen GLM Gemma GPT DeepSeek Erk MiniMax Raon Muse Gemini MiMo Claude Liquid Inkling MiniCPM Kimi Granite Grok Costs: API list prices and recorded token use. Setups on our own GPUs: the maker's list price for the same model (each value's tooltip names it); our hardware cost is not measured. On our GPUs, estimated at a public host's list price for the same weights (the host, the date read and any precision difference are on each setup's page): an API-equivalent estimate, not measured spending. On our GPUs with no public price for the same weights, estimated from GPU time: the run's request time x the GPUs its server held x Runpod's on-demand price for an RTX PRO 6000, $2.49 per GPU-hour (read 5 November 2026): a cloud-equivalent estimate, not measured spending. Log scale.
Cost per 1,000 English questionsper 1M English questions
Estimated dollars per 1,000 questionsper 1M questions: each of the 76 decision models' first English run, ordered by English cost · lower is better
Celeris Perplexity Liquid Fastino Inception Laya Microsoft Elephant meraGPT SubQ Clef Strands Drex Adjudex Dexy Jev Chinese Decision Pointer Verdin vera KnowLine Gemini Mammoth Valen GPT Cygnet Jeffan Winnow vLLM-SR Metask OneJev Copernicus Karpo REAX Logits Open-Jev Cerno GLiClass Sieve StartLux Turion Jev Costs: API list prices and recorded token use. Setups on our own GPUs: the maker's list price for the same model (each value's tooltip names it); our hardware cost is not measured. On our GPUs, estimated at a public host's list price for the same weights (the host, the date read and any precision difference are on each setup's page): an API-equivalent estimate, not measured spending. On our GPUs with no public price for the same weights, estimated from GPU time: the run's request time x the GPUs its server held x Runpod's on-demand price for an RTX PRO 6000, $2.49 per GPU-hour (read 5 November 2026): a cloud-equivalent estimate, not measured spending. No list price: on our own GPUs, with no estimate. ≤ Upper-bound estimate: one $0.001 web search counted per request; Perplexity bills only completed searches. Log scale. No list price: Cerno 9B v1.2, GLiClass v4 Edge, GLiDE 0.4B, GLiDE 0.8B, GLiDE 27B, GLiDE 2B, GLiDE 9B, KnowLine 4B Gen5.5, KnowLine-0.8B-Gen1, KnowLine-2B-Gen1, KnowLine-31B-Gen1.5, OneJev1.6-0.8B, Sieve 31B v3, StartLux-Decision-v1.1-0.8B-Instant, StartLux-Decision-v1.1-2B-Instant, StartLux-Decision-v1.1-4B-Instant, StartLux-Decision-v1.1-9B-Instant and Turion Jev 31B, on our own hardware; the maker lists no price for the same model.
Median latency per request on the 547 English questions
Median time per request: each of the 142 setups' first English run, ordered by English latency · lower is better
Fastino Laya Liquid GLiClass OneJev Strands StartLux Perplexity KnowLine Adjudex Dexy Chinese Decision Pointer Cygnet Verdin vera Valen Jeffan Elephant Winnow vLLM-SR Drex Metask Clef Jev Copernicus Mammoth Karpo Cerno REAX Gemini Logits Gemma Inception Microsoft GPT SubQ Celeris Qwen Turion Jev Open-Jev meraGPT Sieve Erk Claude GLM Raon Kimi DeepSeek MiMo MiniCPM Muse Inkling MiniMax Granite Grok Time per request as measured from our computer during each run; API times include the network. English runs: 1 question per request for 104 runs and 16 for 38; 1 request in flight for 108 runs, 32 for 17, 4 for 14 and 8 for 3. Of the 85 self-hosted setups, KnowLine-31B-Gen1.5, GLiDE 27B, Sieve 31B v3, Turion Jev 31B, StartLux-Decision-v1.1-9B-Instant, GLiDE 9B, Cerno 9B v1.2, StartLux-Decision-v1.1-4B-Instant, KnowLine 4B Gen5.5, KnowLine-2B-Gen1, GLiDE 2B, StartLux-Decision-v1.1-2B-Instant, GLiDE 0.8B, StartLux-Decision-v1.1-0.8B-Instant, GLiDE 0.4B, OneJev1.6-0.8B, KnowLine-0.8B-Gen1 and GLiClass v4 Edge ran their English runs on an NVIDIA DGX Spark, the others on our RTX PRO 6000 GPUs (Our method). Clef and Clef-Flash ran their English runs at the same time on one GPU. Log scale.
Median latency per request on the 547 English questions
Median time per request: each of the 66 chat model setups' first English run, ordered by English latency · lower is better
Fastino Gemma Qwen Gemini Erk Claude GLM Raon Liquid GPT Kimi DeepSeek MiMo MiniCPM Muse Inkling MiniMax Granite Grok Time per request as measured from our computer during each run; API times include the network. English runs: 16 questions per request for 38 runs and 1 for 28; 1 request in flight for 33 runs, 32 for 17, 4 for 13 and 8 for 3. The English runs of the 30 self-hosted setups ran on our RTX PRO 6000 GPUs (Our method). Log scale.
Median latency per request on the 547 English questions
Median time per request: each of the 76 decision models' first English run, ordered by English latency · lower is better
Fastino Laya Liquid GLiClass OneJev Strands StartLux Perplexity KnowLine Adjudex Dexy Chinese Decision Pointer Cygnet Verdin vera Valen Jeffan Elephant Winnow vLLM-SR Drex Metask Clef Jev Copernicus Mammoth Karpo Cerno REAX Gemini Logits Inception Microsoft GPT SubQ Celeris Turion Jev Open-Jev meraGPT Sieve Time per request as measured from our computer during each run; API times include the network. English runs: 1 question per request for 76 runs; 1 request in flight for 75 runs and 4 for 1. Of the 55 self-hosted setups, KnowLine-31B-Gen1.5, GLiDE 27B, Sieve 31B v3, Turion Jev 31B, StartLux-Decision-v1.1-9B-Instant, GLiDE 9B, Cerno 9B v1.2, StartLux-Decision-v1.1-4B-Instant, KnowLine 4B Gen5.5, KnowLine-2B-Gen1, GLiDE 2B, StartLux-Decision-v1.1-2B-Instant, GLiDE 0.8B, StartLux-Decision-v1.1-0.8B-Instant, GLiDE 0.4B, OneJev1.6-0.8B, KnowLine-0.8B-Gen1 and GLiClass v4 Edge ran their English runs on an NVIDIA DGX Spark, the others on our RTX PRO 6000 GPUs (Our method). Clef and Clef-Flash ran their English runs at the same time on one GPU. Log scale.
Limit: a model translated the questions and no person checked them (Our method).
Calibration of holding back by confidence at the 99% threshold is not measured in English.