Chat model
MiniMax-M3.1-Flash high · chat
high reasoning · 16 questions per request
- Accuracy, all 2,198 questions
- 66.5%64.5% to 68.4%, 95% interval; 1,461 right
- Answers kept at 99% confidence
- 375 of 2,19854 wrong; share right 81.7% to 88.8%, 95% interval
- Cost, $ per 1,000 questionsper 1M questions
- $0.814$814Basis and price under Cost
- Latency
- 112.6 smedian request, 16 questions per request
How we called it
- Kind
- Chat model
- Access
- OpenRouter chat API, MiniMax's own endpoint, fallbacks off, output cap 65,536 tokens, reasoning included; JSON mode without the answer schema (the endpoint has no structured outputs), so the prompt adds a paragraph stating the answer layout
- Paid as
- Provider service
- Model ID
minimax/minimax-m3.1-flash- Parameters
- Mixture of experts: 428B total, about 23B active per step. Size class uses the total (428B).mixture of experts, as MiniMax states it; source: huggingface.co/MiniMaxAI/MiniMax-M3.1-Flash; read 9 November 2026
- Confidence from
- Written estimate
- Run
- 862, 8 November 2026
Settings
- Reasoning
- high
- Request format
- 16 questions per request
- Requests at once
- 4
- Temperature
- not recorded
Run record: harness seed, harness commit and prompt version
- Harness seed
- 0
- Harness commit
b0118bbae7ac- Prompt version
254597b8be16
Results
| Questions | Accuracy | 95% interval | Right |
|---|---|---|---|
| All questions | 66.5% | 64.5% to 68.4% | 1,461 of 2,198 |
| Reading | 82.9% | 79.9% to 85.6% | 544 of 656 |
| Grammar | 59.5% | 57.0% to 61.9% | 917 of 1,542 |
Comparison group: CDEFGHIJKL. Compare with another setup.
Invalid answers: 2; read from broken replies: 0; empty replies asked again: 0. Still invalid: 2 with a letter the question does not allow. An empty reply is asked once more; in a broken reply, any letter that is clearly readable is scored and the rest count as wrong. Invalid answers count as wrong (how).
With “Bilmiyorum” (run 864): Invalid answers: 12; read from broken replies: 16; empty replies asked again: 0. Still invalid: 12 with a letter the question does not allow. Without the rule: 28 invalid. (how)
Accuracy by topic
English
The Text tab's 547 kept reading questions translated into English, asked with this setup's Text tab settings. Every setup: English reading; against Turkish: English and Turkish.
- Right answers
- 462 of 547, first English run (863)
- Accuracy
- 84.5%
- Non-hallucination with “I don't know”
- 3.4%: 3 “I don't know” of 88 not right (run 865)
- Answered with “I don't know”
- 99.5%: 544 of 547
- Error rate with “I don't know”
- 15.6%: 85 of 544 answers
- Cost, $ per 1,000 questionsper 1M questions, first English run
- 0.193193at list prices
- Latency, median request, first English run
- 26.7 s; 4 requests at once
- English run
- 863
- Machine, English runs
- hosted API
- Date
- 8 November 2026
- Request format
- 16 questions per request
- Reasoning
- high
- Prompt version
5c3a4849752e- Translations, SHA-256
bf0d47cd385a93050c65906977e199ef7e45d5848dbbdef2bc8d124f4d5abf68
Confidence
- Answers kept at 99% or more
- 375, of which 54 wrong (share right: 81.7% to 88.8%, 95% interval); 1,823 skipped: 1,821 with the model less than 99% sure and 2 without a valid answer
- Calibration gap
- 21.1 points
- AUROC, all questions
- 0.69 0.67 to 0.71
- AUROC, reading
- 0.76 0.71 to 0.80
- AUROC, grammar
- 0.65 0.62 to 0.68
- Brier score
- 0.247
- Accuracy of its surest answers
- most confident fifth: 84.4% right; most confident half: 79.4% right; most confident four fifths: 72.4% right
- Typical score of a right answer
- 96.0%
- Typical score of a wrong answer
- 88.0%
Repeats
We did not repeat this setup.
Latency
- Latency, median request
- 112.6 s (this run)
- Latency, 90th percentile
- 279.2 s
- Requests at once
- 4
- Throughput
- 26.7 questions per minute of request time
- Request time
- 82.4 minutes, over the requests whose answers were kept
Latency: the time from sending one request to receiving its answer, measured from our computer, network and any retries within the request included. Request time: the time at least one request was in flight, over the requests whose answers were kept. It is not the run's elapsed time: it leaves out requests that failed and any pause before a run was resumed.
Cost
- Cost per 1,000 questionsper 1M questions
- $0.814$814
- Whole run
- $1.79
- Basis
- Estimated from published prices and recorded token usage
- Price
- $0.15 input, $0.015 cached input, $0.60 output per 1M tokens (source, 2026-11-08)
- Input tokens per question
- 320
- Output tokens per question
- 1,279