Chat model

GLM-5.3-Flash

reasoning on · 16 questions per request

78.8% of the 2,198 questions right (77.0% to 80.5%, 95% interval). Back to the leaderboard.

How we called it

Kind: Chat model
Access: TensorFold, self-hosted, EXL3 4-bit experts
Paid as: Our own hardware
Model ID: glm-5.3-flash
Parameters: 320B (18B active per token)
Hardware: 4 × RTX PRO 6000 Blackwell (96 GB); EXL3 4-bit experts, FP8 dense layers and KV cache
Confidence from: Written estimate
Run: 160, 11 October 2026

Settings

Reasoning: on
Request format: 16 questions per request
Requests at once: 8
Temperature: not recorded
Harness seed: 0
Harness commit: not recorded
Prompt version: not recorded

Results

Questions	Accuracy	95% interval	Right
All questions	78.8%	77.0% to 80.5%	1,732 of 2,198
Reading	89.3%	86.7% to 91.5%	586 of 656
Grammar	74.3%	72.1% to 76.4%	1,146 of 1,542

Comparison group: m. Compare with another setup.

128 questions got no valid answer; they count as wrong.

Accuracy by topic

Topic	Part	Accuracy	Right
Word and phrase meanings	Reading	88.5%	116 of 131
Sentence meanings	Reading	92.2%	142 of 154
How a writer explains and develops ideas	Reading	78.6%	55 of 70
Topic and main point of a paragraph	Reading	95.9%	94 of 98
How a paragraph fits together	Reading	82.1%	87 of 106
Details that support a paragraph’s main point	Reading	94.8%	92 of 97
Word types, such as nouns and verbs	Grammar	83.6%	117 of 140
Groups of words built around a noun	Grammar	84.6%	55 of 65
Verbs and their forms	Grammar	86.8%	79 of 91
Verb forms used like nouns or modifiers	Grammar	90.3%	65 of 72
Word endings and what they do	Grammar	88.6%	70 of 79
How words are built	Grammar	76.8%	53 of 69
The roles of words and phrases in a sentence	Grammar	84.1%	122 of 145
How verbs show who acts and who receives the action	Grammar	89.2%	66 of 74
Types of sentences	Grammar	84.7%	72 of 85
Questions combining several grammar topics	Grammar	62.7%	138 of 220
Sound patterns and changes	Grammar	0.0%	0 of 77
Spelling	Grammar	64.7%	86 of 133
Punctuation	Grammar	70.1%	82 of 117
Problems in sentence wording	Grammar	80.6%	141 of 175

Confidence

Answers kept at 99% or more: 9, of which 0 wrong (share right: 70.1% to 100.0%, 95% interval); 2,189 skipped: 2,061 with the model less than 99% sure and 128 without a valid answer
Calibration gap: 4.5 points
AUROC, all questions: 0.76 0.73 to 0.79
AUROC, reading: 0.84 0.80 to 0.88
AUROC, grammar: 0.72 0.69 to 0.76
Brier score: 0.120
Accuracy of its surest answers: most confident fifth: 97.4% right; most confident half: 93.3% right; most confident four fifths: 89.4% right
Typical score of a right answer: 85.0%
Typical score of a wrong answer: 70.0%

Repeats

We did not repeat this setup.

Latency

Latency, median request: 102.6 s (this run)
Latency, 90th percentile: 639.3 s
Requests at once: 8
Throughput: 34.2 questions per minute of request time
Request time: 64.3 minutes, over the requests whose answers were kept

Latency: the time from sending one request to receiving its answer, measured from our computer, network and any retries within the request included. Request time: the time at least one request was in flight, over the requests whose answers were kept. It is not the run's elapsed time: it leaves out requests that failed and any pause before a run was resumed.

Cost

$ per 1,000 questions: Not measured
Basis: No dollar estimate; see hardware and time
Input tokens per question: 254
Output tokens per question: 1,010