Chat model

GPT-6 Astra

low reasoning · 16 questions per request

97.6% of the 2,198 questions right (96.9% to 98.2%, 95% interval). Back to the leaderboard.

We also ran this setup at 1 question per request: GPT-6 Astra · low reasoning · 1 question per request.

Which decision model can go first in front of it? See the Routing tab.

How we called it

Kind: Chat model
Access: Codex CLI, ChatGPT subscription
Paid as: Subscription
Model ID: gpt-6-astra
Confidence from: Written estimate
Run: 48, 4 October 2026

Settings

Reasoning: low
Request format: 16 questions per request
Requests at once: 1
Temperature: not recorded
Harness seed: 0
Harness commit: not recorded
Prompt version: not recorded

Reasoning effort levels (OpenAI's model page, read 28 October 2026): low, medium, high, xhigh, max. Top level: max. This run: low.

Results

Questions	Accuracy	95% interval	Right
All questions	97.6%	96.9% to 98.2%	2,145 of 2,198
Reading	97.0%	95.3% to 98.0%	636 of 656
Grammar	97.9%	97.0% to 98.5%	1,509 of 1,542

Comparison group: abc. Compare with another setup.

Accuracy by topic

Topic	Part	Accuracy	Right
Word and phrase meanings	Reading	97.7%	128 of 131
Sentence meanings	Reading	99.4%	153 of 154
How a writer explains and develops ideas	Reading	92.9%	65 of 70
Topic and main point of a paragraph	Reading	96.9%	95 of 98
How a paragraph fits together	Reading	94.3%	100 of 106
Details that support a paragraph’s main point	Reading	97.9%	95 of 97
Word types, such as nouns and verbs	Grammar	99.3%	139 of 140
Groups of words built around a noun	Grammar	98.5%	64 of 65
Verbs and their forms	Grammar	100.0%	91 of 91
Verb forms used like nouns or modifiers	Grammar	98.6%	71 of 72
Word endings and what they do	Grammar	98.7%	78 of 79
How words are built	Grammar	94.2%	65 of 69
The roles of words and phrases in a sentence	Grammar	97.9%	142 of 145
How verbs show who acts and who receives the action	Grammar	98.6%	73 of 74
Types of sentences	Grammar	98.8%	84 of 85
Questions combining several grammar topics	Grammar	95.9%	211 of 220
Sound patterns and changes	Grammar	96.1%	74 of 77
Spelling	Grammar	99.2%	132 of 133
Punctuation	Grammar	96.6%	113 of 117
Problems in sentence wording	Grammar	98.3%	172 of 175

Confidence

Answers kept at 99% or more: 1,725, of which 5 wrong (share right: 99.3% to 99.9%, 95% interval); 473 skipped, the model less than 99% sure
Calibration gap: 0.8 points
AUROC, all questions: 0.92 0.88 to 0.95
AUROC, reading: 0.96 0.93 to 0.98
AUROC, grammar: 0.89 0.84 to 0.94
Brier score: 0.020
Accuracy of its surest answers: most confident fifth: 100.0% right; most confident half: 99.8% right; most confident four fifths: 99.7% right
Typical score of a right answer: 99.0%
Typical score of a wrong answer: 94.0%

Repeats

Run 5 times on a sample of 200 questions.

Answers changed in 5 runs: 1 of 200
Same wrong answer in all 5: 4 of 200
Wrong in at least one run: 5 of 200
Accuracy in each run: 97.5%, 98.0%, 98.0%, 98.0%, 98.0%

Latency

Latency, median request: 13.0 s, measured on a sample (200 questions) over 5 runs
Latency, 90th percentile: 16.8 s
Latency at 1 question per request: 6.14 s median, on 200 questions
Requests at once: 1
Throughput: 61.2 questions per minute of request time
Request time: 36.1 minutes, over the requests whose answers were kept

Latency: the time from sending one request to receiving its answer, measured from our computer, network and any retries within the request included. Request time: the time at least one request was in flight, over the requests whose answers were kept. It is not the run's elapsed time: it leaves out requests that failed and any pause before a run was resumed.

Cost

$ per 1,000 questions: $5.38
Whole run: $11.89
Basis: Subscription run priced as if we had used the API
Price: $10 input, $1 cached input, $12.50 cache writes, $50 output per 1M tokens (source, 2026-10-08)
Input tokens per question: 528
Output tokens per question: 21