Chat model

GPT-6 Luna

low reasoning · 16 questions per request

68.3% of the 2,198 questions right (66.3% to 70.2%, 95% interval). Back to the leaderboard.

How we called it

Kind: Chat model
Access: Codex CLI, ChatGPT subscription
Paid as: Subscription
Model ID: gpt-6-luna
Confidence from: Written estimate
Run: 133, 6 October 2026

Settings

Reasoning: low
Request format: 16 questions per request
Requests at once: 1
Temperature: not recorded
Harness seed: 0
Harness commit: not recorded
Prompt version: not recorded

Results

Questions	Accuracy	95% interval	Right
All questions	68.3%	66.3% to 70.2%	1,501 of 2,198
Reading	87.3%	84.6% to 89.7%	573 of 656
Grammar	60.2%	57.7% to 62.6%	928 of 1,542

Comparison group: pq. Compare with another setup.

Accuracy by topic

Topic	Part	Accuracy	Right
Word and phrase meanings	Reading	87.0%	114 of 131
Sentence meanings	Reading	93.5%	144 of 154
How a writer explains and develops ideas	Reading	74.3%	52 of 70
Topic and main point of a paragraph	Reading	94.9%	93 of 98
How a paragraph fits together	Reading	74.5%	79 of 106
Details that support a paragraph’s main point	Reading	93.8%	91 of 97
Word types, such as nouns and verbs	Grammar	55.0%	77 of 140
Groups of words built around a noun	Grammar	47.7%	31 of 65
Verbs and their forms	Grammar	70.3%	64 of 91
Verb forms used like nouns or modifiers	Grammar	68.1%	49 of 72
Word endings and what they do	Grammar	64.6%	51 of 79
How words are built	Grammar	52.2%	36 of 69
The roles of words and phrases in a sentence	Grammar	53.8%	78 of 145
How verbs show who acts and who receives the action	Grammar	74.3%	55 of 74
Types of sentences	Grammar	64.7%	55 of 85
Questions combining several grammar topics	Grammar	41.4%	91 of 220
Sound patterns and changes	Grammar	37.7%	29 of 77
Spelling	Grammar	82.0%	109 of 133
Punctuation	Grammar	65.0%	76 of 117
Problems in sentence wording	Grammar	72.6%	127 of 175

Confidence

Answers kept at 99% or more: 830, of which 137 wrong (share right: 80.8% to 85.9%, 95% interval); 1,368 skipped, the model less than 99% sure
Calibration gap: 25.7 points
AUROC, all questions: 0.69 0.66 to 0.71
AUROC, reading: 0.72 0.66 to 0.77
AUROC, grammar: 0.65 0.62 to 0.68
Brier score: 0.269
Accuracy of its surest answers: most confident fifth: 84.6% right; most confident half: 80.4% right; most confident four fifths: 73.8% right
Typical score of a right answer: 98.0%
Typical score of a wrong answer: 95.0%

Repeats

We did not repeat this setup.

Latency

Latency, median request: 7.71 s (this run)
Latency, 90th percentile: 11.2 s
Requests at once: 1
Throughput: 106.3 questions per minute of request time
Request time: 20.7 minutes, over the requests whose answers were kept

Latency: the time from sending one request to receiving its answer, measured from our computer, network and any retries within the request included. Request time: the time at least one request was in flight, over the requests whose answers were kept. It is not the run's elapsed time: it leaves out requests that failed and any pause before a run was resumed.

Cost

$ per 1,000 questions: $0.048
Whole run: $0.11
Basis: Subscription run priced as if we had used the API
Price: $0.10 input, $0.01 cached input, $0.125 cache writes, $0.50 output per 1M tokens (source, 2026-10-08)
Input tokens per question: 545
Output tokens per question: 13