Decision model

Drex 2.0

1 question per request

55.7% of the 2,198 questions right (53.6% to 57.8%, 95% interval). Back to the leaderboard.

How we called it

Kind

Decision model

Access

Nace.AI System One API

Paid as

Provider service

Model ID

drex-v2.0

Parameters

35B (3B active per token)

Note

Nace.AI's hosted Drex 2.0 (drex-v2.0, released 6 October 2026), built on Qwen3.6-35B-A3B. It reads text only, so it is not on the Page images tab. Nace's API gives its option probabilities rounded to 4 decimal places, so they sum to 1 only within that rounding; the check before its runs allowed for it.

Confidence from

Option score

Run

334, 31 October 2026

Settings

Reasoning

off

Request format

1 question per request

Requests at once

1

Temperature

not recorded

Harness seed

0

Harness commit

000725f1ff4e

Prompt version

254597b8be16

Results

Comparison group: zABC. Compare with another setup.

Confidence

Answers kept at 99% or more

33, of which 0 wrong (share right: 89.6% to 100.0%, 95% interval); 2,165 skipped, the model less than 99% sure

Calibration gap

4.2 points

AUROC, all questions

0.80 0.79 to 0.82

AUROC, reading

0.88 0.85 to 0.91

AUROC, grammar

0.72 0.69 to 0.75

Brier score

0.179

Accuracy of its surest answers

most confident fifth: 98.0% right; most confident half: 78.5% right; most confident four fifths: 63.0% right

Typical score of a right answer

61.7%

Typical score of a wrong answer

33.5%

Repeats

Run 5 times on all questions.

Answers changed in 5 runs

19 of 2,198

Same wrong answer in all 5

959 of 2,198

Wrong in at least one run

978 of 2,198

Accuracy in each run

55.7%, 55.8%, 55.9%, 55.8%, 55.8%

Latency

Latency, median request

0.325 s (this run)

Latency, 90th percentile

0.395 s

Requests at once

1

Throughput

120.5 questions per minute of request time

Request time

18.2 minutes, over the requests whose answers were kept

Latency: the time from sending one request to receiving its answer, measured from our computer, network and any retries within the request included. Request time: the time at least one request was in flight, over the requests whose answers were kept. It is not the run's elapsed time: it leaves out requests that failed and any pause before a run was resumed.

Cost

$ per 1,000 questions

$0.019

Whole run

$0.042

Basis

Estimated from published prices and recorded token usage

Price

$0.06 input, output free per 1M tokens (source, 2026-10-30)

Input tokens per question

319

Output tokens per question

84

Going first in front of an LLM

Replay of recorded answers, cutoff chosen on half the questions, averaged over the splits. Savings are estimates from API prices; the Method page explains how.