chat model, one letter · v1 set (earlier version)

Claude Opus 5.5

Earlier version

Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
claude-opus-5-5
Provider or endpoint
Claude subscription
Date run
Missing
Price
$4.0 / $20.0 per million tokens in / out; API-equivalent (subscription); price date not recorded
Model type
chat model, one letter
Weights
closed
Runs on
subscription
Reasoning setting
effort low
Confidence source
none
Rules text
short (the v1 rules text; the same stored request text for every model)
Option groups
not recorded; no decision in this data was split
Sampling settings
not sent (API default)
Notes
answer constrained to the option letters by a JSON schema; stored as choice only

Missing B7 fields: option_groups, price.date, date_run

Results in v1 set (earlier version)

Earlier version: plays were listed in the runner's order, which tends to put the best play early. When Gemini 4 Argon answered the same 1,000 positions with the plays shuffled, its error rate changed by +1.9 (95% interval −0.2 to +4.2), so the effect is not proven.

Error rate (PR)
17.314.8 to 19.8
Decisions
1,000 (1,000 scored, 0 invalid)
Engine's best play
62.2%
Blunder rate
13.9%
P95 equity loss
0.187
Mean loss, worst 5%
0.320
Median / P90 s per decision
3.43 / 6.65
$ per 1,000 decisions
Not measured
Risk judgement, margin 0.001
68.1 (95% interval 65.3 to 70.8); risks found 40.8% (145 / 355), right safety 95.4% (352 / 369)
Risk judgement, margin 0.02
69.0 (95% interval 66.1 to 71.9); risks found 43.4% (138 / 318), right safety 94.6% (384 / 406)
No trade-off
276 (left out of every risk metric)

JSON

Risk call card

68.1risk judgement, margin 0.001

right risks145
wrong risks12
needless risks17
missed risks198
right safety352

Hit risk only: being hit. Gammon risk and priming are not in it.

By game phase

PhaseError rate (PR)Decisions
OpeningToo few decisions143
Contact23.0641
Holding gameToo few decisions20
RaceToo few decisions136
Bear-offToo few decisions60