chat model, one letter · League 2

Claude Sonnet 5.5low

FinishedPublished

Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
claude-sonnet-5-5
Provider or endpoint
Claude subscription
Date run
2026-10-19 to 2026-10-26
Price
$2.0 / $10.0 per million tokens in / out; API-equivalent (subscription); price date not recorded
Model type
chat model, one letter
Weights
closed
Runs on
subscription
Reasoning setting
effort low
Confidence source
none
Rules text
long
Option groups
up to 26 options per request (letters A-Z); more legal plays are split into equal groups, a winner per group, then among the winners; in this data 217 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
Sampling settings
not sent (API default)
Notes
Missing

Missing B7 fields: price.date

Results in League 2

Error rate (PR)
20.117.8 to 22.4
Decisions
2,150 (2,150 scored, 0 invalid)
Engine's best play
60.1%
Blunder rate
15.7%
P95 equity loss
0.215
Mean loss, worst 5%
0.379
Median / P90 s per decision
1.51 / 7.37
$ per 1,000 decisions
13.541API-equivalent (subscription)
Risk judgement, margin 0.001
63.5 (95% interval 61.6 to 65.4); risks found 30.9% (204 / 661), right safety 96.2% (883 / 918)
Risk judgement, margin 0.02
64.2 (95% interval 62.1 to 66.2); risks found 33.1% (190 / 574), right safety 95.2% (957 / 1,005)
No trade-off
571 (left out of every risk metric)

JSON

Risk call card

63.5risk judgement, margin 0.001

right risks204
wrong risks24
needless risks35
missed risks433
right safety883

Hit risk only: being hit. Gammon risk and priming are not in it.

Close choices

A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).

Probabilities over plays, not sampled reasoning. Method

League 2: complete letter-readout records are mostly few-option positions, a selected subset.

Decisions
2,150
Eligible (full probability distribution)
0
Left out, by reason
not audited 209; one letter only, or a program 1,941
Close choices
N/A no full probability distribution

By game phase

PhaseError rate (PR)Decisions
Opening17.5313
Contact26.21,340
Holding gameToo few decisions56
Race6.0251
Bear-offToo few decisions190