decision model · v1 set (earlier version)

Cygnet

Earlier version

Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
cygnet
Provider or endpoint
our GPUs (letter-reading server)
Date run
Missing
Price
no price per token; own hardware; price date not recorded
Model type
decision model
Weights
open
Runs on
our GPUs
Reasoning setting
none (letter read directly)
Confidence source
letter probability
Rules text
short (the v1 rules text; the same stored request text for every model)
Option groups
groups of up to 26 options; no decision in this data was split
Sampling settings
Missing
Notes
Google's gemma-4-12B-it, unchanged, behind a server that labels the options with letters and reads each letter's probability

Missing B7 fields: sampling, date_run

Results in v1 set (earlier version)

Earlier version: plays were listed in the runner's order, which tends to put the best play early. When Gemini 4 Argon answered the same 1,000 positions with the plays shuffled, its error rate changed by +1.9 (95% interval −0.2 to +4.2), so the effect is not proven.

Error rate (PR)
38.934.6 to 43.7
Decisions
1,000 (1,000 scored, 0 invalid)
Engine's best play
42.8%
Blunder rate
27.1%
P95 equity loss
0.352
Mean loss, worst 5%
0.572
Median / P90 s per decision
0.78 / 1.12
$ per 1,000 decisions
Not measured
Risk judgement, margin 0.001
55.1 (95% interval 52.0 to 58.0); risks found 25.4% (90 / 355), right safety 84.8% (313 / 369)
Risk judgement, margin 0.02
55.6 (95% interval 52.7 to 58.4); risks found 26.4% (84 / 318), right safety 84.7% (344 / 406)
No trade-off
276 (left out of every risk metric)

JSON

Risk call card

55.1risk judgement, margin 0.001

right risks90
wrong risks37
needless risks56
missed risks228
right safety313

Hit risk only: being hit. Gammon risk and priming are not in it.

By game phase

PhaseError rate (PR)Decisions
OpeningToo few decisions143
Contact45.5641
Holding gameToo few decisions20
RaceToo few decisions136
Bear-offToo few decisions60