decision model · League 1

Cygnet

FinishedPublished

Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
cygnet
Provider or endpoint
NVIDIA RTX PRO 6000 Blackwell (96 GB), bf16; NVIDIA DGX Spark (GB10) from 16 Oct
Date run
2026-10-15 to 2026-10-26
Price
no price per token; estimate: GPU time, at most; source content/own_cost.json (wall clock on our machines × hourly rate); price date not recorded
Model type
decision model
Weights
open
Runs on
our GPUs
Reasoning setting
none (letter read directly)
Confidence source
letter probability
Rules text
long
Option groups
up to 26 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; in this data 123 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
Sampling settings
Missing
Notes
Google's gemma-4-12B-it behind a server that labels the options with letters and reads the letter probabilities; moved to a DGX Spark on 16 Oct 11:04 UTC (20 replayed requests: same play 20/20)

Missing B7 fields: sampling

Results in League 1

Error rate (PR)
38.635.4 to 41.9
Decisions
2,016 (2,016 scored, 0 invalid)
Engine's best play
44.0%
Blunder rate
27.5%
P95 equity loss
0.357
Mean loss, worst 5%
0.570
Median / P90 s per decision
0.37 / 2.96
$ per 1,000 decisions
data-v="11.168">11.168estimate · RTX PRO 6000 Blackwell
Risk judgement, margin 0.001
57.1 (95% interval 55.3 to 58.8); risks found 27.3% (205 / 750), right safety 86.8% (685 / 789)
Risk judgement, margin 0.02
57.3 (95% interval 55.4 to 59.2); risks found 28.2% (192 / 680), right safety 86.4% (742 / 859)
No trade-off
477 (left out of every risk metric)

JSON

Risk call card

57.1risk judgement, margin 0.001

right risks205
wrong risks58
needless risks104
missed risks487
right safety685

Hit risk only: being hit. Gammon risk and priming are not in it.

Close choices

A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).

Probabilities over plays, not sampled reasoning. Method

Decisions
2,016
Eligible (full probability distribution)
1,655
Left out, by reason
asked in groups of options 113; not audited 248
Close choices
21.7% (359 / 1,655)
Ties in the top two
29 decisions (29 at a positive probability, 0 at 0; the two most likely plays tied: 10, the second tied with the third: 19); 13 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
Between a safe play and a riskier one
40.1% (144 / 359) of the close choices; 8.7% (144 / 1,655) of all eligible decisions
Probabilities of the two most likely plays, mean with median
DecisionsNumberp1 (most likely play)p2 (second)p1 + p2
Close choices3590.448median 0.4490.314median 0.3080.762median 0.768
Other decisions1,2960.727median 0.7430.119median 0.1070.846median 0.856
How much probability the two plays hold together: decisions per bin
p1 + p20.0–0.10.1–0.20.2–0.30.3–0.40.4–0.50.5–0.60.6–0.70.7–0.80.8–0.90.9–1.0
Close choices0002154173767478
Other decisions0000629105282410464
What differs between the two most likely plays (several can differ; none is picked as the cause)
DecisionsNumberHitting rollsNewly made pointsPip countsLongest primeNone / 1 / 2 / 3 / 4 differ
Close choices35946.8%168 / 35918.9%68 / 35910.9%39 / 35920.3%73 / 359136 / 130 / 63 / 28 / 2
Other decisions1,29656.9%737 / 1,29631.3%406 / 1,29612.0%155 / 1,29627.0%350 / 1,296310 / 505 / 317 / 147 / 17
Engine values (GNUbg 2-ply, equity), mean with median
DecisionsNumberGap between the two plays' valuesMost likely play: behind the engine's bestSecond most likely play: behind the engine's best
Close choices3590.121median 0.0540.098median 0.0230.110median 0.043
Other decisions1,2960.137median 0.0730.068median 0.0030.145median 0.081
Close choices between a safe play and a riskier one, by position class
MarginIn Risk pays positionsIn Safety pays positions
margin 0.00159.6%87 / 146, under 200 decisions44.2%57 / 129, under 200 decisions
margin 0.0260.9%81 / 133, under 200 decisions44.4%63 / 142, under 200 decisions
Close choices among eligible decisions, by position class and stake band
MarginRisk paysSafety paysNo trade-offRisk pays, small stakeRisk pays, medium stakeRisk pays, big stake
margin 0.00123.8%146 / 61320.3%129 / 63620.7%84 / 40622.0%13 / 59, under 200 decisions22.8%43 / 189, under 200 decisions24.7%90 / 365
margin 0.0224.0%133 / 55420.4%142 / 69520.7%84 / 406N/A0 / 022.8%43 / 189, under 200 decisions24.7%90 / 365

At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.

Missed risks where the risk is worth taking, close choices vs other decisions
MarginClose choices: missed risksOther decisions: missed risks (weighted to the close choices' mix)Difference, close minus otherCrude difference
margin 0.001415 decisions44.7%crude 44.8% (39 / 87), under 200 decisions75.5%crude 76.5% (251 / 328)−30.8 points95% interval −40.8 to −21.2−31.7 points95% interval −39.4 to −23.3
margin 0.02377 decisions42.5%crude 42.0% (34 / 81), under 200 decisions73.8%crude 75.3% (223 / 296)−31.3 points95% interval −43.1 to −19.6−33.4 points95% interval −43.2 to −23.2

By game phase

PhaseError rate (PR)Decisions
Opening39.1298
Contact44.31,302
Holding gameToo few decisions44
Race22.0246
Bear-offToo few decisions126