decision model · League 1
Cygnet
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- cygnet
- Provider or endpoint
- NVIDIA RTX PRO 6000 Blackwell (96 GB), bf16; NVIDIA DGX Spark (GB10) from 16 Oct
- Date run
- 2026-10-15 to 2026-10-26
- Price
- no price per token; estimate: GPU time, at most; source content/own_cost.json (wall clock on our machines × hourly rate); price date not recorded
- Model type
- decision model
- Weights
- open
- Runs on
- our GPUs
- Reasoning setting
- none (letter read directly)
- Confidence source
- letter probability
- Rules text
- long
- Option groups
- up to 26 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; in this data 123 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
- Sampling settings
- Missing
- Notes
- Google's gemma-4-12B-it behind a server that labels the options with letters and reads the letter probabilities; moved to a DGX Spark on 16 Oct 11:04 UTC (20 replayed requests: same play 20/20)
Missing B7 fields: sampling
Results in League 1
- Error rate (PR)
- 38.635.4 to 41.9
- Decisions
- 2,016 (2,016 scored, 0 invalid)
- Engine's best play
- 44.0%
- Blunder rate
- 27.5%
- P95 equity loss
- 0.357
- Mean loss, worst 5%
- 0.570
- Median / P90 s per decision
- 0.37 / 2.96
- $ per 1,000 decisions
- data-v="11.168">11.168estimate · RTX PRO 6000 Blackwell
- Risk judgement, margin 0.001
- 57.1 (95% interval 55.3 to 58.8); risks found 27.3% (205 / 750), right safety 86.8% (685 / 789)
- Risk judgement, margin 0.02
- 57.3 (95% interval 55.4 to 59.2); risks found 28.2% (192 / 680), right safety 86.4% (742 / 859)
- No trade-off
- 477 (left out of every risk metric)
Risk call card
57.1risk judgement, margin 0.001
right risks205
wrong risks58
needless risks104
missed risks487
right safety685
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 2,016
- Eligible (full probability distribution)
- 1,655
- Left out, by reason
- asked in groups of options 113; not audited 248
- Close choices
- 21.7% (359 / 1,655)
- Ties in the top two
- 29 decisions (29 at a positive probability, 0 at 0; the two most likely plays tied: 10, the second tied with the third: 19); 13 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 40.1% (144 / 359) of the close choices; 8.7% (144 / 1,655) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 359 | 0.448median 0.449 | 0.314median 0.308 | 0.762median 0.768 |
| Other decisions | 1,296 | 0.727median 0.743 | 0.119median 0.107 | 0.846median 0.856 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 0 | 2 | 15 | 41 | 73 | 76 | 74 | 78 |
| Other decisions | 0 | 0 | 0 | 0 | 6 | 29 | 105 | 282 | 410 | 464 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 359 | 46.8%168 / 359 | 18.9%68 / 359 | 10.9%39 / 359 | 20.3%73 / 359 | 136 / 130 / 63 / 28 / 2 |
| Other decisions | 1,296 | 56.9%737 / 1,296 | 31.3%406 / 1,296 | 12.0%155 / 1,296 | 27.0%350 / 1,296 | 310 / 505 / 317 / 147 / 17 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 359 | 0.121median 0.054 | 0.098median 0.023 | 0.110median 0.043 |
| Other decisions | 1,296 | 0.137median 0.073 | 0.068median 0.003 | 0.145median 0.081 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 59.6%87 / 146, under 200 decisions | 44.2%57 / 129, under 200 decisions |
| margin 0.02 | 60.9%81 / 133, under 200 decisions | 44.4%63 / 142, under 200 decisions |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 23.8%146 / 613 | 20.3%129 / 636 | 20.7%84 / 406 | 22.0%13 / 59, under 200 decisions | 22.8%43 / 189, under 200 decisions | 24.7%90 / 365 |
| margin 0.02 | 24.0%133 / 554 | 20.4%142 / 695 | 20.7%84 / 406 | N/A0 / 0 | 22.8%43 / 189, under 200 decisions | 24.7%90 / 365 |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001415 decisions | 44.7%crude 44.8% (39 / 87), under 200 decisions | 75.5%crude 76.5% (251 / 328) | −30.8 points95% interval −40.8 to −21.2 | −31.7 points95% interval −39.4 to −23.3 |
| margin 0.02377 decisions | 42.5%crude 42.0% (34 / 81), under 200 decisions | 73.8%crude 75.3% (223 / 296) | −31.3 points95% interval −43.1 to −19.6 | −33.4 points95% interval −43.2 to −23.2 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 39.1 | 298 |
| Contact | 44.3 | 1,302 |
| Holding game | Too few decisions | 44 |
| Race | 22.0 | 246 |
| Bear-off | Too few decisions | 126 |