Setup · League 1
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- Missing
- Provider or endpoint
- Missing
- Date run
- 2026-10-19 to 2026-10-26
- Price
- Missing
- Model type
- Missing
- Weights
- Missing
- Runs on
- Missing
- Reasoning setting
- Missing
- Confidence source
- Missing
- Rules text
- Missing
- Option groups
- not recorded; in this data 147 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
- Sampling settings
- Missing
- Notes
- Missing
- Reasoning, from the logs
- 232 early requests sent no reasoning field. All responses had 0 reasoning tokens.
Missing B7 fields: every B7 field
Results in League 1
- Error rate (PR)
- 35.631.7 to 39.7
- Decisions
- 1,736 (1,736 scored, 0 invalid)
- Engine's best play
- 47.9%
- Blunder rate
- 25.2%
- P95 equity loss
- 0.338
- Mean loss, worst 5%
- 0.556
- Median / P90 s per decision
- 0.48 / 1.63
- $ per 1,000 decisions
- data-v="0.728">0.728estimate · RTX PRO 6000 Blackwell + API
- Risk judgement, margin 0.001
- 59.3 (95% interval 57.1 to 61.5); risks found 29.1% (192 / 659), right safety 89.5% (603 / 674)
- Risk judgement, margin 0.02
- 59.0 (95% interval 56.7 to 61.3); risks found 30.1% (177 / 588), right safety 87.9% (655 / 745)
- No trade-off
- 403 (left out of every risk metric)
Risk call card
59.3risk judgement, margin 0.001
right risks192
wrong risks47
needless risks71
missed risks420
right safety603
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 1,736
- Eligible (full probability distribution)
- 131
- Left out, by reason
- asked in groups of options 41; incomplete distribution (an option without its own probability) 210; not audited 1,354
- Close choices
- 11.5% (15 / 131), under 200 decisions
- Ties in the top two
- 4 decisions (4 at a positive probability, 0 at 0; the two most likely plays tied: 2, the second tied with the third: 2); 2 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 40.0% (6 / 15), under 200 decisions of the close choices; 4.6% (6 / 131), under 200 decisions of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 15 | 0.540median 0.549 | 0.389median 0.376 | 0.928median 0.989 |
| Other decisions | 116 | 0.891median 0.924 | 0.093median 0.072 | 0.983median 0.997 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 0 | 0 | 0 | 0 | 1 | 0 | 3 | 11 |
| Other decisions | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 4 | 112 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 15 | 40.0%6 / 15, under 200 decisions | 6.7%1 / 15, under 200 decisions | 13.3%2 / 15, under 200 decisions | 0.0%0 / 15, under 200 decisions | 9 / 3 / 3 / 0 / 0 |
| Other decisions | 116 | 46.6%54 / 116, under 200 decisions | 11.2%13 / 116, under 200 decisions | 16.4%19 / 116, under 200 decisions | 25.0%29 / 116, under 200 decisions | 34 / 52 / 27 / 3 / 0 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 15 | 0.091median 0.028 | 0.103median 0.037 | 0.036median 0.005 |
| Other decisions | 116 | 0.144median 0.075 | 0.024median 0.000 | 0.130median 0.055 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 100.0%4 / 4, under 200 decisions | 50.0%2 / 4, under 200 decisions |
| margin 0.02 | 100.0%4 / 4, under 200 decisions | 50.0%2 / 4, under 200 decisions |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 19.0%4 / 21, under 200 decisions | 7.7%4 / 52, under 200 decisions | 12.1%7 / 58, under 200 decisions | 0.0%0 / 2, under 200 decisions | 0.0%0 / 8, under 200 decisions | 36.4%4 / 11, under 200 decisions |
| margin 0.02 | 21.1%4 / 19, under 200 decisions | 7.4%4 / 54, under 200 decisions | 12.1%7 / 58, under 200 decisions | N/A0 / 0 | 0.0%0 / 8, under 200 decisions | 36.4%4 / 11, under 200 decisions |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00117 decisions | 66.7%crude 50.0% (2 / 4), under 200 decisions | 70.8%crude 69.2% (9 / 13), under 200 decisions | −4.2 points95% interval −16.7 to +70.0 | −19.2 points95% interval −46.7 to +43.8 |
| margin 0.0216 decisions | 66.7%crude 50.0% (2 / 4), under 200 decisions | 70.8%crude 66.7% (8 / 12), under 200 decisions | −4.2 points95% interval −16.7 to +70.0 | −16.7 points95% interval −46.7 to +53.8 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 33.8 | 255 |
| Contact | 43.4 | 1,141 |
| Holding game | Too few decisions | 29 |
| Race | Too few decisions | 179 |
| Bear-off | Too few decisions | 132 |