Setup · League 1
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- Missing
- Provider or endpoint
- Missing
- Date run
- 2026-10-20 to 2026-10-25
- Price
- Missing
- Model type
- Missing
- Weights
- Missing
- Runs on
- Missing
- Reasoning setting
- Missing
- Confidence source
- Missing
- Rules text
- Missing
- Option groups
- not recorded; in this data 180 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
- Sampling settings
- Missing
- Notes
- Missing
Missing B7 fields: every B7 field
Results in League 1
- Error rate (PR)
- 45.040.7 to 49.9
- Decisions
- 2,005 (2,005 scored, 0 invalid)
- Engine's best play
- 43.7%
- Blunder rate
- 30.7%
- P95 equity loss
- 0.422
- Mean loss, worst 5%
- 0.680
- Median / P90 s per decision
- 1.48 / 2.71
- $ per 1,000 decisions
- data-v="4.551">4.551estimate · RTX PRO 6000 Blackwell
- Risk judgement, margin 0.001
- 57.6 (95% interval 56.1 to 59.0); risks found 23.5% (178 / 756), right safety 91.7% (671 / 732)
- Risk judgement, margin 0.02
- 56.7 (95% interval 55.1 to 58.2); risks found 22.8% (157 / 688), right safety 90.6% (725 / 800)
- No trade-off
- 517 (left out of every risk metric)
Risk call card
57.6risk judgement, margin 0.001
right risks178
wrong risks42
needless risks61
missed risks536
right safety671
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 2,005
- Eligible (full probability distribution)
- 312
- Left out, by reason
- asked in groups of options 33; incomplete distribution (an option without its own probability) 30; not audited 1,630
- Close choices
- 34.9% (109 / 312)
- Ties in the top two
- 1 decisions (1 at a positive probability, 0 at 0; the two most likely plays tied: 1, the second tied with the third: 0); 1 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 42.2% (46 / 109), under 200 decisions of the close choices; 14.7% (46 / 312) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 109 | 0.482median 0.502 | 0.343median 0.338 | 0.825median 0.874 |
| Other decisions | 203 | 0.795median 0.800 | 0.133median 0.124 | 0.928median 0.958 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 0 | 0 | 2 | 9 | 12 | 24 | 15 | 47 |
| Other decisions | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 15 | 33 | 152 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 109 | 48.6%53 / 109, under 200 decisions | 17.4%19 / 109, under 200 decisions | 11.9%13 / 109, under 200 decisions | 24.8%27 / 109, under 200 decisions | 40 / 40 / 17 / 10 / 2 |
| Other decisions | 203 | 56.2%114 / 203 | 27.6%56 / 203 | 7.4%15 / 203 | 28.1%57 / 203 | 47 / 94 / 40 / 20 / 2 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 109 | 0.139median 0.041 | 0.160median 0.031 | 0.105median 0.010 |
| Other decisions | 203 | 0.132median 0.086 | 0.068median 0.001 | 0.129median 0.075 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 72.5%29 / 40, under 200 decisions | 53.1%17 / 32, under 200 decisions |
| margin 0.02 | 71.1%27 / 38, under 200 decisions | 55.9%19 / 34, under 200 decisions |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 36.7%40 / 109, under 200 decisions | 28.3%32 / 113, under 200 decisions | 41.1%37 / 90, under 200 decisions | 22.2%2 / 9, under 200 decisions | 44.8%13 / 29, under 200 decisions | 35.2%25 / 71, under 200 decisions |
| margin 0.02 | 38.0%38 / 100, under 200 decisions | 27.9%34 / 122, under 200 decisions | 41.1%37 / 90, under 200 decisions | N/A0 / 0 | 44.8%13 / 29, under 200 decisions | 35.2%25 / 71, under 200 decisions |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00184 decisions | 64.3%crude 65.5% (19 / 29), under 200 decisions | 82.4%crude 80.0% (44 / 55), under 200 decisions | −18.2 points95% interval −33.8 to +14.3 | −14.5 points95% interval −28.5 to +10.1 |
| margin 0.0278 decisions | 64.0%crude 66.7% (18 / 27), under 200 decisions | 80.3%crude 78.4% (40 / 51), under 200 decisions | −16.3 points95% interval −36.0 to +17.7 | −11.8 points95% interval −28.3 to +10.5 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 35.7 | 310 |
| Contact | 57.2 | 1,259 |
| Holding game | Too few decisions | 28 |
| Race | 18.3 | 273 |
| Bear-off | Too few decisions | 135 |