decision model · League 1
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- Missing
- Provider or endpoint
- NVIDIA RTX PRO 6000 Blackwell (96 GB), bf16
- Date run
- 2026-10-15 to 2026-10-26
- Price
- no price per token; estimate: GPU time, at most; source content/own_cost.json (wall clock on our machines × hourly rate); price date not recorded
- Model type
- decision model
- Weights
- open
- Runs on
- our GPUs
- Reasoning setting
- none (decision model)
- Confidence source
- decision-model probability
- Rules text
- long
- Option groups
- up to 255 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; no decision in this data was split
- Sampling settings
- not sent (decision request)
- Notes
- its server takes no model field; stayed on the same GPU throughout
Results in League 1
- Error rate (PR)
- 65.659.7 to 71.9
- Decisions
- 2,002 (2,002 scored, 0 invalid)
- Engine's best play
- 36.6%
- Blunder rate
- 38.6%
- P95 equity loss
- 0.596
- Mean loss, worst 5%
- 0.846
- Median / P90 s per decision
- 1.57 / 3.38
- $ per 1,000 decisions
- data-v="7.185">7.185estimate · RTX PRO 6000 Blackwell
- Risk judgement, margin 0.001
- 48.5 (95% interval 46.1 to 50.9); risks found 37.3% (264 / 708), right safety 59.8% (455 / 761)
- Risk judgement, margin 0.02
- 48.0 (95% interval 45.5 to 50.7); risks found 37.7% (236 / 626), right safety 58.4% (492 / 843)
- No trade-off
- 533 (left out of every risk metric)
Risk call card
48.5risk judgement, margin 0.001
right risks264
wrong risks198
needless risks306
missed risks246
right safety455
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 2,002
- Eligible (full probability distribution)
- 1,776
- Left out, by reason
- not audited 226
- Close choices
- 98.0% (1,740 / 1,776)
- Ties in the top two
- 0 decisions (0 at a positive probability, 0 at 0; the two most likely plays tied: 0, the second tied with the third: 0); 0 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 37.4% (651 / 1,740) of the close choices; 36.7% (651 / 1,776) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 1,740 | 0.208median 0.151 | 0.175median 0.133 | 0.383median 0.282 |
| Other decisions | 36 | 0.408median 0.313 | 0.172median 0.118 | 0.579median 0.430 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 132 | 463 | 320 | 197 | 125 | 151 | 69 | 98 | 4 | 181 |
| Other decisions | 0 | 0 | 8 | 8 | 6 | 1 | 0 | 0 | 2 | 11 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 1,740 | 55.3%963 / 1,740 | 35.5%617 / 1,740 | 7.5%131 / 1,740 | 30.9%537 / 1,740 | 366 / 717 / 449 / 199 / 9 |
| Other decisions | 36 | 41.7%15 / 36, under 200 decisions | 55.6%20 / 36, under 200 decisions | 8.3%3 / 36, under 200 decisions | 50.0%18 / 36, under 200 decisions | 5 / 12 / 13 / 6 / 0 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 1,740 | 0.147median 0.080 | 0.132median 0.034 | 0.175median 0.086 |
| Other decisions | 36 | 0.147median 0.133 | 0.000median 0.000 | 0.147median 0.133 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 44.0%271 / 616 | 56.5%380 / 672 |
| margin 0.02 | 42.4%230 / 543 | 56.5%421 / 745 |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 99.0%616 / 622 | 97.5%672 / 689 | 97.2%452 / 465 | 98.6%73 / 74, under 200 decisions | 98.5%200 / 203 | 99.4%343 / 345 |
| margin 0.02 | 99.1%543 / 548 | 97.6%745 / 763 | 97.2%452 / 465 | N/A0 / 0 | 98.5%200 / 203 | 99.4%343 / 345 |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001271 decisions | N/A57.2% (155 / 271) | N/AN/A (0 / 0) | N/A: no other decisions in this group | N/A |
| margin 0.02230 decisions | N/A56.5% (130 / 230) | N/AN/A (0 / 0) | N/A: no other decisions in this group | N/A |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 35.4 | 296 |
| Contact | 87.4 | 1,217 |
| Holding game | Too few decisions | 47 |
| Race | 29.5 | 312 |
| Bear-off | Too few decisions | 130 |