decision model · League 1
GPT-6 LunaDecisions API
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- gpt-6-luna
- Provider or endpoint
- OpenAI Decisions API
- Date run
- 2026-10-19 to 2026-10-26
- Price
- $0.1 / $0.0 per million tokens in / out; API price; price date 2026-10-08
- Model type
- decision model
- Weights
- Missing
- Runs on
- API
- Reasoning setting
- none (decision model)
- Confidence source
- decision-model probability
- Rules text
- long
- Option groups
- up to 255 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; no decision in this data was split
- Sampling settings
- not sent (decision request)
- Notes
- OpenAI Decisions API (public beta); an earlier League 1 entry through the ChatGPT subscription (chat model) was removed on 19 Oct
Missing B7 fields: weights
Results in League 1
- Error rate (PR)
- 43.339.7 to 46.9
- Decisions
- 1,962 (1,962 scored, 0 invalid)
- Engine's best play
- 43.2%
- Blunder rate
- 28.9%
- P95 equity loss
- 0.410
- Mean loss, worst 5%
- 0.631
- Median / P90 s per decision
- 0.23 / 0.35
- $ per 1,000 decisions
- 0.335API price
- Risk judgement, margin 0.001
- 55.7 (95% interval 53.8 to 57.6); risks found 21.6% (160 / 740), right safety 89.7% (636 / 709)
- Risk judgement, margin 0.02
- 55.2 (95% interval 53.2 to 57.1); risks found 21.7% (144 / 663), right safety 88.7% (697 / 786)
- No trade-off
- 513 (left out of every risk metric)
Risk call card
55.7risk judgement, margin 0.001
right risks160
wrong risks51
needless risks73
missed risks529
right safety636
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 1,962
- Eligible (full probability distribution)
- 1,741
- Left out, by reason
- not audited 221
- Close choices
- 43.7% (760 / 1,741)
- Ties in the top two
- 123 decisions (122 at a positive probability, 1 at 0; the two most likely plays tied: 29, the second tied with the third: 94); 45 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 38.7% (294 / 760) of the close choices; 16.9% (294 / 1,741) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 760 | 0.409median 0.410 | 0.296median 0.290 | 0.704median 0.710 |
| Other decisions | 981 | 0.670median 0.680 | 0.150median 0.150 | 0.821median 0.840 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 3 | 9 | 29 | 74 | 101 | 139 | 147 | 122 | 136 |
| Other decisions | 0 | 0 | 2 | 8 | 19 | 52 | 121 | 177 | 227 | 375 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 760 | 43.6%331 / 760 | 22.4%170 / 760 | 13.0%99 / 760 | 20.4%155 / 760 | 272 / 290 / 139 / 49 / 10 |
| Other decisions | 981 | 56.8%557 / 981 | 36.3%356 / 981 | 10.6%104 / 981 | 33.2%326 / 981 | 158 / 427 / 284 / 100 / 12 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 760 | 0.107median 0.055 | 0.106median 0.029 | 0.123median 0.049 |
| Other decisions | 981 | 0.146median 0.089 | 0.072median 0.003 | 0.163median 0.100 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 63.0%184 / 292 | 47.4%110 / 232 |
| margin 0.02 | 64.5%169 / 262 | 47.7%125 / 262 |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 44.3%292 / 659 | 36.7%232 / 633 | 52.6%236 / 449 | 41.1%30 / 73, under 200 decisions | 39.4%80 / 203 | 47.5%182 / 383 |
| margin 0.02 | 44.7%262 / 586 | 37.1%262 / 706 | 52.6%236 / 449 | N/A0 / 0 | 39.4%80 / 203 | 47.5%182 / 383 |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001459 decisions | 58.2%crude 58.2% (107 / 184), under 200 decisions | 78.2%crude 80.4% (221 / 275) | −19.9 points95% interval −28.4 to −13.3 | −22.2 points95% interval −28.4 to −16.5 |
| margin 0.02413 decisions | 59.3%crude 59.2% (100 / 169), under 200 decisions | 77.6%crude 79.1% (193 / 244) | −18.3 points95% interval −25.8 to −12.5 | −19.9 points95% interval −25.6 to −13.9 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 40.1 | 293 |
| Contact | 50.9 | 1,272 |
| Holding game | Too few decisions | 30 |
| Race | 19.0 | 231 |
| Bear-off | Too few decisions | 136 |