decision model · League 1
Jev 1.13
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- jev-1.13.0
- Provider or endpoint
- TypeSafe API
- Date run
- 2026-10-19 to 2026-10-26
- Price
- $0.042 / $0.0 per million tokens in / out; API price; price date 2026-10-08
- Model type
- decision model
- Weights
- Missing
- Runs on
- API
- Reasoning setting
- none (decision model)
- Confidence source
- decision-model probability
- Rules text
- long
- Option groups
- up to 255 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; no decision in this data was split
- Sampling settings
- not sent (decision request)
- Notes
- TypeSafe System One API
Missing B7 fields: weights
Results in League 1
- Error rate (PR)
- 34.431.0 to 38.0
- Decisions
- 1,911 (1,911 scored, 0 invalid)
- Engine's best play
- 50.7%
- Blunder rate
- 25.1%
- P95 equity loss
- 0.352
- Mean loss, worst 5%
- 0.534
- Median / P90 s per decision
- 0.22 / 0.26
- $ per 1,000 decisions
- 0.166API price
- Risk judgement, margin 0.001
- 59.5 (95% interval 57.4 to 61.6); risks found 27.0% (190 / 705), right safety 92.1% (710 / 771)
- Risk judgement, margin 0.02
- 58.4 (95% interval 56.2 to 60.8); risks found 26.8% (167 / 622), right safety 90.0% (769 / 854)
- No trade-off
- 435 (left out of every risk metric)
Risk call card
59.5risk judgement, margin 0.001
right risks190
wrong risks57
needless risks61
missed risks458
right safety710
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 1,911
- Eligible (full probability distribution)
- 1,687
- Left out, by reason
- not audited 224
- Close choices
- 42.3% (713 / 1,687)
- Ties in the top two
- 108 decisions (106 at a positive probability, 2 at 0; the two most likely plays tied: 22, the second tied with the third: 86); 44 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 42.2% (301 / 713) of the close choices; 17.8% (301 / 1,687) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 713 | 0.397median 0.390 | 0.284median 0.270 | 0.681median 0.680 |
| Other decisions | 974 | 0.681median 0.687 | 0.148median 0.150 | 0.829median 0.850 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 6 | 30 | 87 | 129 | 136 | 132 | 83 | 110 |
| Other decisions | 0 | 0 | 0 | 0 | 10 | 51 | 121 | 179 | 238 | 375 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 713 | 50.8%362 / 713 | 32.1%229 / 713 | 13.3%95 / 713 | 27.6%197 / 713 | 201 / 268 / 136 / 89 / 19 |
| Other decisions | 974 | 56.9%554 / 974 | 38.8%378 / 974 | 12.1%118 / 974 | 32.4%316 / 974 | 176 / 391 / 261 / 131 / 15 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 713 | 0.113median 0.057 | 0.093median 0.018 | 0.105median 0.046 |
| Other decisions | 974 | 0.154median 0.095 | 0.052median 0.000 | 0.152median 0.090 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 63.7%209 / 328 | 42.0%92 / 219 |
| margin 0.02 | 63.9%186 / 291 | 44.9%115 / 256 |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 52.7%328 / 622 | 32.0%219 / 685 | 43.7%166 / 380 | 52.9%37 / 70, under 200 decisions | 50.0%84 / 168, under 200 decisions | 53.9%207 / 384 |
| margin 0.02 | 52.7%291 / 552 | 33.9%256 / 755 | 43.7%166 / 380 | N/A0 / 0 | 50.0%84 / 168, under 200 decisions | 53.9%207 / 384 |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001423 decisions | 56.8%crude 57.4% (120 / 209) | 79.8%crude 79.9% (171 / 214) | −23.0 points95% interval −29.9 to −15.8 | −22.5 points95% interval −29.9 to −14.6 |
| margin 0.02379 decisions | 56.3%crude 57.0% (106 / 186), under 200 decisions | 78.9%crude 78.2% (151 / 193), under 200 decisions | −22.6 points95% interval −31.4 to −13.9 | −21.2 points95% interval −29.3 to −12.6 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 35.0 | 279 |
| Contact | 40.4 | 1,281 |
| Holding game | Too few decisions | 38 |
| Race | 11.1 | 227 |
| Bear-off | Too few decisions | 86 |