decision model · League 1
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- pplx-decider-v1.2
- Provider or endpoint
- NVIDIA RTX PRO 6000 Blackwell (96 GB), bf16
- Date run
- 2026-10-20 to 2026-10-26
- Price
- no price per token; estimate: GPU time, at most; source content/own_cost.json (wall clock on our machines × hourly rate); price date not recorded
- Model type
- decision model
- Weights
- open
- Runs on
- our GPUs
- Reasoning setting
- none (decision model)
- Confidence source
- decision-model probability
- Rules text
- long
- Option groups
- up to 25 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; in this data 206 decisions were split into groups of 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
- Sampling settings
- not sent (decision request)
- Notes
- Perplexity's pplx-decider-v1.2 through the pplx_decider package its checkpoint ships, one request at a time; joined late (20 Oct); price is Perplexity's list price for the same model
Results in League 1
- Error rate (PR)
- 36.933.9 to 40.1
- Decisions
- 2,028 (2,028 scored, 0 invalid)
- Engine's best play
- 46.6%
- Blunder rate
- 27.9%
- P95 equity loss
- 0.348
- Mean loss, worst 5%
- 0.510
- Median / P90 s per decision
- 0.94 / 2.73
- $ per 1,000 decisions
- data-v="12.227">12.227estimate · RTX PRO 6000 Blackwell
- Risk judgement, margin 0.001
- 57.9 (95% interval 56.0 to 59.9); risks found 33.4% (244 / 731), right safety 82.5% (682 / 827)
- Risk judgement, margin 0.02
- 56.6 (95% interval 54.6 to 58.7); risks found 33.1% (213 / 644), right safety 80.2% (733 / 914)
- No trade-off
- 470 (left out of every risk metric)
Risk call card
57.9risk judgement, margin 0.001
right risks244
wrong risks77
needless risks145
missed risks410
right safety682
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 2,028
- Eligible (full probability distribution)
- 1,599
- Left out, by reason
- asked in groups of options 187; not audited 242
- Close choices
- 53.0% (847 / 1,599)
- Ties in the top two
- 101 decisions (101 at a positive probability, 0 at 0; the two most likely plays tied: 26, the second tied with the third: 75); 71 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 43.7% (370 / 847) of the close choices; 23.1% (370 / 1,599) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 847 | 0.356median 0.342 | 0.259median 0.244 | 0.615median 0.586 |
| Other decisions | 752 | 0.605median 0.599 | 0.157median 0.152 | 0.762median 0.780 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 40 | 103 | 144 | 155 | 128 | 95 | 73 | 109 |
| Other decisions | 0 | 0 | 1 | 16 | 54 | 83 | 123 | 129 | 142 | 204 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 847 | 53.1%450 / 847 | 24.4%207 / 847 | 13.0%110 / 847 | 27.4%232 / 847 | 226 / 341 / 197 / 68 / 15 |
| Other decisions | 752 | 60.5%455 / 752 | 33.5%252 / 752 | 15.7%118 / 752 | 29.5%222 / 752 | 131 / 309 / 215 / 80 / 17 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 847 | 0.131median 0.075 | 0.087median 0.029 | 0.112median 0.035 |
| Other decisions | 752 | 0.154median 0.096 | 0.045median 0.000 | 0.155median 0.090 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 61.7%205 / 332 | 53.1%165 / 311 |
| margin 0.02 | 61.6%178 / 289 | 54.2%192 / 354 |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 59.2%332 / 561 | 47.9%311 / 649 | 52.4%204 / 389 | 64.2%43 / 67, under 200 decisions | 55.9%99 / 177, under 200 decisions | 59.9%190 / 317 |
| margin 0.02 | 58.5%289 / 494 | 49.4%354 / 716 | 52.4%204 / 389 | N/A0 / 0 | 55.9%99 / 177, under 200 decisions | 59.9%190 / 317 |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001373 decisions | 56.1%crude 56.1% (115 / 205) | 67.1%crude 72.0% (121 / 168), under 200 decisions | −11.0 points95% interval −23.1 to −1.2 | −15.9 points95% interval −26.9 to −6.4 |
| margin 0.02328 decisions | 58.2%crude 57.9% (103 / 178), under 200 decisions | 68.0%crude 72.0% (108 / 150), under 200 decisions | −9.8 points95% interval −20.0 to −0.3 | −14.1 points95% interval −23.4 to −4.5 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 26.1 | 287 |
| Contact | 46.1 | 1,304 |
| Holding game | Too few decisions | 47 |
| Race | 18.5 | 274 |
| Bear-off | Too few decisions | 116 |