decision model · League 1
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- pplx-decider-v1.1-27b
- Provider or endpoint
- NVIDIA RTX PRO 6000 Blackwell (96 GB), bf16; NVIDIA DGX Spark (GB10) from 16 Oct
- Date run
- 2026-10-15 to 2026-10-26
- Price
- no price per token; estimate: GPU time, at most; source content/own_cost.json (wall clock on our machines × hourly rate); price date not recorded
- Model type
- decision model
- Weights
- open
- Runs on
- our GPUs
- Reasoning setting
- none (decision model)
- Confidence source
- decision-model probability
- Rules text
- long
- Option groups
- up to 25 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; in this data 226 decisions were split into groups of 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
- Sampling settings
- not sent (decision request)
- Notes
- Perplexity's released checkpoint on our GPUs; moved to a DGX Spark on 16 Oct 11:04 UTC (same checkpoint and server arguments; 20 replayed requests: same play 20/20)
Missing B7 fields: price.date
Results in League 1
- Error rate (PR)
- 38.835.3 to 42.5
- Decisions
- 2,052 (2,052 scored, 0 invalid)
- Engine's best play
- 47.1%
- Blunder rate
- 26.8%
- P95 equity loss
- 0.389
- Mean loss, worst 5%
- 0.569
- Median / P90 s per decision
- 1.97 / 7.48
- $ per 1,000 decisions
- data-v="6.593">6.593estimate · RTX PRO 6000 Blackwell
- Risk judgement, margin 0.001
- 56.9 (95% interval 54.9 to 58.8); risks found 33.1% (233 / 703), right safety 80.7% (682 / 845)
- Risk judgement, margin 0.02
- 56.7 (95% interval 54.7 to 58.7); risks found 33.7% (211 / 626), right safety 79.7% (735 / 922)
- No trade-off
- 504 (left out of every risk metric)
Risk call card
56.9risk judgement, margin 0.001
right risks233
wrong risks67
needless risks163
missed risks403
right safety682
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 2,052
- Eligible (full probability distribution)
- 1,633
- Left out, by reason
- asked in groups of options 202; not audited 217
- Close choices
- 40.0% (654 / 1,633)
- Ties in the top two
- 80 decisions (80 at a positive probability, 0 at 0; the two most likely plays tied: 40, the second tied with the third: 40); 56 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 46.2% (302 / 654) of the close choices; 18.5% (302 / 1,633) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 654 | 0.391median 0.385 | 0.279median 0.270 | 0.670median 0.665 |
| Other decisions | 979 | 0.717median 0.729 | 0.134median 0.131 | 0.850median 0.892 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 4 | 29 | 80 | 121 | 150 | 120 | 72 | 78 |
| Other decisions | 0 | 0 | 0 | 2 | 19 | 53 | 98 | 137 | 202 | 468 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 654 | 54.4%356 / 654 | 26.8%175 / 654 | 14.8%97 / 654 | 28.3%185 / 654 | 160 / 262 / 154 / 69 / 9 |
| Other decisions | 979 | 54.3%532 / 979 | 33.9%332 / 979 | 12.4%121 / 979 | 30.8%302 / 979 | 205 / 394 / 259 / 109 / 12 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 654 | 0.134median 0.072 | 0.101median 0.034 | 0.110median 0.049 |
| Other decisions | 979 | 0.143median 0.079 | 0.053median 0.000 | 0.140median 0.065 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 66.4%170 / 256 | 50.2%132 / 263 |
| margin 0.02 | 66.5%151 / 227 | 51.7%151 / 292 |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 48.5%256 / 528 | 39.0%263 / 674 | 31.3%135 / 431 | 50.9%29 / 57, under 200 decisions | 44.3%74 / 167, under 200 decisions | 50.3%153 / 304 |
| margin 0.02 | 48.2%227 / 471 | 39.9%292 / 731 | 31.3%135 / 431 | N/A0 / 0 | 44.3%74 / 167, under 200 decisions | 50.3%153 / 304 |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001365 decisions | 58.3%crude 57.1% (97 / 170), under 200 decisions | 76.2%crude 75.4% (147 / 195), under 200 decisions | −17.9 points95% interval −27.0 to −5.9 | −18.3 points95% interval −26.9 to −9.7 |
| margin 0.02326 decisions | 57.6%crude 56.3% (85 / 151), under 200 decisions | 76.7%crude 75.4% (132 / 175), under 200 decisions | −19.0 points95% interval −29.0 to −6.3 | −19.1 points95% interval −28.8 to −9.3 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 38.0 | 304 |
| Contact | 46.1 | 1,277 |
| Holding game | Too few decisions | 75 |
| Race | 20.8 | 251 |
| Bear-off | Too few decisions | 145 |