decision model · League 1
Perplexity Decider Flash v1.2
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- pplx-decider-flash-v1.2
- Provider or endpoint
- NVIDIA RTX PRO 6000 Blackwell (96 GB), bf16
- Date run
- 2026-10-19 to 2026-10-26
- Price
- no price per token; estimate: GPU time, at most; source content/own_cost.json (wall clock on our machines × hourly rate); price date not recorded
- Model type
- decision model
- Weights
- open
- Runs on
- our GPUs
- Reasoning setting
- none (decision model)
- Confidence source
- decision-model probability
- Rules text
- long
- Option groups
- up to 25 options per request; more legal plays are split into equal groups, a winner per group, then among the winners; in this data 224 decisions were split into groups of 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25
- Sampling settings
- not sent (decision request)
- Notes
- Perplexity's pplx-decider-flash-v1.2 (from Qwen3.5-4B) through the pplx_decider package its checkpoint ships, one request at a time; price is Perplexity's list price for the same model
Results in League 1
- Error rate (PR)
- 59.654.9 to 64.5
- Decisions
- 2,006 (2,006 scored, 0 invalid)
- Engine's best play
- 37.4%
- Blunder rate
- 37.5%
- P95 equity loss
- 0.531
- Mean loss, worst 5%
- 0.765
- Median / P90 s per decision
- 0.56 / 0.96
- $ per 1,000 decisions
- data-v="12.267">12.267estimate · RTX PRO 6000 Blackwell
- Risk judgement, margin 0.001
- 49.1 (95% interval 46.5 to 51.6); risks found 31.3% (223 / 712), right safety 66.9% (519 / 776)
- Risk judgement, margin 0.02
- 47.7 (95% interval 45.0 to 50.1); risks found 30.6% (192 / 627), right safety 64.8% (558 / 861)
- No trade-off
- 518 (left out of every risk metric)
Risk call card
49.1risk judgement, margin 0.001
right risks223
wrong risks185
needless risks257
missed risks304
right safety519
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 2,006
- Eligible (full probability distribution)
- 1,629
- Left out, by reason
- asked in groups of options 198; not audited 179
- Close choices
- 68.1% (1,110 / 1,629)
- Ties in the top two
- 120 decisions (120 at a positive probability, 0 at 0; the two most likely plays tied: 43, the second tied with the third: 82); 90 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 42.7% (474 / 1,110) of the close choices; 29.1% (474 / 1,629) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 1,110 | 0.300median 0.269 | 0.221median 0.199 | 0.521median 0.468 |
| Other decisions | 519 | 0.525median 0.526 | 0.163median 0.153 | 0.688median 0.688 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 32 | 184 | 224 | 169 | 130 | 112 | 91 | 51 | 117 |
| Other decisions | 0 | 0 | 11 | 52 | 51 | 77 | 76 | 67 | 83 | 102 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 1,110 | 60.1%667 / 1,110 | 28.7%319 / 1,110 | 11.9%132 / 1,110 | 24.3%270 / 1,110 | 236 / 495 / 259 / 105 / 15 |
| Other decisions | 519 | 60.3%313 / 519 | 42.6%221 / 519 | 11.2%58 / 519 | 28.1%146 / 519 | 85 / 206 / 160 / 60 / 8 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 1,110 | 0.145median 0.078 | 0.124median 0.045 | 0.152median 0.065 |
| Other decisions | 519 | 0.172median 0.108 | 0.065median 0.000 | 0.167median 0.101 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 56.8%245 / 431 | 59.8%229 / 383 |
| margin 0.02 | 56.8%220 / 387 | 59.5%254 / 427 |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 75.7%431 / 569 | 62.3%383 / 615 | 66.5%296 / 445 | 65.7%44 / 67, under 200 decisions | 74.6%138 / 185, under 200 decisions | 78.5%249 / 317 |
| margin 0.02 | 77.1%387 / 502 | 62.6%427 / 682 | 66.5%296 / 445 | N/A0 / 0 | 74.6%138 / 185, under 200 decisions | 78.5%249 / 317 |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001344 decisions | 55.7%crude 55.9% (137 / 245) | 75.9%crude 74.7% (74 / 99), under 200 decisions | −20.2 points95% interval −32.7 to −5.3 | −18.8 points95% interval −29.8 to −8.1 |
| margin 0.02301 decisions | 55.2%crude 55.0% (121 / 220) | 76.1%crude 75.3% (61 / 81), under 200 decisions | −20.9 points95% interval −33.0 to −5.9 | −20.3 points95% interval −31.1 to −8.7 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 50.9 | 291 |
| Contact | 71.1 | 1,283 |
| Holding game | Too few decisions | 25 |
| Race | 34.3 | 283 |
| Bear-off | Too few decisions | 124 |