chat model, answer-letter probabilities · League 2
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- deepseek-v4.1-flash
- Provider or endpoint
- Fireworks via OpenRouter (quantization not published) until 20 Oct 21:56 UTC, then 4 × NVIDIA RTX PRO 6000 Blackwell (96 GB) (SGLang, published FP4/FP8 weights, 17 Oct revision)
- Date run
- 2026-10-19 to 2026-10-26
- Price
- $0.3 / $1.2 per million tokens in / out; estimate: GPU time, at most; price date not recorded
- Model type
- chat model, answer-letter probabilities
- Weights
- open
- Runs on
- API
- Reasoning setting
- thinking on, reasoning effort low
- Confidence source
- letter probability
- Rules text
- long
- Option groups
- up to 26 options per request (letters A-Z); more legal plays are split into equal groups, a winner per group, then among the winners; in this data 295 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
- Sampling settings
- temperature 1.0
- Notes
- Fireworks via OpenRouter (model ID deepseek/deepseek-v4.1-flash, quantization not published) until 20 Oct 21:56 UTC, then the published weights on our GPUs (SGLang); letter probabilities read from the answer after the reasoning; the price stays Fireworks' list price for comparison
- Earlier name
- DeepSeek-V4.1-Flash (on) (renamed on 28 Oct 2026 to the reasoning effort its requests sent)
Missing B7 fields: price.date
Results in League 2
- Error rate (PR)
- 19.016.6 to 21.3
- Decisions
- 2,226 (2,188 scored, 38 invalid)
- Engine's best play
- 59.7%
- Blunder rate
- 13.9%
- P95 equity loss
- 0.197
- Mean loss, worst 5%
- 0.372
- Median / P90 s per decision
- 43.8 / 173.1
- $ per 1,000 decisions
- data-v="47.092">47.092estimate · RTX PRO 6000 Blackwell + API
- Risk judgement, margin 0.001
- 69.0 (95% interval 67.3 to 70.9); risks found 45.0% (338 / 751), right safety 93.0% (891 / 958)
- Risk judgement, margin 0.02
- 70.2 (95% interval 68.2 to 72.2); risks found 49.2% (315 / 640), right safety 91.2% (975 / 1,069)
- No trade-off
- 479 (left out of every risk metric)
Risk call card
69.0risk judgement, margin 0.001
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
League 2: complete letter-readout records are mostly few-option positions, a selected subset.
- Decisions
- 2,226
- Eligible (full probability distribution)
- 408
- Left out, by reason
- invalid answers 38; asked in groups of options 245; incomplete distribution (an option without its own probability) 1,270; not audited 265
- Close choices
- 0.0% (0 / 408)
- Ties in the top two
- 14 decisions (14 at a positive probability, 0 at 0; the two most likely plays tied: 0, the second tied with the third: 14); 0 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- N/A (0 / 0) of the close choices; 0.0% (0 / 408) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 0 | N/A | N/A | N/A |
| Other decisions | 408 | 1.000median 1.000 | 0.000median 0.000 | 1.000median 1.000 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 |
| Other decisions | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 408 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 0 | N/A0 / 0 | N/A0 / 0 | N/A0 / 0 | N/A0 / 0 | 0 / 0 / 0 / 0 / 0 |
| Other decisions | 408 | 43.6%178 / 408 | 14.7%60 / 408 | 16.4%67 / 408 | 28.2%115 / 408 | 122 / 170 / 99 / 16 / 1 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 0 | N/A | N/A | N/A |
| Other decisions | 408 | 0.152median 0.065 | 0.014median 0.000 | 0.149median 0.052 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | N/A0 / 0 | N/A0 / 0 |
| margin 0.02 | N/A0 / 0 | N/A0 / 0 |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 0.0%0 / 54, under 200 decisions | 0.0%0 / 150, under 200 decisions | 0.0%0 / 204 | 0.0%0 / 9, under 200 decisions | 0.0%0 / 20, under 200 decisions | 0.0%0 / 25, under 200 decisions |
| margin 0.02 | 0.0%0 / 45, under 200 decisions | 0.0%0 / 159, under 200 decisions | 0.0%0 / 204 | N/A0 / 0 | 0.0%0 / 20, under 200 decisions | 0.0%0 / 25, under 200 decisions |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00145 decisions | N/AN/A (0 / 0) | 71.1%71.1% (32 / 45), under 200 decisions | N/A: no close choices in this group | N/A |
| margin 0.0236 decisions | N/AN/A (0 / 0) | 63.9%63.9% (23 / 36), under 200 decisions | N/A: no close choices in this group | N/A |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 16.9 | 296 |
| Contact | 23.2 | 1,434 |
| Holding game | Too few decisions | 86 |
| Race | 2.3 | 231 |
| Bear-off | Too few decisions | 141 |