chat model, answer-letter probabilities · League 2

DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release

FinishedPublished

Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
deepseek-v4.1-flash
Provider or endpoint
Fireworks via OpenRouter (quantization not published) until 20 Oct 21:56 UTC, then 4 × NVIDIA RTX PRO 6000 Blackwell (96 GB) (SGLang, published FP4/FP8 weights, 17 Oct revision)
Date run
2026-10-19 to 2026-10-26
Price
$0.3 / $1.2 per million tokens in / out; estimate: GPU time, at most; price date not recorded
Model type
chat model, answer-letter probabilities
Weights
open
Runs on
API
Reasoning setting
thinking on, reasoning effort low
Confidence source
letter probability
Rules text
long
Option groups
up to 26 options per request (letters A-Z); more legal plays are split into equal groups, a winner per group, then among the winners; in this data 295 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
Sampling settings
temperature 1.0
Notes
Fireworks via OpenRouter (model ID deepseek/deepseek-v4.1-flash, quantization not published) until 20 Oct 21:56 UTC, then the published weights on our GPUs (SGLang); letter probabilities read from the answer after the reasoning; the price stays Fireworks' list price for comparison
Earlier name
DeepSeek-V4.1-Flash (on) (renamed on 28 Oct 2026 to the reasoning effort its requests sent)

Missing B7 fields: price.date

Results in League 2

Error rate (PR)
19.016.6 to 21.3
Decisions
2,226 (2,188 scored, 38 invalid)
Engine's best play
59.7%
Blunder rate
13.9%
P95 equity loss
0.197
Mean loss, worst 5%
0.372
Median / P90 s per decision
43.8 / 173.1
$ per 1,000 decisions
data-v="47.092">47.092estimate · RTX PRO 6000 Blackwell + API
Risk judgement, margin 0.001
69.0 (95% interval 67.3 to 70.9); risks found 45.0% (338 / 751), right safety 93.0% (891 / 958)
Risk judgement, margin 0.02
70.2 (95% interval 68.2 to 72.2); risks found 49.2% (315 / 640), right safety 91.2% (975 / 1,069)
No trade-off
479 (left out of every risk metric)

JSON

Risk call card

69.0risk judgement, margin 0.001

right risks338
wrong risks45
needless risks67
missed risks368
right safety891

Hit risk only: being hit. Gammon risk and priming are not in it.

Close choices

A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).

Probabilities over plays, not sampled reasoning. Method

League 2: complete letter-readout records are mostly few-option positions, a selected subset.

Decisions
2,226
Eligible (full probability distribution)
408
Left out, by reason
invalid answers 38; asked in groups of options 245; incomplete distribution (an option without its own probability) 1,270; not audited 265
Close choices
0.0% (0 / 408)
Ties in the top two
14 decisions (14 at a positive probability, 0 at 0; the two most likely plays tied: 0, the second tied with the third: 14); 0 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
Between a safe play and a riskier one
N/A (0 / 0) of the close choices; 0.0% (0 / 408) of all eligible decisions
Probabilities of the two most likely plays, mean with median
DecisionsNumberp1 (most likely play)p2 (second)p1 + p2
Close choices0N/AN/AN/A
Other decisions4081.000median 1.0000.000median 0.0001.000median 1.000
How much probability the two plays hold together: decisions per bin
p1 + p20.0–0.10.1–0.20.2–0.30.3–0.40.4–0.50.5–0.60.6–0.70.7–0.80.8–0.90.9–1.0
Close choices0000000000
Other decisions000000000408
What differs between the two most likely plays (several can differ; none is picked as the cause)
DecisionsNumberHitting rollsNewly made pointsPip countsLongest primeNone / 1 / 2 / 3 / 4 differ
Close choices0N/A0 / 0N/A0 / 0N/A0 / 0N/A0 / 00 / 0 / 0 / 0 / 0
Other decisions40843.6%178 / 40814.7%60 / 40816.4%67 / 40828.2%115 / 408122 / 170 / 99 / 16 / 1
Engine values (GNUbg 2-ply, equity), mean with median
DecisionsNumberGap between the two plays' valuesMost likely play: behind the engine's bestSecond most likely play: behind the engine's best
Close choices0N/AN/AN/A
Other decisions4080.152median 0.0650.014median 0.0000.149median 0.052
Close choices between a safe play and a riskier one, by position class
MarginIn Risk pays positionsIn Safety pays positions
margin 0.001N/A0 / 0N/A0 / 0
margin 0.02N/A0 / 0N/A0 / 0
Close choices among eligible decisions, by position class and stake band
MarginRisk paysSafety paysNo trade-offRisk pays, small stakeRisk pays, medium stakeRisk pays, big stake
margin 0.0010.0%0 / 54, under 200 decisions0.0%0 / 150, under 200 decisions0.0%0 / 2040.0%0 / 9, under 200 decisions0.0%0 / 20, under 200 decisions0.0%0 / 25, under 200 decisions
margin 0.020.0%0 / 45, under 200 decisions0.0%0 / 159, under 200 decisions0.0%0 / 204N/A0 / 00.0%0 / 20, under 200 decisions0.0%0 / 25, under 200 decisions

At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.

Missed risks where the risk is worth taking, close choices vs other decisions
MarginClose choices: missed risksOther decisions: missed risks (weighted to the close choices' mix)Difference, close minus otherCrude difference
margin 0.00145 decisionsN/AN/A (0 / 0)71.1%71.1% (32 / 45), under 200 decisionsN/A: no close choices in this groupN/A
margin 0.0236 decisionsN/AN/A (0 / 0)63.9%63.9% (23 / 36), under 200 decisionsN/A: no close choices in this groupN/A

By game phase

PhaseError rate (PR)Decisions
Opening16.9296
Contact23.21,434
Holding gameToo few decisions86
Race2.3231
Bear-offToo few decisions141