chat model, answer-letter probabilities · League 2

Qwen3.8 Flashoff

FinishedPublished

Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
qwen/qwen3.8-flash
Provider or endpoint
Alibaba via OpenRouter, quantization not published
Date run
2026-10-20 to 2026-10-26
Price
$0.15 / $0.47 per million tokens in / out; API price; price date not recorded
Model type
chat model, answer-letter probabilities
Weights
Missing
Runs on
API
Reasoning setting
reasoning off
Confidence source
letter probability
Rules text
long
Option groups
up to 26 options per request (letters A-Z); more legal plays are split into equal groups, a winner per group, then among the winners; in this data 171 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
Sampling settings
temperature 1.0
Notes
OpenRouter, pinned to Alibaba (its only provider); replaces Qwen3.8 Flash Next, which is not on OpenRouter; quantization not published

Missing B7 fields: weights, price.date

Results in League 2

Error rate (PR)
33.630.8 to 36.7
Decisions
2,027 (2,027 scored, 0 invalid)
Engine's best play
44.6%
Blunder rate
25.3%
P95 equity loss
0.308
Mean loss, worst 5%
0.533
Median / P90 s per decision
1.70 / 7.47
$ per 1,000 decisions
0.284API price
Risk judgement, margin 0.001
59.5 (95% interval 57.2 to 61.7); risks found 34.4% (243 / 706), right safety 84.6% (613 / 725)
Risk judgement, margin 0.02
59.3 (95% interval 57.1 to 61.5); risks found 35.5% (222 / 625), right safety 83.0% (669 / 806)
No trade-off
596 (left out of every risk metric)

JSON

Risk call card

59.5risk judgement, margin 0.001

right risks243
wrong risks91
needless risks112
missed risks372
right safety613

Hit risk only: being hit. Gammon risk and priming are not in it.

Close choices

A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).

Probabilities over plays, not sampled reasoning. Method

League 2: complete letter-readout records are mostly few-option positions, a selected subset.

Decisions
2,027
Eligible (full probability distribution)
615
Left out, by reason
asked in groups of options 145; incomplete distribution (an option without its own probability) 1,024; not audited 243
Close choices
40.3% (248 / 615)
Ties in the top two
46 decisions (46 at a positive probability, 0 at 0; the two most likely plays tied: 25, the second tied with the third: 21); 30 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
Between a safe play and a riskier one
29.0% (72 / 248) of the close choices; 11.7% (72 / 615) of all eligible decisions
Probabilities of the two most likely plays, mean with median
DecisionsNumberp1 (most likely play)p2 (second)p1 + p2
Close choices2480.496median 0.5000.354median 0.3490.850median 0.855
Other decisions3670.720median 0.7310.192median 0.1940.911median 0.919
How much probability the two plays hold together: decisions per bin
p1 + p20.0–0.10.1–0.20.2–0.30.3–0.40.4–0.50.5–0.60.6–0.70.7–0.80.8–0.90.9–1.0
Close choices00001426596098
Other decisions000000437110216
What differs between the two most likely plays (several can differ; none is picked as the cause)
DecisionsNumberHitting rollsNewly made pointsPip countsLongest primeNone / 1 / 2 / 3 / 4 differ
Close choices24829.0%72 / 24810.5%26 / 2486.0%15 / 24816.9%42 / 248136 / 76 / 29 / 7 / 0
Other decisions36739.2%144 / 36713.1%48 / 36711.7%43 / 36727.8%102 / 367121 / 163 / 76 / 6 / 1
Engine values (GNUbg 2-ply, equity), mean with median
DecisionsNumberGap between the two plays' valuesMost likely play: behind the engine's bestSecond most likely play: behind the engine's best
Close choices2480.073median 0.0210.044median 0.0010.058median 0.003
Other decisions3670.124median 0.0480.026median 0.0000.109median 0.030
Close choices between a safe play and a riskier one, by position class
MarginIn Risk pays positionsIn Safety pays positions
margin 0.00175.0%27 / 36, under 200 decisions75.0%45 / 60, under 200 decisions
margin 0.0279.3%23 / 29, under 200 decisions73.1%49 / 67, under 200 decisions
Close choices among eligible decisions, by position class and stake band
MarginRisk paysSafety paysNo trade-offRisk pays, small stakeRisk pays, medium stakeRisk pays, big stake
margin 0.00143.4%36 / 83, under 200 decisions31.7%60 / 189, under 200 decisions44.3%152 / 34353.8%7 / 13, under 200 decisions37.1%13 / 35, under 200 decisions45.7%16 / 35, under 200 decisions
margin 0.0241.4%29 / 70, under 200 decisions33.2%67 / 20244.3%152 / 343N/A0 / 037.1%13 / 35, under 200 decisions45.7%16 / 35, under 200 decisions

At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.

Missed risks where the risk is worth taking, close choices vs other decisions
MarginClose choices: missed risksOther decisions: missed risks (weighted to the close choices' mix)Difference, close minus otherCrude difference
margin 0.00165 decisions74.1%crude 74.1% (20 / 27), under 200 decisions88.1%crude 89.5% (34 / 38), under 200 decisions−14.1 points95% interval −37.9 to +9.4−15.4 points95% interval −33.9 to +5.6
margin 0.0255 decisions69.6%crude 69.6% (16 / 23), under 200 decisions86.1%crude 87.5% (28 / 32), under 200 decisions−16.5 points95% interval −42.9 to +10.7−17.9 points95% interval −38.1 to +4.9

By game phase

PhaseError rate (PR)Decisions
Opening36.8298
Contact37.11,337
Holding gameToo few decisions27
Race20.6249
Bear-offToo few decisions116