chat model, answer-letter probabilities · League 2
GLM-5.3low
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- z-ai/glm-5.3
- Provider or endpoint
- Fireworks via OpenRouter, quantization not published
- Date run
- 2026-10-19 to 2026-10-26
- Price
- $1.4 / $4.4 per million tokens in / out; API price; price date not recorded
- Model type
- chat model, answer-letter probabilities
- Weights
- open
- Runs on
- API
- Reasoning setting
- reasoning effort low (reasoning is mandatory on this endpoint, so no off setup)
- Confidence source
- letter probability
- Rules text
- long
- Option groups
- up to 26 options per request (letters A-Z); more legal plays are split into equal groups, a winner per group, then among the winners; in this data 203 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
- Sampling settings
- temperature 1.0
- Notes
- OpenRouter, pinned to Fireworks (no fallbacks); quantization not published
- Earlier name
- GLM-5.3 (on) (renamed on 28 Oct 2026 to the reasoning effort its requests sent)
Missing B7 fields: price.date
Results in League 2
- Error rate (PR)
- 30.927.8 to 34.0
- Decisions
- 1,989 (1,989 scored, 0 invalid)
- Engine's best play
- 50.2%
- Blunder rate
- 21.7%
- P95 equity loss
- 0.310
- Mean loss, worst 5%
- 0.512
- Median / P90 s per decision
- 1.75 / 7.48
- $ per 1,000 decisions
- 2.917API price
- Risk judgement, margin 0.001
- 61.7 (95% interval 59.7 to 63.8); risks found 30.7% (205 / 668), right safety 92.7% (726 / 783)
- Risk judgement, margin 0.02
- 61.6 (95% interval 59.5 to 63.9); risks found 31.8% (184 / 579), right safety 91.4% (797 / 872)
- No trade-off
- 538 (left out of every risk metric)
Risk call card
61.7risk judgement, margin 0.001
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
League 2: complete letter-readout records are mostly few-option positions, a selected subset.
- Decisions
- 1,989
- Eligible (full probability distribution)
- 527
- Left out, by reason
- asked in groups of options 184; incomplete distribution (an option without its own probability) 1,090; not audited 187; one letter only, or a program 1
- Close choices
- 5.5% (29 / 527)
- Ties in the top two
- 17 decisions (17 at a positive probability, 0 at 0; the two most likely plays tied: 4, the second tied with the third: 13); 4 of them close choices. Ties are broken by the order the plays were shown, never by the engine.
- Between a safe play and a riskier one
- 31.0% (9 / 29), under 200 decisions of the close choices; 1.7% (9 / 527) of all eligible decisions
| Decisions | Number | p1 (most likely play) | p2 (second) | p1 + p2 |
|---|---|---|---|---|
| Close choices | 29 | 0.528median 0.525 | 0.382median 0.392 | 0.910median 0.963 |
| Other decisions | 498 | 0.954median 0.982 | 0.036median 0.012 | 0.990median 0.998 |
| p1 + p2 | 0.0–0.1 | 0.1–0.2 | 0.2–0.3 | 0.3–0.4 | 0.4–0.5 | 0.5–0.6 | 0.6–0.7 | 0.7–0.8 | 0.8–0.9 | 0.9–1.0 |
|---|---|---|---|---|---|---|---|---|---|---|
| Close choices | 0 | 0 | 0 | 0 | 0 | 0 | 3 | 4 | 2 | 20 |
| Other decisions | 0 | 0 | 0 | 0 | 0 | 0 | 0 | 2 | 7 | 489 |
| Decisions | Number | Hitting rolls | Newly made points | Pip counts | Longest prime | None / 1 / 2 / 3 / 4 differ |
|---|---|---|---|---|---|---|
| Close choices | 29 | 31.0%9 / 29, under 200 decisions | 17.2%5 / 29, under 200 decisions | 3.4%1 / 29, under 200 decisions | 17.2%5 / 29, under 200 decisions | 17 / 7 / 2 / 3 / 0 |
| Other decisions | 498 | 40.6%202 / 498 | 14.5%72 / 498 | 13.3%66 / 498 | 25.7%128 / 498 | 160 / 223 / 100 / 15 / 0 |
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 29 | 0.113median 0.041 | 0.078median 0.008 | 0.037median 0.000 |
| Other decisions | 498 | 0.134median 0.054 | 0.026median 0.000 | 0.127median 0.042 |
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 100.0%3 / 3, under 200 decisions | 60.0%6 / 10, under 200 decisions |
| margin 0.02 | 100.0%3 / 3, under 200 decisions | 60.0%6 / 10, under 200 decisions |
| Margin | Risk pays | Safety pays | No trade-off | Risk pays, small stake | Risk pays, medium stake | Risk pays, big stake |
|---|---|---|---|---|---|---|
| margin 0.001 | 4.0%3 / 75, under 200 decisions | 5.6%10 / 178, under 200 decisions | 5.8%16 / 274 | 0.0%0 / 12, under 200 decisions | 8.0%2 / 25, under 200 decisions | 2.6%1 / 38, under 200 decisions |
| margin 0.02 | 4.8%3 / 63, under 200 decisions | 5.3%10 / 190, under 200 decisions | 5.8%16 / 274 | N/A0 / 0 | 8.0%2 / 25, under 200 decisions | 2.6%1 / 38, under 200 decisions |
At the 0.02 margin a Risk pays position has a stake of at least 0.02, so the small band is empty. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00162 decisions | 0.0%crude 0.0% (0 / 3), under 200 decisions | 84.7%crude 83.1% (49 / 59), under 200 decisions | −84.7 points95% interval −100.0 to −70.0 | −83.1 points95% interval −90.9 to −74.5 |
| margin 0.0251 decisions | 0.0%crude 0.0% (0 / 3), under 200 decisions | 84.7%crude 81.2% (39 / 48), under 200 decisions | −84.7 points95% interval −100.0 to −70.0 | −81.2 points95% interval −90.9 to −71.4 |
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | 29.0 | 297 |
| Contact | 36.4 | 1,257 |
| Holding game | Too few decisions | 50 |
| Race | 21.3 | 254 |
| Bear-off | Too few decisions | 131 |