chat model, one letter · League 3
Kimi K3.1 Prooff
FinishedPublished
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- moonshotai/kimi-k3.1-pro
- Provider or endpoint
- Moonshot AI's standard endpoint via OpenRouter (endpoint tag moonshotai/mxfp4, quantization mxfp4; not the highspeed endpoint), provider pinned, no fallbacks
- Date run
- 24–27 Oct 2026
- Price
- $2.0 / $10.0 per million tokens in / out; API price; price date 2026-10-24
- Model type
- chat model, one letter
- Weights
- Missing
- Runs on
- API
- Reasoning setting
- reasoning off: every request sent reasoning_effort "none" (the lowest effort OpenRouter lists for this model on 2026-10-24); every response reported 0 reasoning tokens
- Confidence source
- none
- Rules text
- long
- Option groups
- up to 26 options per request (letters A-Z); more legal plays are split into equal groups, a winner per group, then among the winners; in this data 131 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
- Sampling settings
- temperature 1.0
- Notes
- Moonshot's Kimi K3.1 Pro through OpenRouter, pinned to Moonshot AI's standard endpoint (mxfp4) for every one of its 1,462 league decisions; never on our GPUs. Late entry to League 3 on 24 Oct 2026 (its two missing pairings were played on 27 Oct). Every request sent reasoning_effort "none"; all 1,933 responses reported 0 reasoning tokens and carried no reasoning text. The endpoint returned no token probabilities, so it answers with one letter. Prompts are billed at list price; a separate $30 budget cap applied to Kimi.
Missing B7 fields: weights (not verified for K3.1 Pro)
Results in League 3
- Error rate (PR)
- 28.024.5 to 31.8
- Decisions
- 1,167 (1,167 scored, 0 invalid)
- Engine's best play
- 51.2%
- Blunder rate
- 21.7%
- P95 equity loss
- 0.267
- Mean loss, worst 5%
- 0.448
- Median / P90 s per decision
- 2.11 / 7.23
- $ per 1,000 decisions
- 3.982API price
- Risk judgement, margin 0.001
- 65.9 (95% interval 63.2 to 68.9); risks found 42.4% (153 / 361), right safety 89.5% (416 / 465)
- Risk judgement, margin 0.02
- 66.7 (95% interval 63.7 to 69.8); risks found 44.7% (143 / 320), right safety 88.7% (449 / 506)
- No trade-off
- 341 (left out of every risk metric)
Risk call card
65.9risk judgement, margin 0.001
right risks153
wrong risks31
needless risks49
missed risks177
right safety416
Hit risk only: being hit. Gammon risk and priming are not in it.
Close choices
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).
Probabilities over plays, not sampled reasoning. Method
- Decisions
- 1,167
- Eligible (full probability distribution)
- 0
- Left out, by reason
- one letter only, or a program 1,167
- Close choices
- N/A no full probability distribution
By game phase
| Phase | Error rate (PR) | Decisions |
|---|---|---|
| Opening | Too few decisions | 168 |
| Contact | 36.6 | 717 |
| Holding game | Too few decisions | 53 |
| Race | Too few decisions | 158 |
| Bear-off | Too few decisions | 71 |