chat model, one letter · League 3

Kimi K3.1 Prooff

FinishedPublished

Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
moonshotai/kimi-k3.1-pro
Provider or endpoint
Moonshot AI's standard endpoint via OpenRouter (endpoint tag moonshotai/mxfp4, quantization mxfp4; not the highspeed endpoint), provider pinned, no fallbacks
Date run
24–27 Oct 2026
Price
$2.0 / $10.0 per million tokens in / out; API price; price date 2026-10-24
Model type
chat model, one letter
Weights
Missing
Runs on
API
Reasoning setting
reasoning off: every request sent reasoning_effort "none" (the lowest effort OpenRouter lists for this model on 2026-10-24); every response reported 0 reasoning tokens
Confidence source
none
Rules text
long
Option groups
up to 26 options per request (letters A-Z); more legal plays are split into equal groups, a winner per group, then among the winners; in this data 131 decisions were split into groups of 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26
Sampling settings
temperature 1.0
Notes
Moonshot's Kimi K3.1 Pro through OpenRouter, pinned to Moonshot AI's standard endpoint (mxfp4) for every one of its 1,462 league decisions; never on our GPUs. Late entry to League 3 on 24 Oct 2026 (its two missing pairings were played on 27 Oct). Every request sent reasoning_effort "none"; all 1,933 responses reported 0 reasoning tokens and carried no reasoning text. The endpoint returned no token probabilities, so it answers with one letter. Prompts are billed at list price; a separate $30 budget cap applied to Kimi.

Missing B7 fields: weights (not verified for K3.1 Pro)

Results in League 3

Error rate (PR)
28.024.5 to 31.8
Decisions
1,167 (1,167 scored, 0 invalid)
Engine's best play
51.2%
Blunder rate
21.7%
P95 equity loss
0.267
Mean loss, worst 5%
0.448
Median / P90 s per decision
2.11 / 7.23
$ per 1,000 decisions
3.982API price
Risk judgement, margin 0.001
65.9 (95% interval 63.2 to 68.9); risks found 42.4% (153 / 361), right safety 89.5% (416 / 465)
Risk judgement, margin 0.02
66.7 (95% interval 63.7 to 69.8); risks found 44.7% (143 / 320), right safety 88.7% (449 / 506)
No trade-off
341 (left out of every risk metric)

JSON

Risk call card

65.9risk judgement, margin 0.001

right risks153
wrong risks31
needless risks49
missed risks177
right safety416

Hit risk only: being hit. Gammon risk and priming are not in it.

Close choices

A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split).

Probabilities over plays, not sampled reasoning. Method

Decisions
1,167
Eligible (full probability distribution)
0
Left out, by reason
one letter only, or a program 1,167
Close choices
N/A no full probability distribution

By game phase

PhaseError rate (PR)Decisions
OpeningToo few decisions168
Contact36.6717
Holding gameToo few decisions53
RaceToo few decisions158
Bear-offToo few decisions71