Leagues
Leagues
Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.
Starting groups
Decision starting group, the original League 1
Swipe for more columns →
| Model and setting | Place | Match points | Played | Won-drawn-lost | Score for-against | Cup place · league moves |
|---|---|---|---|---|---|---|
| GNUbg weakprogram, a reference row: an official place in the match standings, not a model type | 1 | 48 | 18 | 16-0-2 | 101-36 | |
| d1-ultra | 2 | 48 | 18 | 16-0-2 | 90-43 | Hype Cup seed 5Nackgammon knockout↑ now in League 2round 1: League 1 → 2 (promoted) |
| GLiDEAPI | 3 | 37 | 18 | 12-1-5 | 77-49 | Hype Cup seed 6Nackgammon knockout↑ now in League 2round 2: League 1 → 2 (promoted) |
| Jev 1.13 | 4 | 36 | 18 | 12-0-6 | 81-47 | |
| Jev 1.14 | 5 | 36 | 18 | 12-0-6 | 77-52 | ↑ now in League 2round 3: League 1 → 2 (promoted) |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 6 | 36 | 18 | 12-0-6 | 84-46 | |
| Cygnet | 7 | 34 | 18 | 11-1-6 | 75-59 | |
| Clef | 8 | 32 | 18 | 10-2-6 | 69-53 | ↑ now in League 2round 4: League 1 → 2 (promoted) · round 5: League 2 → 1 (relegated) · round 6: League 1 → 2 (final swap (ladder ended)) |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 9 | 30 | 18 | 10-0-8 | 69-55 | |
| celeris-1-decision | 10 | 30 | 18 | 9-3-6 | 74-63 | ↕ back in League 1round 5: League 1 → 2 (promoted) · round 6: League 2 → 1 (final swap (ladder ended)) |
| SubQ Snap Preview | 11 | 28 | 18 | 9-1-8 | 76-58 | |
| GPT-6 LunaDecisions API | 12 | 27 | 18 | 9-0-9 | 62-65 | |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 13 | 27 | 18 | 9-0-9 | 69-60 | |
| Mercury Decide | 14 | 19 | 18 | 6-1-11 | 51-78 | |
| Winnow-12B | 15 | 12 | 18 | 4-0-14 | 51-77 | |
| Safestprogram, a reference row: an official place in the match standings, not a model type | 16 | 12 | 18 | 3-3-12 | 50-83 | |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 17 | 8 | 18 | 2-2-14 | 38-96 | |
| Perplexity Decider Flash v1.2 | 18 | 6 | 18 | 2-0-16 | 36-97 | |
| Randomprogram, a reference row: an official place in the match standings, not a model type | 19 | 0 | 18 | 0-0-18 | 6-119 |
Level on points, ranked by the tiebreak:
- 48 points: GNUbg weak, d1-ultra, in that order on head-to-head: d1-ultra 5–6 GNUbg weak (match #192).
- 36 points: Jev 1.13, Jev 1.14, Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell, in that order on the mini-table of their matches: Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell 0–7 Jev 1.13 (match #416); Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell 3–4 Jev 1.14 (match #415); Jev 1.13 5–1 Jev 1.14 (match #287).
- 30 points: GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell, celeris-1-decision, in that order on head-to-head: celeris-1-decision 1–5 GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell (match #493).
- 27 points: GPT-6 LunaDecisions API, Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark, in that order on head-to-head: Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark 2–4 GPT-6 LunaDecisions API (match #254).
- 12 points: Winnow-12B, Safest, in that order on head-to-head: Winnow-12B 5–3 Safest (match #238).
Workhorse starting group, the original League 2
Swipe for more columns →
| Model and setting | Place | Match points | Played | Won-drawn-lost | Score for-against | Cup place · league moves |
|---|---|---|---|---|---|---|
| GPT-6.1 Sollow | 1 | 48 | 18 | 16-0-2 | 92-37 | Hype Cup seed 3Nackgammon knockout↑ now in League 3round 1: League 2 → 3 (promoted) |
| GPT-6.1 Solhigh | 2 | 42 | 18 | 14-0-4 | 80-41 | Hype Cup seed 4Nackgammon knockout↑ now in League 3round 2: League 2 → 3 (promoted) |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 3 | 39 | 18 | 13-0-5 | 79-54 | Hype Cup seed 8Nackgammon knockout |
| Claude Sonnet 5.5high | 4 | 39 | 18 | 13-0-5 | 82-46 | |
| Gemini 3.8 Flashlow | 5 | 38 | 18 | 12-2-4 | 84-47 | ↕ back in League 2round 3: League 2 → 3 (promoted) · round 4: League 3 → 2 (relegated) · round 5: League 2 → 3 (promoted) · round 6: League 3 → 2 (final swap (ladder ended)) |
| Claude Sonnet 5.5low | 6 | 37 | 18 | 12-1-5 | 82-48 | |
| DeepSeek-V4.1-Flashlow · API · 1017 | 7 | 37 | 18 | 12-1-5 | 88-54 | |
| Gemini 3.8 Flashhigh | 8 | 36 | 18 | 12-0-6 | 77-48 | |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 9 | 33 | 18 | 11-0-7 | 77-55 | |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 10 | 27 | 18 | 9-0-9 | 58-61 | |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 11 | 25 | 18 | 8-1-9 | 63-59 | |
| GNUbg weakprogram, a reference row: an official place in the match standings, not a model type | 12 | 24 | 18 | 8-0-10 | 67-66 | |
| GLM-5.3low | 13 | 21 | 18 | 7-0-11 | 56-78 | |
| DeepSeek-V4.1-Flashoff · API · 1017 | 14 | 15 | 18 | 5-0-13 | 56-82 | |
| Qwen3.8 Flashoff | 15 | 15 | 18 | 5-0-13 | 50-86 | |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 16 | 13 | 18 | 4-1-13 | 56-83 | ↓ now in League 1round 4: League 2 → 1 (relegated) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 12 | 18 | 4-0-14 | 37-91 | ↓ now in League 1round 3: League 2 → 1 (relegated) |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 9 | 18 | 3-0-15 | 39-98 | ↓ now in League 1round 2: League 2 → 1 (relegated) |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 19 | 0 | 18 | 0-0-18 | 13-102 | ↓ now in League 1round 1: League 2 → 1 (relegated) |
Level on points, ranked by the tiebreak:
- 39 points: Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark, Claude Sonnet 5.5high, in that order on head-to-head: Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark 4–2 Claude Sonnet 5.5high (match #202).
- 37 points: Claude Sonnet 5.5low, DeepSeek-V4.1-Flashlow · API · 1017, in that order on head-to-head: Claude Sonnet 5.5low 7–4 DeepSeek-V4.1-Flashlow · API · 1017 (match #613).
- 15 points: DeepSeek-V4.1-Flashoff · API · 1017, Qwen3.8 Flashoff, in that order on head-to-head: DeepSeek-V4.1-Flashoff · API · 1017 6–1 Qwen3.8 Flashoff (match #629).
Frontier starting group, the original League 3
Swipe for more columns →
| Model and setting | Place | Match points | Played | Won-drawn-lost | Score for-against | Cup place · league moves |
|---|---|---|---|---|---|---|
| GPT-6 Astramedium | 1 | 27 | 11 | 9-0-2 | 49-30 | Hype Cup seed 1Nackgammon knockout |
| Claude Fable 5.1high | 2 | 24 | 11 | 8-0-3 | 51-31 | Hype Cup seed 2Nackgammon knockout |
| Grok 4.7high | 3 | 19 | 11 | 6-1-4 | 39-40 | Hype Cup seed 7Nackgammon knockout |
| Claude Opus 5.5high | 4 | 18 | 11 | 6-0-5 | 48-37 | |
| GPT-6 Astralow | 5 | 17 | 11 | 5-2-4 | 44-33 | |
| Gemini 4 Argonhigh | 6 | 16 | 11 | 5-1-5 | 36-43 | |
| Gemini 4 Argonlow | 7 | 16 | 11 | 5-1-5 | 38-45 | ↕ back in League 3round 3: League 3 → 2 (relegated) · round 4: League 2 → 3 (promoted) · round 5: League 3 → 2 (relegated) · round 6: League 2 → 3 (final swap (ladder ended)) |
| Claude Opus 5.5low | 8 | 15 | 11 | 5-0-6 | 38-42 | |
| Kimi K3.1 Prooff | 9 | 15 | 11 | 5-0-6 | 38-38 | |
| Grok 4.7low | 10 | 12 | 11 | 4-0-7 | 33-40 | ↓ now in League 2round 2: League 3 → 2 (relegated) |
| GNUbg weakprogram, a reference row: an official place in the match standings, not a model type | 11 | 8 | 11 | 2-2-7 | 32-54 | |
| Claude Fable 5.1low | 12 | 7 | 11 | 2-1-8 | 37-50 | ↓ now in League 2round 1: League 3 → 2 (relegated) |
Level on points, ranked by the tiebreak:
- 16 points: Gemini 4 Argonhigh, Gemini 4 Argonlow, in that order on head-to-head: Gemini 4 Argonlow 2–4 Gemini 4 Argonhigh (match #432).
- 15 points: Claude Opus 5.5low, Kimi K3.1 Prooff, in that order on head-to-head: Kimi K3.1 Prooff 4–5 Claude Opus 5.5low (match #603).
Groups reflect model profiles, not tournament rankings.
These are fixed starting cohorts, not current ladder ranks or mutually exclusive universal model types.
Each setup stays in the group it started in, the original league it played in before any league move, with that group's own match points: only matches between setups of the same starting group count, wherever they were played (league season, ladder round or gap match); late entries are left out. 3 points a win, 1 a draw. Ties: points, then head-to-head (a mini-table of the matches among three or more), then the lower error rate (rule set on 28 Oct, after the ties were known; no place here needed the error rate). GNUbg weak, the anchor program, is in all three groups. A setup that moved between leagues keeps its row here; where it plays now is in After league moves.
Badges: Hype Cup seed a Cup place (Cups) · ↑ now in League N ↓ now in League N ↕ back in League N league moves, from the ladder's records. Match points are match results only; they are not the Risk, Quality or Confidence measures.
After league moves
League 1
FinishedPublishedSwipe for more columns →
| Model and setting | Place | Played | Won-drawn-lost | Points | Games won-lost | Gammons won | To play | Starting group | League moves |
|---|---|---|---|---|---|---|---|---|---|
| GNUbg weakprogram · CPU | 1 | 18 | 16-0-2 | 48 | 54-22 | 35 | 0 | Decision, Workhorse, Frontier program, original Leagues 1, 2, 3 | |
| celeris-1-decisiondecision model · API | 2 | 18 | 12-2-4 | 38 | 57-39 | 17 | 0 | Decision original League 1 | ↕ back in League 1round 5: League 1 → 2 (promoted) · round 6: League 2 → 1 (final swap (ladder ended)) |
| SubQ Snap Previewdecision model · API | 3 | 18 | 12-1-5 | 37 | 66-38 | 14 | 0 | Decision original League 1 | |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 4 | 18 | 12-1-5 | 37 | 60-40 | 15 | 0 | Decision original League 1 | |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 5 | 18 | 12-0-6 | 36 | 60-36 | 23 | 0 | Decision original League 1 | |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 6 | 18 | 12-0-6 | 36 | 59-39 | 18 | 0 | Decision original League 1 | |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 7 | 18 | 12-0-6 | 36 | 58-28 | 17 | 0 | Workhorse original League 2 | ↓ now in League 1round 4: League 2 → 1 (relegated) |
| Jev 1.13decision model · API | 8 | 18 | 12-0-6 | 36 | 55-39 | 26 | 0 | Decision original League 1 | |
| GPT-6 LunaDecisions APIdecision model · API | 9 | 18 | 11-1-6 | 34 | 51-47 | 11 | 0 | Decision original League 1 | |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 10 | 18 | 10-0-8 | 30 | 54-48 | 23 | 0 | Decision original League 1 | |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 11 | 18 | 10-0-8 | 30 | 50-50 | 15 | 0 | Workhorse original League 2 | ↓ now in League 1round 2: League 2 → 1 (relegated) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 12 | 18 | 9-1-8 | 28 | 51-53 | 9 | 0 | Workhorse original League 2 | ↓ now in League 1round 3: League 2 → 1 (relegated) |
| Mercury Decidedecision model · API | 13 | 18 | 8-1-9 | 25 | 43-53 | 10 | 0 | Decision original League 1 | |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 14 | 18 | 7-0-11 | 21 | 47-53 | 14 | 0 | Decision original League 1 | |
| Safestprogram · CPU | 15 | 18 | 4-2-12 | 14 | 41-65 | 10 | 0 | Decision program, original League 1 | |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 16 | 18 | 3-0-15 | 9 | 31-63 | 11 | 0 | Workhorse original League 2 | ↓ now in League 1round 1: League 2 → 1 (relegated) |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 17 | 18 | 2-1-15 | 7 | 30-70 | 5 | 0 | Decision original League 1 | |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 18 | 18 | 2-0-16 | 6 | 33-65 | 7 | 0 | Decision original League 1 | |
| Randomprogram · CPU | 19 | 18 | 0-0-18 | 0 | 6-58 | 0 | 0 | Decision program, original League 1 |
171 of 171 matches finished, 906 games; started 2026-10-15, finished 2026-10-26. 36,995 of 36,995 decisions engine-scored so far.
Notes
- Won: GNUbg weak, 48 points from 16-0-2 in 18 matches.
- Lowest error rate in league play: Jev 1.13, 34.4 (31.0 to 38.0); no clear difference in league play from GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell, DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release, celeris-1-decision, Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell, Cygnet, Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark.
- LFM2.5-24B-A2B, off → on: lower error rate in league play.
- League comparisons are not paired: each setup met different positions. Paired comparisons come from the shared positions.
- The anchor is GNU Backgammon 1.08 Beginner, x86-64 build; runner changes during the leagues are listed in Method.
Decision quality
Swipe for more columns →
| Model and setting | Error rate (PR), 95% interval | Band on the human scale | Engine's best play | Blunder rate | Median / P90 s per decision | $ per 1,000 decisions | Decisions scored | Invalid answers |
|---|---|---|---|---|---|---|---|---|
| GNUbg weakprogram · CPU | 21.418.8 to 24.3 | Casual (approx.) | 52.0% | 19.0% | 0.01 / 0.01 | Not measured | 1,744 | 0 |
| Jev 1.13decision model · API | 34.431.0 to 38.0 | Distracted (approx.) | 50.7% | 25.1% | 0.22 / 0.26 | 0.166API price | 1,911 | 0 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 35.031.6 to 39.0 | Distracted (approx.) | 47.1% | 24.6% | 1.41 / 2.75 | 10.301estimate · RTX PRO 6000 Blackwell | 2,003†1 split into groups | 0 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 35.631.7 to 39.7 | Distracted (approx.) | 47.9% | 25.2% | 0.48 / 1.63 | 0.728estimate · RTX PRO 6000 Blackwell + API | 1,736†147 split into groups | 0 |
| celeris-1-decisiondecision model · API | 35.632.0 to 39.4 | Distracted (approx.) | 45.8% | 25.8% | 0.17 / 0.33 | 0.150API price | 1,922 | 0 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 36.933.9 to 40.1 | Distracted (approx.) | 46.6% | 27.9% | 0.94 / 2.73 | 12.227estimate · RTX PRO 6000 Blackwell | 2,028†206 split into groups | 0 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 38.635.4 to 41.9 | Distracted (approx.) | 44.0% | 27.5% | 0.37 / 2.96 | 11.168estimate · RTX PRO 6000 Blackwell | 2,016†123 split into groups | 0 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 38.835.3 to 42.5 | Distracted (approx.) | 47.1% | 26.8% | 1.97 / 7.48 | 6.593estimate · RTX PRO 6000 Blackwell | 2,052†226 split into groups | 0 |
| GPT-6 LunaDecisions APIdecision model · API | 43.339.7 to 46.9 | Distracted (approx.) | 43.2% | 28.9% | 0.23 / 0.35 | 0.335API price | 1,962 | 0 |
| SubQ Snap Previewdecision model · API | 43.939.3 to 48.8 | Distracted (approx.) | 42.3% | 31.8% | 0.24 / 0.32 | 0.193API price | 2,200†1 split into groups | 0 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 45.040.7 to 49.9 | Distracted (approx.) | 43.7% | 30.7% | 1.48 / 2.71 | 4.551estimate · RTX PRO 6000 Blackwell | 2,005†180 split into groups | 0 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 47.744.1 to 51.2 | Distracted (approx.) | 39.6% | 32.4% | 324.8 / 668.6 | 38.716estimate · RTX PRO 6000 Blackwell | 2,193†229 split into groups | 0 |
| Mercury Decidedecision model · API | 47.943.3 to 52.9 | Distracted (approx.) | 42.2% | 30.3% | 0.48 / 0.77 | 0.071API price | 1,853 | 0 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 49.745.9 to 53.6 | Distracted (approx.) | 38.9% | 32.3% | 2.46 / 5.45 | 9.615estimate · RTX PRO 6000 Blackwell | 2,015†92 split into groups | 0 |
| Safestprogram · CPU | 51.647.4 to 56.2 | Distracted (approx.) | 37.3% | 35.1% | 0.00 / 0.00 | Not measured | 2,058 | 0 |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 59.654.9 to 64.5 | Distracted (approx.) | 37.4% | 37.5% | 0.56 / 0.96 | 12.267estimate · RTX PRO 6000 Blackwell | 2,006†224 split into groups | 0 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 61.556.3 to 66.8 | Distracted (approx.) | 32.2% | 40.8% | 0.14 / 0.36 | 9.132estimate · RTX PRO 6000 Blackwell | 1,952†182 split into groups | 0 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 65.659.7 to 71.9 | Distracted (approx.) | 36.6% | 38.6% | 1.57 / 3.38 | 7.185estimate · RTX PRO 6000 Blackwell | 2,002 | 0 |
| Randomprogram · CPU | 121.1113.3 to 128.5 | Distracted (approx.) | 16.5% | 61.5% | 0.00 / 0.00 | Not measured | 1,337 | 0 |
† Setups got different inputs: when a position had more legal plays than a setup takes in one request, its plays were split into groups and the group winners compared. The rows marked † had such decisions; the others saw every play at once. Every setup got the same long rules text.
Time and cost
Swipe for more columns →
| Model and setting | $ per 1,000 decisions | Median / P90 s per decision | Mean retry/wait s | Over the time budget | Tokens in / out per decision | Cost coverage | Decisions |
|---|---|---|---|---|---|---|---|
| GNUbg weakprogram · CPU | Not measured | 0.01 / 0.01 | 0.00 | 0.0% | Not measured | 0% | 1,744 |
| Jev 1.13decision model · API | 0.166API price | 0.22 / 0.26 | 0.00 | 0.0% | 3,945 / 257 | 100% | 1,911 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 10.301estimate · RTX PRO 6000 Blackwell | 1.41 / 2.75 | 0.00 | 0.0% | 3,800 / 0 | 100% | 2,003 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 0.728estimate · RTX PRO 6000 Blackwell + API | 0.48 / 1.63 | 15.5 | 0.1% | 3,960 / 9 | 100% | 1,736 |
| celeris-1-decisiondecision model · API | 0.150API price | 0.17 / 0.33 | 0.00 | 0.0% | 3,747 / 0 | 100% | 1,922 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 12.227estimate · RTX PRO 6000 Blackwell | 0.94 / 2.73 | 1.84 | 0.0% | 4,331 / 0 | 100% | 2,028 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 11.168estimate · RTX PRO 6000 Blackwell | 0.37 / 2.96 | 0.00 | 0.0% | 4,593 / 1 | 100% | 2,016 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 6.593estimate · RTX PRO 6000 Blackwell | 1.97 / 7.48 | 3.86 | 0.6% | 4,488 / 0 | 100% | 2,052 |
| GPT-6 LunaDecisions APIdecision model · API | 0.335API price | 0.23 / 0.35 | 0.01 | 0.0% | 3,351 / 0 | 100% | 1,962 |
| SubQ Snap Previewdecision model · API | 0.193API price | 0.24 / 0.32 | 0.00 | 0.0% | 3,866 / 0 | 100% | 2,200 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4.551estimate · RTX PRO 6000 Blackwell | 1.48 / 2.71 | 0.42 | 0.0% | 4,266 / 3 | 100% | 2,005 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 38.716estimate · RTX PRO 6000 Blackwell | 324.8 / 668.6 | 127.7 | 100.0% | 28,836 / 21,668 | 100% | 2,193 |
| Mercury Decidedecision model · API | 0.071API price | 0.48 / 0.77 | 0.01 | 0.0% | 3,567 / 1 | 100% | 1,853 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 9.615estimate · RTX PRO 6000 Blackwell | 2.46 / 5.45 | 0.00 | 0.1% | 4,144 / 0 | 100% | 2,015 |
| Safestprogram · CPU | Not measured | 0.00 / 0.00 | 0.00 | 0.0% | Not measured | 0% | 2,058 |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 12.267estimate · RTX PRO 6000 Blackwell | 0.56 / 0.96 | 0.70 | 0.0% | 4,492 / 0 | 100% | 2,006 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 9.132estimate · RTX PRO 6000 Blackwell | 0.14 / 0.36 | 0.00 | 0.0% | 3,882 / 1 | 100% | 1,952 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 7.185estimate · RTX PRO 6000 Blackwell | 1.57 / 3.38 | 2.02 | 0.0% | 35,473 / 0 | 100% | 2,002 |
| Randomprogram · CPU | Not measured | 0.00 / 0.00 | 0.00 | 0.0% | Not measured | 0% | 1,337 |
Matches (171)
Swipe for more columns →
| Match | Matchday | Score | Status | Games |
|---|---|---|---|---|
| SubQ Snap Preview v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 1 | 5–6 | Finished | 8 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v GNUbg weak | 2 | 0–7 | Finished | 4 |
| Random v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 3 | 0–6 | Finished | 4 |
| Safest v SubQ Snap Preview | 4 | 0–6 | Finished | 6 |
| Cygnet v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 4 | 5–1 | Finished | 6 |
| GNUbg weak v Safest | 5 | 7–3 | Finished | 6 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5 | 7–1 | Finished | 4 |
| GNUbg weak v Random | 6 | 5–0 | Finished | 2 |
| Safest v Random | 7 | 7–0 | Finished | 4 |
| Cygnet v GNUbg weak | 7 | 0–7 | Finished | 4 |
| SubQ Snap Preview v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 8 | 6–0 | Finished | 6 |
| Cygnet v Random | 8 | 5–0 | Finished | 2 |
| GNUbg weak v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 8 | 5–1 | Finished | 4 |
| Random v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 9 | 0–7 | Finished | 4 |
| Safest v Cygnet | 9 | 4–4 | Finished | 8 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Cygnet | 10 | 6–3 | Finished | 6 |
| Safest v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 11 | 4–3 | Finished | 6 |
| SubQ Snap Preview v GNUbg weak | 11 | 1–5 | Finished | 4 |
| Safest v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 12 | 5–5 | Finished | 8 |
| Random v SubQ Snap Preview | 12 | 0–8 | Finished | 4 |
| SubQ Snap Preview v Cygnet | 13 | 3–7 | Finished | 6 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v GNUbg weak | 15 | 0–5 | Finished | 2 |
| Jev 1.13 v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 6–3 | Finished | 8 |
| Jev 1.13 v SubQ Snap Preview | 16 | 2–5 | Finished | 6 |
| Jev 1.13 v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 16 | 6–0 | Finished | 4 |
| Cygnet v Jev 1.13 | 16 | 2–4 | Finished | 6 |
| Random v Jev 1.13 | 16 | 0–6 | Finished | 2 |
| Jev 1.13 v Safest | 16 | 5–4 | Finished | 6 |
| Jev 1.13 v GNUbg weak | 16 | 6–0 | Finished | 4 |
| GNUbg weak v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 7–2 | Finished | 4 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 16 | 8–4 | Finished | 6 |
| GNUbg weak v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 5–0 | Finished | 4 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 17 | 4–3 | Finished | 6 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 1–5 | Finished | 6 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GPT-6 LunaDecisions API | 18 | 2–4 | Finished | 6 |
| GPT-6 LunaDecisions API v SubQ Snap Preview | 18 | 0–7 | Finished | 4 |
| GPT-6 LunaDecisions API v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 18 | 5–3 | Finished | 8 |
| GPT-6 LunaDecisions API v Cygnet | 18 | 5–4 | Finished | 6 |
| GPT-6 LunaDecisions API v Random | 18 | 6–1 | Finished | 4 |
| Safest v GPT-6 LunaDecisions API | 18 | 0–5 | Finished | 4 |
| GPT-6 LunaDecisions API v GNUbg weak | 18 | 1–5 | Finished | 4 |
| Jev 1.13 v GPT-6 LunaDecisions API | 18 | 4–5 | Finished | 6 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 0–7 | Finished | 4 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v SubQ Snap Preview | 18 | 2–4 | Finished | 6 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 18 | 4–5 | Finished | 6 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v Cygnet | 18 | 2–4 | Finished | 6 |
| Random v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 18 | 1–6 | Finished | 4 |
| Safest v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 18 | 5–3 | Finished | 4 |
| Jev 1.13 v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 18 | 7–1 | Finished | 6 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v GPT-6 LunaDecisions API | 18 | 0–5 | Finished | 4 |
| Perplexity Decider Flash v1.2 v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 18 | 2–8 | Finished | 6 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v Winnow-12B | 18 | 4–6 | Finished | 6 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v Mercury Decide | 18 | 0–6 | Finished | 6 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 18 | 4–7 | Finished | 8 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 18 | 5–3 | Finished | 6 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 4–5 | Finished | 6 |
| SubQ Snap Preview v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 6–1 | Finished | 6 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 18 | 6–1 | Finished | 6 |
| Cygnet v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 4–5 | Finished | 6 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Random | 18 | 5–0 | Finished | 2 |
| Safest v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 2–4 | Finished | 6 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Jev 1.13 | 18 | 0–6 | Finished | 4 |
| GPT-6 LunaDecisions API v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 4–3 | Finished | 6 |
| Perplexity Decider Flash v1.2 v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 3–7 | Finished | 6 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Winnow-12B | 18 | 3–6 | Finished | 6 |
| Mercury Decide v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 1–5 | Finished | 6 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 2–5 | Finished | 6 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 6–3 | Finished | 6 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 5–8 | Finished | 8 |
| SubQ Snap Preview v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 6–2 | Finished | 6 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 3–6 | Finished | 6 |
| Cygnet v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 6–0 | Finished | 6 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Random | 18 | 6–0 | Finished | 4 |
| Safest v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 2–5 | Finished | 6 |
| Jev 1.13 v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 3–4 | Finished | 6 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GPT-6 LunaDecisions API | 18 | 4–4 | Finished | 8 |
| Perplexity Decider Flash v1.2 v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 3–6 | Finished | 6 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Winnow-12B | 18 | 2–5 | Finished | 6 |
| Mercury Decide v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 4–3 | Finished | 6 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 18 | 5–3 | Finished | 8 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 18 | 5–2 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v GNUbg weak | 18 | 1–5 | Finished | 4 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 18 | 0–6 | Finished | 4 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 2–6 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 8–0 | Finished | 4 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Perplexity Decider Flash v1.2 | 19 | 5–3 | Finished | 6 |
| SubQ Snap Preview v Perplexity Decider Flash v1.2 | 19 | 5–1 | Finished | 4 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider Flash v1.2 | 19 | 5–3 | Finished | 8 |
| Cygnet v Perplexity Decider Flash v1.2 | 19 | 6–1 | Finished | 4 |
| Perplexity Decider Flash v1.2 v Random | 19 | 7–3 | Finished | 6 |
| Safest v Perplexity Decider Flash v1.2 | 19 | 5–3 | Finished | 6 |
| GNUbg weak v Perplexity Decider Flash v1.2 | 19 | 9–0 | Finished | 4 |
| Jev 1.13 v Perplexity Decider Flash v1.2 | 19 | 5–3 | Finished | 6 |
| Perplexity Decider Flash v1.2 v GPT-6 LunaDecisions API | 19 | 1–6 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 19 | 6–2 | Finished | 4 |
| SubQ Snap Preview v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 2–4 | Finished | 6 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 0–5 | Finished | 4 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v Cygnet | 19 | 3–4 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v Random | 19 | 5–0 | Finished | 2 |
| Safest v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 0–5 | Finished | 4 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v Jev 1.13 | 19 | 5–3 | Finished | 6 |
| GPT-6 LunaDecisions API v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 0–6 | Finished | 4 |
| Perplexity Decider Flash v1.2 v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 1–5 | Finished | 4 |
| Winnow-12B v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 1–5 | Finished | 6 |
| Mercury Decide v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 4–3 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 19 | 6–0 | Finished | 4 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 8–3 | Finished | 6 |
| celeris-1-decision v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 19 | 3–4 | Finished | 6 |
| SubQ Snap Preview v celeris-1-decision | 19 | 4–4 | Finished | 8 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v celeris-1-decision | 19 | 1–5 | Finished | 4 |
| celeris-1-decision v Cygnet | 19 | 2–5 | Finished | 6 |
| celeris-1-decision v Random | 19 | 7–0 | Finished | 4 |
| Safest v celeris-1-decision | 19 | 2–4 | Finished | 6 |
| GNUbg weak v celeris-1-decision | 19 | 3–5 | Finished | 4 |
| Jev 1.13 v celeris-1-decision | 19 | 2–5 | Finished | 6 |
| celeris-1-decision v GPT-6 LunaDecisions API | 19 | 6–3 | Finished | 6 |
| celeris-1-decision v Perplexity Decider Flash v1.2 | 19 | 5–3 | Finished | 6 |
| celeris-1-decision v Winnow-12B | 19 | 5–3 | Finished | 6 |
| celeris-1-decision v Mercury Decide | 19 | 5–5 | Finished | 8 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v celeris-1-decision | 19 | 6–2 | Finished | 4 |
| celeris-1-decision v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 19 | 1–5 | Finished | 4 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell v celeris-1-decision | 19 | 1–5 | Finished | 4 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v celeris-1-decision | 19 | 2–4 | Finished | 6 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v celeris-1-decision | 19 | 0–5 | Finished | 2 |
| celeris-1-decision v DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 19 | 5–3 | Finished | 6 |
| Winnow-12B v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 20 | 1–5 | Finished | 4 |
| Winnow-12B v SubQ Snap Preview | 20 | 2–7 | Finished | 6 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v Winnow-12B | 20 | 1–5 | Finished | 6 |
| Cygnet v Winnow-12B | 20 | 6–2 | Finished | 4 |
| Random v Winnow-12B | 20 | 0–7 | Finished | 4 |
| Winnow-12B v Safest | 20 | 5–3 | Finished | 8 |
| Winnow-12B v GNUbg weak | 20 | 4–5 | Finished | 6 |
| Winnow-12B v Jev 1.13 | 20 | 2–5 | Finished | 4 |
| Winnow-12B v GPT-6 LunaDecisions API | 20 | 4–3 | Finished | 6 |
| Perplexity Decider Flash v1.2 v Winnow-12B | 20 | 5–3 | Finished | 6 |
| Mercury Decide v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 21 | 2–5 | Finished | 6 |
| SubQ Snap Preview v Mercury Decide | 21 | 5–2 | Finished | 6 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v Mercury Decide | 21 | 3–7 | Finished | 6 |
| Mercury Decide v Cygnet | 21 | 4–5 | Finished | 6 |
| Mercury Decide v Random | 21 | 6–1 | Finished | 4 |
| Mercury Decide v Safest | 21 | 4–3 | Finished | 6 |
| Mercury Decide v GNUbg weak | 21 | 0–8 | Finished | 4 |
| Jev 1.13 v Mercury Decide | 21 | 6–0 | Finished | 4 |
| Mercury Decide v GPT-6 LunaDecisions API | 21 | 2–5 | Finished | 6 |
| Perplexity Decider Flash v1.2 v Mercury Decide | 21 | 1–5 | Finished | 4 |
| Mercury Decide v Winnow-12B | 21 | 4–3 | Finished | 6 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 23 | 4–3 | Finished | 6 |
| SubQ Snap Preview v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 23 | 0–7 | Finished | 6 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 23 | 0–6 | Finished | 4 |
| Cygnet v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 23 | 4–2 | Finished | 6 |
| Random v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 23 | 0–8 | Finished | 4 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v Safest | 23 | 4–3 | Finished | 6 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v GNUbg weak | 23 | 4–5 | Finished | 6 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v Jev 1.13 | 23 | 0–7 | Finished | 4 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v GPT-6 LunaDecisions API | 23 | 6–0 | Finished | 4 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider Flash v1.2 | 23 | 8–2 | Finished | 6 |
| Winnow-12B v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 23 | 0–5 | Finished | 4 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell v Mercury Decide | 23 | 7–0 | Finished | 4 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 24 | 4–3 | Finished | 6 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v SubQ Snap Preview | 24 | 2–4 | Finished | 6 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 24 | 2–5 | Finished | 6 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v Cygnet | 24 | 4–5 | Finished | 6 |
| Random v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 24 | 0–8 | Finished | 4 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v Safest | 24 | 5–1 | Finished | 6 |
| GNUbg weak v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 24 | 7–1 | Finished | 6 |
| Jev 1.13 v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 24 | 2–5 | Finished | 6 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v GPT-6 LunaDecisions API | 24 | 3–4 | Finished | 6 |
| Perplexity Decider Flash v1.2 v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 24 | 0–5 | Finished | 4 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v Winnow-12B | 24 | 5–2 | Finished | 6 |
| Mercury Decide v GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 24 | 0–5 | Finished | 2 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell v Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 24 | 2–6 | Finished | 6 |
This league table's matches. Every match of every stage (ladder rounds, gap matches, late entries, the Hype Cup, the Nackgammon knockout) and the voided ones: Matches.
League 2
FinishedPublishedSwipe for more columns →
| Model and setting | Place | Played | Won-drawn-lost | Points | Games won-lost | Gammons won | To play | Starting group | League moves |
|---|---|---|---|---|---|---|---|---|---|
| Gemini 3.8 Flashlowchat model, one letter · subscription | 1 | 18 | 14-2-2 | 44 | 63-39 | 22 | 0 | Workhorse original League 2 | ↕ back in League 2round 3: League 2 → 3 (promoted) · round 4: League 3 → 2 (relegated) · round 5: League 2 → 3 (promoted) · round 6: League 3 → 2 (final swap (ladder ended)) |
| Claude Sonnet 5.5lowchat model, one letter · subscription | 2 | 18 | 13-1-4 | 40 | 65-41 | 17 | 0 | Workhorse original League 2 | |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 3 | 18 | 13-0-5 | 39 | 62-42 | 11 | 0 | Workhorse original League 2 | |
| DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API | 4 | 18 | 12-1-5 | 37 | 62-46 | 23 | 0 | Workhorse original League 2 | |
| Claude Sonnet 5.5highchat model, one letter · subscription | 5 | 18 | 12-0-6 | 36 | 61-41 | 11 | 0 | Workhorse original League 2 | |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API | 6 | 18 | 12-0-6 | 36 | 59-43 | 15 | 0 | Workhorse original League 2 | |
| Gemini 3.8 Flashhighchat model, one letter · subscription | 7 | 18 | 12-0-6 | 36 | 57-47 | 11 | 0 | Workhorse original League 2 | |
| Grok 4.7low | 8 | 18 | 11-0-7 | 33 | 59-43 | 14 | 0 | Frontier original League 3 | ↓ now in League 2round 2: League 3 → 2 (relegated) |
| Claude Fable 5.1low | 9 | 18 | 10-0-8 | 30 | 56-42 | 13 | 0 | Frontier original League 3 | ↓ now in League 2round 1: League 3 → 2 (relegated) |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 10 | 18 | 10-0-8 | 30 | 52-42 | 14 | 0 | Workhorse original League 2 | |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 11 | 18 | 9-0-9 | 27 | 53-45 | 6 | 0 | Workhorse original League 2 | |
| GNUbg weakprogram · CPU | 12 | 18 | 8-0-10 | 24 | 42-52 | 21 | 0 | Decision, Workhorse, Frontier program, original Leagues 1, 2, 3 | |
| GLM-5.3lowchat model, answer-letter probabilities · API | 13 | 18 | 7-0-11 | 21 | 39-61 | 12 | 0 | Workhorse original League 2 | |
| d1-ultra | 14 | 18 | 6-1-11 | 19 | 46-56 | 10 | 0 | Decision original League 1 | ↑ now in League 2round 1: League 1 → 2 (promoted) |
| DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API | 15 | 18 | 5-1-12 | 16 | 39-63 | 13 | 0 | Workhorse original League 2 | |
| GLiDEAPI | 16 | 18 | 5-0-13 | 15 | 45-59 | 9 | 0 | Decision original League 1 | ↑ now in League 2round 2: League 1 → 2 (promoted) |
| Jev 1.14 | 17 | 18 | 5-0-13 | 15 | 34-58 | 8 | 0 | Decision original League 1 | ↑ now in League 2round 3: League 1 → 2 (promoted) |
| Qwen3.8 Flashoffchat model, answer-letter probabilities · API | 18 | 18 | 2-0-16 | 6 | 33-67 | 8 | 0 | Workhorse original League 2 | |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 19 | 18 | 2-0-16 | 6 | 27-67 | 8 | 0 | Decision original League 1 | ↑ now in League 2round 4: League 1 → 2 (promoted) · round 5: League 2 → 1 (relegated) · round 6: League 1 → 2 (final swap (ladder ended)) |
171 of 171 matches finished, 954 games; started 2026-10-19, finished 2026-10-26. 39,538 of 39,538 decisions engine-scored so far.
Notes
- Won: Gemini 3.8 Flashlow, 44 points from 14-2-2 in 18 matches.
- Lowest error rate in league play: Gemini 3.8 Flashhigh, 14.9 (12.9 to 17.0); no clear difference in league play from Gemini 3.8 Flashlow, Claude Sonnet 5.5high, DeepSeek-V4.1-Flashlow · API · 1017, Claude Fable 5.1low, DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release, Grok 4.7low.
- Claude Sonnet 5.5, low → high: no clear difference in league play.
- DeepSeek-V4.1-Flash1017, off → low: lower error rate in league play.
- Gemini 3.8 Flash, low → high: no clear difference in league play.
- Gemma 4 31B, off → on: no clear difference in league play.
- League comparisons are not paired: each setup met different positions. Paired comparisons come from the shared positions.
- The anchor is GNU Backgammon 1.08 Beginner, x86-64 build; runner changes during the leagues are listed in Method.
Decision quality
Swipe for more columns →
| Model and setting | Error rate (PR), 95% interval | Band on the human scale | Engine's best play | Blunder rate | Median / P90 s per decision | $ per 1,000 decisions | Decisions scored | Invalid answers |
|---|---|---|---|---|---|---|---|---|
| Gemini 3.8 Flashhighchat model, one letter · subscription | 14.912.9 to 17.0 | Intermediate (approx.) | 63.1% | 11.0% | 9.59 / 44.4 | 22.352API-equivalent (subscription) | 2,168†237 split into groups | 0 |
| Gemini 3.8 Flashlowchat model, one letter · subscription | 16.314.5 to 18.2 | Intermediate (approx.) | 59.0% | 13.5% | 2.16 / 6.87 | 4.123API-equivalent (subscription) | 2,053†202 split into groups | 0 |
| Claude Sonnet 5.5highchat model, one letter · subscription | 16.714.9 to 18.7 | Intermediate (approx.) | 62.5% | 13.3% | 6.63 / 17.6 | 19.847API-equivalent (subscription) | 2,164†257 split into groups | 0 |
| DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API | 17.115.1 to 19.2 | Intermediate (approx.) | 61.8% | 13.4% | 22.3 / 107.6 | 4.869API price | 2,222†258 split into groups | 12 |
| Claude Fable 5.1low | 17.915.9 to 20.1 | Casual (approx.) | 58.7% | 15.1% | 1.54 / 6.34 | 62.613API-equivalent (subscription) | 2,008†232 split into groups | 0 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API | 19.016.6 to 21.3 | Casual (approx.) | 59.7% | 13.9% | 43.8 / 173.1 | 47.092estimate · RTX PRO 6000 Blackwell + API | 2,188†295 split into groups | 38 |
| Grok 4.7low | 19.116.9 to 21.4 | Casual (approx.) | 60.8% | 15.1% | 24.3 / 110.2 | 32.813API-equivalent (subscription) | 2,152†250 split into groups | 0 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 19.617.5 to 21.7 | Casual (approx.) | 59.1% | 15.6% | 120.6 / 394.3 | 34.943estimate · RTX PRO 6000 Blackwell | 2,122†252 split into groups | 0 |
| Claude Sonnet 5.5lowchat model, one letter · subscription | 20.117.8 to 22.4 | Casual (approx.) | 60.1% | 15.7% | 1.51 / 7.37 | 13.541API-equivalent (subscription) | 2,150†217 split into groups | 0 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 22.519.9 to 25.3 | Casual (approx.) | 54.4% | 18.2% | 0.31 / 4.03 | 8.215estimate · RTX PRO 6000 Blackwell | 1,847†183 split into groups | 0 |
| GNUbg weakprogram · CPU | 27.024.1 to 29.8 | Beginner (approx.) | 46.2% | 22.1% | 0.01 / 0.01 | Not measured | 2,136 | 0 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 29.026.5 to 31.8 | Beginner (approx.) | 52.8% | 22.4% | 171.1 / 566.7 | 39.023estimate · RTX PRO 6000 Blackwell + API | 2,108†217 split into groups | 0 |
| d1-ultra | 30.327.3 to 33.4 | Distracted (approx.) | 48.6% | 22.4% | 0.23 / 0.41 | 0.145API price | 2,071†1 split into groups | 0 |
| GLM-5.3lowchat model, answer-letter probabilities · API | 30.927.8 to 34.0 | Distracted (approx.) | 50.2% | 21.7% | 1.75 / 7.48 | 2.917API price | 1,989†203 split into groups | 0 |
| Jev 1.14 | 31.728.3 to 35.7 | Distracted (approx.) | 46.2% | 24.7% | 0.19 / 0.24 | 0.173API price | 1,912†2 split into groups | 0 |
| GLiDEAPI | 33.230.2 to 36.4 | Distracted (approx.) | 48.4% | 24.1% | 0.48 / 0.79 | 0.153API price | 2,159†2 split into groups | 0 |
| Qwen3.8 Flashoffchat model, answer-letter probabilities · API | 33.630.8 to 36.7 | Distracted (approx.) | 44.6% | 25.3% | 1.70 / 7.47 | 0.284API price | 2,027†171 split into groups | 0 |
| DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API | 35.231.8 to 38.8 | Distracted (approx.) | 47.7% | 23.6% | 0.84 / 1.39 | 0.244API price | 2,034†169 split into groups | 0 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 36.533.5 to 39.6 | Distracted (approx.) | 44.5% | 27.1% | 1.13 / 5.41 | 5.736estimate · RTX PRO 6000 Blackwell | 1,978†22 split into groups | 0 |
† Setups got different inputs: when a position had more legal plays than a setup takes in one request, its plays were split into groups and the group winners compared. The rows marked † had such decisions; the others saw every play at once. Every setup got the same long rules text.
Time and cost
Swipe for more columns →
| Model and setting | $ per 1,000 decisions | Median / P90 s per decision | Mean retry/wait s | Over the time budget | Tokens in / out per decision | Cost coverage | Decisions |
|---|---|---|---|---|---|---|---|
| Gemini 3.8 Flashhighchat model, one letter · subscription | 22.352API-equivalent (subscription) | 9.59 / 44.4 | 2.29 | 18.4% | 4,344 / 5,092 | 100% | 2,168 |
| Gemini 3.8 Flashlowchat model, one letter · subscription | 4.123API-equivalent (subscription) | 2.16 / 6.87 | 0.04 | 0.7% | 4,311 / 237 | 100% | 2,053 |
| Claude Sonnet 5.5highchat model, one letter · subscription | 19.847API-equivalent (subscription) | 6.63 / 17.6 | 0.26 | 3.4% | 5,664 / 852 | 100% | 2,164 |
| DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API | 4.869API price | 22.3 / 107.6 | 12.1 | 43.4% | 4,215 / 7,173 | 100% | 2,234 |
| Claude Fable 5.1low | 62.613API-equivalent (subscription) | 1.54 / 6.34 | 0.04 | 1.1% | 5,728 / 107 | 100% | 2,008 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API | 47.092estimate · RTX PRO 6000 Blackwell + API | 43.8 / 173.1 | 1.99 | 63.7% | 4,364 / 12,487 | 100% | 2,226 |
| Grok 4.7low | 32.813API-equivalent (subscription) | 24.3 / 110.2 | 14.0 | 42.7% | 5,999 / 3,469 | 100% | 2,152 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 34.943estimate · RTX PRO 6000 Blackwell | 120.6 / 394.3 | 15.3 | 96.4% | 14,746 / 5,599 | 100% | 2,122 |
| Claude Sonnet 5.5lowchat model, one letter · subscription | 13.541API-equivalent (subscription) | 1.51 / 7.37 | 0.01 | 0.5% | 5,579 / 238 | 100% | 2,150 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 8.215estimate · RTX PRO 6000 Blackwell | 0.31 / 4.03 | 0.90 | 1.6% | 4,167 / 1 | 100% | 1,847 |
| GNUbg weakprogram · CPU | Not measured | 0.01 / 0.01 | 0.00 | 0.0% | Not measured | 0% | 2,136 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 39.023estimate · RTX PRO 6000 Blackwell + API | 171.1 / 566.7 | 24.0 | 97.7% | 14,723 / 5,772 | 100% | 2,108 |
| d1-ultra | 0.145API price | 0.23 / 0.41 | 0.00 | 0.0% | 3,461 / 0 | 100% | 2,071 |
| GLM-5.3lowchat model, answer-letter probabilities · API | 2.917API price | 1.75 / 7.48 | 6.69 | 0.6% | 4,005 / 148 | 100% | 1,989 |
| Jev 1.14 | 0.173API price | 0.19 / 0.24 | 0.00 | 0.0% | 4,116 / 292 | 100% | 1,912 |
| GLiDEAPI | 0.153API price | 0.48 / 0.79 | 0.00 | 0.0% | 3,829 / 0 | 100% | 2,159 |
| Qwen3.8 Flashoffchat model, answer-letter probabilities · API | 0.284API price | 1.70 / 7.47 | 8.97 | 0.3% | 4,182 / 10 | 100% | 2,027 |
| DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API | 0.244API price | 0.84 / 1.39 | 24.2 | 0.0% | 3,846 / 1 | 100% | 2,034 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 5.736estimate · RTX PRO 6000 Blackwell | 1.13 / 5.41 | 2.15 | 0.2% | 4,028 / 0 | 100% | 1,978 |
Matches (171)
Swipe for more columns →
| Match | Matchday | Score | Status | Games |
|---|---|---|---|---|
| Gemini 3.8 Flashhigh v GLM-5.3low | 1 | 5–3 | Finished | 6 |
| Gemini 3.8 Flashhigh v Claude Sonnet 5.5low | 2 | 5–4 | Finished | 8 |
| GNUbg weak v Gemini 3.8 Flashhigh | 3 | 5–0 | Finished | 4 |
| Claude Sonnet 5.5high v GLM-5.3low | 3 | 5–3 | Finished | 8 |
| Gemini 3.8 Flashhigh v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 4 | 3–4 | Finished | 6 |
| Claude Sonnet 5.5high v Claude Sonnet 5.5low | 4 | 4–2 | Finished | 6 |
| GLM-5.3low v Claude Sonnet 5.5low | 5 | 0–5 | Finished | 4 |
| GNUbg weak v Claude Sonnet 5.5high | 5 | 0–8 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v Claude Sonnet 5.5high | 6 | 5–1 | Finished | 4 |
| GLM-5.3low v GNUbg weak | 6 | 2–5 | Finished | 4 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v GLM-5.3low | 7 | 3–4 | Finished | 6 |
| GNUbg weak v Claude Sonnet 5.5low | 7 | 3–4 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v Claude Sonnet 5.5low | 8 | 1–7 | Finished | 6 |
| Gemini 3.8 Flashhigh v Claude Sonnet 5.5high | 9 | 4–6 | Finished | 8 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v GNUbg weak | 9 | 4–3 | Finished | 6 |
| Qwen3.8 Flashoff v Gemini 3.8 Flashhigh | 10 | 2–5 | Finished | 6 |
| Claude Sonnet 5.5low v Qwen3.8 Flashoff | 10 | 6–1 | Finished | 6 |
| Claude Sonnet 5.5high v Qwen3.8 Flashoff | 10 | 4–3 | Finished | 6 |
| Qwen3.8 Flashoff v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 10 | 3–6 | Finished | 6 |
| Qwen3.8 Flashoff v GLM-5.3low | 10 | 3–4 | Finished | 6 |
| Qwen3.8 Flashoff v GNUbg weak | 10 | 5–3 | Finished | 6 |
| d1-ultra v GNUbg weak | 10 | 5–6 | Finished | 6 |
| Claude Fable 5.1low v GNUbg weak | 11 | 3–6 | Finished | 6 |
| GNUbg weak v GLiDEAPI | 12 | 5–1 | Finished | 4 |
| d1-ultra v GLiDEAPI | 12 | 5–3 | Finished | 6 |
| Gemini 3.8 Flashhigh v Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 13 | 2–5 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Claude Sonnet 5.5low | 13 | 3–4 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Claude Sonnet 5.5high | 13 | 4–2 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 13 | 1–7 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GLM-5.3low | 13 | 4–2 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GNUbg weak | 13 | 6–4 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Qwen3.8 Flashoff | 13 | 7–1 | Finished | 6 |
| GNUbg weak v Grok 4.7low | 13 | 2–5 | Finished | 6 |
| Claude Fable 5.1low v Grok 4.7low | 13 | 2–4 | Finished | 6 |
| Gemini 3.8 Flashhigh v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 14 | 4–2 | Finished | 6 |
| Claude Sonnet 5.5low v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 14 | 2–5 | Finished | 4 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Claude Sonnet 5.5high | 14 | 2–4 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 14 | 5–2 | Finished | 4 |
| GLM-5.3low v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 14 | 3–5 | Finished | 4 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GNUbg weak | 14 | 2–5 | Finished | 4 |
| Qwen3.8 Flashoff v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 14 | 1–6 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 14 | 5–2 | Finished | 6 |
| Gemini 3.8 Flashhigh v GLiDEAPI | 15 | 1–6 | Finished | 6 |
| Claude Sonnet 5.5low v GLiDEAPI | 15 | 5–3 | Finished | 6 |
| Claude Sonnet 5.5high v GLiDEAPI | 15 | 5–2 | Finished | 4 |
| GLiDEAPI v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 15 | 2–5 | Finished | 6 |
| GLiDEAPI v GLM-5.3low | 15 | 3–4 | Finished | 6 |
| GLiDEAPI v Qwen3.8 Flashoff | 15 | 4–3 | Finished | 6 |
| GLiDEAPI v Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 15 | 2–4 | Finished | 6 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GLiDEAPI | 15 | 4–2 | Finished | 6 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GLiDEAPI | 15 | 6–1 | Finished | 6 |
| Claude Fable 5.1low v GLiDEAPI | 15 | 6–0 | Finished | 4 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Gemini 3.8 Flashhigh | 16 | 1–5 | Finished | 6 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Claude Sonnet 5.5low | 16 | 2–5 | Finished | 6 |
| Claude Sonnet 5.5high v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 3–4 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 5–1 | Finished | 4 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GLM-5.3low | 16 | 5–1 | Finished | 4 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v GNUbg weak | 16 | 5–1 | Finished | 4 |
| Qwen3.8 Flashoff v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 5–3 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 5–0 | Finished | 4 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 2–4 | Finished | 6 |
| Gemini 3.8 Flashhigh v d1-ultra | 16 | 5–1 | Finished | 4 |
| Claude Sonnet 5.5low v d1-ultra | 16 | 5–1 | Finished | 4 |
| d1-ultra v Claude Sonnet 5.5high | 16 | 3–4 | Finished | 6 |
| d1-ultra v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 16 | 2–8 | Finished | 6 |
| GLM-5.3low v d1-ultra | 16 | 5–4 | Finished | 6 |
| Qwen3.8 Flashoff v d1-ultra | 16 | 2–4 | Finished | 6 |
| d1-ultra v Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 2–4 | Finished | 6 |
| d1-ultra v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 2–5 | Finished | 6 |
| d1-ultra v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 4–3 | Finished | 6 |
| Gemini 3.8 Flashhigh v Grok 4.7low | 16 | 5–1 | Finished | 4 |
| Grok 4.7low v Claude Sonnet 5.5low | 16 | 2–5 | Finished | 6 |
| Grok 4.7low v Claude Sonnet 5.5high | 16 | 7–2 | Finished | 6 |
| Grok 4.7low v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 16 | 5–3 | Finished | 8 |
| Grok 4.7low v GLM-5.3low | 16 | 4–5 | Finished | 6 |
| Qwen3.8 Flashoff v Grok 4.7low | 16 | 1–5 | Finished | 4 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Grok 4.7low | 16 | 2–4 | Finished | 6 |
| Grok 4.7low v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 5–1 | Finished | 4 |
| Grok 4.7low v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 16 | 3–4 | Finished | 6 |
| Grok 4.7low v d1-ultra | 16 | 3–4 | Finished | 6 |
| GLiDEAPI v Grok 4.7low | 16 | 3–5 | Finished | 8 |
| GNUbg weak v Jev 1.14 | 16 | 7–2 | Finished | 4 |
| d1-ultra v Jev 1.14 | 16 | 4–2 | Finished | 6 |
| GLiDEAPI v Jev 1.14 | 16 | 5–3 | Finished | 6 |
| Claude Fable 5.1low v Gemini 3.8 Flashhigh | 17 | 4–5 | Finished | 6 |
| Claude Sonnet 5.5low v Claude Fable 5.1low | 17 | 1–5 | Finished | 6 |
| Claude Fable 5.1low v Claude Sonnet 5.5high | 17 | 5–1 | Finished | 6 |
| Claude Fable 5.1low v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 17 | 1–5 | Finished | 4 |
| GLM-5.3low v Claude Fable 5.1low | 17 | 4–3 | Finished | 6 |
| Qwen3.8 Flashoff v Claude Fable 5.1low | 17 | 2–7 | Finished | 6 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Claude Fable 5.1low | 17 | 3–5 | Finished | 6 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Claude Fable 5.1low | 17 | 6–1 | Finished | 6 |
| Claude Fable 5.1low v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 4–2 | Finished | 6 |
| d1-ultra v Claude Fable 5.1low | 17 | 1–5 | Finished | 4 |
| Gemini 3.8 Flashhigh v DeepSeek-V4.1-Flashoff · API · 1017 | 17 | 5–0 | Finished | 4 |
| Claude Sonnet 5.5low v DeepSeek-V4.1-Flashoff · API · 1017 | 17 | 5–3 | Finished | 6 |
| Claude Sonnet 5.5high v DeepSeek-V4.1-Flashoff · API · 1017 | 17 | 6–0 | Finished | 4 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v DeepSeek-V4.1-Flashoff · API · 1017 | 17 | 2–4 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API · 1017 v GLM-5.3low | 17 | 6–7 | Finished | 8 |
| DeepSeek-V4.1-Flashoff · API · 1017 v GNUbg weak | 17 | 6–5 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API · 1017 v Qwen3.8 Flashoff | 17 | 6–1 | Finished | 4 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v DeepSeek-V4.1-Flashoff · API · 1017 | 17 | 5–3 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API · 1017 v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 2–5 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API · 1017 v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 4–3 | Finished | 6 |
| d1-ultra v DeepSeek-V4.1-Flashoff · API · 1017 | 17 | 5–5 | Finished | 8 |
| DeepSeek-V4.1-Flashoff · API · 1017 v Claude Fable 5.1low | 17 | 1–8 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API · 1017 v GLiDEAPI | 17 | 4–6 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API · 1017 v Grok 4.7low | 17 | 1–6 | Finished | 6 |
| Clef v Gemini 3.8 Flashhigh | 17 | 2–4 | Finished | 6 |
| Clef v Claude Sonnet 5.5low | 17 | 2–6 | Finished | 4 |
| Claude Sonnet 5.5high v Clef | 17 | 5–1 | Finished | 4 |
| Clef v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 17 | 0–6 | Finished | 6 |
| GLM-5.3low v Clef | 17 | 2–5 | Finished | 4 |
| Clef v GNUbg weak | 17 | 2–5 | Finished | 6 |
| Qwen3.8 Flashoff v Clef | 17 | 4–5 | Finished | 6 |
| Clef v Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 4–6 | Finished | 6 |
| Clef v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 17 | 0–5 | Finished | 4 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Clef | 17 | 5–1 | Finished | 4 |
| Clef v d1-ultra | 17 | 1–5 | Finished | 4 |
| Claude Fable 5.1low v Clef | 17 | 7–2 | Finished | 6 |
| GLiDEAPI v Clef | 17 | 6–1 | Finished | 6 |
| Grok 4.7low v Clef | 17 | 7–1 | Finished | 4 |
| Clef v DeepSeek-V4.1-Flashoff · API · 1017 | 17 | 4–6 | Finished | 6 |
| Clef v DeepSeek-V4.1-Flashlow · API · 1017 | 17 | 0–8 | Finished | 6 |
| Clef v Jev 1.14 | 17 | 3–4 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API · 1017 v Gemini 3.8 Flashhigh | 18 | 5–4 | Finished | 8 |
| Claude Sonnet 5.5low v DeepSeek-V4.1-Flashlow · API · 1017 | 18 | 7–4 | Finished | 8 |
| DeepSeek-V4.1-Flashlow · API · 1017 v Claude Sonnet 5.5high | 18 | 2–5 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API · 1017 v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 18 | 6–2 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API · 1017 v GLM-5.3low | 18 | 5–2 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API · 1017 v GNUbg weak | 18 | 7–2 | Finished | 6 |
| Qwen3.8 Flashoff v DeepSeek-V4.1-Flashlow · API · 1017 | 18 | 2–5 | Finished | 4 |
| DeepSeek-V4.1-Flashlow · API · 1017 v Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 7–1 | Finished | 6 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v DeepSeek-V4.1-Flashlow · API · 1017 | 18 | 4–2 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API · 1017 v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 2–5 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API · 1017 v d1-ultra | 18 | 5–3 | Finished | 6 |
| Claude Fable 5.1low v DeepSeek-V4.1-Flashlow · API · 1017 | 18 | 1–5 | Finished | 4 |
| GLiDEAPI v DeepSeek-V4.1-Flashlow · API · 1017 | 18 | 2–7 | Finished | 6 |
| DeepSeek-V4.1-Flashlow · API · 1017 v Grok 4.7low | 18 | 4–3 | Finished | 6 |
| DeepSeek-V4.1-Flashoff · API · 1017 v DeepSeek-V4.1-Flashlow · API · 1017 | 18 | 1–7 | Finished | 4 |
| Jev 1.14 v Gemini 3.8 Flashhigh | 18 | 1–5 | Finished | 4 |
| Claude Sonnet 5.5low v Jev 1.14 | 18 | 7–0 | Finished | 6 |
| Jev 1.14 v Claude Sonnet 5.5high | 18 | 0–6 | Finished | 4 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release v Jev 1.14 | 18 | 7–2 | Finished | 6 |
| GLM-5.3low v Jev 1.14 | 18 | 2–5 | Finished | 6 |
| Qwen3.8 Flashoff v Jev 1.14 | 18 | 1–5 | Finished | 6 |
| Jev 1.14 v Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 3–4 | Finished | 6 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Jev 1.14 | 18 | 5–1 | Finished | 4 |
| Jev 1.14 v Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 1–5 | Finished | 6 |
| Jev 1.14 v Claude Fable 5.1low | 18 | 0–6 | Finished | 4 |
| Grok 4.7low v Jev 1.14 | 18 | 5–2 | Finished | 4 |
| DeepSeek-V4.1-Flashoff · API · 1017 v Jev 1.14 | 18 | 0–6 | Finished | 4 |
| DeepSeek-V4.1-Flashlow · API · 1017 v Jev 1.14 | 18 | 3–5 | Finished | 6 |
| Gemini 3.8 Flashlow v Gemini 3.8 Flashhigh | 18 | 2–4 | Finished | 6 |
| Claude Sonnet 5.5low v Gemini 3.8 Flashlow | 18 | 4–4 | Finished | 8 |
| Claude Sonnet 5.5high v Gemini 3.8 Flashlow | 18 | 2–5 | Finished | 6 |
| Gemini 3.8 Flashlow v DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 18 | 6–2 | Finished | 6 |
| GLM-5.3low v Gemini 3.8 Flashlow | 18 | 1–5 | Finished | 4 |
| Gemini 3.8 Flashlow v GNUbg weak | 18 | 8–0 | Finished | 4 |
| Qwen3.8 Flashoff v Gemini 3.8 Flashlow | 18 | 1–6 | Finished | 4 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Gemini 3.8 Flashlow | 18 | 5–1 | Finished | 4 |
| Gemini 3.8 Flashlow v Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 18 | 7–3 | Finished | 6 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark v Gemini 3.8 Flashlow | 18 | 3–5 | Finished | 6 |
| Gemini 3.8 Flashlow v d1-ultra | 18 | 5–2 | Finished | 6 |
| Claude Fable 5.1low v Gemini 3.8 Flashlow | 18 | 1–5 | Finished | 6 |
| GLiDEAPI v Gemini 3.8 Flashlow | 18 | 3–5 | Finished | 6 |
| Gemini 3.8 Flashlow v Grok 4.7low | 18 | 5–2 | Finished | 6 |
| Gemini 3.8 Flashlow v DeepSeek-V4.1-Flashoff · API · 1017 | 18 | 5–3 | Finished | 6 |
| Gemini 3.8 Flashlow v DeepSeek-V4.1-Flashlow · API · 1017 | 18 | 5–5 | Finished | 8 |
| Gemini 3.8 Flashlow v Jev 1.14 | 18 | 5–0 | Finished | 4 |
| Clef v Gemini 3.8 Flashlow | 18 | 2–5 | Finished | 6 |
This league table's matches. Every match of every stage (ladder rounds, gap matches, late entries, the Hype Cup, the Nackgammon knockout) and the voided ones: Matches.
League 3
FinishedPublishedSwipe for more columns →
| Model and setting | Place | Played | Won-drawn-lost | Points | Games won-lost | Gammons won | To play | Starting group | League moves |
|---|---|---|---|---|---|---|---|---|---|
| GPT-6.1 Sollow | 1 | 11 | 9-0-2 | 27 | 41-25 | 11 | 0 | Workhorse original League 2 | ↑ now in League 3round 1: League 2 → 3 (promoted) |
| GPT-6 Astramediumchat model, one letter · subscription | 2 | 11 | 8-0-3 | 24 | 36-26 | 12 | 0 | Frontier original League 3 | |
| GPT-6.1 Solhigh | 3 | 11 | 7-0-4 | 21 | 38-26 | 8 | 0 | Workhorse original League 2 | ↑ now in League 3round 2: League 2 → 3 (promoted) |
| Grok 4.7highchat model, one letter · subscription | 4 | 11 | 6-1-4 | 19 | 34-34 | 3 | 0 | Frontier original League 3 | |
| Claude Opus 5.5highchat model, one letter · subscription | 5 | 11 | 6-0-5 | 18 | 39-31 | 7 | 0 | Frontier original League 3 | |
| Claude Fable 5.1highchat model, one letter · subscription | 6 | 11 | 6-0-5 | 18 | 32-28 | 10 | 0 | Frontier original League 3 | |
| GPT-6 Astralowchat model, one letter · subscription | 7 | 11 | 4-1-6 | 13 | 30-30 | 7 | 0 | Frontier original League 3 | |
| Gemini 4 Argonhighchat model, one letter · subscription | 8 | 11 | 4-1-6 | 13 | 27-39 | 5 | 0 | Frontier original League 3 | |
| Claude Opus 5.5lowchat model, one letter · subscription | 9 | 11 | 4-0-7 | 12 | 31-37 | 7 | 0 | Frontier original League 3 | |
| Kimi K3.1 Prooffchat model, one letter · API | 10 | 11 | 4-0-7 | 12 | 27-31 | 8 | 0 | Frontier original League 3 | |
| Gemini 4 Argonlowchat model, one letter · subscription | 11 | 11 | 3-1-7 | 10 | 27-43 | 5 | 0 | Frontier original League 3 | ↕ back in League 3round 3: League 3 → 2 (relegated) · round 4: League 2 → 3 (promoted) · round 5: League 3 → 2 (relegated) · round 6: League 2 → 3 (final swap (ladder ended)) |
| GNUbg weakprogram · CPU | 12 | 11 | 2-2-7 | 8 | 26-38 | 7 | 0 | Decision, Workhorse, Frontier program, original Leagues 1, 2, 3 |
66 of 66 matches finished, 388 games; started 2026-10-20, finished 2026-10-26. 16,355 of 16,355 decisions engine-scored so far.
Notes
- Won: GPT-6.1 Sollow, 27 points from 9-0-2 in 11 matches.
- Lowest error rate in league play: GPT-6.1 Solhigh, 9.9 (8.6 to 11.3); no clear difference in league play from GPT-6 Astramedium, GPT-6 Astralow, GPT-6.1 Sollow, Gemini 4 Argonhigh.
- Claude Opus 5.5, low → high: no clear difference in league play.
- GPT-6 Astra, low → medium: no clear difference in league play.
- GPT-6.1 Sol, low → high: no clear difference in league play.
- Gemini 4 Argon, low → high: no clear difference in league play.
- League comparisons are not paired: each setup met different positions. Paired comparisons come from the shared positions.
- The anchor is GNU Backgammon 1.08 Beginner, x86-64 build; runner changes during the leagues are listed in Method.
Decision quality
Swipe for more columns →
| Model and setting | Error rate (PR), 95% interval | Band on the human scale | Engine's best play | Blunder rate | Median / P90 s per decision | $ per 1,000 decisions | Decisions scored | Invalid answers |
|---|---|---|---|---|---|---|---|---|
| GPT-6.1 Solhigh | 9.98.6 to 11.3 | Advanced (approx.) | 64.4% | 7.8% | 13.1 / 56.5 | 15.573API-equivalent (subscription) | 1,389†242 split into groups | 0 |
| GPT-6 Astramediumchat model, one letter · subscription | 11.19.5 to 12.9 | Advanced (approx.) | 65.2% | 8.6% | 4.40 / 15.1 | 50.268API-equivalent (subscription) | 1,362†182 split into groups | 0 |
| GPT-6 Astralowchat model, one letter · subscription | 11.89.5 to 14.2 | Advanced (approx.) | 66.3% | 9.4% | 3.94 / 14.0 | 49.737API-equivalent (subscription) | 1,251†169 split into groups | 0 |
| GPT-6.1 Sollow | 12.110.5 to 13.7 | Advanced (approx.) | 65.5% | 9.4% | 4.33 / 14.3 | 10.228API-equivalent (subscription) | 1,344†212 split into groups | 0 |
| Gemini 4 Argonhighchat model, one letter · subscription | 12.610.5 to 14.8 | Intermediate (approx.) | 64.6% | 8.9% | 27.5 / 148.6 | 64.401API-equivalent (subscription) | 1,332†208 split into groups | 36 |
| Claude Fable 5.1highchat model, one letter · subscription | 14.011.5 to 16.3 | Intermediate (approx.) | 64.2% | 11.7% | 12.7 / 47.1 | 136.268API-equivalent (subscription) | 1,286†213 split into groups | 0 |
| Claude Opus 5.5highchat model, one letter · subscription | 14.412.1 to 17.0 | Intermediate (approx.) | 65.9% | 11.4% | 6.35 / 18.5 | 36.400API-equivalent (subscription) | 1,477†178 split into groups | 0 |
| Claude Opus 5.5lowchat model, one letter · subscription | 15.413.1 to 18.0 | Intermediate (approx.) | 61.1% | 13.7% | 3.92 / 10.6 | 29.062API-equivalent (subscription) | 1,411†178 split into groups | 0 |
| Gemini 4 Argonlowchat model, one letter · subscription | 16.513.7 to 19.5 | Intermediate (approx.) | 60.3% | 12.4% | 3.82 / 12.2 | 10.204API-equivalent (subscription) | 1,484†204 split into groups | 0 |
| Grok 4.7highchat model, one letter · subscription | 18.815.9 to 21.8 | Casual (approx.) | 62.5% | 14.0% | 122.3 / 588.6 | 117.379API-equivalent (subscription) | 1,337†195 split into groups | 0 |
| Kimi K3.1 Prooffchat model, one letter · API | 28.024.5 to 31.8 | Beginner (approx.) | 51.2% | 21.7% | 2.11 / 7.23 | 3.982API price | 1,167†131 split into groups | 0 |
| GNUbg weakprogram · CPU | 28.125.0 to 31.4 | Beginner (approx.) | 44.5% | 23.3% | 0.01 / 0.01 | Not measured | 1,479 | 0 |
† Setups got different inputs: when a position had more legal plays than a setup takes in one request, its plays were split into groups and the group winners compared. The rows marked † had such decisions; the others saw every play at once. Every setup got the same long rules text.
Time and cost
Swipe for more columns →
| Model and setting | $ per 1,000 decisions | Median / P90 s per decision | Mean retry/wait s | Over the time budget | Tokens in / out per decision | Cost coverage | Decisions |
|---|---|---|---|---|---|---|---|
| GPT-6.1 Solhigh | 15.573API-equivalent (subscription) | 13.1 / 56.5 | 5.69 | 24.5% | 5,217 / 514 | 100% | 1,389 |
| GPT-6 Astramediumchat model, one letter · subscription | 50.268API-equivalent (subscription) | 4.40 / 15.1 | 0.41 | 2.6% | 4,471 / 111 | 100% | 1,362 |
| GPT-6 Astralowchat model, one letter · subscription | 49.737API-equivalent (subscription) | 3.94 / 14.0 | 0.23 | 1.6% | 4,486 / 98 | 100% | 1,251 |
| GPT-6.1 Sollow | 10.228API-equivalent (subscription) | 4.33 / 14.3 | 0.44 | 1.5% | 4,543 / 114 | 100% | 1,344 |
| Gemini 4 Argonhighchat model, one letter · subscription | 64.401API-equivalent (subscription) | 27.5 / 148.6 | 6.93 | 47.1% | 5,117 / 5,417 | 100% | 1,368 |
| Claude Fable 5.1highchat model, one letter · subscription | 136.268API-equivalent (subscription) | 12.7 / 47.1 | 0.50 | 19.4% | 6,502 / 1,425 | 100% | 1,286 |
| Claude Opus 5.5highchat model, one letter · subscription | 36.400API-equivalent (subscription) | 6.35 / 18.5 | 0.51 | 4.3% | 5,861 / 648 | 100% | 1,477 |
| Claude Opus 5.5lowchat model, one letter · subscription | 29.062API-equivalent (subscription) | 3.92 / 10.6 | 0.03 | 0.6% | 5,896 / 274 | 100% | 1,411 |
| Gemini 4 Argonlowchat model, one letter · subscription | 10.204API-equivalent (subscription) | 3.82 / 12.2 | 0.12 | 1.3% | 4,970 / 27 | 100% | 1,484 |
| Grok 4.7highchat model, one letter · subscription | 117.379API-equivalent (subscription) | 122.3 / 588.6 | 2.21 | 85.9% | 6,400 / 17,430 | 100% | 1,337 |
| Kimi K3.1 Prooffchat model, one letter · API | 3.982API price | 2.11 / 7.23 | 0.03 | 0.1% | 4,353 / 20 | 100% | 1,167 |
| GNUbg weakprogram · CPU | Not measured | 0.01 / 0.01 | 0.00 | 0.0% | Not measured | 0% | 1,479 |
Matches (66)
Swipe for more columns →
| Match | Matchday | Score | Status | Games |
|---|---|---|---|---|
| Claude Fable 5.1high v Claude Opus 5.5high | 1 | 6–5 | Finished | 6 |
| Gemini 4 Argonhigh v Grok 4.7high | 1 | 4–3 | Finished | 6 |
| GPT-6 Astralow v GPT-6 Astramedium | 2 | 2–5 | Finished | 6 |
| Claude Fable 5.1high v Grok 4.7high | 2 | 5–1 | Finished | 6 |
| GPT-6 Astramedium v Claude Opus 5.5low | 3 | 2–5 | Finished | 6 |
| GNUbg weak v GPT-6 Astralow | 3 | 1–5 | Finished | 4 |
| Gemini 4 Argonhigh v GPT-6 Astralow | 4 | 5–2 | Finished | 6 |
| GNUbg weak v GPT-6 Astramedium | 4 | 4–5 | Finished | 6 |
| Claude Opus 5.5high v Claude Opus 5.5low | 4 | 5–2 | Finished | 6 |
| Claude Opus 5.5low v Grok 4.7high | 5 | 3–4 | Finished | 6 |
| Gemini 4 Argonhigh v GNUbg weak | 5 | 4–4 | Finished | 8 |
| Claude Opus 5.5high v GPT-6 Astramedium | 5 | 4–2 | Finished | 6 |
| GPT-6 Astralow v Claude Fable 5.1high | 5 | 5–2 | Finished | 6 |
| Grok 4.7high v Claude Opus 5.5high | 6 | 4–2 | Finished | 6 |
| GNUbg weak v Claude Fable 5.1high | 6 | 0–6 | Finished | 4 |
| Gemini 4 Argonhigh v GPT-6 Astramedium | 6 | 5–6 | Finished | 8 |
| Grok 4.7high v GPT-6 Astramedium | 7 | 1–5 | Finished | 4 |
| Claude Fable 5.1high v Gemini 4 Argonhigh | 7 | 5–0 | Finished | 4 |
| GPT-6 Astramedium v Claude Fable 5.1high | 8 | 5–3 | Finished | 4 |
| GPT-6 Astralow v Claude Opus 5.5low | 8 | 5–0 | Finished | 4 |
| GPT-6.1 Sollow v GNUbg weak | 8 | 5–4 | Finished | 6 |
| GPT-6 Astralow v Claude Opus 5.5high | 9 | 2–5 | Finished | 6 |
| GNUbg weak v Claude Opus 5.5low | 9 | 6–5 | Finished | 8 |
| GNUbg weak v GPT-6.1 Solhigh | 9 | 5–1 | Finished | 4 |
| GPT-6.1 Sollow v GPT-6.1 Solhigh | 9 | 5–2 | Finished | 6 |
| Gemini 4 Argonhigh v Claude Opus 5.5low | 10 | 2–4 | Finished | 6 |
| GNUbg weak v Claude Opus 5.5high | 10 | 1–6 | Finished | 6 |
| Grok 4.7high v GPT-6 Astralow | 10 | 4–2 | Finished | 6 |
| GPT-6 Astralow v GPT-6.1 Sollow | 10 | 1–5 | Finished | 4 |
| GPT-6 Astramedium v GPT-6.1 Sollow | 10 | 5–1 | Finished | 6 |
| GPT-6.1 Sollow v Claude Opus 5.5low | 10 | 7–6 | Finished | 8 |
| Claude Opus 5.5high v GPT-6.1 Sollow | 10 | 5–3 | Finished | 8 |
| GPT-6.1 Sollow v Claude Fable 5.1high | 10 | 8–1 | Finished | 6 |
| Gemini 4 Argonhigh v GPT-6.1 Sollow | 10 | 2–5 | Finished | 6 |
| Grok 4.7high v GPT-6.1 Sollow | 10 | 0–6 | Finished | 6 |
| Claude Opus 5.5low v Claude Fable 5.1high | 11 | 3–4 | Finished | 6 |
| Gemini 4 Argonhigh v Claude Opus 5.5high | 11 | 5–4 | Finished | 6 |
| GNUbg weak v Grok 4.7high | 11 | 5–5 | Finished | 8 |
| GPT-6 Astralow v GPT-6.1 Solhigh | 11 | 4–6 | Finished | 6 |
| GPT-6 Astramedium v GPT-6.1 Solhigh | 11 | 3–5 | Finished | 6 |
| Claude Opus 5.5low v GPT-6.1 Solhigh | 11 | 1–5 | Finished | 6 |
| GPT-6.1 Solhigh v Claude Opus 5.5high | 11 | 3–4 | Finished | 6 |
| GPT-6.1 Solhigh v Claude Fable 5.1high | 11 | 4–2 | Finished | 6 |
| GPT-6.1 Solhigh v Gemini 4 Argonhigh | 11 | 6–0 | Finished | 4 |
| GPT-6.1 Solhigh v Grok 4.7high | 11 | 3–5 | Finished | 8 |
| Gemini 4 Argonlow v GPT-6 Astralow | 11 | 5–5 | Finished | 8 |
| Gemini 4 Argonlow v GPT-6 Astramedium | 11 | 0–5 | Finished | 4 |
| Claude Opus 5.5low v Gemini 4 Argonlow | 11 | 4–2 | Finished | 6 |
| Claude Opus 5.5high v Gemini 4 Argonlow | 11 | 3–5 | Finished | 8 |
| Claude Fable 5.1high v Gemini 4 Argonlow | 11 | 5–6 | Finished | 8 |
| Gemini 4 Argonlow v Gemini 4 Argonhigh | 11 | 2–4 | Finished | 6 |
| Gemini 4 Argonlow v Grok 4.7high | 11 | 1–5 | Finished | 6 |
| GNUbg weak v Gemini 4 Argonlow | 11 | 3–4 | Finished | 6 |
| GPT-6.1 Sollow v Gemini 4 Argonlow | 11 | 4–3 | Finished | 6 |
| Gemini 4 Argonlow v GPT-6.1 Solhigh | 11 | 2–5 | Finished | 6 |
| Kimi K3.1 Prooff v Gemini 4 Argonlow | 11 | 5–2 | Finished | 6 |
| Kimi K3.1 Prooff v GPT-6 Astralow | 12 | 2–5 | Finished | 4 |
| GPT-6 Astramedium v Kimi K3.1 Prooff | 12 | 6–2 | Finished | 6 |
| Kimi K3.1 Prooff v Claude Opus 5.5low | 12 | 4–5 | Finished | 6 |
| Kimi K3.1 Prooff v Claude Opus 5.5high | 12 | 4–3 | Finished | 6 |
| Kimi K3.1 Prooff v Claude Fable 5.1high | 12 | 1–5 | Finished | 4 |
| Kimi K3.1 Prooff v Gemini 4 Argonhigh | 12 | 7–1 | Finished | 6 |
| Grok 4.7high v Kimi K3.1 Prooff | 12 | 5–2 | Finished | 6 |
| GNUbg weak v Kimi K3.1 Prooff | 12 | 0–6 | Finished | 4 |
| GPT-6.1 Sollow v Kimi K3.1 Prooff | 12 | 5–1 | Finished | 4 |
| GPT-6.1 Solhigh v Kimi K3.1 Prooff | 12 | 8–1 | Finished | 6 |
This league table's matches. Every match of every stage (ladder rounds, gap matches, late entries, the Hype Cup, the Nackgammon knockout) and the voided ones: Matches.
The leagues as they stand now, after 24 moves in ladder rounds 1–6 (the ladder ended on 26 Oct; its final swap stands). These leagues mix starting groups, so they keep their numbers: League 1, League 2, League 3. Each row says its starting group and, for a setup that moved, its recorded moves. The leaderboard's League keys and links such as ?league=league-3 refer to these leagues.
Cups
Hype Cup
FinishedQuarter-finals
Quarter-final 1Played
- 1GPT-6 Astramedium (winner)6
- 8Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark1
2026-10-29: match #770
Quarter-final 2Played
- 2Claude Fable 5.1high (winner)6
- 6GLiDEAPI0
2026-10-28: match #767
Quarter-final 3Stored
- 3GPT-6.1 Sollow (winner)6
- 7Grok 4.7high0
Played in Ladder round 1, League 3, 2026-10-24: match #550
Quarter-final 4Stored
- 4GPT-6.1 Solhigh (winner)7
- 5d1-ultra0
Played in Ladder round 1, League 2, 2026-10-23: match #512
Semi-finals
Semi-final AStored
- 1GPT-6 Astramedium3
- 4GPT-6.1 Solhigh (winner)5
Played in Ladder round 2, League 3, 2026-10-24: match #555
Semi-final BStored
- 2Claude Fable 5.1high1
- 3GPT-6.1 Sollow (winner)8
Played in Ladder round 1, League 3, 2026-10-23: match #516
Final
FinalStored
- 4GPT-6.1 Solhigh2
- 3GPT-6.1 Sollow (winner)5
Played in League 2 season, 2026-10-19: match #126
Played (2): a new Cup match. Stored (5): Two setups play each other once: a pairing already played in a league stands, and the bracket shows where it was played. A level match goes to the setup with the lower error rate in that match.
Roster. Teams from the original-league tables: each setup in its original league, only matches within it count (3 points a win, 1 a draw, 0 a loss; tiebreak: head-to-head, then the lower error rate; rule set on 28 Oct, after the ties were known). The top 2 models of each table and the two best 3rd places by error rate; programs and late entries are not eligible.
Nackgammon knockout
FinishedQuarter-finals
Quarter-final 1Played
- 1GPT-6 Astramedium (winner)4
- 8Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark2
2026-10-29: knockout match #3
Quarter-final 2Played
- 2Claude Fable 5.1high (winner)6
- 6GLiDEAPI0
2026-10-28: knockout match #1
Quarter-final 3Played
- 3GPT-6.1 Sollow5
- 7Grok 4.7high (winner)8
2026-10-29: knockout match #4
Quarter-final 4Played
- 4GPT-6.1 Solhigh (winner)6
- 5d1-ultra1
2026-10-28: knockout match #2
Semi-finals
Semi-final APlayed
- 1GPT-6 Astramedium2
- 4GPT-6.1 Solhigh (winner)5
2026-10-29: knockout match #5
Semi-final BPlayed
- 2Claude Fable 5.1high (winner)5
- 7Grok 4.7high4
2026-10-29: knockout match #6
Final
FinalPlayed
- 4GPT-6.1 Solhigh (winner)5
- 2Claude Fable 5.1high4
2026-10-29: knockout match #7
Nackgammon start; all 7 matches new (stored standard results don't count). The same 8 teams and the same draw, every game from the Nackgammon start; otherwise the league match rules.
Every match of both, with the leagues' matches: Matches. The Cup is an event. The error rate is the ranking.
Knockout events, apart from the league tables: their places and results are not league points. The Hype Cup took its 8 teams from the starting groups' tables (its roster rule is under its bracket); the Nackgammon knockout used the same 8 teams and the same draw. A Cup place is not a promotion.
Late entries (not in standings)
Swipe for more columns →
| Setup | Entry date | League field played | Matches played | Record (W–D–L) | Error rate (PR), 95% interval | Risk judgement, margin 0.001 | Ability d′ | Caution c (+ cautious, − bold) | Engine's best play | Median s per decision | $ per 1,000 decisions | Invalid answers | Cut off at the output limit | Route and effort sent |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 AstrahighPublished | 2026-10-28first match 15:35 UTC | League 3 (original table)12 setups | 12 / 12 | 10–1–1won–drawn–lost | 8.87.4 to 10.3 | 74.471.8 to 77.0 | 1.871.68 to 2.08 · risks taken 267 / 465 Risk pays · needless 29 / 643 Safety pays | +0.75+0.64 to +0.87 · cautious | 67.6% | 9.83 | 59.356API-equivalent (subscription), $10.0 per million input tokens, $50.0 per million output tokens | 0of 1,489 decisions | 00 of 1,489 decisions sent an output limit; 0 over 32,768 output tokens without one | chatgpt-subscriptionopenai/gpt-6-astra · effort high in 1,489 of 1,489 decisions |
| SubQ Snap 1.1PublishedThis pair may be SubQ Snap 1.1 against itself: under the 'subq-snap-preview' name the API returned answers identical to SubQ Snap 1.1 (1,000 identical paired positions: letters and probabilities). | 2026-11-02first match 18:29 UTC | League 1 (original table)19 setups | 19 / 19 | 12–0–7won–drawn–lost | 40.236.7 to 43.8 | 56.754.8 to 58.6 | 0.710.61 to 0.82 · risks taken 412 / 798 Risk pays · needless 253 / 1,003 Safety pays | +0.31+0.25 to +0.38 · cautious | 42.9% | 0.23 | 0.150API price (billed); output free, $0.04 per million input tokens; docs.subq.ai/using-the-api/models, read 2026-11-01 | 0of 2,326 decisions | 00 of 2,326 decisions sent an output limit; 0 over 32,768 output tokens without one | subqsubq/subq-snap-1.1 · effort not in the requests |
| Elephant-1Published | 2026-11-02first match 18:29 UTC | League 1 (original table)19 setups | 19 / 19 | 11–0–8won–drawn–lost | 44.640.3 to 48.9 | 57.755.8 to 59.6 | 0.910.80 to 1.04 · risks taken 337 / 812 Risk pays · needless 102 / 784 Safety pays | +0.67+0.60 to +0.74 · cautious | 41.8% | 0.26 | 0.685estimate · RTX PRO 6000 Blackwell | 0of 2,044 decisions | 00 of 2,044 decisions sent an output limit; 0 over 32,768 output tokens without one | local (baselabs_server.py)baselabs/baselabs/elephant-1 · effort not in the requests |
| Mammoth-1Published | 2026-11-02first match 18:29 UTC | League 1 (original table)19 setups | 19 / 19 | 16–1–2won–drawn–lost | 28.526.2 to 31.1 | 63.261.2 to 65.2 | 1.100.97 to 1.23 · risks taken 361 / 720 Risk pays · needless 120 / 880 Safety pays | +0.55+0.48 to +0.62 · cautious | 49.0% | 0.72 | 0.999estimate · RTX PRO 6000 Blackwell | 0of 2,021 decisions | 00 of 2,021 decisions sent an output limit; 0 over 32,768 output tokens without one | local (baselabs_server.py)baselabs/baselabs/mammoth-1 · effort not in the requests |
| Drex Max v1.1Published | 2026-11-06first match 00:49 UTC | League 1 (original table)19 setups | 18 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | 12–2–4won–drawn–lost | 30.928.2 to 33.7 | 59.157.0 to 61.3 | 0.740.60 to 0.87 · risks taken 297 / 702 Risk pays · needless 164 / 939 Safety pays | +0.56+0.49 to +0.64 · cautious | 46.8% | 0.44 | 0.265list price; output not billed, $0.06 per million input tokens; nace.ai/drex and console.nace.ai/docs/drex/reference/pricing, read 2026-11-05billed at Nace's launch price, $0.03 per million input tokens: 0.132 | 0of 2,154 decisions | 00 of 2,154 decisions sent an output limit; 0 over 32,768 output tokens without one | nacenace/drex-max-v1.1 · effort not in the requests |
| Drex 2.0Published | 2026-11-06first match 00:49 UTC | League 1 (original table)19 setups | 18 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | 10–0–8won–drawn–lost | 38.435.6 to 41.4 | 58.155.7 to 60.4 | 0.890.73 to 1.04 · risks taken 311 / 699 Risk pays · needless 112 / 735 Safety pays | +0.58+0.51 to +0.65 · cautious | 45.2% | 0.42 | 0.258list price (billed); output not billed, $0.06 per million input tokens; console.nace.ai/docs/drex/reference/pricing, read 2026-11-05 | 0of 1,937 decisions | 00 of 1,937 decisions sent an output limit; 0 over 32,768 output tokens without one | nacenace/drex-v2.0 · effort not in the requests |
| Drex MaxNVIDIA RTX PRO 6000 BlackwellPublished | 2026-11-06first match 01:02 UTC | League 1 (original table)19 setups | 18 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | 11–0–7won–drawn–lost | 30.427.4 to 33.4 | 62.961.1 to 64.7 | 1.080.95 to 1.22 · risks taken 373 / 761 Risk pays · needless 121 / 896 Safety pays | +0.56+0.48 to +0.65 · cautious | 48.2% | 0.68 | 1.285estimate · RTX PRO 6000 Blackwell | 0of 2,148 decisions | 00 of 2,148 decisions sent an output limit; 0 over 32,768 output tokens without one | local (drex_max_server.py)drex-max/drex-max-v1 · effort not in the requests |
| Drex 2.0NVIDIA RTX PRO 6000 BlackwellPublished | 2026-11-06first match 07:44 UTC | League 1 (original table)19 setups | 18 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | 7–0–11won–drawn–lost | 42.939.1 to 47.1 | 53.050.5 to 55.2 | 0.570.43 to 0.71 · risks taken 286 / 720 Risk pays · needless 148 / 729 Safety pays | +0.54+0.47 to +0.63 · cautious | 42.7% | 0.40 | 0.782estimate · RTX PRO 6000 Blackwell | 0of 1,993 decisions | 00 of 1,993 decisions sent an output limit; 0 over 32,768 output tokens without one | local (drex-v2.0 serve.py)drex-local/drex-v2.0 · effort not in the requests |
| REAX 1.3Published | 2026-11-07first match 11:33 UTC | League 1 (original table)19 setups | 18 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | 11–1–6won–drawn–lost | 27.824.9 to 30.6 | 60.558.7 to 62.4 | 0.930.78 to 1.06 · risks taken 249 / 666 Risk pays · needless 90 / 850 Safety pays | +0.78+0.70 to +0.87 · cautious | 51.7% | 1.74 | 1.545estimate · RTX PRO 6000 Blackwell | 0of 1,998 decisions | 00 of 1,998 decisions sent an output limit; 0 over 32,768 output tokens without one | local (reax13 serve.py)reax/reax13 · effort not in the requests |
| Muse EmbermaxPublished | 2026-11-10first match 14:44 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 13–0–4won–drawn–lost | 20.117.6 to 22.7 | 66.063.9 to 68.2 | 1.341.17 to 1.53 · risks taken 241 / 596 Risk pays · needless 40 / 711 Safety pays | +0.91+0.83 to +1.01 · cautious | 58.1% | 15.3 | 6.863API price (billed), $0.1 per million input tokens, $0.5 per million output tokens; dev.meta.ai/models/muse-ember and OpenRouter's endpoints list for meta/muse-ember (Meta: $0.10/M prompt, $0.50/M completion), read 2026-11-10, read 2026-11-10 | 46of 1,713 decisions | 451,713 of 1,713 decisions sent an output limit; 0 over 32,768 output tokens without one | openrouter:metaopenai-compatible/meta/muse-ember · effort max in 1,713 of 1,713 decisions |
| KnowLine-31B-Gen1.5NVIDIA DGX Spark (GB10)Published | 2026-11-10first match 20:35 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 13–1–3won–drawn–lost | 26.223.0 to 29.4 | 61.659.6 to 63.5 | 1.140.96 to 1.33 · risks taken 212 / 649 Risk pays · needless 45 / 803 Safety pays | +1.02+0.94 to +1.11 · cautious | 54.6% | 3.45 | No priceno price (our GPU); no list price published for KnowLine-31B-Gen1.5, read 2026-11-10 | 0of 1,928 decisions | 00 of 1,928 decisions sent an output limit; 0 over 32,768 output tokens without one | local (knowline_server.py)knowline/m · effort not in the requests |
| KnowLine 4B Gen5.5Published | 2026-11-11first match 11:48 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 5–1–11won–drawn–lost | 53.648.8 to 58.6 | 49.747.0 to 52.3 | 0.590.47 to 0.72 · risks taken 448 / 778 Risk pays · needless 281 / 813 Safety pays | +0.10+0.02 to +0.18 · cautious | 36.9% | 0.28 | 0.054estimate · DGX Spark | 0of 2,153 decisions | 00 of 2,153 decisions sent an output limit; 0 over 32,768 output tokens without one | local (knowline_server.py)knowline/m · effort not in the requests |
| StartLux-Decision-v1.1-2B-InstantPublished | 2026-11-11first match 12:42 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 3–0–14won–drawn–lost | 50.045.4 to 54.8 | 53.851.3 to 56.6 | 0.700.54 to 0.86 · risks taken 330 / 630 Risk pays · needless 158 / 608 Safety pays | +0.29+0.20 to +0.38 · cautious | 36.2% | 0.21 | 0.052estimate · DGX Spark | 0of 1,771 decisions | 00 of 1,771 decisions sent an output limit; 0 over 32,768 output tokens without one | local (startlux-decide serve)startlux · effort not in the requests |
| KnowLine-0.8B-Gen1Published | 2026-11-11first match 18:27 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 1–1–15won–drawn–lost | 71.966.2 to 77.9 | 45.242.5 to 47.7 | 0.480.33 to 0.62 · risks taken 379 / 637 Risk pays · needless 240 / 594 Safety pays | +0.00−0.07 to +0.08 · cautious | 30.3% | 0.11 | 0.049estimate · DGX Spark | 0of 1,728 decisions | 00 of 1,728 decisions sent an output limit; 0 over 32,768 output tokens without one | local (knowline_server.py)knowline/m · effort not in the requests |
| Gemini Decision 1Published | 2026-11-11first match 19:28 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 13–0–4won–drawn–lost | 28.025.0 to 31.1 | 64.762.2 to 67.2 | 1.271.11 to 1.46 · risks taken 283 / 535 Risk pays · needless 84 / 732 Safety pays | +0.56+0.48 to +0.66 · cautious | 52.3% | 0.24 | 0.382API price (billed), $0.1 per million input tokens; aistudio.google.com/docs/pricing#gemini-decision and openrouter.ai/google/gemini-decision-1 ($0.10/M input, no output tokens), read 2026-11-11; cost is the billed cost per decision (turns.cost_usd), read 2026-11-11 | 0of 1,704 decisions | 00 of 1,704 decisions sent an output limit; 0 over 32,768 output tokens without one | openrouter (google ai studio)openrouter-decisions/google/gemini-decision-1 · effort not in the requests |
| StartLux-Decision-v1.1-0.8B-InstantPublished | 2026-11-11first match 19:49 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 3–0–14won–drawn–lost | 64.759.3 to 70.3 | 44.642.3 to 46.9 | 0.650.50 to 0.80 · risks taken 488 / 665 Risk pays · needless 351 / 715 Safety pays | −0.30−0.36 to −0.24 · bold | 32.5% | 0.13 | 0.054estimate · DGX Spark | 0of 1,882 decisions | 00 of 1,882 decisions sent an output limit; 0 over 32,768 output tokens without one | local (startlux-decide serve)startlux · effort not in the requests |
| OneJev1.6-0.8BPublished | 2026-11-11first match 21:19 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 0–0–17won–drawn–lost | 82.575.2 to 90.8 | 42.539.3 to 45.8 | 0.450.27 to 0.61 · risks taken 365 / 606 Risk pays · needless 270 / 634 Safety pays | −0.04−0.11 to +0.05 · bold | 28.1% | 0.10 | 0.052estimate · DGX Spark | 0of 1,763 decisions | 00 of 1,763 decisions sent an output limit; 0 over 32,768 output tokens without one | local (flide serve)onejev · effort not in the requests |
| GLiClass v4 EdgePublished | 2026-11-12first match 08:43 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 0–0–17won–drawn–lost | 133.5118.5 to 148.8 | 21.518.6 to 24.6 | 0.340.13 to 0.57 · risks taken 461 / 508 Risk pays · needless 400 / 478 Safety pays | −1.15−1.29 to −1.02 · bold | 15.7% | 1.33 | 0.174estimate · DGX Spark | 0of 1,405 decisions | 00 of 1,405 decisions sent an output limit; 0 over 32,768 output tokens without one | local (gliclass_server.py)gliclass · effort not in the requests |
| KnowLine-2B-Gen1Published | 2026-11-12first match 16:50 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 3–0–14won–drawn–lost | 54.548.9 to 60.8 | 51.148.7 to 53.5 | 0.590.45 to 0.72 · risks taken 337 / 676 Risk pays · needless 195 / 703 Safety pays | +0.30+0.22 to +0.38 · cautious | 35.6% | 0.15 | 0.049estimate · DGX Spark | 0of 1,955 decisions | 00 of 1,955 decisions sent an output limit; 0 over 32,768 output tokens without one | local (knowline_server.py)knowline/m · effort not in the requests |
| GLiDE 2BPublished | 2026-11-12first match 16:58 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 1–0–16won–drawn–lost | 67.461.0 to 74.5 | 46.743.6 to 49.6 | 0.650.49 to 0.81 · risks taken 378 / 581 Risk pays · needless 233 / 587 Safety pays | −0.06−0.15 to +0.03 · bold | 30.3% | 0.33 | 0.058estimate · DGX Spark | 0of 1,676 decisions | 00 of 1,676 decisions sent an output limit; 0 over 32,768 output tokens without one | local (glide.server)glide · effort not in the requests |
| GLiDE 0.8BPublished | 2026-11-12first match 17:08 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 2–1–14won–drawn–lost | 80.471.8 to 89.3 | 41.738.9 to 44.3 | 0.580.44 to 0.72 · risks taken 464 / 671 Risk pays · needless 311 / 663 Safety pays | −0.21−0.29 to −0.13 · bold | 26.6% | 0.22 | 0.051estimate · DGX Spark | 0of 1,832 decisions | 00 of 1,832 decisions sent an output limit; 0 over 32,768 output tokens without one | local (glide.server)glide · effort not in the requests |
| GLiDE 0.4BPublished | 2026-11-12first match 17:18 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 1–0–16won–drawn–lost | 113.2102.1 to 124.1 | 28.925.6 to 32.1 | 0.630.42 to 0.85 · risks taken 361 / 441 Risk pays · needless 304 / 498 Safety pays | −0.59−0.72 to −0.49 · bold | 17.7% | 0.13 | 0.057estimate · DGX Spark | 0of 1,355 decisions | 00 of 1,355 decisions sent an output limit; 0 over 32,768 output tokens without one | local (glide.server)glide-0.4b · effort not in the requests |
| Sieve 31B v3Published | 2026-11-12first match 19:29 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 15–0–2won–drawn–lost | 27.524.1 to 31.6 | 62.760.6 to 64.9 | 1.040.88 to 1.22 · risks taken 228 / 568 Risk pays · needless 76 / 772 Safety pays | +0.77+0.69 to +0.85 · cautious | 53.8% | 11.1 | 0.794estimate · DGX Spark | 0of 1,740 decisions | 00 of 1,740 decisions sent an output limit; 0 over 32,768 output tokens without one | local (sieve-serve)sieve · effort not in the requests |
| Turion Jev 31BPublished | 2026-11-13first match 12:30 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 13–1–3won–drawn–lost | 29.726.5 to 33.3 | 59.657.7 to 61.4 | 1.050.87 to 1.26 · risks taken 184 / 674 Risk pays · needless 35 / 715 Safety pays | +1.13+1.04 to +1.24 · cautious | 51.8% | 3.95 | 0.293estimate · DGX Spark | 0of 1,855 decisions | 00 of 1,855 decisions sent an output limit; 0 over 32,768 output tokens without one | local (turion_jev.serve)turion-jev · effort not in the requests |
| StartLux-Decision-v1.1-4B-InstantPublished | 2026-11-13first match 16:16 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 10–0–7won–drawn–lost | 40.136.7 to 43.7 | 57.255.0 to 59.3 | 0.790.67 to 0.92 · risks taken 370 / 777 Risk pays · needless 145 / 739 Safety pays | +0.46+0.36 to +0.55 · cautious | 42.4% | 0.44 | 0.072estimate · DGX Spark | 0of 1,989 decisions | 00 of 1,989 decisions sent an output limit; 0 over 32,768 output tokens without one | local (startlux-decide serve)startlux · effort not in the requests |
| Cerno 9B v1.2Published | 2026-11-13first match 17:22 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 6–1–10won–drawn–lost | 48.544.5 to 52.6 | 56.153.9 to 58.2 | 0.820.69 to 0.97 · risks taken 428 / 734 Risk pays · needless 232 / 857 Safety pays | +0.20+0.13 to +0.27 · cautious | 40.6% | 1.96 | 0.139estimate · DGX Spark | 0of 2,108 decisions | 00 of 2,108 decisions sent an output limit; 0 over 32,768 output tokens without one | local (cerno_server.py around CernoEngine)cerno · effort not in the requests |
| d1-2BNVIDIA RTX PRO 6000 BlackwellPublished | 2026-11-14first match 16:12 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 5–0–12won–drawn–lost | 47.142.8 to 51.5 | 55.252.8 to 57.5 | 0.750.62 to 0.89 · risks taken 349 / 688 Risk pays · needless 180 / 776 Safety pays | +0.36+0.29 to +0.43 · cautious | 39.7% | 0.06 | 0.467estimate · RTX PRO 6000 Blackwell | 0of 1,980 decisions | 00 of 1,980 decisions sent an output limit; 0 over 32,768 output tokens without one | local (d1_server.py)d1-local · effort not in the requests |
| d1-Omni-2BNVIDIA RTX PRO 6000 BlackwellPublished | 2026-11-14first match 19:47 UTC | League 1 (original table)19 setups | 17 / 19GPT-6 LunaDecisions API: outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit)Mercury Decide: outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | 9–0–8won–drawn–lost | 49.843.5 to 57.4 | 53.250.9 to 55.6 | 0.610.48 to 0.75 · risks taken 287 / 619 Risk pays · needless 175 / 728 Safety pays | +0.40+0.32 to +0.48 · cautious | 40.4% | 0.06 | 0.657estimate · RTX PRO 6000 Blackwell | 0of 1,710 decisions | 00 of 1,710 decisions sent an output limit; 0 over 32,768 output tokens without one | local (d1_server.py)d1-local/LiquidAI/d1-Omni-2B · effort not in the requests |
A late entry joins an original league table after it was played and plays every setup of that table once, in matches of its own. Its games count in no standings table, no league table, no Hype Cup input and no other setup's numbers. The numbers here come from its own late-entry games only, computed as for a setup in a league. Late entries are ranked on the risk leaderboard; they do not change league match-points standings, Cups, or other setups' results.
GPT-6 Astrahigh: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GNUbg weak | 5–0 | Won | 758 | 2026-10-28 |
| GPT-6 Astralow | 5–2 | Won | 759 | 2026-10-28 |
| GPT-6 Astramedium | 4–3 | Won | 760 | 2026-10-28 |
| Gemini 4 Argonhigh | 5–1 | Won | 761 | 2026-10-28 |
| Gemini 4 Argonlow | 6–3 | Won | 762 | 2026-10-28 |
| Claude Fable 5.1low | 5–1 | Won | 763 | 2026-10-28 |
| Claude Fable 5.1high | 0–7 | Lost | 764 | 2026-10-28 |
| Claude Opus 5.5low | 4–3 | Won | 765 | 2026-10-28 |
| Claude Opus 5.5high | 5–5 | Draw | 766 | 2026-10-28 |
| Grok 4.7high | 4–2 | Won | 768 | 2026-10-29 |
| Grok 4.7low | 6–1 | Won | 769 | 2026-10-28 |
| Kimi K3.1 Prooff | 6–2 | Won | 771 | 2026-10-30 |
SubQ Snap 1.1: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| SubQ Snap PreviewThis pair may be SubQ Snap 1.1 against itself: under the 'subq-snap-preview' name the API returned answers identical to SubQ Snap 1.1 (1,000 identical paired positions: letters and probabilities). | 7–4 | Won | 772 | 2026-11-02 |
| Jev 1.14 | 4–2 | Won | 773 | 2026-11-02 |
| celeris-1-decision | 0–6 | Lost | 774 | 2026-11-02 |
| GLiDEAPI | 2–5 | Lost | 775 | 2026-11-02 |
| Jev 1.13 | 6–0 | Won | 784 | 2026-11-02 |
| GPT-6 LunaDecisions API | 4–5 | Lost | 785 | 2026-11-02 |
| Safest | 6–2 | Won | 786 | 2026-11-02 |
| Mercury Decide | 5–1 | Won | 787 | 2026-11-02 |
| Winnow-12B | 5–0 | Won | 788 | 2026-11-02 |
| d1-ultra | 2–5 | Lost | 789 | 2026-11-02 |
| GNUbg weak | 0–7 | Lost | 790 | 2026-11-02 |
| Random | 9–0 | Won | 791 | 2026-11-02 |
| Perplexity Decider Flash v1.2 | 5–3 | Won | 792 | 2026-11-02 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 4–2 | Won | 793 | 2026-11-02 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 2–7 | Lost | 795 | 2026-11-02 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–3 | Won | 817 | 2026-11-02 |
| Clef | 2–6 | Lost | 822 | 2026-11-02 |
| Cygnet | 5–2 | Won | 824 | 2026-11-02 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 7–1 | Won | 826 | 2026-11-02 |
Elephant-1: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 2–6 | Lost | 776 | 2026-11-02 |
| Jev 1.14 | 3–5 | Lost | 777 | 2026-11-02 |
| celeris-1-decision | 4–3 | Won | 778 | 2026-11-02 |
| GLiDEAPI | 5–2 | Won | 779 | 2026-11-02 |
| Jev 1.13 | 2–4 | Lost | 794 | 2026-11-02 |
| GPT-6 LunaDecisions API | 5–0 | Won | 796 | 2026-11-02 |
| Safest | 5–3 | Won | 797 | 2026-11-02 |
| Mercury Decide | 6–1 | Won | 798 | 2026-11-02 |
| Winnow-12B | 1–5 | Lost | 799 | 2026-11-02 |
| d1-ultra | 3–4 | Lost | 800 | 2026-11-02 |
| GNUbg weak | 1–5 | Lost | 801 | 2026-11-02 |
| Random | 8–0 | Won | 802 | 2026-11-02 |
| Perplexity Decider Flash v1.2 | 5–3 | Won | 803 | 2026-11-02 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 804 | 2026-11-02 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 3–4 | Lost | 805 | 2026-11-02 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–3 | Won | 819 | 2026-11-02 |
| Cygnet | 5–6 | Lost | 823 | 2026-11-02 |
| Clef | 6–1 | Won | 825 | 2026-11-02 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 7–2 | Won | 827 | 2026-11-02 |
Mammoth-1: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 5–2 | Won | 780 | 2026-11-02 |
| celeris-1-decision | 4–2 | Won | 781 | 2026-11-02 |
| GLiDEAPI | 5–2 | Won | 782 | 2026-11-02 |
| Jev 1.14 | 4–3 | Won | 783 | 2026-11-02 |
| Jev 1.13 | 5–2 | Won | 806 | 2026-11-02 |
| GPT-6 LunaDecisions API | 6–4 | Won | 807 | 2026-11-02 |
| Safest | 5–5 | Draw | 808 | 2026-11-02 |
| Mercury Decide | 6–2 | Won | 809 | 2026-11-02 |
| Winnow-12B | 5–1 | Won | 810 | 2026-11-02 |
| d1-ultra | 4–5 | Lost | 811 | 2026-11-02 |
| GNUbg weak | 2–6 | Lost | 812 | 2026-11-02 |
| Random | 5–0 | Won | 813 | 2026-11-02 |
| Perplexity Decider Flash v1.2 | 5–1 | Won | 814 | 2026-11-02 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 4–2 | Won | 815 | 2026-11-02 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 4–3 | Won | 816 | 2026-11-02 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 5–1 | Won | 818 | 2026-11-02 |
| Cygnet | 6–0 | Won | 820 | 2026-11-02 |
| Clef | 5–1 | Won | 821 | 2026-11-02 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–1 | Won | 828 | 2026-11-02 |
Drex Max v1.1: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| Jev 1.13 | 5–5 | Draw | 830 | 2026-11-06 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 3–5 | Lost | 831 | 2026-11-06 |
| Mercury Decide | 5–0 | Won | 832 | 2026-11-06 |
| GLiDEAPI | 2–5 | Lost | 839 | 2026-11-06 |
| Random | 6–0 | Won | 842 | 2026-11-06 |
| celeris-1-decision | 6–1 | Won | 844 | 2026-11-06 |
| Jev 1.14 | 7–2 | Won | 850 | 2026-11-06 |
| GNUbg weak | 4–3 | Won | 851 | 2026-11-06 |
| Safest | 6–2 | Won | 854 | 2026-11-06 |
| d1-ultra | 4–3 | Won | 862 | 2026-11-06 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 4–5 | Lost | 865 | 2026-11-06 |
| Perplexity Decider Flash v1.2 | 6–1 | Won | 866 | 2026-11-06 |
| Winnow-12B | 5–2 | Won | 869 | 2026-11-06 |
| Cygnet | 3–4 | Lost | 871 | 2026-11-06 |
| Clef | 6–5 | Won | 875 | 2026-11-06 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 6–6 | Draw | 878 | 2026-11-06 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–0 | Won | 880 | 2026-11-06 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 5–4 | Won | 884 | 2026-11-06 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A |
Drex 2.0: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 0–5 | Lost | 833 | 2026-11-06 |
| Mercury Decide | 5–2 | Won | 834 | 2026-11-06 |
| Jev 1.13 | 6–2 | Won | 836 | 2026-11-06 |
| GLiDEAPI | 3–5 | Lost | 837 | 2026-11-06 |
| Random | 5–0 | Won | 838 | 2026-11-06 |
| celeris-1-decision | 1–5 | Lost | 840 | 2026-11-06 |
| Jev 1.14 | 5–3 | Won | 841 | 2026-11-06 |
| GNUbg weak | 5–2 | Won | 843 | 2026-11-06 |
| Safest | 3–5 | Lost | 849 | 2026-11-06 |
| d1-ultra | 3–5 | Lost | 852 | 2026-11-06 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 853 | 2026-11-06 |
| Perplexity Decider Flash v1.2 | 5–1 | Won | 858 | 2026-11-06 |
| Winnow-12B | 7–6 | Won | 860 | 2026-11-06 |
| Cygnet | 1–5 | Lost | 863 | 2026-11-06 |
| Clef | 0–5 | Lost | 876 | 2026-11-06 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 7–4 | Won | 877 | 2026-11-06 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–3 | Won | 881 | 2026-11-06 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0–5 | Lost | 883 | 2026-11-06 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A |
Drex MaxNVIDIA RTX PRO 6000 Blackwell: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| Mercury Decide | 5–1 | Won | 845 | 2026-11-06 |
| Jev 1.13 | 3–4 | Lost | 847 | 2026-11-06 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 5–4 | Won | 848 | 2026-11-06 |
| GLiDEAPI | 3–5 | Lost | 855 | 2026-11-06 |
| Random | 5–0 | Won | 856 | 2026-11-06 |
| celeris-1-decision | 4–2 | Won | 857 | 2026-11-06 |
| Jev 1.14 | 3–5 | Lost | 859 | 2026-11-06 |
| GNUbg weak | 0–5 | Lost | 861 | 2026-11-06 |
| Safest | 5–3 | Won | 864 | 2026-11-06 |
| d1-ultra | 6–1 | Won | 867 | 2026-11-06 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 2–5 | Lost | 868 | 2026-11-06 |
| Perplexity Decider Flash v1.2 | 6–2 | Won | 870 | 2026-11-06 |
| Winnow-12B | 7–1 | Won | 872 | 2026-11-06 |
| Cygnet | 3–4 | Lost | 873 | 2026-11-06 |
| Clef | 8–3 | Won | 874 | 2026-11-06 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 5–3 | Won | 879 | 2026-11-06 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 6–0 | Won | 882 | 2026-11-06 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 3–4 | Lost | 885 | 2026-11-06 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A |
Drex 2.0NVIDIA RTX PRO 6000 Blackwell: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 2–5 | Lost | 887 | 2026-11-06 |
| Jev 1.13 | 3–5 | Lost | 888 | 2026-11-06 |
| Mercury Decide | 6–2 | Won | 889 | 2026-11-06 |
| GLiDEAPI | 4–3 | Won | 890 | 2026-11-06 |
| Random | 8–0 | Won | 891 | 2026-11-06 |
| celeris-1-decision | 3–5 | Lost | 892 | 2026-11-06 |
| Jev 1.14 | 2–4 | Lost | 893 | 2026-11-06 |
| GNUbg weak | 0–7 | Lost | 894 | 2026-11-06 |
| Safest | 5–3 | Won | 895 | 2026-11-06 |
| d1-ultra | 1–5 | Lost | 896 | 2026-11-06 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 0–5 | Lost | 897 | 2026-11-06 |
| Perplexity Decider Flash v1.2 | 6–3 | Won | 898 | 2026-11-06 |
| Winnow-12B | 2–5 | Lost | 899 | 2026-11-06 |
| Cygnet | 2–6 | Lost | 900 | 2026-11-06 |
| Clef | 2–5 | Lost | 901 | 2026-11-06 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 902 | 2026-11-06 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–0 | Won | 903 | 2026-11-06 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–5 | Lost | 904 | 2026-11-06 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A |
REAX 1.3: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| Random | 7–0 | Won | 905 | 2026-11-07 |
| GNUbg weak | 1–5 | Lost | 906 | 2026-11-07 |
| Safest | 6–0 | Won | 907 | 2026-11-07 |
| Cygnet | 2–5 | Lost | 908 | 2026-11-07 |
| Winnow-12B | 6–3 | Won | 909 | 2026-11-07 |
| Perplexity Decider Flash v1.2 | 5–0 | Won | 910 | 2026-11-07 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 6–0 | Won | 911 | 2026-11-07 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 6–2 | Won | 912 | 2026-11-07 |
| Clef | 6–0 | Won | 913 | 2026-11-07 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–1 | Won | 914 | 2026-11-07 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 1–5 | Lost | 915 | 2026-11-07 |
| celeris-1-decision | 4–5 | Lost | 916 | 2026-11-07 |
| GLiDEAPI | 5–5 | Draw | 917 | 2026-11-07 |
| Jev 1.13 | 5–0 | Won | 918 | 2026-11-07 |
| Jev 1.14 | 5–6 | Lost | 919 | 2026-11-07 |
| d1-ultra | 5–3 | Won | 920 | 2026-11-07 |
| Mercury Decide | 6–1 | Won | 921 | 2026-11-07 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 2–6 | Lost | 922 | 2026-11-07 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A |
Muse Embermax: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 4–2 | Won | 923 | 2026-11-10 |
| celeris-1-decision | 2–4 | Lost | 924 | 2026-11-10 |
| GNUbg weak | 2–6 | Lost | 925 | 2026-11-10 |
| Random | 5–0 | Won | 926 | 2026-11-10 |
| Safest | 7–1 | Won | 927 | 2026-11-10 |
| d1-ultra | 7–1 | Won | 928 | 2026-11-10 |
| Jev 1.14 | 5–2 | Won | 929 | 2026-11-10 |
| Jev 1.13 | 7–2 | Won | 930 | 2026-11-10 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 3–4 | Lost | 931 | 2026-11-10 |
| Cygnet | 7–1 | Won | 955 | 2026-11-11 |
| Winnow-12B | 7–0 | Won | 956 | 2026-11-11 |
| Perplexity Decider Flash v1.2 | 6–0 | Won | 967 | 2026-11-11 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 6–0 | Won | 968 | 2026-11-11 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 3–4 | Lost | 969 | 2026-11-11 |
| Clef | 4–2 | Won | 980 | 2026-11-11 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–1 | Won | 981 | 2026-11-11 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 5–2 | Won | 982 | 2026-11-11 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
KnowLine-31B-Gen1.5NVIDIA DGX Spark (GB10): results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 4–3 | Won | 937 | 2026-11-10 |
| celeris-1-decision | 2–5 | Lost | 938 | 2026-11-10 |
| GNUbg weak | 1–5 | Lost | 939 | 2026-11-10 |
| Random | 8–0 | Won | 940 | 2026-11-10 |
| Safest | 6–2 | Won | 941 | 2026-11-10 |
| d1-ultra | 3–4 | Lost | 942 | 2026-11-10 |
| Jev 1.14 | 6–1 | Won | 943 | 2026-11-10 |
| Jev 1.13 | 4–2 | Won | 944 | 2026-11-10 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 6–0 | Won | 945 | 2026-11-10 |
| Cygnet | 5–2 | Won | 947 | 2026-11-11 |
| Winnow-12B | 6–1 | Won | 948 | 2026-11-11 |
| Perplexity Decider Flash v1.2 | 6–1 | Won | 949 | 2026-11-11 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 950 | 2026-11-11 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 5–1 | Won | 951 | 2026-11-11 |
| Clef | 6–2 | Won | 952 | 2026-11-11 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 7–2 | Won | 953 | 2026-11-11 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–4 | Draw | 954 | 2026-11-11 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
KnowLine 4B Gen5.5: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 4–5 | Lost | 957 | 2026-11-11 |
| celeris-1-decision | 2–6 | Lost | 958 | 2026-11-11 |
| GNUbg weak | 0–9 | Lost | 959 | 2026-11-11 |
| Random | 8–2 | Won | 960 | 2026-11-11 |
| Safest | 2–5 | Lost | 961 | 2026-11-11 |
| d1-ultra | 0–5 | Lost | 962 | 2026-11-11 |
| Jev 1.14 | 0–5 | Lost | 963 | 2026-11-11 |
| Jev 1.13 | 2–5 | Lost | 964 | 2026-11-11 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 6–6 | Draw | 965 | 2026-11-11 |
| Cygnet | 6–2 | Won | 1371 | 2026-11-14 |
| Winnow-12B | 3–5 | Lost | 1372 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 4–2 | Won | 1373 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1374 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1375 | 2026-11-14 |
| Clef | 0–7 | Lost | 1376 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 4–2 | Won | 1377 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–2 | Won | 1378 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
StartLux-Decision-v1.1-2B-Instant: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 1–5 | Lost | 970 | 2026-11-11 |
| celeris-1-decision | 0–5 | Lost | 971 | 2026-11-11 |
| GNUbg weak | 0–6 | Lost | 972 | 2026-11-11 |
| Random | 5–1 | Won | 973 | 2026-11-11 |
| Safest | 5–3 | Won | 974 | 2026-11-11 |
| d1-ultra | 1–5 | Lost | 975 | 2026-11-11 |
| Jev 1.14 | 0–5 | Lost | 976 | 2026-11-11 |
| Jev 1.13 | 1–7 | Lost | 977 | 2026-11-11 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 2–5 | Lost | 978 | 2026-11-11 |
| Cygnet | 2–4 | Lost | 1379 | 2026-11-14 |
| Winnow-12B | 0–6 | Lost | 1380 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 4–5 | Lost | 1381 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 3–5 | Lost | 1382 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 1383 | 2026-11-14 |
| Clef | 3–5 | Lost | 1384 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 3–5 | Lost | 1385 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 1–6 | Lost | 1386 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
KnowLine-0.8B-Gen1: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 0–7 | Lost | 983 | 2026-11-11 |
| celeris-1-decision | 3–6 | Lost | 984 | 2026-11-11 |
| GNUbg weak | 0–5 | Lost | 985 | 2026-11-11 |
| Random | 6–3 | Won | 986 | 2026-11-11 |
| Safest | 3–4 | Lost | 987 | 2026-11-11 |
| d1-ultra | 2–5 | Lost | 988 | 2026-11-11 |
| Jev 1.14 | 0–5 | Lost | 989 | 2026-11-11 |
| Jev 1.13 | 0–5 | Lost | 990 | 2026-11-11 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 3–4 | Lost | 991 | 2026-11-11 |
| Cygnet | 1–6 | Lost | 1331 | 2026-11-14 |
| Winnow-12B | 2–4 | Lost | 1332 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 2–6 | Lost | 1333 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 0–6 | Lost | 1334 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 2–7 | Lost | 1335 | 2026-11-14 |
| Clef | 0–5 | Lost | 1336 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–5 | Draw | 1337 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 2–8 | Lost | 1338 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
Gemini Decision 1: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 5–1 | Won | 993 | 2026-11-11 |
| celeris-1-decision | 2–6 | Lost | 994 | 2026-11-11 |
| GNUbg weak | 1–5 | Lost | 995 | 2026-11-11 |
| Random | 5–0 | Won | 996 | 2026-11-11 |
| Safest | 7–1 | Won | 997 | 2026-11-11 |
| d1-ultra | 4–3 | Won | 998 | 2026-11-11 |
| Jev 1.14 | 6–2 | Won | 999 | 2026-11-11 |
| Jev 1.13 | 3–5 | Lost | 1005 | 2026-11-11 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 1–5 | Lost | 1013 | 2026-11-11 |
| Cygnet | 6–0 | Won | 1053 | 2026-11-11 |
| Winnow-12B | 6–1 | Won | 1055 | 2026-11-11 |
| Perplexity Decider Flash v1.2 | 5–0 | Won | 1056 | 2026-11-11 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 4–2 | Won | 1057 | 2026-11-11 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 1060 | 2026-11-11 |
| Clef | 6–1 | Won | 1061 | 2026-11-11 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–3 | Won | 1062 | 2026-11-11 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–3 | Won | 1063 | 2026-11-11 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
StartLux-Decision-v1.1-0.8B-Instant: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 2–6 | Lost | 1000 | 2026-11-11 |
| celeris-1-decision | 1–5 | Lost | 1001 | 2026-11-11 |
| GNUbg weak | 1–6 | Lost | 1003 | 2026-11-11 |
| Random | 8–0 | Won | 1006 | 2026-11-11 |
| Safest | 1–5 | Lost | 1007 | 2026-11-11 |
| d1-ultra | 2–5 | Lost | 1009 | 2026-11-11 |
| Jev 1.14 | 1–5 | Lost | 1010 | 2026-11-11 |
| Jev 1.13 | 1–5 | Lost | 1012 | 2026-11-11 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 0–5 | Lost | 1014 | 2026-11-11 |
| Cygnet | 2–6 | Lost | 1339 | 2026-11-14 |
| Winnow-12B | 2–7 | Lost | 1340 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 3–7 | Lost | 1341 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 6–3 | Won | 1342 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 2–5 | Lost | 1343 | 2026-11-14 |
| Clef | 0–5 | Lost | 1344 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 4–2 | Won | 1345 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0–5 | Lost | 1346 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
OneJev1.6-0.8B: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 0–5 | Lost | 1041 | 2026-11-11 |
| celeris-1-decision | 0–6 | Lost | 1042 | 2026-11-11 |
| GNUbg weak | 0–5 | Lost | 1043 | 2026-11-11 |
| Random | 4–5 | Lost | 1044 | 2026-11-11 |
| Safest | 1–7 | Lost | 1045 | 2026-11-11 |
| d1-ultra | 1–6 | Lost | 1046 | 2026-11-11 |
| Jev 1.14 | 1–8 | Lost | 1047 | 2026-11-11 |
| Jev 1.13 | 1–6 | Lost | 1048 | 2026-11-11 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 2–5 | Lost | 1049 | 2026-11-11 |
| Cygnet | 1–5 | Lost | 1347 | 2026-11-14 |
| Winnow-12B | 0–7 | Lost | 1348 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 1–6 | Lost | 1349 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 0–8 | Lost | 1350 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 0–7 | Lost | 1351 | 2026-11-14 |
| Clef | 1–7 | Lost | 1352 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 4–5 | Lost | 1353 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0–7 | Lost | 1354 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
GLiClass v4 Edge: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 0–8 | Lost | 1141 | 2026-11-12 |
| celeris-1-decision | 1–8 | Lost | 1142 | 2026-11-12 |
| GNUbg weak | 0–5 | Lost | 1143 | 2026-11-12 |
| Random | 1–5 | Lost | 1144 | 2026-11-12 |
| Safest | 0–7 | Lost | 1165 | 2026-11-12 |
| d1-ultra | 0–7 | Lost | 1166 | 2026-11-12 |
| Jev 1.14 | 0–6 | Lost | 1167 | 2026-11-12 |
| Jev 1.13 | 0–6 | Lost | 1168 | 2026-11-12 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 0–7 | Lost | 1169 | 2026-11-12 |
| Cygnet | 0–5 | Lost | 1363 | 2026-11-14 |
| Winnow-12B | 1–6 | Lost | 1364 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 0–6 | Lost | 1365 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 1–5 | Lost | 1366 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 0–7 | Lost | 1367 | 2026-11-14 |
| Clef | 0–8 | Lost | 1368 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 0–7 | Lost | 1369 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0–5 | Lost | 1370 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
KnowLine-2B-Gen1: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 3–5 | Lost | 1181 | 2026-11-12 |
| celeris-1-decision | 2–5 | Lost | 1182 | 2026-11-12 |
| GNUbg weak | 0–6 | Lost | 1183 | 2026-11-12 |
| Random | 5–2 | Won | 1184 | 2026-11-12 |
| Safest | 5–3 | Won | 1185 | 2026-11-12 |
| d1-ultra | 1–5 | Lost | 1186 | 2026-11-12 |
| Jev 1.14 | 3–5 | Lost | 1187 | 2026-11-12 |
| Jev 1.13 | 2–5 | Lost | 1188 | 2026-11-12 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 2–6 | Lost | 1189 | 2026-11-12 |
| Cygnet | 1–5 | Lost | 1355 | 2026-11-14 |
| Winnow-12B | 2–5 | Lost | 1356 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 3–4 | Lost | 1357 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1358 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 2–5 | Lost | 1359 | 2026-11-14 |
| Clef | 5–2 | Won | 1360 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 3–6 | Lost | 1361 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 3–4 | Lost | 1362 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
GLiDE 2B: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 1–7 | Lost | 1191 | 2026-11-12 |
| celeris-1-decision | 2–4 | Lost | 1192 | 2026-11-12 |
| GNUbg weak | 0–6 | Lost | 1193 | 2026-11-12 |
| Random | 5–0 | Won | 1194 | 2026-11-12 |
| Safest | 1–6 | Lost | 1195 | 2026-11-12 |
| d1-ultra | 0–7 | Lost | 1196 | 2026-11-12 |
| Jev 1.14 | 0–5 | Lost | 1197 | 2026-11-12 |
| Jev 1.13 | 0–5 | Lost | 1198 | 2026-11-12 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 1–6 | Lost | 1199 | 2026-11-12 |
| Cygnet | 2–6 | Lost | 1298 | 2026-11-13 |
| Winnow-12B | 4–6 | Lost | 1299 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 1–5 | Lost | 1300 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 1–5 | Lost | 1301 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 2–5 | Lost | 1302 | 2026-11-14 |
| Clef | 0–6 | Lost | 1303 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 3–4 | Lost | 1304 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 1–6 | Lost | 1305 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
GLiDE 0.8B: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 0–6 | Lost | 1201 | 2026-11-12 |
| celeris-1-decision | 0–7 | Lost | 1202 | 2026-11-12 |
| GNUbg weak | 1–7 | Lost | 1203 | 2026-11-12 |
| Random | 6–5 | Won | 1204 | 2026-11-12 |
| Safest | 4–4 | Draw | 1205 | 2026-11-12 |
| d1-ultra | 2–4 | Lost | 1206 | 2026-11-12 |
| Jev 1.14 | 1–7 | Lost | 1207 | 2026-11-12 |
| Jev 1.13 | 0–5 | Lost | 1208 | 2026-11-12 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 4–5 | Lost | 1209 | 2026-11-12 |
| Cygnet | 4–3 | Won | 1315 | 2026-11-14 |
| Winnow-12B | 2–6 | Lost | 1316 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 3–4 | Lost | 1317 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1318 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1319 | 2026-11-14 |
| Clef | 1–6 | Lost | 1320 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 2–5 | Lost | 1321 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 1–6 | Lost | 1322 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
GLiDE 0.4B: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 0–7 | Lost | 1211 | 2026-11-12 |
| celeris-1-decision | 0–8 | Lost | 1212 | 2026-11-12 |
| GNUbg weak | 0–6 | Lost | 1213 | 2026-11-12 |
| Random | 7–3 | Won | 1214 | 2026-11-12 |
| Safest | 0–6 | Lost | 1215 | 2026-11-12 |
| d1-ultra | 0–8 | Lost | 1216 | 2026-11-12 |
| Jev 1.14 | 0–7 | Lost | 1217 | 2026-11-12 |
| Jev 1.13 | 0–7 | Lost | 1218 | 2026-11-12 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 0–6 | Lost | 1219 | 2026-11-12 |
| Cygnet | 1–5 | Lost | 1323 | 2026-11-14 |
| Winnow-12B | 0–6 | Lost | 1324 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 1–5 | Lost | 1325 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 0–9 | Lost | 1326 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 0–8 | Lost | 1327 | 2026-11-14 |
| Clef | 0–5 | Lost | 1328 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1329 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0–5 | Lost | 1330 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
Sieve 31B v3: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 4–3 | Won | 1221 | 2026-11-12 |
| celeris-1-decision | 6–2 | Won | 1222 | 2026-11-12 |
| GNUbg weak | 4–3 | Won | 1223 | 2026-11-12 |
| Random | 9–0 | Won | 1224 | 2026-11-12 |
| Safest | 5–1 | Won | 1225 | 2026-11-12 |
| d1-ultra | 3–4 | Lost | 1226 | 2026-11-12 |
| Jev 1.14 | 4–3 | Won | 1227 | 2026-11-12 |
| Jev 1.13 | 5–1 | Won | 1228 | 2026-11-12 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 3–5 | Lost | 1229 | 2026-11-12 |
| Cygnet | 5–2 | Won | 1411 | 2026-11-14 |
| Winnow-12B | 5–0 | Won | 1412 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 5–3 | Won | 1413 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 1414 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 6–0 | Won | 1415 | 2026-11-14 |
| Clef | 5–1 | Won | 1416 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–1 | Won | 1417 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 5–1 | Won | 1418 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
Turion Jev 31B: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 6–0 | Won | 1241 | 2026-11-13 |
| celeris-1-decision | 5–3 | Won | 1242 | 2026-11-13 |
| GNUbg weak | 6–0 | Won | 1243 | 2026-11-13 |
| Random | 7–0 | Won | 1244 | 2026-11-13 |
| Safest | 5–2 | Won | 1245 | 2026-11-13 |
| d1-ultra | 2–5 | Lost | 1246 | 2026-11-13 |
| Jev 1.14 | 5–2 | Won | 1247 | 2026-11-13 |
| Jev 1.13 | 3–4 | Lost | 1248 | 2026-11-13 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 5–5 | Draw | 1249 | 2026-11-13 |
| Cygnet | 4–3 | Won | 1403 | 2026-11-14 |
| Winnow-12B | 7–0 | Won | 1404 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 5–0 | Won | 1405 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 5–4 | Won | 1406 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 6–1 | Won | 1407 | 2026-11-14 |
| Clef | 5–6 | Lost | 1408 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 1409 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 6–3 | Won | 1410 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
StartLux-Decision-v1.1-4B-Instant: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 4–2 | Won | 1251 | 2026-11-13 |
| celeris-1-decision | 4–2 | Won | 1252 | 2026-11-13 |
| GNUbg weak | 1–6 | Lost | 1253 | 2026-11-13 |
| Random | 6–0 | Won | 1254 | 2026-11-13 |
| Safest | 5–0 | Won | 1255 | 2026-11-13 |
| d1-ultra | 1–5 | Lost | 1256 | 2026-11-13 |
| Jev 1.14 | 6–3 | Won | 1257 | 2026-11-13 |
| Jev 1.13 | 7–5 | Won | 1258 | 2026-11-13 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 5–3 | Won | 1259 | 2026-11-13 |
| Cygnet | 5–2 | Won | 1387 | 2026-11-14 |
| Winnow-12B | 1–6 | Lost | 1388 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 6–1 | Won | 1389 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1390 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 2–6 | Lost | 1391 | 2026-11-14 |
| Clef | 3–4 | Lost | 1392 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–3 | Won | 1393 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–6 | Lost | 1394 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
Cerno 9B v1.2: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 4–6 | Lost | 1261 | 2026-11-13 |
| celeris-1-decision | 2–6 | Lost | 1262 | 2026-11-13 |
| GNUbg weak | 1–7 | Lost | 1263 | 2026-11-13 |
| Random | 8–0 | Won | 1264 | 2026-11-13 |
| Safest | 6–1 | Won | 1265 | 2026-11-13 |
| d1-ultra | 2–6 | Lost | 1266 | 2026-11-13 |
| Jev 1.14 | 3–6 | Lost | 1267 | 2026-11-13 |
| Jev 1.13 | 1–6 | Lost | 1268 | 2026-11-13 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 3–5 | Lost | 1269 | 2026-11-13 |
| Cygnet | 5–5 | Draw | 1395 | 2026-11-14 |
| Winnow-12B | 2–4 | Lost | 1396 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 6–1 | Won | 1397 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 5–4 | Won | 1398 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 4–5 | Lost | 1399 | 2026-11-14 |
| Clef | 5–4 | Won | 1400 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 6–1 | Won | 1401 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4–6 | Lost | 1402 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
d1-2BNVIDIA RTX PRO 6000 Blackwell: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 1–7 | Lost | 1419 | 2026-11-14 |
| celeris-1-decision | 2–5 | Lost | 1420 | 2026-11-14 |
| GNUbg weak | 3–4 | Lost | 1421 | 2026-11-14 |
| Random | 6–1 | Won | 1422 | 2026-11-14 |
| Safest | 5–2 | Won | 1423 | 2026-11-14 |
| d1-ultra | 1–6 | Lost | 1424 | 2026-11-14 |
| Jev 1.14 | 2–5 | Lost | 1425 | 2026-11-14 |
| Jev 1.13 | 2–5 | Lost | 1426 | 2026-11-14 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 1–5 | Lost | 1427 | 2026-11-14 |
| Cygnet | 5–2 | Won | 1429 | 2026-11-14 |
| Winnow-12B | 2–6 | Lost | 1430 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 4–2 | Won | 1431 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 4–5 | Lost | 1432 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 2–6 | Lost | 1433 | 2026-11-14 |
| Clef | 2–5 | Lost | 1434 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 1435 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 1–6 | Lost | 1436 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
d1-Omni-2BNVIDIA RTX PRO 6000 Blackwell: results per opponent
Swipe for more columns →
| Opponent | Score | Result | # | Finished |
|---|---|---|---|---|
| GLiDEAPI | 2–4 | Lost | 1437 | 2026-11-14 |
| celeris-1-decision | 6–3 | Won | 1438 | 2026-11-14 |
| GNUbg weak | 1–6 | Lost | 1439 | 2026-11-14 |
| Random | 5–1 | Won | 1440 | 2026-11-14 |
| Safest | 1–5 | Lost | 1441 | 2026-11-14 |
| d1-ultra | 6–1 | Won | 1442 | 2026-11-14 |
| Jev 1.14 | 5–2 | Won | 1443 | 2026-11-14 |
| Jev 1.13 | 6–1 | Won | 1444 | 2026-11-14 |
| SubQ Snap PreviewThe opponent was requested as SubQ Snap Preview; under that name the API returned answers identical to SubQ Snap 1.1 on 1,000 paired positions, so the opponent's model identity is not verified. | 3–6 | Lost | 1445 | 2026-11-14 |
| Cygnet | 0–5 | Lost | 1447 | 2026-11-14 |
| Winnow-12B | 0–6 | Lost | 1448 | 2026-11-14 |
| Perplexity Decider Flash v1.2 | 5–2 | Won | 1449 | 2026-11-14 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 7–2 | Won | 1450 | 2026-11-14 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 1–6 | Lost | 1451 | 2026-11-14 |
| Clef | 7–3 | Won | 1452 | 2026-11-14 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 5–2 | Won | 1453 | 2026-11-14 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0–5 | Lost | 1454 | 2026-11-14 |
| GPT-6 LunaDecisions API | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent's API out of credit) | N/A | |
| Mercury Decide | No score | Outside the requested scope (Ümit, 12 Nov); not played (opponent no longer served on OpenRouter) | N/A |
Error rate and risk judgement: 95% intervals, whole games resampled. Ability d′ and caution c at the 0.001 margin: rates corrected by adding 0.5 to the counts and 1 to the totals; 95% intervals, blocks of two seat-swapped games resampled. Cut off: answers that stopped at an output limit the request sent (finish reason length, stop reason max_tokens, finishReason MAX_TOKENS, incomplete max_output_tokens), in the stored response or error. Effort: the reasoning effort the requests sent. Engine's best play: the share of its scored decisions that chose the engine's best play (equity loss under the best-play threshold). Time: median request seconds per decision (waits between retries not included). Cost per 1,000 decisions: the recorded tokens at the price shown, as labelled there (list or billed price; for a setup on our GPUs the hosted model's list price, not billed). Every match of every stage: Matches.
Risk judgement
Risk calls per setup: was each risk the right one, and was safe right when it played safe? 100 = every call right, 50 = always safe; the line on each bar is its 95% interval (games resampled) · higher is better
Leagues played different positions; close scores across leagues need not mean a real difference.
League 1
League 2
League 3
Analysis pending Shared positions: Analysis pending.
OpenAIGoogleAnthropicxAIDeepSeekMoonshotLiquid AIAlibabaInceptionCloudflareZ.aiFastinoPerplexitySubquadraticCelerisEldanRingTypeSafeZefan CaiBlockbrain Labsprogram (reference row)95% interval 50 = always safe
Ranks show observed scores; small gaps need not be meaningful. More caveats: Method
The Phase filter does not apply to these charts: all phases.
Error rate
How much equity each move gives away, judged by the engine, not by wins: error rate (PR) with its 95% interval as the line on each bar, log scale · lower is better
Leagues played different positions; close scores across leagues need not mean a real difference.
League 1
League 2
League 3
Analysis pending Shared positions: Analysis pending.
OpenAIGoogleAnthropicxAIDeepSeekMoonshotLiquid AIAlibabaInceptionCloudflareZ.aiFastinoPerplexitySubquadraticCelerisEldanRingTypeSafeZefan CaiBlockbrain Labsprogram (reference row)95% interval a program's error rate (reference row, named above the axis)
Setups played different positions; league comparisons are not paired. More caveats: Method
The Phase filter does not apply to these charts: all phases.
Cost
Dollars per 1,000 decisionsper 1M decisions, log scale · lower is better
Leagues played different positions; close scores across leagues need not mean a real difference.
League 1
League 2
League 3
Analysis pending Shared positions: Analysis pending.
Programs, Not measured: GNUbg weak (League 1, 2, 3); Random (League 1); Safest (League 1).
OpenAIGoogleAnthropicxAIDeepSeekMoonshotLiquid AIAlibabaInceptionCloudflareZ.aiFastinoPerplexitySubquadraticCelerisEldanRingTypeSafeZefan CaiBlockbrain Labsprogram (reference row)
Setups on our own machines: an estimate, GPU time, at most (wall-clock time × hourly rate, an upper bound); each setup's price label and how its cost was counted are in its row's tooltip and in the Cost tab. More caveats: Method
The Phase filter does not apply to these charts: all phases.
Time
Median seconds per decision (P90 beside it), log scale · lower is better
Leagues played different positions; close scores across leagues need not mean a real difference.
League 1
League 2
League 3
Analysis pending Shared positions: Analysis pending.
OpenAIGoogleAnthropicxAIDeepSeekMoonshotLiquid AIAlibabaInceptionCloudflareZ.aiFastinoPerplexitySubquadraticCelerisEldanRingTypeSafeZefan CaiBlockbrain Labsprogram (reference row)
Setups played different positions; league comparisons are not paired. More caveats: Method
The Phase filter does not apply to these charts: all phases.
Every number, setup by setup
Leagues played different positions; close scores across leagues need not mean a real difference.
Swipe for more columns →
| Model and setting | League | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Error rate by risk-call category (PR) | Decisions with a trade-off | Big risks found | Big needless risks | Extra hitting rolls: right / wrong and needless | Confidence on the call: ranking score for right risk | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astralowchat | League 3 | 75.873.1 to 78.5 | 57.2%226 / 395 | 94.4%486 / 515 | 84.0%226 / 269 | 39.2%155 / 395 | 5.6%29 / 515 | 3.5%14 / 395 | 11.8needless risks 0.64 · missed risks 6.63 · wrong risks 0.69 · right risks 1.81 · right safety 1.03 · No trade-off 1.00 | 910Risk pays 395 · Safety pays 515 · No trade-off 341 | 68.9%153 / 222 | 1.6%8 / 515 | 9.7 / Too few decisions226 / 43 decisions | N/A | 1.851.63 to 2.14 · risks taken 240 / 395 Risk pays · needless 29 / 515 Safety pays | +0.65+0.57 to +0.76 · cautious | 94.2%226 / 240 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.21β 0.68 · needless risks lost 0.056 on average (29) · missed risks 0.107 (155) |
| GPT-6 Astramediumchat | League 3 | 74.771.9 to 77.3 | 54.9%245 / 446 | 94.4%542 / 574 | 84.8%245 / 289 | 42.4%189 / 446 | 5.6%32 / 574 | 2.7%12 / 446 | 11.1needless risks 0.92 · missed risks 6.28 · wrong risks 0.34 · right risks 1.58 · right safety 1.31 · No trade-off 0.67 | 1,020Risk pays 446 · Safety pays 574 · No trade-off 342 | 70.5%170 / 241 | 1.6%9 / 574 | 8.7 / Too few decisions245 / 44 decisions | N/A | 1.781.59 to 2.00 · risks taken 257 / 446 Risk pays · needless 32 / 574 Safety pays | +0.70+0.60 to +0.82 · cautious | 95.3%245 / 257 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.06β 1.11 · needless risks lost 0.078 on average (32) · missed risks 0.091 (189) |
| Gemini 4 Argonhighchat | League 3 | 74.172.0 to 76.1 | 50.8%224 / 441 | 97.4%484 / 497 | 88.5%224 / 253 | 45.6%201 / 441 | 2.6%13 / 497 | 3.6%16 / 441 | 12.6needless risks 0.18 · missed risks 8.05 · wrong risks 0.55 · right risks 1.70 · right safety 1.34 · No trade-off 0.75 | 938Risk pays 441 · Safety pays 497 · No trade-off 394 | 66.4%148 / 223 | 0.4%2 / 497 | 11.0 / Too few decisions224 / 29 decisions | N/A | 2.041.82 to 2.36 · risks taken 240 / 441 Risk pays · needless 13 / 497 Safety pays | +0.91+0.77 to +1.09 · cautious | 93.3%224 / 240 | unstable: fewer than 200 needless risks behind a mean cost −0.47β 0.38 · needless risks lost 0.036 on average (13) · missed risks 0.107 (201) |
| GPT-6.1 Sollowchat | League 3 | 72.769.9 to 75.5 | 51.7%198 / 383 | 93.8%602 / 642 | 78.0%198 / 254 | 44.1%169 / 383 | 6.2%40 / 642 | 4.2%16 / 383 | 12.1needless risks 1.00 · missed risks 6.27 · wrong risks 0.83 · right risks 1.62 · right safety 1.78 · No trade-off 0.62 | 1,025Risk pays 383 · Safety pays 642 · No trade-off 319 | 65.0%134 / 206 | 1.6%10 / 642 | Too few decisions / Too few decisions198 / 56 decisions | N/A | 1.681.51 to 1.88 · risks taken 214 / 383 Risk pays · needless 40 / 642 Safety pays | +0.69+0.58 to +0.83 · cautious | 92.5%198 / 214 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.07β 1.13 · needless risks lost 0.067 on average (40) · missed risks 0.100 (169) |
| Claude Fable 5.1highchat | League 3 | 72.369.8 to 74.7 | 50.7%215 / 424 | 94.0%529 / 563 | 77.1%215 / 279 | 42.2%179 / 424 | 6.0%34 / 563 | 7.1%30 / 424 | 14.0needless risks 1.38 · missed risks 6.65 · wrong risks 1.53 · right risks 1.48 · right safety 2.23 · No trade-off 0.68 | 987Risk pays 424 · Safety pays 563 · No trade-off 299 | 65.5%144 / 220 | 2.7%15 / 563 | 9.7 / Too few decisions215 / 64 decisions | N/A | 1.741.56 to 1.96 · risks taken 245 / 424 Risk pays · needless 34 / 563 Safety pays | +0.67+0.55 to +0.81 · cautious | 87.8%215 / 245 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.22β 1.46 · needless risks lost 0.105 on average (34) · missed risks 0.096 (179) |
| GPT-6.1 Solhighchat | League 3 | 72.269.5 to 74.9 | 50.6%212 / 419 | 93.7%568 / 606 | 78.2%212 / 271 | 44.4%186 / 419 | 6.3%38 / 606 | 5.0%21 / 419 | 9.9needless risks 0.89 · missed risks 4.75 · wrong risks 0.90 · right risks 1.27 · right safety 1.53 · No trade-off 0.56 | 1,025Risk pays 419 · Safety pays 606 · No trade-off 364 | 71.2%148 / 208 | 1.7%10 / 606 | 10.8 / Too few decisions212 / 59 decisions | N/A | 1.671.42 to 1.94 · risks taken 233 / 419 Risk pays · needless 38 / 606 Safety pays | +0.69+0.60 to +0.79 · cautious | 91.0%212 / 233 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.17β 1.32 · needless risks lost 0.065 on average (38) · missed risks 0.071 (186) |
| Claude Opus 5.5highchat | League 3 | 70.067.3 to 72.6 | 43.4%199 / 459 | 96.6%590 / 611 | 81.6%199 / 244 | 51.4%236 / 459 | 3.4%21 / 611 | 5.2%24 / 459 | 14.4needless risks 0.62 · missed risks 8.96 · wrong risks 1.23 · right risks 1.26 · right safety 1.55 · No trade-off 0.81 | 1,070Risk pays 459 · Safety pays 611 · No trade-off 407 | 56.8%146 / 257 | 1.5%9 / 611 | Too few decisions / Too few decisions199 / 45 decisions | N/A | 1.771.53 to 2.05 · risks taken 223 / 459 Risk pays · needless 21 / 611 Safety pays | +0.92+0.79 to +1.08 · cautious | 89.2%199 / 223 | unstable: fewer than 200 needless risks behind a mean cost +0.02β 1.04 · needless risks lost 0.088 on average (21) · missed risks 0.112 (236) |
| Gemini 4 Argonlowchat | League 3 | 69.266.2 to 72.3 | 47.8%209 / 437 | 90.7%543 / 599 | 71.6%209 / 292 | 46.0%201 / 437 | 9.3%56 / 599 | 6.2%27 / 437 | 16.5needless risks 2.37 · missed risks 8.03 · wrong risks 1.23 · right risks 1.35 · right safety 2.53 · No trade-off 0.94 | 1,036Risk pays 437 · Safety pays 599 · No trade-off 448 | 59.1%137 / 232 | 4.0%24 / 599 | 10.8 / Too few decisions209 / 83 decisions | N/A | 1.421.22 to 1.64 · risks taken 236 / 437 Risk pays · needless 56 / 599 Safety pays | +0.61+0.52 to +0.70 · cautious | 88.6%209 / 236 | unstable: fewer than 200 needless risks behind a mean cost +0.26β 1.46 · needless risks lost 0.126 on average (56) · missed risks 0.119 (201) |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat | League 2 | 69.067.3 to 70.9 | 45.0%338 / 751 | 93.0%891 / 958 | 75.1%338 / 450 | 49.0%368 / 751 | 7.0%67 / 958 | 6.0%45 / 751 | 19.0needless risks 2.27 · missed risks 8.56 · wrong risks 1.93 · right risks 2.41 · right safety 2.89 · No trade-off 0.93 | 1,709Risk pays 751 · Safety pays 958 · No trade-off 479 | 62.8%255 / 406 | 3.4%33 / 958 | 9.1 / Too few decisions338 / 112 decisions | 0.53371 risks with a probability‡; 79 split into groups left out; mean confidence on missed risks 100.0% (304) | 1.501.36 to 1.64 · risks taken 383 / 751 Risk pays · needless 67 / 958 Safety pays | +0.72+0.63 to +0.81 · cautious | 88.3%338 / 383 | unstable: fewer than 200 needless risks behind a mean cost +0.41β 1.86 · needless risks lost 0.148 on average (67) · missed risks 0.102 (368) |
| Grok 4.7highchat | League 3 | 68.566.1 to 71.1 | 44.0%190 / 432 | 93.0%454 / 488 | 76.6%190 / 248 | 50.5%218 / 432 | 7.0%34 / 488 | 5.6%24 / 432 | 18.8needless risks 1.31 · missed risks 9.66 · wrong risks 2.39 · right risks 3.33 · right safety 1.53 · No trade-off 0.56 | 920Risk pays 432 · Safety pays 488 · No trade-off 417 | 53.8%129 / 240 | 2.7%13 / 488 | Too few decisions / Too few decisions190 / 58 decisions | N/A | 1.461.26 to 1.70 · risks taken 214 / 432 Risk pays · needless 34 / 488 Safety pays | +0.74+0.64 to +0.86 · cautious | 88.8%190 / 214 | unstable: fewer than 200 needless risks behind a mean cost −0.01β 0.99 · needless risks lost 0.103 on average (34) · missed risks 0.118 (218) |
| DeepSeek-V4.1-Flashlow · API · 1017chat | League 2 | 68.566.4 to 70.6 | 42.5%321 / 755 | 94.5%872 / 923 | 77.7%321 / 413 | 52.1%393 / 755 | 5.5%51 / 923 | 5.4%41 / 755 | 17.1needless risks 1.48 · missed risks 9.93 · wrong risks 1.57 · right risks 1.68 · right safety 1.63 · No trade-off 0.86 | 1,678Risk pays 755 · Safety pays 923 · No trade-off 544 | 55.4%226 / 408 | 2.8%26 / 923 | 9.2 / Too few decisions321 / 92 decisions | 0.51340 risks with a probability‡; 73 split into groups left out; mean confidence on missed risks 100.0% (337) | 1.541.39 to 1.70 · risks taken 362 / 755 Risk pays · needless 51 / 923 Safety pays | +0.82+0.74 to +0.91 · cautious | 88.7%321 / 362 | unstable: fewer than 200 needless risks behind a mean cost +0.22β 1.40 · needless risks lost 0.129 on average (51) · missed risks 0.112 (393) |
| Claude Sonnet 5.5highchat | League 2 | 68.266.3 to 70.0 | 41.8%308 / 737 | 94.6%841 / 889 | 81.9%308 / 376 | 55.5%409 / 737 | 5.4%48 / 889 | 2.7%20 / 737 | 16.7needless risks 1.12 · missed risks 10.76 · wrong risks 0.64 · right risks 1.70 · right safety 1.58 · No trade-off 0.92 | 1,626Risk pays 737 · Safety pays 889 · No trade-off 538 | 56.2%230 / 409 | 1.9%17 / 889 | 8.6 / Too few decisions308 / 68 decisions | N/A | 1.461.33 to 1.61 · risks taken 328 / 737 Risk pays · needless 48 / 889 Safety pays | +0.87+0.80 to +0.95 · cautious | 93.9%308 / 328 | unstable: fewer than 200 needless risks behind a mean cost +0.05β 1.07 · needless risks lost 0.101 on average (48) · missed risks 0.114 (409) |
| Claude Fable 5.1lowchat | League 2 | 67.865.6 to 69.9 | 41.0%270 / 658 | 94.6%781 / 826 | 77.4%270 / 349 | 53.8%354 / 658 | 5.4%45 / 826 | 5.2%34 / 658 | 17.9needless risks 1.59 · missed risks 8.82 · wrong risks 1.49 · right risks 1.74 · right safety 2.68 · No trade-off 1.61 | 1,484Risk pays 658 · Safety pays 826 · No trade-off 524 | 58.2%198 / 340 | 3.4%28 / 826 | 9.3 / Too few decisions270 / 79 decisions | N/A | 1.501.33 to 1.70 · risks taken 304 / 658 Risk pays · needless 45 / 826 Safety pays | +0.85+0.76 to +0.94 · cautious | 88.8%270 / 304 | unstable: fewer than 200 needless risks behind a mean cost +0.38β 1.78 · needless risks lost 0.142 on average (45) · missed risks 0.100 (354) |
| Grok 4.7lowchat | League 2 | 67.766.0 to 69.5 | 42.3%295 / 697 | 93.0%852 / 916 | 71.4%295 / 413 | 49.9%348 / 697 | 7.0%64 / 916 | 7.7%54 / 697 | 19.1needless risks 1.99 · missed risks 10.40 · wrong risks 2.28 · right risks 1.90 · right safety 2.04 · No trade-off 0.51 | 1,613Risk pays 697 · Safety pays 916 · No trade-off 539 | 51.4%201 / 391 | 3.4%31 / 916 | 9.3 / Too few decisions295 / 118 decisions | N/A | 1.481.34 to 1.63 · risks taken 349 / 697 Risk pays · needless 64 / 916 Safety pays | +0.74+0.67 to +0.81 · cautious | 84.5%295 / 349 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.37 · needless risks lost 0.134 on average (64) · missed risks 0.129 (348) |
| Claude Opus 5.5lowchat | League 3 | 66.964.7 to 69.2 | 40.1%187 / 466 | 93.8%541 / 577 | 77.3%187 / 242 | 55.8%260 / 466 | 6.2%36 / 577 | 4.1%19 / 466 | 15.4needless risks 1.67 · missed risks 8.98 · wrong risks 1.10 · right risks 1.00 · right safety 2.14 · No trade-off 0.55 | 1,043Risk pays 466 · Safety pays 577 · No trade-off 368 | 53.0%131 / 247 | 3.5%20 / 577 | Too few decisions / Too few decisions187 / 55 decisions | N/A | 1.381.17 to 1.62 · risks taken 206 / 466 Risk pays · needless 36 / 577 Safety pays | +0.84+0.72 to +0.97 · cautious | 90.8%187 / 206 | unstable: fewer than 200 needless risks behind a mean cost +0.37β 1.66 · needless risks lost 0.131 on average (36) · missed risks 0.097 (260) |
| Gemini 3.8 Flashhighchat | League 2 | 66.965.2 to 68.5 | 36.1%264 / 731 | 97.6%858 / 879 | 87.4%264 / 302 | 61.6%450 / 731 | 2.4%21 / 879 | 2.3%17 / 731 | 14.9needless risks 0.42 · missed risks 10.77 · wrong risks 0.59 · right risks 1.16 · right safety 1.50 · No trade-off 0.50 | 1,610Risk pays 731 · Safety pays 879 · No trade-off 558 | 52.9%193 / 365 | 0.5%4 / 879 | 9.2 / Too few decisions264 / 38 decisions | N/A | 1.681.49 to 1.88 · risks taken 281 / 731 Risk pays · needless 21 / 879 Safety pays | +1.13+1.05 to +1.23 · cautious | 94.0%264 / 281 | unstable: fewer than 200 needless risks behind a mean cost −0.00β 0.99 · needless risks lost 0.086 on average (21) · missed risks 0.104 (450) |
| Kimi K3.1 Prooffchat | League 3 | 65.963.2 to 68.9 | 42.4%153 / 361 | 89.5%416 / 465 | 65.7%153 / 233 | 49.0%177 / 361 | 10.5%49 / 465 | 8.6%31 / 361 | 28.0needless risks 4.62 · missed risks 10.08 · wrong risks 3.16 · right risks 2.22 · right safety 4.08 · No trade-off 3.89 | 826Risk pays 361 · Safety pays 465 · No trade-off 341 | 51.4%109 / 212 | 6.7%31 / 465 | Too few decisions / Too few decisions153 / 80 decisions | N/A | 1.271.11 to 1.45 · risks taken 184 / 361 Risk pays · needless 49 / 465 Safety pays | +0.61+0.51 to +0.72 · cautious | 83.2%153 / 184 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.60β 2.13 · needless risks lost 0.220 on average (49) · missed risks 0.133 (177) |
| Gemini 3.8 Flashlowchat | League 2 | 65.163.5 to 66.8 | 33.8%214 / 633 | 96.5%877 / 909 | 80.8%214 / 265 | 63.2%400 / 633 | 3.5%32 / 909 | 3.0%19 / 633 | 16.3needless risks 0.80 · missed risks 9.90 · wrong risks 0.74 · right risks 0.79 · right safety 2.82 · No trade-off 1.29 | 1,542Risk pays 633 · Safety pays 909 · No trade-off 511 | 49.5%151 / 305 | 1.7%15 / 909 | 9.7 / Too few decisions214 / 51 decisions | N/A | 1.471.33 to 1.64 · risks taken 233 / 633 Risk pays · needless 32 / 909 Safety pays | +1.07+0.99 to +1.17 · cautious | 91.8%214 / 233 | unstable: fewer than 200 needless risks behind a mean cost +0.25β 1.45 · needless risks lost 0.102 on average (32) · missed risks 0.102 (400) |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | 64.562.9 to 66.2 | 31.9%223 / 700 | 97.1%878 / 904 | 82.0%223 / 272 | 64.9%454 / 700 | 2.9%26 / 904 | 3.3%23 / 700 | 19.6needless risks 0.54 · missed risks 12.79 · wrong risks 1.21 · right risks 0.96 · right safety 3.00 · No trade-off 1.08 | 1,604Risk pays 700 · Safety pays 904 · No trade-off 518 | 43.7%164 / 375 | 1.2%11 / 904 | 10.3 / Too few decisions223 / 49 decisions | 0.52225 risks with a probability‡; 47 split into groups left out; mean confidence on missed risks 99.7% (385) | 1.511.36 to 1.69 · risks taken 246 / 700 Risk pays · needless 26 / 904 Safety pays | +1.14+1.05 to +1.23 · cautious | 90.7%223 / 246 | unstable: fewer than 200 needless risks behind a mean cost −0.03β 0.95 · needless risks lost 0.088 on average (26) · missed risks 0.120 (454) |
| d1-ultradecision | League 2 | 64.061.7 to 66.3 | 42.0%289 / 688 | 86.0%702 / 816 | 62.7%289 / 461 | 49.6%341 / 688 | 14.0%114 / 816 | 8.4%58 / 688 | 30.3needless risks 3.66 · missed risks 13.25 · wrong risks 3.02 · right risks 2.37 · right safety 4.52 · No trade-off 3.51 | 1,504Risk pays 688 · Safety pays 816 · No trade-off 567 | 48.5%180 / 371 | 7.4%60 / 816 | 9.9 / Too few decisions289 / 172 decisions | 0.64461 risks with a probability‡; mean confidence on missed risks 34.2% (340) | 1.090.92 to 1.26 · risks taken 347 / 688 Risk pays · needless 114 / 816 Safety pays | +0.53+0.46 to +0.61 · cautious | 83.3%289 / 347 | unstable: fewer than 200 needless risks behind a mean cost −0.02β 0.98 · needless risks lost 0.133 on average (114) · missed risks 0.161 (341) |
| Claude Sonnet 5.5lowchat | League 2 | 63.561.6 to 65.4 | 30.9%204 / 661 | 96.2%883 / 918 | 77.6%204 / 263 | 65.5%433 / 661 | 3.8%35 / 918 | 3.6%24 / 661 | 20.1needless risks 1.22 · missed risks 12.96 · wrong risks 0.96 · right risks 1.07 · right safety 2.67 · No trade-off 1.26 | 1,579Risk pays 661 · Safety pays 918 · No trade-off 571 | 41.9%145 / 346 | 2.7%25 / 918 | 8.7 / Too few decisions204 / 59 decisions | N/A | 1.371.19 to 1.58 · risks taken 228 / 661 Risk pays · needless 35 / 918 Safety pays | +1.08+0.99 to +1.19 · cautious | 89.5%204 / 228 | unstable: fewer than 200 needless risks behind a mean cost +0.35β 1.62 · needless risks lost 0.150 on average (35) · missed risks 0.129 (433) |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | 62.660.4 to 65.0 | 33.2%194 / 584 | 92.1%732 / 795 | 63.4%194 / 306 | 58.4%341 / 584 | 7.9%63 / 795 | 8.4%49 / 584 | 22.5needless risks 2.06 · missed risks 11.12 · wrong risks 2.12 · right risks 1.55 · right safety 3.07 · No trade-off 2.57 | 1,379Risk pays 584 · Safety pays 795 · No trade-off 468 | 44.7%136 / 304 | 4.9%39 / 795 | Too few decisions / Too few decisions194 / 112 decisions | 0.60261 risks with a probability‡; 45 split into groups left out; mean confidence on missed risks 97.5% (299) | 1.201.04 to 1.36 · risks taken 243 / 584 Risk pays · needless 63 / 795 Safety pays | +0.81+0.72 to +0.90 · cautious | 79.8%194 / 243 | unstable: fewer than 200 needless risks behind a mean cost +0.26β 1.36 · needless risks lost 0.121 on average (63) · missed risks 0.120 (341) |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | 62.060.1 to 63.9 | 33.8%242 / 715 | 90.2%735 / 815 | 64.0%242 / 378 | 58.3%417 / 715 | 9.8%80 / 815 | 7.8%56 / 715 | 29.0needless risks 2.89 · missed risks 14.82 · wrong risks 2.55 · right risks 1.94 · right safety 3.41 · No trade-off 3.43 | 1,530Risk pays 715 · Safety pays 815 · No trade-off 578 | 42.2%174 / 412 | 6.0%49 / 815 | 10.0 / Too few decisions242 / 136 decisions | 0.56332 risks with a probability‡; 46 split into groups left out; mean confidence on missed risks 100.0% (355) | 1.080.92 to 1.24 · risks taken 298 / 715 Risk pays · needless 80 / 815 Safety pays | +0.75+0.66 to +0.84 · cautious | 81.2%242 / 298 | unstable: fewer than 200 needless risks behind a mean cost +0.14β 1.16 · needless risks lost 0.153 on average (80) · missed risks 0.150 (417) |
| GLM-5.3lowchat | League 2 | 61.759.7 to 63.8 | 30.7%205 / 668 | 92.7%726 / 783 | 67.7%205 / 303 | 63.2%422 / 668 | 7.3%57 / 783 | 6.1%41 / 668 | 30.9needless risks 1.87 · missed risks 17.40 · wrong risks 2.14 · right risks 1.78 · right safety 3.99 · No trade-off 3.70 | 1,451Risk pays 668 · Safety pays 783 · No trade-off 538 | 36.8%141 / 383 | 4.5%35 / 783 | 8.8 / Too few decisions205 / 98 decisions | 0.52259 risks with a probability‡; 43 split into groups left out; mean confidence on missed risks 91.3% (373) | 1.120.95 to 1.29 · risks taken 246 / 668 Risk pays · needless 57 / 783 Safety pays | +0.89+0.81 to +0.98 · cautious | 83.3%205 / 246 | unstable: fewer than 200 needless risks behind a mean cost −0.06β 0.93 · needless risks lost 0.130 on average (57) · missed risks 0.164 (422) |
| GLiDEAPIdecision | League 2 | 60.458.3 to 62.6 | 36.9%249 / 675 | 84.0%756 / 900 | 52.4%249 / 475 | 51.0%344 / 675 | 16.0%144 / 900 | 12.1%82 / 675 | 33.2needless risks 7.18 · missed risks 10.97 · wrong risks 4.79 · right risks 2.03 · right safety 4.97 · No trade-off 3.24 | 1,575Risk pays 675 · Safety pays 900 · No trade-off 584 | 45.7%169 / 370 | 11.9%107 / 900 | 12.2 / 9.9249 / 226 decisions | 0.58475 risks with a probability‡; mean confidence on missed risks 39.7% (343) | 0.970.85 to 1.09 · risks taken 331 / 675 Risk pays · needless 144 / 900 Safety pays | +0.51+0.43 to +0.58 · cautious | 75.2%249 / 331 | unstable: fewer than 200 needless risks behind a mean cost +0.76β 2.08 · needless risks lost 0.215 on average (144) · missed risks 0.138 (344) |
| Jev 1.13decision | League 1 | 59.557.4 to 61.6 | 27.0%190 / 705 | 92.1%710 / 771 | 61.7%190 / 308 | 65.0%458 / 705 | 7.9%61 / 771 | 8.1%57 / 705 | 34.4needless risks 2.10 · missed risks 21.41 · wrong risks 3.52 · right risks 2.12 · right safety 2.66 · No trade-off 2.60 | 1,476Risk pays 705 · Safety pays 771 · No trade-off 435 | 33.0%143 / 433 | 3.8%29 / 771 | Too few decisions / Too few decisions190 / 118 decisions | 0.53308 risks with a probability‡; mean confidence on missed risks 52.2% (458) | 1.020.82 to 1.26 · risks taken 247 / 705 Risk pays · needless 61 / 771 Safety pays | +0.90+0.81 to +1.00 · cautious | 76.9%190 / 247 | unstable: fewer than 200 needless risks behind a mean cost −0.21β 0.80 · needless risks lost 0.131 on average (61) · missed risks 0.179 (458) |
| Qwen3.8 Flashoffchat | League 2 | 59.557.2 to 61.7 | 34.4%243 / 706 | 84.6%613 / 725 | 54.5%243 / 446 | 52.7%372 / 706 | 15.4%112 / 725 | 12.9%91 / 706 | 33.6needless risks 4.64 · missed risks 14.39 · wrong risks 3.96 · right risks 3.46 · right safety 2.96 · No trade-off 4.17 | 1,431Risk pays 706 · Safety pays 725 · No trade-off 596 | 42.5%166 / 391 | 9.5%69 / 725 | 7.7 / 9.5243 / 203 decisions | 0.57386 risks with a probability‡; 60 split into groups left out; mean confidence on missed risks 54.5% (328) | 0.950.79 to 1.12 · risks taken 334 / 706 Risk pays · needless 112 / 725 Safety pays | +0.54+0.46 to +0.62 · cautious | 72.8%243 / 334 | unstable: fewer than 200 needless risks behind a mean cost +0.10β 1.10 · needless risks lost 0.168 on average (112) · missed risks 0.157 (372) |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat | League 1 | 59.357.1 to 61.5 | 29.1%192 / 659 | 89.5%603 / 674 | 61.9%192 / 310 | 63.7%420 / 659 | 10.5%71 / 674 | 7.1%47 / 659 | 35.6needless risks 2.94 · missed risks 20.07 · wrong risks 3.16 · right risks 3.52 · right safety 3.53 · No trade-off 2.34 | 1,333Risk pays 659 · Safety pays 674 · No trade-off 403 | 38.3%150 / 392 | 5.6%38 / 674 | Too few decisions / Too few decisions192 / 118 decisions | 0.54269 risks with a probability‡; 41 split into groups left out; mean confidence on missed risks 72.5% (378) | 0.900.74 to 1.06 · risks taken 239 / 659 Risk pays · needless 71 / 674 Safety pays | +0.80+0.72 to +0.89 · cautious | 80.3%192 / 239 | unstable: fewer than 200 needless risks behind a mean cost −0.13β 0.89 · needless risks lost 0.144 on average (71) · missed risks 0.166 (420) |
| Jev 1.14decision | League 2 | 59.357.1 to 61.4 | 29.1%184 / 632 | 89.4%657 / 735 | 59.5%184 / 309 | 63.4%401 / 632 | 10.6%78 / 735 | 7.4%47 / 632 | 31.7needless risks 2.84 · missed risks 16.00 · wrong risks 2.91 · right risks 1.72 · right safety 4.37 · No trade-off 3.89 | 1,367Risk pays 632 · Safety pays 735 · No trade-off 545 | 33.2%118 / 355 | 6.1%45 / 735 | Too few decisions / Too few decisions184 / 125 decisions | 0.49309 risks with a probability‡; mean confidence on missed risks 48.2% (401) | 0.900.73 to 1.07 · risks taken 231 / 632 Risk pays · needless 78 / 735 Safety pays | +0.79+0.73 to +0.86 · cautious | 79.7%184 / 231 | unstable: fewer than 200 needless risks behind a mean cost +0.07β 1.06 · needless risks lost 0.139 on average (78) · missed risks 0.153 (401) |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 58.856.7 to 60.8 | 33.0%219 / 664 | 84.6%698 / 825 | 54.8%219 / 400 | 58.9%391 / 664 | 15.4%127 / 825 | 8.1%54 / 664 | 35.0needless risks 5.15 · missed risks 16.67 · wrong risks 2.72 · right risks 1.56 · right safety 5.81 · No trade-off 3.09 | 1,489Risk pays 664 · Safety pays 825 · No trade-off 514 | 38.4%139 / 362 | 9.1%75 / 825 | 12.4 / Too few decisions219 / 181 decisions | 0.60400 risks with a probability‡; mean confidence on missed risks 35.5% (391) | 0.790.67 to 0.93 · risks taken 273 / 664 Risk pays · needless 127 / 825 Safety pays | +0.62+0.54 to +0.71 · cautious | 80.2%219 / 273 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.18 · needless risks lost 0.162 on average (127) · missed risks 0.171 (391) |
| DeepSeek-V4.1-Flashoff · API · 1017chat | League 2 | 58.556.6 to 60.5 | 25.4%181 / 713 | 91.7%683 / 745 | 62.8%181 / 288 | 68.3%487 / 713 | 8.3%62 / 745 | 6.3%45 / 713 | 35.2needless risks 2.49 · missed risks 21.43 · wrong risks 2.50 · right risks 1.61 · right safety 3.75 · No trade-off 3.47 | 1,458Risk pays 713 · Safety pays 745 · No trade-off 576 | 33.3%138 / 415 | 5.1%38 / 745 | Too few decisions / Too few decisions181 / 107 decisions | 0.58262 risks with a probability‡; 26 split into groups left out; mean confidence on missed risks 81.0% (437) | 0.900.74 to 1.07 · risks taken 226 / 713 Risk pays · needless 62 / 745 Safety pays | +0.93+0.84 to +1.02 · cautious | 80.1%181 / 226 | unstable: fewer than 200 needless risks behind a mean cost −0.05β 0.96 · needless risks lost 0.164 on average (62) · missed risks 0.179 (487) |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 57.956.0 to 59.9 | 33.4%244 / 731 | 82.5%682 / 827 | 52.4%244 / 466 | 56.1%410 / 731 | 17.5%145 / 827 | 10.5%77 / 731 | 36.9needless risks 6.42 · missed risks 16.25 · wrong risks 3.85 · right risks 2.17 · right safety 4.74 · No trade-off 3.51 | 1,558Risk pays 731 · Safety pays 827 · No trade-off 470 | 38.9%162 / 416 | 12.2%101 / 827 | 10.1 / 8.9244 / 222 decisions | 0.55416 risks with a probability‡; 50 split into groups left out; mean confidence on missed risks 42.4% (337) | 0.780.67 to 0.89 · risks taken 321 / 731 Risk pays · needless 145 / 827 Safety pays | +0.54+0.48 to +0.61 · cautious | 76.0%244 / 321 | unstable: fewer than 200 needless risks behind a mean cost +0.30β 1.26 · needless risks lost 0.180 on average (145) · missed risks 0.161 (410) |
| celeris-1-decisiondecision | League 1 | 57.855.9 to 59.7 | 23.9%163 / 683 | 91.7%749 / 817 | 60.8%163 / 268 | 70.7%483 / 683 | 8.3%68 / 817 | 5.4%37 / 683 | 35.6needless risks 2.30 · missed risks 20.00 · wrong risks 2.07 · right risks 1.37 · right safety 5.50 · No trade-off 4.40 | 1,500Risk pays 683 · Safety pays 817 · No trade-off 422 | 32.1%120 / 374 | 4.7%38 / 817 | Too few decisions / Too few decisions163 / 105 decisions | 0.52268 risks with a probability‡; mean confidence on missed risks 67.9% (483) | 0.840.67 to 1.03 · risks taken 200 / 683 Risk pays · needless 68 / 817 Safety pays | +0.96+0.88 to +1.05 · cautious | 81.5%163 / 200 | unstable: fewer than 200 needless risks behind a mean cost −0.03β 0.97 · needless risks lost 0.130 on average (68) · missed risks 0.159 (483) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 1 | 57.656.1 to 59.0 | 23.5%178 / 756 | 91.7%671 / 732 | 63.3%178 / 281 | 70.9%536 / 756 | 8.3%61 / 732 | 5.6%42 / 756 | 45.0needless risks 3.20 · missed risks 26.37 · wrong risks 2.82 · right risks 2.92 · right safety 5.75 · No trade-off 3.98 | 1,488Risk pays 756 · Safety pays 732 · No trade-off 517 | 26.3%124 / 471 | 6.3%46 / 732 | Too few decisions / Too few decisions178 / 103 decisions | 0.50244 risks with a probability‡; 37 split into groups left out; mean confidence on missed risks 70.5% (477) | 0.830.70 to 0.98 · risks taken 220 / 756 Risk pays · needless 61 / 732 Safety pays | +0.96+0.89 to +1.04 · cautious | 80.9%178 / 220 | unstable: fewer than 200 needless risks behind a mean cost +0.04β 1.03 · needless risks lost 0.210 on average (61) · missed risks 0.197 (536) |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 2 | 57.555.4 to 59.8 | 39.5%263 / 666 | 75.6%603 / 798 | 47.0%263 / 560 | 45.2%301 / 666 | 24.4%195 / 798 | 15.3%102 / 666 | 36.5needless risks 8.85 · missed risks 11.85 · wrong risks 5.48 · right risks 2.26 · right safety 4.18 · No trade-off 3.92 | 1,464Risk pays 666 · Safety pays 798 · No trade-off 514 | 47.4%182 / 384 | 15.4%123 / 798 | 9.3 / 10.7263 / 297 decisions | 0.49552 risks with a probability‡; 8 split into groups left out; mean confidence on missed risks 22.6% (297) | 0.810.67 to 0.94 · risks taken 365 / 666 Risk pays · needless 195 / 798 Safety pays | +0.29+0.20 to +0.37 · cautious | 72.1%263 / 365 | unstable: fewer than 200 needless risks behind a mean cost +0.40β 1.38 · needless risks lost 0.180 on average (195) · missed risks 0.156 (301) |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 1 | 57.155.3 to 58.8 | 27.3%205 / 750 | 86.8%685 / 789 | 55.9%205 / 367 | 64.9%487 / 750 | 13.2%104 / 789 | 7.7%58 / 750 | 38.6needless risks 4.14 · missed risks 19.98 · wrong risks 3.19 · right risks 2.79 · right safety 4.26 · No trade-off 4.20 | 1,539Risk pays 750 · Safety pays 789 · No trade-off 477 | 35.6%159 / 447 | 8.5%67 / 789 | 7.6 / Too few decisions205 / 162 decisions | 0.56339 risks with a probability‡; 28 split into groups left out; mean confidence on missed risks 65.9% (452) | 0.730.60 to 0.88 · risks taken 263 / 750 Risk pays · needless 104 / 789 Safety pays | +0.75+0.67 to +0.83 · cautious | 77.9%205 / 263 | unstable: fewer than 200 needless risks behind a mean cost +0.03β 1.02 · needless risks lost 0.160 on average (104) · missed risks 0.165 (487) |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 1 | 56.954.9 to 58.8 | 33.1%233 / 703 | 80.7%682 / 845 | 50.3%233 / 463 | 57.3%403 / 703 | 19.3%163 / 845 | 9.5%67 / 703 | 38.8needless risks 8.32 · missed risks 16.99 · wrong risks 3.72 · right risks 2.11 · right safety 3.90 · No trade-off 3.75 | 1,548Risk pays 703 · Safety pays 845 · No trade-off 504 | 39.9%160 / 401 | 14.2%120 / 845 | 10.1 / 10.1233 / 230 decisions | 0.55400 risks with a probability‡; 63 split into groups left out; mean confidence on missed risks 53.3% (340) | 0.680.56 to 0.81 · risks taken 300 / 703 Risk pays · needless 163 / 845 Safety pays | +0.53+0.46 to +0.59 · cautious | 77.7%233 / 300 | unstable: fewer than 200 needless risks behind a mean cost +0.55β 1.45 · needless risks lost 0.209 on average (163) · missed risks 0.173 (403) |
| SubQ Snap Previewdecision | League 1 | 56.553.8 to 59.1 | 37.9%297 / 783 | 75.1%673 / 896 | 45.9%297 / 647 | 45.8%359 / 783 | 24.9%223 / 896 | 16.2%127 / 783 | 43.9needless risks 10.40 · missed risks 14.35 · wrong risks 7.22 · right risks 3.21 · right safety 5.11 · No trade-off 3.58 | 1,679Risk pays 783 · Safety pays 896 · No trade-off 521 | 45.9%201 / 438 | 17.4%156 / 896 | 9.6 / 10.9297 / 350 decisions | 0.61647 risks with a probability‡; mean confidence on missed risks 44.3% (358) | 0.780.65 to 0.93 · risks taken 424 / 783 Risk pays · needless 223 / 896 Safety pays | +0.29+0.22 to +0.36 · cautious | 70.0%297 / 424 | +0.37β 1.33 · needless risks lost 0.205 on average (223) · missed risks 0.176 (359) |
| GPT-6 LunaDecisions APIdecision | League 1 | 55.753.8 to 57.6 | 21.6%160 / 740 | 89.7%636 / 709 | 56.3%160 / 284 | 71.5%529 / 740 | 10.3%73 / 709 | 6.9%51 / 740 | 43.3needless risks 3.67 · missed risks 25.05 · wrong risks 3.45 · right risks 2.58 · right safety 4.50 · No trade-off 4.07 | 1,449Risk pays 740 · Safety pays 709 · No trade-off 513 | 28.6%123 / 430 | 7.5%53 / 709 | Too few decisions / Too few decisions160 / 124 decisions | 0.60284 risks with a probability‡; mean confidence on missed risks 54.6% (529) | 0.690.54 to 0.87 · risks taken 211 / 740 Risk pays · needless 73 / 709 Safety pays | +0.91+0.83 to +1.00 · cautious | 75.8%160 / 211 | unstable: fewer than 200 needless risks behind a mean cost +0.02β 1.02 · needless risks lost 0.197 on average (73) · missed risks 0.186 (529) |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 55.252.9 to 57.2 | 33.7%263 / 780 | 76.6%573 / 748 | 48.7%263 / 540 | 53.2%415 / 780 | 23.4%175 / 748 | 13.1%102 / 780 | 49.7needless risks 10.05 · missed risks 19.93 · wrong risks 6.06 · right risks 3.45 · right safety 5.37 · No trade-off 4.86 | 1,528Risk pays 780 · Safety pays 748 · No trade-off 487 | 39.9%176 / 441 | 16.4%123 / 748 | 9.8 / 10.2263 / 277 decisions | 0.62518 risks with a probability‡; 22 split into groups left out; mean confidence on missed risks 59.7% (390) | 0.640.53 to 0.77 · risks taken 365 / 780 Risk pays · needless 175 / 748 Safety pays | +0.40+0.32 to +0.49 · cautious | 72.1%263 / 365 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.15 · needless risks lost 0.231 on average (175) · missed risks 0.194 (415) |
| Mercury Decidedecision | League 1 | 53.451.5 to 55.2 | 18.7%129 / 690 | 88.1%583 / 662 | 51.2%129 / 252 | 74.9%517 / 690 | 11.9%79 / 662 | 6.4%44 / 690 | 47.9needless risks 5.59 · missed risks 29.17 · wrong risks 2.52 · right risks 1.67 · right safety 4.44 · No trade-off 4.55 | 1,352Risk pays 690 · Safety pays 662 · No trade-off 501 | 21.8%91 / 417 | 7.7%51 / 662 | Too few decisions / Too few decisions129 / 123 decisions | 0.63252 risks with a probability‡; mean confidence on missed risks 83.3% (517) | 0.500.36 to 0.67 · risks taken 173 / 690 Risk pays · needless 79 / 662 Safety pays | +0.92+0.84 to +1.02 · cautious | 74.6%129 / 173 | unstable: fewer than 200 needless risks behind a mean cost +0.37β 1.20 · needless risks lost 0.262 on average (79) · missed risks 0.209 (517) |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 1 | 52.450.3 to 54.5 | 31.5%248 / 788 | 73.4%647 / 882 | 38.2%248 / 650 | 47.3%373 / 788 | 26.6%235 / 882 | 21.2%167 / 788 | 47.7needless risks 13.29 · missed risks 12.21 · wrong risks 10.08 · right risks 2.73 · right safety 4.23 · No trade-off 5.15 | 1,670Risk pays 788 · Safety pays 882 · No trade-off 523 | 44.8%182 / 406 | 19.2%169 / 882 | 10.5 / 12.3248 / 402 decisions | 0.52572 risks with a probability‡; 78 split into groups left out; mean confidence on missed risks 100.0% (309) | 0.690.56 to 0.82 · risks taken 415 / 788 Risk pays · needless 235 / 882 Safety pays | +0.28+0.21 to +0.34 · cautious | 59.8%248 / 415 | +0.96β 1.93 · needless risks lost 0.248 on average (235) · missed risks 0.144 (373) |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 49.146.5 to 51.6 | 31.3%223 / 712 | 66.9%519 / 776 | 33.5%223 / 665 | 42.7%304 / 712 | 33.1%257 / 776 | 26.0%185 / 712 | 59.6needless risks 16.04 · missed risks 13.43 · wrong risks 13.80 · right risks 3.10 · right safety 6.17 · No trade-off 7.11 | 1,488Risk pays 712 · Safety pays 776 · No trade-off 518 | 41.9%167 / 399 | 25.3%196 / 776 | 10.7 / 13.4223 / 442 decisions | 0.56565 risks with a probability‡; 100 split into groups left out; mean confidence on missed risks 32.5% (270) | 0.620.51 to 0.74 · risks taken 408 / 712 Risk pays · needless 257 / 776 Safety pays | +0.13+0.05 to +0.20 · cautious | 54.7%223 / 408 | +0.70β 1.54 · needless risks lost 0.250 on average (257) · missed risks 0.177 (304) |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 48.546.1 to 50.9 | 37.3%264 / 708 | 59.8%455 / 761 | 34.4%264 / 768 | 34.7%246 / 708 | 40.2%306 / 761 | 28.0%198 / 708 | 65.6needless risks 21.42 · missed risks 12.49 · wrong risks 17.29 · right risks 3.05 · right safety 4.95 · No trade-off 6.35 | 1,469Risk pays 708 · Safety pays 761 · No trade-off 533 | 42.8%170 / 397 | 31.0%236 / 761 | 11.2 / 11.9264 / 504 decisions | 0.53768 risks with a probability‡; mean confidence on missed risks 14.9% (246) | 0.640.52 to 0.76 · risks taken 462 / 708 Risk pays · needless 306 / 761 Safety pays | −0.07−0.14 to −0.01 · bold | 57.1%264 / 462 | +0.62β 1.48 · needless risks lost 0.280 on average (306) · missed risks 0.203 (246) |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwellchat | League 1 | 45.743.2 to 48.1 | 28.8%206 / 715 | 62.7%470 / 750 | 31.8%206 / 648 | 48.5%347 / 715 | 37.3%280 / 750 | 22.7%162 / 715 | 61.5needless risks 18.70 · missed risks 14.79 · wrong risks 13.15 · right risks 2.81 · right safety 5.11 · No trade-off 6.90 | 1,465Risk pays 715 · Safety pays 750 · No trade-off 487 | 35.6%147 / 413 | 27.5%206 / 750 | 9.9 / 12.8206 / 442 decisions | 0.52573 risks with a probability‡; 75 split into groups left out; mean confidence on missed risks 39.2% (316) | 0.360.23 to 0.49 · risks taken 368 / 715 Risk pays · needless 280 / 750 Safety pays | +0.14+0.06 to +0.22 · cautious | 56.0%206 / 368 | +1.38β 1.64 · needless risks lost 0.261 on average (280) · missed risks 0.166 (347) |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 1 | 67.163.8 to 70.3 | 66.6%410 / 616 | 67.6%533 / 788 | 50.9%410 / 805 | 10.7%66 / 616 | 32.4%255 / 788 | 22.7%140 / 616 | 21.4needless risks 8.80 · missed risks 1.22 · wrong risks 5.99 · right risks 2.66 · right safety 1.66 · No trade-off 1.07 | 1,404Risk pays 616 · Safety pays 788 · No trade-off 340 | 82.7%254 / 307 | 18.0%142 / 788 | 11.2 / 13.1410 / 395 decisions | N/A | 1.701.53 to 1.89 · risks taken 550 / 616 Risk pays · needless 255 / 788 Safety pays | −0.39−0.48 to −0.30 · bold | 74.5%410 / 550 | unstable: fewer than 200 missed risks behind a mean cost +0.51β 2.39 · needless risks lost 0.120 on average (255) · missed risks 0.065 (66) |
| Randomprogram · reference row, never ranked · a random legal play | League 1 | 24.922.3 to 27.4 | 24.8%112 / 452 | 25.0%121 / 484 | 15.4%112 / 728 | 19.2%87 / 452 | 75.0%363 / 484 | 56.0%253 / 452 | 121.1needless risks 54.22 · missed risks 8.20 · wrong risks 38.00 · right risks 3.36 · right safety 3.88 · No trade-off 13.44 | 936Risk pays 452 · Safety pays 484 · No trade-off 401 | 29.2%78 / 267 | 68.0%329 / 484 | Too few decisions / 16.4112 / 616 decisions | N/A | 0.190.05 to 0.34 · risks taken 365 / 452 Risk pays · needless 363 / 484 Safety pays | −0.77−0.89 to −0.65 · bold | 30.7%112 / 365 | unstable: fewer than 200 missed risks behind a mean cost +2.73β 1.70 · needless risks lost 0.399 on average (363) · missed risks 0.252 (87) |
| Safestprogram · reference row, never ranked · always plays safe | League 1 | 50.050.0 to 50.0 | 0.0%0 / 785 | 100.0%749 / 749 | N/A0 / 0 | 100.0%785 / 785 | 0.0%0 / 749 | 0.0%0 / 785 | 51.6needless risks 0.00 · missed risks 34.95 · wrong risks 0.00 · right risks 0.00 · right safety 8.45 · No trade-off 8.25 | 1,534Risk pays 785 · Safety pays 749 · No trade-off 524 | 0.0%0 / 460 | 0.0%0 / 749 | 0 / 00 / 0 decisions | N/A | −0.01−0.05 to 0.02 · risks taken 0 / 785 Risk pays · needless 0 / 749 Safety pays | +3.22+3.20 to +3.23 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.183 (785) |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 2 | 62.359.5 to 64.8 | 62.7%483 / 770 | 61.9%558 / 902 | 48.2%483 / 1,003 | 14.4%111 / 770 | 38.1%344 / 902 | 22.9%176 / 770 | 27.0needless risks 12.05 · missed risks 2.19 · wrong risks 6.03 · right risks 2.93 · right safety 1.97 · No trade-off 1.80 | 1,672Risk pays 770 · Safety pays 902 · No trade-off 464 | 82.4%327 / 397 | 23.4%211 / 902 | 11.2 / 13.5483 / 520 decisions | N/A | 1.361.22 to 1.50 · risks taken 659 / 770 Risk pays · needless 344 / 902 Safety pays | −0.38−0.46 to −0.30 · bold | 73.3%483 / 659 | unstable: fewer than 200 missed risks behind a mean cost +0.54β 2.08 · needless risks lost 0.150 on average (344) · missed risks 0.084 (111) |
| Random (computed)program · reference row, never ranked · computed: every legal play equally likely | League 2 | 27.026.7 to 27.2 | 25.1%3,300.3 / 13,140 | 28.8%4,655.0 / 16,136 | 14.7%3,300.3 / 22,427.9 | 16.7%2,193.1 / 13,140 | 71.2%11,481.0 / 16,136 | 58.2%7,646.6 / 13,140 | 117.9needless risks 57.31 · missed risks 5.40 · wrong risks 40.03 · right risks 3.23 · right safety 3.91 · No trade-off 8.04 | 29,276Risk pays 13,140 · Safety pays 16,136 · No trade-off 10,212 | 31.6%2,248.9 / 7,127 | 63.1%10,184.8 / 16,136 | 15.4 / 17.83,300.3 / 19,127.7 decisions | N/A | 0.410.39 to 0.42 · risks taken 10,946.9 / 13,140 Risk pays · needless 11,481.0 / 16,136 Safety pays | −0.76−0.77 to −0.75 · bold | 30.1%3,300.3 / 10,946.9 | +2.23β 2.49 · needless risks lost 0.394 on average (11,481.0) · missed risks 0.195 (2,193.1) |
| Safest (computed)program · reference row, never ranked · computed: always the best safe play | League 2 | 50.050.0 to 50.0 | 0.0%0 / 13,140 | 100.0%16,136 / 16,136 | N/A0 / 0 | 100.0%13,140 / 13,140 | 0.0%0 / 16,136 | 0.0%0 / 13,140 | 25.2needless risks 0.00 · missed risks 25.23 · wrong risks 0.00 · right risks 0.00 · right safety 0.00 · No trade-off 0.00 | 29,276Risk pays 13,140 · Safety pays 16,136 · No trade-off 10,212 | 0.0%0 / 7,127 | 0.0%0 / 16,136 | 0 / 00 / 0 decisions | N/A | 0.050.04 to 0.06 · risks taken 0 / 13,140 Risk pays · needless 0 / 16,136 Safety pays | +3.98+3.98 to +3.98 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.152 (13,140) |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 3 | 62.158.2 to 65.9 | 58.8%288 / 490 | 65.5%415 / 634 | 45.9%288 / 627 | 16.7%82 / 490 | 34.5%219 / 634 | 24.5%120 / 490 | 28.1needless risks 11.03 · missed risks 2.16 · wrong risks 6.12 · right risks 4.03 · right safety 2.52 · No trade-off 2.22 | 1,124Risk pays 490 · Safety pays 634 · No trade-off 355 | 82.8%202 / 244 | 21.5%136 / 634 | 12.5 / 14.5288 / 339 decisions | N/A | 1.361.16 to 1.57 · risks taken 408 / 490 Risk pays · needless 219 / 634 Safety pays | −0.28−0.36 to −0.20 · bold | 70.6%288 / 408 | unstable: fewer than 200 missed risks behind a mean cost +0.67β 2.47 · needless risks lost 0.149 on average (219) · missed risks 0.078 (82) |
| Random (computed)program · reference row, never ranked · computed: every legal play equally likely | League 3 | 27.627.1 to 28.0 | 25.1%1,293.3 / 5,153 | 30.1%2,036.5 / 6,771 | 14.4%1,293.3 / 8,984.9 | 17.5%902.5 / 5,153 | 69.9%4,734.5 / 6,771 | 57.4%2,957.2 / 5,153 | 117.6needless risks 59.90 · missed risks 5.67 · wrong risks 36.45 · right risks 3.08 · right safety 4.60 · No trade-off 7.94 | 11,924Risk pays 5,153 · Safety pays 6,771 · No trade-off 4,395 | 31.4%862.9 / 2,752 | 62.8%4,253.1 / 6,771 | 15.1 / 17.71,293.3 / 7,691.6 decisions | N/A | 0.410.39 to 0.44 · risks taken 4,250.5 / 5,153 Risk pays · needless 4,734.5 / 6,771 Safety pays | −0.73−0.74 to −0.71 · bold | 30.4%1,293.3 / 4,250.5 | +2.36β 2.65 · needless risks lost 0.413 on average (4,734.5) · missed risks 0.205 (902.5) |
| Safest (computed)program · reference row, never ranked · computed: always the best safe play | League 3 | 50.050.0 to 50.0 | 0.0%0 / 5,153 | 100.0%6,771 / 6,771 | N/A0 / 0 | 100.0%5,153 / 5,153 | 0.0%0 / 6,771 | 0.0%0 / 5,153 | 23.5needless risks 0.00 · missed risks 23.47 · wrong risks 0.00 · right risks 0.00 · right safety 0.00 · No trade-off 0.00 | 11,924Risk pays 5,153 · Safety pays 6,771 · No trade-off 4,395 | 0.0%0 / 2,752 | 0.0%0 / 6,771 | 0 / 00 / 0 decisions | N/A | 0.070.06 to 0.08 · risks taken 0 / 5,153 Risk pays · needless 0 / 6,771 Safety pays | +3.76+3.75 to +3.77 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.149 (5,153) |
Hit risk only: being hit. Gammon risk and priming are not in it.
Setups met different positions; risk judgement is not adjusted for how hard they were. Paired comparisons wait for the shared positions.
Big risks found: stake above 0.08. Big needless risks: the chosen play lost more than 0.08.
‡ Risks with a probability: decisions whose plays were split into groups have no single probability and are left out; their count is shown per row. Setups that return no probabilities show N/A. Confidence on the call: ranking score only, no calibration gap (the probability is for the chosen play, not the risk). Definitions
Ability, caution and the SDT reference: A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity. Rates corrected by adding 0.5 to the counts and 1 to the totals; a setup without Risk pays or Safety pays decisions shows N/A. Intervals: 95%, blocks of two seat-swapped games resampled. Definitions
The same at the 0.02 margin (robustness)
Swipe for more columns →
| Model and setting | League | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 4 Argonhighchat | League 3 | 75.973.4 to 78.1 | 56.4%215 / 381 | 95.3%531 / 557 | 85.0%215 / 253 | 40.4%154 / 381 | 4.7%26 / 557 | 3.1%12 / 381 | 1.911.73 to 2.11 · risks taken 227 / 381 Risk pays · needless 26 / 557 Safety pays | +0.71+0.59 to +0.84 · cautious | 94.7%215 / 227 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.49β 0.40 · needless risks lost 0.036 on average (26) · missed risks 0.132 (154) |
| GPT-6 Astralowchat | League 3 | 74.971.9 to 77.9 | 58.1%209 / 360 | 91.8%505 / 550 | 77.7%209 / 269 | 37.8%136 / 360 | 8.2%45 / 550 | 4.2%15 / 360 | 1.701.52 to 1.88 · risks taken 224 / 360 Risk pays · needless 45 / 550 Safety pays | +0.54+0.47 to +0.62 · cautious | 93.3%209 / 224 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.44β 0.47 · needless risks lost 0.037 on average (45) · missed risks 0.118 (136) |
| GPT-6 Astramediumchat | League 3 | 74.972.4 to 77.3 | 56.5%225 / 398 | 93.2%580 / 622 | 77.9%225 / 289 | 37.9%151 / 398 | 6.8%42 / 622 | 5.5%22 / 398 | 1.801.62 to 1.99 · risks taken 247 / 398 Risk pays · needless 42 / 622 Safety pays | +0.59+0.50 to +0.70 · cautious | 91.1%225 / 247 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.09β 0.85 · needless risks lost 0.059 on average (42) · missed risks 0.109 (151) |
| GPT-6.1 Sollowchat | League 3 | 74.270.9 to 77.4 | 56.2%182 / 324 | 92.2%646 / 701 | 71.7%182 / 254 | 38.6%125 / 324 | 7.8%55 / 701 | 5.2%17 / 324 | 1.701.49 to 1.90 · risks taken 199 / 324 Risk pays · needless 55 / 701 Safety pays | +0.56+0.46 to +0.68 · cautious | 91.5%182 / 199 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.06β 0.90 · needless risks lost 0.054 on average (55) · missed risks 0.129 (125) |
| GPT-6.1 Solhighchat | League 3 | 72.469.7 to 75.1 | 52.4%188 / 359 | 92.5%616 / 666 | 69.4%188 / 271 | 38.4%138 / 359 | 7.5%50 / 666 | 9.2%33 / 359 | 1.731.52 to 1.96 · risks taken 221 / 359 Risk pays · needless 50 / 666 Safety pays | +0.57+0.48 to +0.67 · cautious | 85.1%188 / 221 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.07β 1.12 · needless risks lost 0.054 on average (50) · missed risks 0.090 (138) |
| Claude Fable 5.1highchat | League 3 | 72.169.5 to 74.9 | 52.7%195 / 370 | 91.6%565 / 617 | 69.9%195 / 279 | 38.6%143 / 370 | 8.4%52 / 617 | 8.6%32 / 370 | 1.661.49 to 1.86 · risks taken 227 / 370 Risk pays · needless 52 / 617 Safety pays | +0.54+0.43 to +0.67 · cautious | 85.9%195 / 227 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.06β 1.11 · needless risks lost 0.076 on average (52) · missed risks 0.114 (143) |
| Claude Opus 5.5highchat | League 3 | 70.867.8 to 73.8 | 46.3%188 / 406 | 95.3%633 / 664 | 77.0%188 / 244 | 47.5%193 / 406 | 4.7%31 / 664 | 6.2%25 / 406 | 1.731.49 to 2.01 · risks taken 213 / 406 Risk pays · needless 31 / 664 Safety pays | +0.80+0.68 to +0.94 · cautious | 88.3%188 / 213 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.14β 0.78 · needless risks lost 0.064 on average (31) · missed risks 0.134 (193) |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat | League 2 | 70.268.2 to 72.2 | 49.2%315 / 640 | 91.2%975 / 1,069 | 70.0%315 / 450 | 44.4%284 / 640 | 8.8%94 / 1,069 | 6.4%41 / 640 | 1.491.36 to 1.63 · risks taken 356 / 640 Risk pays · needless 94 / 1,069 Safety pays | +0.60+0.53 to +0.68 · cautious | 88.5%315 / 356 | unstable: fewer than 200 needless risks behind a mean cost +0.33β 1.64 · needless risks lost 0.125 on average (94) · missed risks 0.127 (284) |
| Claude Sonnet 5.5highchat | League 2 | 69.067.0 to 71.1 | 44.0%292 / 664 | 94.0%904 / 962 | 77.7%292 / 376 | 52.1%346 / 664 | 6.0%58 / 962 | 3.9%26 / 664 | 1.501.37 to 1.65 · risks taken 318 / 664 Risk pays · needless 58 / 962 Safety pays | +0.80+0.72 to +0.89 · cautious | 91.8%292 / 318 | unstable: fewer than 200 needless risks behind a mean cost −0.05β 0.93 · needless risks lost 0.084 on average (58) · missed risks 0.132 (346) |
| DeepSeek-V4.1-Flashlow · API · 1017chat | League 2 | 68.966.5 to 71.3 | 44.8%293 / 654 | 93.0%952 / 1,024 | 70.9%293 / 413 | 47.9%313 / 654 | 7.0%72 / 1,024 | 7.3%48 / 654 | 1.521.36 to 1.70 · risks taken 341 / 654 Risk pays · needless 72 / 1,024 Safety pays | +0.71+0.63 to +0.79 · cautious | 85.9%293 / 341 | unstable: fewer than 200 needless risks behind a mean cost +0.08β 1.13 · needless risks lost 0.097 on average (72) · missed risks 0.135 (313) |
| Gemini 4 Argonlowchat | League 3 | 68.765.6 to 71.9 | 49.6%188 / 379 | 87.8%577 / 657 | 64.4%188 / 292 | 44.1%167 / 379 | 12.2%80 / 657 | 6.3%24 / 379 | 1.311.13 to 1.50 · risks taken 212 / 379 Risk pays · needless 80 / 657 Safety pays | +0.51+0.42 to +0.60 · cautious | 88.7%188 / 212 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.10β 1.14 · needless risks lost 0.092 on average (80) · missed risks 0.140 (167) |
| Claude Fable 5.1lowchat | League 2 | 68.566.3 to 70.8 | 43.7%248 / 567 | 93.2%855 / 917 | 71.1%248 / 349 | 49.4%280 / 567 | 6.8%62 / 917 | 6.9%39 / 567 | 1.511.35 to 1.68 · risks taken 287 / 567 Risk pays · needless 62 / 917 Safety pays | +0.74+0.65 to +0.83 · cautious | 86.4%248 / 287 | unstable: fewer than 200 needless risks behind a mean cost +0.33β 1.65 · needless risks lost 0.122 on average (62) · missed risks 0.120 (280) |
| Grok 4.7highchat | League 3 | 67.964.9 to 70.9 | 44.6%171 / 383 | 91.1%489 / 537 | 69.0%171 / 248 | 47.8%183 / 383 | 8.9%48 / 537 | 7.6%29 / 383 | 1.401.20 to 1.60 · risks taken 200 / 383 Risk pays · needless 48 / 537 Safety pays | +0.64+0.55 to +0.75 · cautious | 85.5%171 / 200 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.13β 0.83 · needless risks lost 0.082 on average (48) · missed risks 0.137 (183) |
| Gemini 3.8 Flashhighchat | League 2 | 67.665.5 to 69.6 | 39.2%239 / 610 | 96.1%961 / 1,000 | 79.1%239 / 302 | 56.9%347 / 610 | 3.9%39 / 1,000 | 3.9%24 / 610 | 1.581.40 to 1.79 · risks taken 263 / 610 Risk pays · needless 39 / 1,000 Safety pays | +0.97+0.89 to +1.05 · cautious | 90.9%239 / 263 | unstable: fewer than 200 needless risks behind a mean cost −0.29β 0.63 · needless risks lost 0.049 on average (39) · missed risks 0.128 (347) |
| Claude Opus 5.5lowchat | League 3 | 67.164.5 to 69.5 | 41.9%168 / 401 | 92.4%593 / 642 | 69.4%168 / 242 | 51.9%208 / 401 | 7.6%49 / 642 | 6.2%25 / 401 | 1.381.17 to 1.61 · risks taken 193 / 401 Risk pays · needless 49 / 642 Safety pays | +0.74+0.63 to +0.87 · cautious | 87.0%168 / 193 | unstable: fewer than 200 needless risks behind a mean cost +0.30β 1.51 · needless risks lost 0.108 on average (49) · missed risks 0.115 (208) |
| Grok 4.7lowchat | League 2 | 67.064.9 to 69.1 | 42.8%263 / 614 | 91.1%910 / 999 | 63.7%263 / 413 | 47.2%290 / 614 | 8.9%89 / 999 | 9.9%61 / 614 | 1.411.26 to 1.57 · risks taken 324 / 614 Risk pays · needless 89 / 999 Safety pays | +0.64+0.57 to +0.71 · cautious | 81.2%263 / 324 | unstable: fewer than 200 needless risks behind a mean cost +0.08β 1.13 · needless risks lost 0.104 on average (89) · missed risks 0.150 (290) |
| Gemini 3.8 Flashlowchat | League 2 | 66.864.9 to 68.8 | 37.6%203 / 540 | 96.0%962 / 1,002 | 76.6%203 / 265 | 58.3%315 / 540 | 4.0%40 / 1,002 | 4.1%22 / 540 | 1.541.38 to 1.71 · risks taken 225 / 540 Risk pays · needless 40 / 1,002 Safety pays | +0.98+0.90 to +1.07 · cautious | 90.2%203 / 225 | unstable: fewer than 200 needless risks behind a mean cost +0.22β 1.40 · needless risks lost 0.093 on average (40) · missed risks 0.124 (315) |
| Kimi K3.1 Prooffchat | League 3 | 66.763.7 to 69.8 | 44.7%143 / 320 | 88.7%449 / 506 | 61.4%143 / 233 | 45.0%144 / 320 | 11.3%57 / 506 | 10.3%33 / 320 | 1.331.18 to 1.49 · risks taken 176 / 320 Risk pays · needless 57 / 506 Safety pays | +0.54+0.44 to +0.64 · cautious | 81.2%143 / 176 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.55β 2.09 · needless risks lost 0.209 on average (57) · missed risks 0.158 (144) |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | 65.663.8 to 67.5 | 34.7%213 / 613 | 96.5%956 / 991 | 78.3%213 / 272 | 61.3%376 / 613 | 3.5%35 / 991 | 3.9%24 / 613 | 1.511.35 to 1.68 · risks taken 237 / 613 Risk pays · needless 35 / 991 Safety pays | +1.04+0.96 to +1.14 · cautious | 89.9%213 / 237 | unstable: fewer than 200 needless risks behind a mean cost +0.07β 1.11 · needless risks lost 0.095 on average (35) · missed risks 0.139 (376) |
| Claude Sonnet 5.5lowchat | League 2 | 64.262.1 to 66.2 | 33.1%190 / 574 | 95.2%957 / 1,005 | 72.2%190 / 263 | 62.5%359 / 574 | 4.8%48 / 1,005 | 4.4%25 / 574 | 1.341.18 to 1.53 · risks taken 215 / 574 Risk pays · needless 48 / 1,005 Safety pays | +0.99+0.91 to +1.09 · cautious | 88.4%190 / 215 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.33 · needless risks lost 0.115 on average (48) · missed risks 0.151 (359) |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | 63.261.0 to 65.6 | 35.5%182 / 513 | 90.9%787 / 866 | 59.5%182 / 306 | 55.8%286 / 513 | 9.1%79 / 866 | 8.8%45 / 513 | 1.191.03 to 1.35 · risks taken 227 / 513 Risk pays · needless 79 / 866 Safety pays | +0.74+0.66 to +0.82 · cautious | 80.2%182 / 227 | unstable: fewer than 200 needless risks behind a mean cost +0.22β 1.30 · needless risks lost 0.106 on average (79) · missed risks 0.138 (286) |
| d1-ultradecision | League 2 | 63.060.8 to 65.3 | 41.8%254 / 607 | 84.2%755 / 897 | 55.1%254 / 461 | 47.4%288 / 607 | 15.8%142 / 897 | 10.7%65 / 607 | 1.060.91 to 1.22 · risks taken 319 / 607 Risk pays · needless 142 / 897 Safety pays | +0.47+0.40 to +0.54 · cautious | 79.6%254 / 319 | unstable: fewer than 200 needless risks behind a mean cost −0.02β 0.98 · needless risks lost 0.122 on average (142) · missed risks 0.185 (288) |
| GLM-5.3lowchat | League 2 | 61.659.5 to 63.9 | 31.8%184 / 579 | 91.4%797 / 872 | 60.7%184 / 303 | 60.6%351 / 579 | 8.6%75 / 872 | 7.6%44 / 579 | 1.090.91 to 1.27 · risks taken 228 / 579 Risk pays · needless 75 / 872 Safety pays | +0.82+0.74 to +0.89 · cautious | 80.7%184 / 228 | unstable: fewer than 200 needless risks behind a mean cost −0.12β 0.88 · needless risks lost 0.113 on average (75) · missed risks 0.193 (351) |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | 61.259.2 to 63.2 | 34.4%217 / 630 | 87.9%791 / 900 | 57.4%217 / 378 | 57.3%361 / 630 | 12.1%109 / 900 | 8.3%52 / 630 | 0.980.83 to 1.15 · risks taken 269 / 630 Risk pays · needless 109 / 900 Safety pays | +0.68+0.59 to +0.77 · cautious | 80.7%217 / 269 | unstable: fewer than 200 needless risks behind a mean cost +0.11β 1.11 · needless risks lost 0.131 on average (109) · missed risks 0.169 (361) |
| GLiDEAPIdecision | League 2 | 60.558.2 to 62.6 | 38.4%228 / 593 | 82.5%810 / 982 | 48.0%228 / 475 | 48.9%290 / 593 | 17.5%172 / 982 | 12.6%75 / 593 | 0.960.86 to 1.07 · risks taken 303 / 593 Risk pays · needless 172 / 982 Safety pays | +0.45+0.38 to +0.53 · cautious | 75.2%228 / 303 | unstable: fewer than 200 needless risks behind a mean cost +0.73β 2.01 · needless risks lost 0.190 on average (172) · missed risks 0.157 (290) |
| Qwen3.8 Flashoffchat | League 2 | 59.357.1 to 61.5 | 35.5%222 / 625 | 83.0%669 / 806 | 49.8%222 / 446 | 50.6%316 / 625 | 17.0%137 / 806 | 13.9%87 / 625 | 0.940.79 to 1.08 · risks taken 309 / 625 Risk pays · needless 137 / 806 Safety pays | +0.48+0.41 to +0.56 · cautious | 71.8%222 / 309 | unstable: fewer than 200 needless risks behind a mean cost +0.06β 1.05 · needless risks lost 0.147 on average (137) · missed risks 0.180 (316) |
| Jev 1.14decision | League 2 | 59.257.0 to 61.4 | 29.9%167 / 559 | 88.5%715 / 808 | 54.0%167 / 309 | 61.4%343 / 559 | 11.5%93 / 808 | 8.8%49 / 559 | 0.910.75 to 1.08 · risks taken 216 / 559 Risk pays · needless 93 / 808 Safety pays | +0.74+0.68 to +0.81 · cautious | 77.3%167 / 216 | unstable: fewer than 200 needless risks behind a mean cost −0.00β 1.00 · needless risks lost 0.121 on average (93) · missed risks 0.174 (343) |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat | League 1 | 59.056.7 to 61.3 | 30.1%177 / 588 | 87.9%655 / 745 | 57.1%177 / 310 | 62.6%368 / 588 | 12.1%90 / 745 | 7.3%43 / 588 | 0.850.72 to 0.98 · risks taken 220 / 588 Risk pays · needless 90 / 745 Safety pays | +0.74+0.66 to +0.83 · cautious | 80.5%177 / 220 | unstable: fewer than 200 needless risks behind a mean cost −0.20β 0.84 · needless risks lost 0.124 on average (90) · missed risks 0.187 (368) |
| DeepSeek-V4.1-Flashoff · API · 1017chat | League 2 | 58.856.7 to 60.9 | 26.5%168 / 635 | 91.1%750 / 823 | 58.3%168 / 288 | 66.1%420 / 635 | 8.9%73 / 823 | 7.4%47 / 635 | 0.930.77 to 1.10 · risks taken 215 / 635 Risk pays · needless 73 / 823 Safety pays | +0.88+0.79 to +0.97 · cautious | 78.1%168 / 215 | unstable: fewer than 200 needless risks behind a mean cost −0.09β 0.92 · needless risks lost 0.145 on average (73) · missed risks 0.204 (420) |
| Jev 1.13decision | League 1 | 58.456.2 to 60.8 | 26.8%167 / 622 | 90.0%769 / 854 | 54.2%167 / 308 | 64.1%399 / 622 | 10.0%85 / 854 | 9.0%56 / 622 | 0.920.72 to 1.13 · risks taken 223 / 622 Risk pays · needless 85 / 854 Safety pays | +0.82+0.74 to +0.91 · cautious | 74.9%167 / 223 | unstable: fewer than 200 needless risks behind a mean cost −0.36β 0.72 · needless risks lost 0.106 on average (85) · missed risks 0.202 (399) |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 57.955.4 to 60.4 | 33.1%192 / 580 | 82.7%752 / 909 | 48.0%192 / 400 | 58.1%337 / 580 | 17.3%157 / 909 | 8.8%51 / 580 | 0.740.59 to 0.89 · risks taken 243 / 580 Risk pays · needless 157 / 909 Safety pays | +0.57+0.49 to +0.66 · cautious | 79.0%192 / 243 | unstable: fewer than 200 needless risks behind a mean cost +0.15β 1.11 · needless risks lost 0.138 on average (157) · missed risks 0.194 (337) |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 2 | 57.655.2 to 60.0 | 40.6%240 / 591 | 74.6%651 / 873 | 42.9%240 / 560 | 42.8%253 / 591 | 25.4%222 / 873 | 16.6%98 / 591 | 0.840.70 to 0.99 · risks taken 338 / 591 Risk pays · needless 222 / 873 Safety pays | +0.24+0.15 to +0.32 · cautious | 71.0%240 / 338 | +0.39β 1.39 · needless risks lost 0.168 on average (222) · missed risks 0.178 (253) |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 1 | 57.355.4 to 59.2 | 28.2%192 / 680 | 86.4%742 / 859 | 52.3%192 / 367 | 63.2%430 / 680 | 13.6%117 / 859 | 8.5%58 / 680 | 0.760.63 to 0.89 · risks taken 250 / 680 Risk pays · needless 117 / 859 Safety pays | +0.72+0.64 to +0.80 · cautious | 76.8%192 / 250 | unstable: fewer than 200 needless risks behind a mean cost +0.05β 1.04 · needless risks lost 0.151 on average (117) · missed risks 0.184 (430) |
| celeris-1-decisiondecision | League 1 | 57.355.4 to 59.1 | 24.1%141 / 586 | 90.5%827 / 914 | 52.6%141 / 268 | 69.1%405 / 586 | 9.5%87 / 914 | 6.8%40 / 586 | 0.810.65 to 0.98 · risks taken 181 / 586 Risk pays · needless 87 / 914 Safety pays | +0.90+0.82 to +0.99 · cautious | 77.9%141 / 181 | unstable: fewer than 200 needless risks behind a mean cost −0.10β 0.92 · needless risks lost 0.107 on average (87) · missed risks 0.181 (405) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 1 | 56.755.1 to 58.2 | 22.8%157 / 688 | 90.6%725 / 800 | 55.9%157 / 281 | 70.1%482 / 688 | 9.4%75 / 800 | 7.1%49 / 688 | 0.790.66 to 0.93 · risks taken 206 / 688 Risk pays · needless 75 / 800 Safety pays | +0.92+0.85 to +0.99 · cautious | 76.2%157 / 206 | unstable: fewer than 200 needless risks behind a mean cost −0.03β 0.98 · needless risks lost 0.180 on average (75) · missed risks 0.215 (482) |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 1 | 56.754.7 to 58.7 | 33.7%211 / 626 | 79.7%735 / 922 | 45.6%211 / 463 | 55.9%350 / 626 | 20.3%187 / 922 | 10.4%65 / 626 | 0.680.55 to 0.82 · risks taken 276 / 626 Risk pays · needless 187 / 922 Safety pays | +0.49+0.42 to +0.56 · cautious | 76.4%211 / 276 | unstable: fewer than 200 needless risks behind a mean cost +0.55β 1.46 · needless risks lost 0.190 on average (187) · missed risks 0.192 (350) |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 56.654.6 to 58.7 | 33.1%213 / 644 | 80.2%733 / 914 | 45.7%213 / 466 | 55.7%359 / 644 | 19.8%181 / 914 | 11.2%72 / 644 | 0.700.60 to 0.81 · risks taken 285 / 644 Risk pays · needless 181 / 914 Safety pays | +0.50+0.42 to +0.57 · cautious | 74.7%213 / 285 | unstable: fewer than 200 needless risks behind a mean cost +0.30β 1.23 · needless risks lost 0.155 on average (181) · missed risks 0.179 (359) |
| SubQ Snap Previewdecision | League 1 | 55.953.1 to 58.8 | 38.4%263 / 685 | 73.4%730 / 994 | 40.6%263 / 647 | 44.1%302 / 685 | 26.6%264 / 994 | 17.5%120 / 685 | 0.770.63 to 0.93 · risks taken 383 / 685 Risk pays · needless 264 / 994 Safety pays | +0.24+0.16 to +0.31 · cautious | 68.7%263 / 383 | +0.40β 1.37 · needless risks lost 0.189 on average (264) · missed risks 0.201 (302) |
| GPT-6 LunaDecisions APIdecision | League 1 | 55.253.2 to 57.1 | 21.7%144 / 663 | 88.7%697 / 786 | 50.7%144 / 284 | 70.6%468 / 663 | 11.3%89 / 786 | 7.7%51 / 663 | 0.670.50 to 0.84 · risks taken 195 / 663 Risk pays · needless 89 / 786 Safety pays | +0.87+0.79 to +0.96 · cautious | 73.8%144 / 195 | unstable: fewer than 200 needless risks behind a mean cost +0.05β 1.03 · needless risks lost 0.179 on average (89) · missed risks 0.205 (468) |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 54.552.7 to 56.2 | 33.6%233 / 693 | 75.3%629 / 835 | 43.1%233 / 540 | 51.8%359 / 693 | 24.7%206 / 835 | 14.6%101 / 693 | 0.640.52 to 0.76 · risks taken 334 / 693 Risk pays · needless 206 / 835 Safety pays | +0.36+0.28 to +0.45 · cautious | 69.8%233 / 334 | +0.24β 1.17 · needless risks lost 0.211 on average (206) · missed risks 0.218 (359) |
| Mercury Decidedecision | League 1 | 53.151.3 to 55.1 | 18.4%115 / 625 | 87.8%638 / 727 | 45.6%115 / 252 | 73.9%462 / 625 | 12.2%89 / 727 | 7.7%48 / 625 | 0.520.37 to 0.67 · risks taken 163 / 625 Risk pays · needless 89 / 727 Safety pays | +0.90+0.82 to +0.99 · cautious | 70.6%115 / 163 | unstable: fewer than 200 needless risks behind a mean cost +0.32β 1.18 · needless risks lost 0.233 on average (89) · missed risks 0.230 (462) |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 1 | 52.450.2 to 54.6 | 32.2%224 / 695 | 72.6%708 / 975 | 34.5%224 / 650 | 44.9%312 / 695 | 27.4%267 / 975 | 22.9%159 / 695 | 0.730.61 to 0.85 · risks taken 383 / 695 Risk pays · needless 267 / 975 Safety pays | +0.24+0.18 to +0.30 · cautious | 58.5%224 / 383 | +1.01β 2.09 · needless risks lost 0.243 on average (267) · missed risks 0.163 (312) |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 48.045.5 to 50.7 | 37.7%236 / 626 | 58.4%492 / 843 | 30.7%236 / 768 | 33.4%209 / 626 | 41.6%351 / 843 | 28.9%181 / 626 | 0.640.50 to 0.78 · risks taken 417 / 626 Risk pays · needless 351 / 843 Safety pays | −0.11−0.18 to −0.04 · bold | 56.6%236 / 417 | +0.72β 1.58 · needless risks lost 0.271 on average (351) · missed risks 0.231 (209) |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | 47.745.0 to 50.1 | 30.6%192 / 627 | 64.8%558 / 861 | 28.9%192 / 665 | 42.3%265 / 627 | 35.2%303 / 861 | 27.1%170 / 627 | 0.570.46 to 0.70 · risks taken 362 / 627 Risk pays · needless 303 / 861 Safety pays | +0.09+0.02 to +0.17 · cautious | 53.0%192 / 362 | +0.82β 1.60 · needless risks lost 0.232 on average (303) · missed risks 0.198 (265) |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwellchat | League 1 | 44.942.6 to 47.6 | 28.4%181 / 637 | 61.5%509 / 828 | 27.9%181 / 648 | 48.4%308 / 637 | 38.5%319 / 828 | 23.2%148 / 637 | 0.330.20 to 0.46 · risks taken 329 / 637 Risk pays · needless 319 / 828 Safety pays | +0.13+0.05 to +0.21 · cautious | 55.0%181 / 329 | +1.71β 1.77 · needless risks lost 0.249 on average (319) · missed risks 0.183 (308) |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 1 | 64.361.1 to 67.5 | 66.1%345 / 522 | 62.6%552 / 882 | 42.9%345 / 805 | 9.0%47 / 522 | 37.4%330 / 882 | 24.9%130 / 522 | 1.661.47 to 1.88 · risks taken 475 / 522 Risk pays · needless 330 / 882 Safety pays | −0.51−0.62 to −0.41 · bold | 72.6%345 / 475 | unstable: fewer than 200 missed risks behind a mean cost +0.47β 2.19 · needless risks lost 0.105 on average (330) · missed risks 0.081 (47) |
| Randomprogram · reference row, never ranked · a random legal play | League 1 | 23.721.0 to 26.3 | 22.7%94 / 414 | 24.7%129 / 522 | 12.9%94 / 728 | 19.1%79 / 414 | 75.3%393 / 522 | 58.2%241 / 414 | 0.190.05 to 0.35 · risks taken 335 / 414 Risk pays · needless 393 / 522 Safety pays | −0.78−0.90 to −0.66 · bold | 28.1%94 / 335 | unstable: fewer than 200 missed risks behind a mean cost +2.95β 1.75 · needless risks lost 0.377 on average (393) · missed risks 0.271 (79) |
| Safestprogram · reference row, never ranked · always plays safe | League 1 | 50.050.0 to 50.0 | 0.0%0 / 707 | 100.0%827 / 827 | N/A0 / 0 | 100.0%707 / 707 | 0.0%0 / 827 | 0.0%0 / 707 | 0.040.01 to 0.08 · risks taken 0 / 707 Risk pays · needless 0 / 827 Safety pays | +3.21+3.20 to +3.23 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.200 (707) |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 2 | 61.758.7 to 64.6 | 64.8%420 / 648 | 58.5%599 / 1,024 | 41.9%420 / 1,003 | 10.8%70 / 648 | 41.5%425 / 1,024 | 24.4%158 / 648 | 1.451.31 to 1.61 · risks taken 578 / 648 Risk pays · needless 425 / 1,024 Safety pays | −0.51−0.60 to −0.42 · bold | 72.7%420 / 578 | unstable: fewer than 200 missed risks behind a mean cost +0.36β 1.68 · needless risks lost 0.130 on average (425) · missed risks 0.122 (70) |
| Random (computed)program · reference row, never ranked · computed: every legal play equally likely | League 2 | 25.925.6 to 26.1 | 23.7%2,710.4 / 11,456 | 28.1%5,001.1 / 17,820 | 12.1%2,710.4 / 22,427.9 | 16.1%1,847.0 / 11,456 | 71.9%12,818.9 / 17,820 | 60.2%6,898.7 / 11,456 | 0.410.39 to 0.43 · risks taken 9,609.0 / 11,456 Risk pays · needless 12,818.9 / 17,820 Safety pays | −0.79−0.79 to −0.78 · bold | 28.2%2,710.4 / 9,609.0 | +2.43β 2.69 · needless risks lost 0.381 on average (12,818.9) · missed risks 0.220 (1,847.0) |
| Safest (computed)program · reference row, never ranked · computed: always the best safe play | League 2 | 50.050.0 to 50.0 | 0.0%0 / 11,456 | 100.0%17,820 / 17,820 | N/A0 / 0 | 100.0%11,456 / 11,456 | 0.0%0 / 17,820 | 0.0%0 / 11,456 | 0.110.10 to 0.11 · risks taken 0 / 11,456 Risk pays · needless 0 / 17,820 Safety pays | +3.98+3.97 to +3.98 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.172 (11,456) |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 3 | 60.056.4 to 63.4 | 58.1%247 / 425 | 61.8%432 / 699 | 39.4%247 / 627 | 15.3%65 / 425 | 38.2%267 / 699 | 26.6%113 / 425 | 1.321.14 to 1.52 · risks taken 360 / 425 Risk pays · needless 267 / 699 Safety pays | −0.36−0.45 to −0.27 · bold | 68.6%247 / 360 | unstable: fewer than 200 missed risks behind a mean cost +0.66β 2.38 · needless risks lost 0.132 on average (267) · missed risks 0.091 (65) |
| Random (computed)program · reference row, never ranked · computed: every legal play equally likely | League 3 | 26.425.9 to 26.9 | 23.4%1,054.8 / 4,506 | 29.4%2,178.6 / 7,418 | 11.7%1,054.8 / 8,984.9 | 16.9%760.5 / 4,506 | 70.6%5,239.4 / 7,418 | 59.7%2,690.7 / 4,506 | 0.420.39 to 0.44 · risks taken 3,745.5 / 4,506 Risk pays · needless 5,239.4 / 7,418 Safety pays | −0.75−0.77 to −0.73 · bold | 28.2%1,054.8 / 3,745.5 | +2.51β 2.84 · needless risks lost 0.398 on average (5,239.4) · missed risks 0.231 (760.5) |
| Safest (computed)program · reference row, never ranked · computed: always the best safe play | League 3 | 50.050.0 to 50.0 | 0.0%0 / 4,506 | 100.0%7,418 / 7,418 | N/A0 / 0 | 100.0%4,506 / 4,506 | 0.0%0 / 7,418 | 0.0%0 / 4,506 | 0.120.11 to 0.14 · risks taken 0 / 4,506 Risk pays · needless 0 / 7,418 Safety pays | +3.76+3.75 to +3.76 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.169 (4,506) |
Ability and caution by reasoning setting (UNPAIRED) are in each league's view: League 1, 2 or 3.
League 1
Swipe for more columns →
| Model and setting | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Error rate by risk-call category (PR) | Decisions with a trade-off | Big risks found | Big needless risks | Extra hitting rolls: right / wrong and needless | Confidence on the call: ranking score for right risk | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Jev 1.13decision model · API | 59.557.4 to 61.6 | 27.0%190 / 705 | 92.1%710 / 771 | 61.7%190 / 308 | 65.0%458 / 705 | 7.9%61 / 771 | 8.1%57 / 705 | 34.4needless risks 2.10 · missed risks 21.41 · wrong risks 3.52 · right risks 2.12 · right safety 2.66 · No trade-off 2.60 | 1,476Risk pays 705 · Safety pays 771 · No trade-off 435 | 33.0%143 / 433 | 3.8%29 / 771 | Too few decisions / Too few decisions190 / 118 decisions | 0.53308 risks with a probability‡; mean confidence on missed risks 52.2% (458) | 1.020.82 to 1.26 · risks taken 247 / 705 Risk pays · needless 61 / 771 Safety pays | +0.90+0.81 to +1.00 · cautious | 76.9%190 / 247 | unstable: fewer than 200 needless risks behind a mean cost −0.21β 0.80 · needless risks lost 0.131 on average (61) · missed risks 0.179 (458) |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 59.357.1 to 61.5 | 29.1%192 / 659 | 89.5%603 / 674 | 61.9%192 / 310 | 63.7%420 / 659 | 10.5%71 / 674 | 7.1%47 / 659 | 35.6needless risks 2.94 · missed risks 20.07 · wrong risks 3.16 · right risks 3.52 · right safety 3.53 · No trade-off 2.34 | 1,333Risk pays 659 · Safety pays 674 · No trade-off 403 | 38.3%150 / 392 | 5.6%38 / 674 | Too few decisions / Too few decisions192 / 118 decisions | 0.54269 risks with a probability‡; 41 split into groups left out; mean confidence on missed risks 72.5% (378) | 0.900.74 to 1.06 · risks taken 239 / 659 Risk pays · needless 71 / 674 Safety pays | +0.80+0.72 to +0.89 · cautious | 80.3%192 / 239 | unstable: fewer than 200 needless risks behind a mean cost −0.13β 0.89 · needless risks lost 0.144 on average (71) · missed risks 0.166 (420) |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 58.856.7 to 60.8 | 33.0%219 / 664 | 84.6%698 / 825 | 54.8%219 / 400 | 58.9%391 / 664 | 15.4%127 / 825 | 8.1%54 / 664 | 35.0needless risks 5.15 · missed risks 16.67 · wrong risks 2.72 · right risks 1.56 · right safety 5.81 · No trade-off 3.09 | 1,489Risk pays 664 · Safety pays 825 · No trade-off 514 | 38.4%139 / 362 | 9.1%75 / 825 | 12.4 / Too few decisions219 / 181 decisions | 0.60400 risks with a probability‡; mean confidence on missed risks 35.5% (391) | 0.790.67 to 0.93 · risks taken 273 / 664 Risk pays · needless 127 / 825 Safety pays | +0.62+0.54 to +0.71 · cautious | 80.2%219 / 273 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.18 · needless risks lost 0.162 on average (127) · missed risks 0.171 (391) |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 57.956.0 to 59.9 | 33.4%244 / 731 | 82.5%682 / 827 | 52.4%244 / 466 | 56.1%410 / 731 | 17.5%145 / 827 | 10.5%77 / 731 | 36.9needless risks 6.42 · missed risks 16.25 · wrong risks 3.85 · right risks 2.17 · right safety 4.74 · No trade-off 3.51 | 1,558Risk pays 731 · Safety pays 827 · No trade-off 470 | 38.9%162 / 416 | 12.2%101 / 827 | 10.1 / 8.9244 / 222 decisions | 0.55416 risks with a probability‡; 50 split into groups left out; mean confidence on missed risks 42.4% (337) | 0.780.67 to 0.89 · risks taken 321 / 731 Risk pays · needless 145 / 827 Safety pays | +0.54+0.48 to +0.61 · cautious | 76.0%244 / 321 | unstable: fewer than 200 needless risks behind a mean cost +0.30β 1.26 · needless risks lost 0.180 on average (145) · missed risks 0.161 (410) |
| celeris-1-decisiondecision model · API | 57.855.9 to 59.7 | 23.9%163 / 683 | 91.7%749 / 817 | 60.8%163 / 268 | 70.7%483 / 683 | 8.3%68 / 817 | 5.4%37 / 683 | 35.6needless risks 2.30 · missed risks 20.00 · wrong risks 2.07 · right risks 1.37 · right safety 5.50 · No trade-off 4.40 | 1,500Risk pays 683 · Safety pays 817 · No trade-off 422 | 32.1%120 / 374 | 4.7%38 / 817 | Too few decisions / Too few decisions163 / 105 decisions | 0.52268 risks with a probability‡; mean confidence on missed risks 67.9% (483) | 0.840.67 to 1.03 · risks taken 200 / 683 Risk pays · needless 68 / 817 Safety pays | +0.96+0.88 to +1.05 · cautious | 81.5%163 / 200 | unstable: fewer than 200 needless risks behind a mean cost −0.03β 0.97 · needless risks lost 0.130 on average (68) · missed risks 0.159 (483) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 57.656.1 to 59.0 | 23.5%178 / 756 | 91.7%671 / 732 | 63.3%178 / 281 | 70.9%536 / 756 | 8.3%61 / 732 | 5.6%42 / 756 | 45.0needless risks 3.20 · missed risks 26.37 · wrong risks 2.82 · right risks 2.92 · right safety 5.75 · No trade-off 3.98 | 1,488Risk pays 756 · Safety pays 732 · No trade-off 517 | 26.3%124 / 471 | 6.3%46 / 732 | Too few decisions / Too few decisions178 / 103 decisions | 0.50244 risks with a probability‡; 37 split into groups left out; mean confidence on missed risks 70.5% (477) | 0.830.70 to 0.98 · risks taken 220 / 756 Risk pays · needless 61 / 732 Safety pays | +0.96+0.89 to +1.04 · cautious | 80.9%178 / 220 | unstable: fewer than 200 needless risks behind a mean cost +0.04β 1.03 · needless risks lost 0.210 on average (61) · missed risks 0.197 (536) |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 57.155.3 to 58.8 | 27.3%205 / 750 | 86.8%685 / 789 | 55.9%205 / 367 | 64.9%487 / 750 | 13.2%104 / 789 | 7.7%58 / 750 | 38.6needless risks 4.14 · missed risks 19.98 · wrong risks 3.19 · right risks 2.79 · right safety 4.26 · No trade-off 4.20 | 1,539Risk pays 750 · Safety pays 789 · No trade-off 477 | 35.6%159 / 447 | 8.5%67 / 789 | 7.6 / Too few decisions205 / 162 decisions | 0.56339 risks with a probability‡; 28 split into groups left out; mean confidence on missed risks 65.9% (452) | 0.730.60 to 0.88 · risks taken 263 / 750 Risk pays · needless 104 / 789 Safety pays | +0.75+0.67 to +0.83 · cautious | 77.9%205 / 263 | unstable: fewer than 200 needless risks behind a mean cost +0.03β 1.02 · needless risks lost 0.160 on average (104) · missed risks 0.165 (487) |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 56.954.9 to 58.8 | 33.1%233 / 703 | 80.7%682 / 845 | 50.3%233 / 463 | 57.3%403 / 703 | 19.3%163 / 845 | 9.5%67 / 703 | 38.8needless risks 8.32 · missed risks 16.99 · wrong risks 3.72 · right risks 2.11 · right safety 3.90 · No trade-off 3.75 | 1,548Risk pays 703 · Safety pays 845 · No trade-off 504 | 39.9%160 / 401 | 14.2%120 / 845 | 10.1 / 10.1233 / 230 decisions | 0.55400 risks with a probability‡; 63 split into groups left out; mean confidence on missed risks 53.3% (340) | 0.680.56 to 0.81 · risks taken 300 / 703 Risk pays · needless 163 / 845 Safety pays | +0.53+0.46 to +0.59 · cautious | 77.7%233 / 300 | unstable: fewer than 200 needless risks behind a mean cost +0.55β 1.45 · needless risks lost 0.209 on average (163) · missed risks 0.173 (403) |
| SubQ Snap Previewdecision model · API | 56.553.8 to 59.1 | 37.9%297 / 783 | 75.1%673 / 896 | 45.9%297 / 647 | 45.8%359 / 783 | 24.9%223 / 896 | 16.2%127 / 783 | 43.9needless risks 10.40 · missed risks 14.35 · wrong risks 7.22 · right risks 3.21 · right safety 5.11 · No trade-off 3.58 | 1,679Risk pays 783 · Safety pays 896 · No trade-off 521 | 45.9%201 / 438 | 17.4%156 / 896 | 9.6 / 10.9297 / 350 decisions | 0.61647 risks with a probability‡; mean confidence on missed risks 44.3% (358) | 0.780.65 to 0.93 · risks taken 424 / 783 Risk pays · needless 223 / 896 Safety pays | +0.29+0.22 to +0.36 · cautious | 70.0%297 / 424 | +0.37β 1.33 · needless risks lost 0.205 on average (223) · missed risks 0.176 (359) |
| GPT-6 LunaDecisions APIdecision model · API | 55.753.8 to 57.6 | 21.6%160 / 740 | 89.7%636 / 709 | 56.3%160 / 284 | 71.5%529 / 740 | 10.3%73 / 709 | 6.9%51 / 740 | 43.3needless risks 3.67 · missed risks 25.05 · wrong risks 3.45 · right risks 2.58 · right safety 4.50 · No trade-off 4.07 | 1,449Risk pays 740 · Safety pays 709 · No trade-off 513 | 28.6%123 / 430 | 7.5%53 / 709 | Too few decisions / Too few decisions160 / 124 decisions | 0.60284 risks with a probability‡; mean confidence on missed risks 54.6% (529) | 0.690.54 to 0.87 · risks taken 211 / 740 Risk pays · needless 73 / 709 Safety pays | +0.91+0.83 to +1.00 · cautious | 75.8%160 / 211 | unstable: fewer than 200 needless risks behind a mean cost +0.02β 1.02 · needless risks lost 0.197 on average (73) · missed risks 0.186 (529) |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 55.252.9 to 57.2 | 33.7%263 / 780 | 76.6%573 / 748 | 48.7%263 / 540 | 53.2%415 / 780 | 23.4%175 / 748 | 13.1%102 / 780 | 49.7needless risks 10.05 · missed risks 19.93 · wrong risks 6.06 · right risks 3.45 · right safety 5.37 · No trade-off 4.86 | 1,528Risk pays 780 · Safety pays 748 · No trade-off 487 | 39.9%176 / 441 | 16.4%123 / 748 | 9.8 / 10.2263 / 277 decisions | 0.62518 risks with a probability‡; 22 split into groups left out; mean confidence on missed risks 59.7% (390) | 0.640.53 to 0.77 · risks taken 365 / 780 Risk pays · needless 175 / 748 Safety pays | +0.40+0.32 to +0.49 · cautious | 72.1%263 / 365 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.15 · needless risks lost 0.231 on average (175) · missed risks 0.194 (415) |
| Mercury Decidedecision model · API | 53.451.5 to 55.2 | 18.7%129 / 690 | 88.1%583 / 662 | 51.2%129 / 252 | 74.9%517 / 690 | 11.9%79 / 662 | 6.4%44 / 690 | 47.9needless risks 5.59 · missed risks 29.17 · wrong risks 2.52 · right risks 1.67 · right safety 4.44 · No trade-off 4.55 | 1,352Risk pays 690 · Safety pays 662 · No trade-off 501 | 21.8%91 / 417 | 7.7%51 / 662 | Too few decisions / Too few decisions129 / 123 decisions | 0.63252 risks with a probability‡; mean confidence on missed risks 83.3% (517) | 0.500.36 to 0.67 · risks taken 173 / 690 Risk pays · needless 79 / 662 Safety pays | +0.92+0.84 to +1.02 · cautious | 74.6%129 / 173 | unstable: fewer than 200 needless risks behind a mean cost +0.37β 1.20 · needless risks lost 0.262 on average (79) · missed risks 0.209 (517) |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 52.450.3 to 54.5 | 31.5%248 / 788 | 73.4%647 / 882 | 38.2%248 / 650 | 47.3%373 / 788 | 26.6%235 / 882 | 21.2%167 / 788 | 47.7needless risks 13.29 · missed risks 12.21 · wrong risks 10.08 · right risks 2.73 · right safety 4.23 · No trade-off 5.15 | 1,670Risk pays 788 · Safety pays 882 · No trade-off 523 | 44.8%182 / 406 | 19.2%169 / 882 | 10.5 / 12.3248 / 402 decisions | 0.52572 risks with a probability‡; 78 split into groups left out; mean confidence on missed risks 100.0% (309) | 0.690.56 to 0.82 · risks taken 415 / 788 Risk pays · needless 235 / 882 Safety pays | +0.28+0.21 to +0.34 · cautious | 59.8%248 / 415 | +0.96β 1.93 · needless risks lost 0.248 on average (235) · missed risks 0.144 (373) |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 49.146.5 to 51.6 | 31.3%223 / 712 | 66.9%519 / 776 | 33.5%223 / 665 | 42.7%304 / 712 | 33.1%257 / 776 | 26.0%185 / 712 | 59.6needless risks 16.04 · missed risks 13.43 · wrong risks 13.80 · right risks 3.10 · right safety 6.17 · No trade-off 7.11 | 1,488Risk pays 712 · Safety pays 776 · No trade-off 518 | 41.9%167 / 399 | 25.3%196 / 776 | 10.7 / 13.4223 / 442 decisions | 0.56565 risks with a probability‡; 100 split into groups left out; mean confidence on missed risks 32.5% (270) | 0.620.51 to 0.74 · risks taken 408 / 712 Risk pays · needless 257 / 776 Safety pays | +0.13+0.05 to +0.20 · cautious | 54.7%223 / 408 | +0.70β 1.54 · needless risks lost 0.250 on average (257) · missed risks 0.177 (304) |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 48.546.1 to 50.9 | 37.3%264 / 708 | 59.8%455 / 761 | 34.4%264 / 768 | 34.7%246 / 708 | 40.2%306 / 761 | 28.0%198 / 708 | 65.6needless risks 21.42 · missed risks 12.49 · wrong risks 17.29 · right risks 3.05 · right safety 4.95 · No trade-off 6.35 | 1,469Risk pays 708 · Safety pays 761 · No trade-off 533 | 42.8%170 / 397 | 31.0%236 / 761 | 11.2 / 11.9264 / 504 decisions | 0.53768 risks with a probability‡; mean confidence on missed risks 14.9% (246) | 0.640.52 to 0.76 · risks taken 462 / 708 Risk pays · needless 306 / 761 Safety pays | −0.07−0.14 to −0.01 · bold | 57.1%264 / 462 | +0.62β 1.48 · needless risks lost 0.280 on average (306) · missed risks 0.203 (246) |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 45.743.2 to 48.1 | 28.8%206 / 715 | 62.7%470 / 750 | 31.8%206 / 648 | 48.5%347 / 715 | 37.3%280 / 750 | 22.7%162 / 715 | 61.5needless risks 18.70 · missed risks 14.79 · wrong risks 13.15 · right risks 2.81 · right safety 5.11 · No trade-off 6.90 | 1,465Risk pays 715 · Safety pays 750 · No trade-off 487 | 35.6%147 / 413 | 27.5%206 / 750 | 9.9 / 12.8206 / 442 decisions | 0.52573 risks with a probability‡; 75 split into groups left out; mean confidence on missed risks 39.2% (316) | 0.360.23 to 0.49 · risks taken 368 / 715 Risk pays · needless 280 / 750 Safety pays | +0.14+0.06 to +0.22 · cautious | 56.0%206 / 368 | +1.38β 1.64 · needless risks lost 0.261 on average (280) · missed risks 0.166 (347) |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 67.163.8 to 70.3 | 66.6%410 / 616 | 67.6%533 / 788 | 50.9%410 / 805 | 10.7%66 / 616 | 32.4%255 / 788 | 22.7%140 / 616 | 21.4needless risks 8.80 · missed risks 1.22 · wrong risks 5.99 · right risks 2.66 · right safety 1.66 · No trade-off 1.07 | 1,404Risk pays 616 · Safety pays 788 · No trade-off 340 | 82.7%254 / 307 | 18.0%142 / 788 | 11.2 / 13.1410 / 395 decisions | N/A | 1.701.53 to 1.89 · risks taken 550 / 616 Risk pays · needless 255 / 788 Safety pays | −0.39−0.48 to −0.30 · bold | 74.5%410 / 550 | unstable: fewer than 200 missed risks behind a mean cost +0.51β 2.39 · needless risks lost 0.120 on average (255) · missed risks 0.065 (66) |
| Randomprogram · CPU | 24.922.3 to 27.4 | 24.8%112 / 452 | 25.0%121 / 484 | 15.4%112 / 728 | 19.2%87 / 452 | 75.0%363 / 484 | 56.0%253 / 452 | 121.1needless risks 54.22 · missed risks 8.20 · wrong risks 38.00 · right risks 3.36 · right safety 3.88 · No trade-off 13.44 | 936Risk pays 452 · Safety pays 484 · No trade-off 401 | 29.2%78 / 267 | 68.0%329 / 484 | Too few decisions / 16.4112 / 616 decisions | N/A | 0.190.05 to 0.34 · risks taken 365 / 452 Risk pays · needless 363 / 484 Safety pays | −0.77−0.89 to −0.65 · bold | 30.7%112 / 365 | unstable: fewer than 200 missed risks behind a mean cost +2.73β 1.70 · needless risks lost 0.399 on average (363) · missed risks 0.252 (87) |
| Safestprogram · CPU | 50.050.0 to 50.0 | 0.0%0 / 785 | 100.0%749 / 749 | N/A0 / 0 | 100.0%785 / 785 | 0.0%0 / 749 | 0.0%0 / 785 | 51.6needless risks 0.00 · missed risks 34.95 · wrong risks 0.00 · right risks 0.00 · right safety 8.45 · No trade-off 8.25 | 1,534Risk pays 785 · Safety pays 749 · No trade-off 524 | 0.0%0 / 460 | 0.0%0 / 749 | 0 / 00 / 0 decisions | N/A | −0.01−0.05 to 0.02 · risks taken 0 / 785 Risk pays · needless 0 / 749 Safety pays | +3.22+3.20 to +3.23 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.183 (785) |
Hit risk only: being hit. Gammon risk and priming are not in it.
Setups met different positions; risk judgement is not adjusted for how hard they were. Paired comparisons wait for the shared positions.
Big risks found: stake above 0.08. Big needless risks: the chosen play lost more than 0.08.
‡ Risks with a probability: decisions whose plays were split into groups have no single probability and are left out; their count is shown per row. Setups that return no probabilities show N/A. Confidence on the call: ranking score only, no calibration gap (the probability is for the chosen play, not the risk). Definitions
Ability, caution and the SDT reference: A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity. Rates corrected by adding 0.5 to the counts and 1 to the totals; a setup without Risk pays or Safety pays decisions shows N/A. Intervals: 95%, blocks of two seat-swapped games resampled. Definitions
The same at the 0.02 margin (robustness)
Swipe for more columns →
| Model and setting | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 59.056.7 to 61.3 | 30.1%177 / 588 | 87.9%655 / 745 | 57.1%177 / 310 | 62.6%368 / 588 | 12.1%90 / 745 | 7.3%43 / 588 | 0.850.72 to 0.98 · risks taken 220 / 588 Risk pays · needless 90 / 745 Safety pays | +0.74+0.66 to +0.83 · cautious | 80.5%177 / 220 | unstable: fewer than 200 needless risks behind a mean cost −0.20β 0.84 · needless risks lost 0.124 on average (90) · missed risks 0.187 (368) |
| Jev 1.13decision model · API | 58.456.2 to 60.8 | 26.8%167 / 622 | 90.0%769 / 854 | 54.2%167 / 308 | 64.1%399 / 622 | 10.0%85 / 854 | 9.0%56 / 622 | 0.920.72 to 1.13 · risks taken 223 / 622 Risk pays · needless 85 / 854 Safety pays | +0.82+0.74 to +0.91 · cautious | 74.9%167 / 223 | unstable: fewer than 200 needless risks behind a mean cost −0.36β 0.72 · needless risks lost 0.106 on average (85) · missed risks 0.202 (399) |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 57.955.4 to 60.4 | 33.1%192 / 580 | 82.7%752 / 909 | 48.0%192 / 400 | 58.1%337 / 580 | 17.3%157 / 909 | 8.8%51 / 580 | 0.740.59 to 0.89 · risks taken 243 / 580 Risk pays · needless 157 / 909 Safety pays | +0.57+0.49 to +0.66 · cautious | 79.0%192 / 243 | unstable: fewer than 200 needless risks behind a mean cost +0.15β 1.11 · needless risks lost 0.138 on average (157) · missed risks 0.194 (337) |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 57.355.4 to 59.2 | 28.2%192 / 680 | 86.4%742 / 859 | 52.3%192 / 367 | 63.2%430 / 680 | 13.6%117 / 859 | 8.5%58 / 680 | 0.760.63 to 0.89 · risks taken 250 / 680 Risk pays · needless 117 / 859 Safety pays | +0.72+0.64 to +0.80 · cautious | 76.8%192 / 250 | unstable: fewer than 200 needless risks behind a mean cost +0.05β 1.04 · needless risks lost 0.151 on average (117) · missed risks 0.184 (430) |
| celeris-1-decisiondecision model · API | 57.355.4 to 59.1 | 24.1%141 / 586 | 90.5%827 / 914 | 52.6%141 / 268 | 69.1%405 / 586 | 9.5%87 / 914 | 6.8%40 / 586 | 0.810.65 to 0.98 · risks taken 181 / 586 Risk pays · needless 87 / 914 Safety pays | +0.90+0.82 to +0.99 · cautious | 77.9%141 / 181 | unstable: fewer than 200 needless risks behind a mean cost −0.10β 0.92 · needless risks lost 0.107 on average (87) · missed risks 0.181 (405) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 56.755.1 to 58.2 | 22.8%157 / 688 | 90.6%725 / 800 | 55.9%157 / 281 | 70.1%482 / 688 | 9.4%75 / 800 | 7.1%49 / 688 | 0.790.66 to 0.93 · risks taken 206 / 688 Risk pays · needless 75 / 800 Safety pays | +0.92+0.85 to +0.99 · cautious | 76.2%157 / 206 | unstable: fewer than 200 needless risks behind a mean cost −0.03β 0.98 · needless risks lost 0.180 on average (75) · missed risks 0.215 (482) |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 56.754.7 to 58.7 | 33.7%211 / 626 | 79.7%735 / 922 | 45.6%211 / 463 | 55.9%350 / 626 | 20.3%187 / 922 | 10.4%65 / 626 | 0.680.55 to 0.82 · risks taken 276 / 626 Risk pays · needless 187 / 922 Safety pays | +0.49+0.42 to +0.56 · cautious | 76.4%211 / 276 | unstable: fewer than 200 needless risks behind a mean cost +0.55β 1.46 · needless risks lost 0.190 on average (187) · missed risks 0.192 (350) |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 56.654.6 to 58.7 | 33.1%213 / 644 | 80.2%733 / 914 | 45.7%213 / 466 | 55.7%359 / 644 | 19.8%181 / 914 | 11.2%72 / 644 | 0.700.60 to 0.81 · risks taken 285 / 644 Risk pays · needless 181 / 914 Safety pays | +0.50+0.42 to +0.57 · cautious | 74.7%213 / 285 | unstable: fewer than 200 needless risks behind a mean cost +0.30β 1.23 · needless risks lost 0.155 on average (181) · missed risks 0.179 (359) |
| SubQ Snap Previewdecision model · API | 55.953.1 to 58.8 | 38.4%263 / 685 | 73.4%730 / 994 | 40.6%263 / 647 | 44.1%302 / 685 | 26.6%264 / 994 | 17.5%120 / 685 | 0.770.63 to 0.93 · risks taken 383 / 685 Risk pays · needless 264 / 994 Safety pays | +0.24+0.16 to +0.31 · cautious | 68.7%263 / 383 | +0.40β 1.37 · needless risks lost 0.189 on average (264) · missed risks 0.201 (302) |
| GPT-6 LunaDecisions APIdecision model · API | 55.253.2 to 57.1 | 21.7%144 / 663 | 88.7%697 / 786 | 50.7%144 / 284 | 70.6%468 / 663 | 11.3%89 / 786 | 7.7%51 / 663 | 0.670.50 to 0.84 · risks taken 195 / 663 Risk pays · needless 89 / 786 Safety pays | +0.87+0.79 to +0.96 · cautious | 73.8%144 / 195 | unstable: fewer than 200 needless risks behind a mean cost +0.05β 1.03 · needless risks lost 0.179 on average (89) · missed risks 0.205 (468) |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 54.552.7 to 56.2 | 33.6%233 / 693 | 75.3%629 / 835 | 43.1%233 / 540 | 51.8%359 / 693 | 24.7%206 / 835 | 14.6%101 / 693 | 0.640.52 to 0.76 · risks taken 334 / 693 Risk pays · needless 206 / 835 Safety pays | +0.36+0.28 to +0.45 · cautious | 69.8%233 / 334 | +0.24β 1.17 · needless risks lost 0.211 on average (206) · missed risks 0.218 (359) |
| Mercury Decidedecision model · API | 53.151.3 to 55.1 | 18.4%115 / 625 | 87.8%638 / 727 | 45.6%115 / 252 | 73.9%462 / 625 | 12.2%89 / 727 | 7.7%48 / 625 | 0.520.37 to 0.67 · risks taken 163 / 625 Risk pays · needless 89 / 727 Safety pays | +0.90+0.82 to +0.99 · cautious | 70.6%115 / 163 | unstable: fewer than 200 needless risks behind a mean cost +0.32β 1.18 · needless risks lost 0.233 on average (89) · missed risks 0.230 (462) |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 52.450.2 to 54.6 | 32.2%224 / 695 | 72.6%708 / 975 | 34.5%224 / 650 | 44.9%312 / 695 | 27.4%267 / 975 | 22.9%159 / 695 | 0.730.61 to 0.85 · risks taken 383 / 695 Risk pays · needless 267 / 975 Safety pays | +0.24+0.18 to +0.30 · cautious | 58.5%224 / 383 | +1.01β 2.09 · needless risks lost 0.243 on average (267) · missed risks 0.163 (312) |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 48.045.5 to 50.7 | 37.7%236 / 626 | 58.4%492 / 843 | 30.7%236 / 768 | 33.4%209 / 626 | 41.6%351 / 843 | 28.9%181 / 626 | 0.640.50 to 0.78 · risks taken 417 / 626 Risk pays · needless 351 / 843 Safety pays | −0.11−0.18 to −0.04 · bold | 56.6%236 / 417 | +0.72β 1.58 · needless risks lost 0.271 on average (351) · missed risks 0.231 (209) |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 47.745.0 to 50.1 | 30.6%192 / 627 | 64.8%558 / 861 | 28.9%192 / 665 | 42.3%265 / 627 | 35.2%303 / 861 | 27.1%170 / 627 | 0.570.46 to 0.70 · risks taken 362 / 627 Risk pays · needless 303 / 861 Safety pays | +0.09+0.02 to +0.17 · cautious | 53.0%192 / 362 | +0.82β 1.60 · needless risks lost 0.232 on average (303) · missed risks 0.198 (265) |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 44.942.6 to 47.6 | 28.4%181 / 637 | 61.5%509 / 828 | 27.9%181 / 648 | 48.4%308 / 637 | 38.5%319 / 828 | 23.2%148 / 637 | 0.330.20 to 0.46 · risks taken 329 / 637 Risk pays · needless 319 / 828 Safety pays | +0.13+0.05 to +0.21 · cautious | 55.0%181 / 329 | +1.71β 1.77 · needless risks lost 0.249 on average (319) · missed risks 0.183 (308) |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 64.361.1 to 67.5 | 66.1%345 / 522 | 62.6%552 / 882 | 42.9%345 / 805 | 9.0%47 / 522 | 37.4%330 / 882 | 24.9%130 / 522 | 1.661.47 to 1.88 · risks taken 475 / 522 Risk pays · needless 330 / 882 Safety pays | −0.51−0.62 to −0.41 · bold | 72.6%345 / 475 | unstable: fewer than 200 missed risks behind a mean cost +0.47β 2.19 · needless risks lost 0.105 on average (330) · missed risks 0.081 (47) |
| Randomprogram · CPU | 23.721.0 to 26.3 | 22.7%94 / 414 | 24.7%129 / 522 | 12.9%94 / 728 | 19.1%79 / 414 | 75.3%393 / 522 | 58.2%241 / 414 | 0.190.05 to 0.35 · risks taken 335 / 414 Risk pays · needless 393 / 522 Safety pays | −0.78−0.90 to −0.66 · bold | 28.1%94 / 335 | unstable: fewer than 200 missed risks behind a mean cost +2.95β 1.75 · needless risks lost 0.377 on average (393) · missed risks 0.271 (79) |
| Safestprogram · CPU | 50.050.0 to 50.0 | 0.0%0 / 707 | 100.0%827 / 827 | N/A0 / 0 | 100.0%707 / 707 | 0.0%0 / 827 | 0.0%0 / 707 | 0.040.01 to 0.08 · risks taken 0 / 707 Risk pays · needless 0 / 827 Safety pays | +3.21+3.20 to +3.23 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.200 (707) |
Ability and caution by reasoning setting (UNPAIRED)
Swipe for more columns →
| Model | Ability d′, 0.001 | Caution c, 0.001 | Ability d′, 0.02 | Caution c, 0.02 | Median s per decision | Output tokens per decision (mean) |
|---|---|---|---|---|---|---|
| LFM2.5-24B-A2Boff → on · UNPAIRED | 0.36 → 0.69change +0.33 | +0.14 → +0.28change +0.14 | 0.33 → 0.73change +0.40 | +0.13 → +0.24change +0.11 | 0.14 → 324.8+324.7 s | 1 → 21,668 |
UNPAIRED: league games are not the same positions, so a change here is not a paired effect of the setting. Output tokens include reasoning tokens where the provider counts them there.
The Phase filter does not apply to this table: all phases.
League 2
Swipe for more columns →
| Model and setting | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Error rate by risk-call category (PR) | Decisions with a trade-off | Big risks found | Big needless risks | Extra hitting rolls: right / wrong and needless | Confidence on the call: ranking score for right risk | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 69.067.3 to 70.9 | 45.0%338 / 751 | 93.0%891 / 958 | 75.1%338 / 450 | 49.0%368 / 751 | 7.0%67 / 958 | 6.0%45 / 751 | 19.0needless risks 2.27 · missed risks 8.56 · wrong risks 1.93 · right risks 2.41 · right safety 2.89 · No trade-off 0.93 | 1,709Risk pays 751 · Safety pays 958 · No trade-off 479 | 62.8%255 / 406 | 3.4%33 / 958 | 9.1 / Too few decisions338 / 112 decisions | 0.53371 risks with a probability‡; 79 split into groups left out; mean confidence on missed risks 100.0% (304) | 1.501.36 to 1.64 · risks taken 383 / 751 Risk pays · needless 67 / 958 Safety pays | +0.72+0.63 to +0.81 · cautious | 88.3%338 / 383 | unstable: fewer than 200 needless risks behind a mean cost +0.41β 1.86 · needless risks lost 0.148 on average (67) · missed risks 0.102 (368) |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | 68.566.4 to 70.6 | 42.5%321 / 755 | 94.5%872 / 923 | 77.7%321 / 413 | 52.1%393 / 755 | 5.5%51 / 923 | 5.4%41 / 755 | 17.1needless risks 1.48 · missed risks 9.93 · wrong risks 1.57 · right risks 1.68 · right safety 1.63 · No trade-off 0.86 | 1,678Risk pays 755 · Safety pays 923 · No trade-off 544 | 55.4%226 / 408 | 2.8%26 / 923 | 9.2 / Too few decisions321 / 92 decisions | 0.51340 risks with a probability‡; 73 split into groups left out; mean confidence on missed risks 100.0% (337) | 1.541.39 to 1.70 · risks taken 362 / 755 Risk pays · needless 51 / 923 Safety pays | +0.82+0.74 to +0.91 · cautious | 88.7%321 / 362 | unstable: fewer than 200 needless risks behind a mean cost +0.22β 1.40 · needless risks lost 0.129 on average (51) · missed risks 0.112 (393) |
| Claude Sonnet 5.5highreasoning high · one letter · subscription | 68.266.3 to 70.0 | 41.8%308 / 737 | 94.6%841 / 889 | 81.9%308 / 376 | 55.5%409 / 737 | 5.4%48 / 889 | 2.7%20 / 737 | 16.7needless risks 1.12 · missed risks 10.76 · wrong risks 0.64 · right risks 1.70 · right safety 1.58 · No trade-off 0.92 | 1,626Risk pays 737 · Safety pays 889 · No trade-off 538 | 56.2%230 / 409 | 1.9%17 / 889 | 8.6 / Too few decisions308 / 68 decisions | N/A | 1.461.33 to 1.61 · risks taken 328 / 737 Risk pays · needless 48 / 889 Safety pays | +0.87+0.80 to +0.95 · cautious | 93.9%308 / 328 | unstable: fewer than 200 needless risks behind a mean cost +0.05β 1.07 · needless risks lost 0.101 on average (48) · missed risks 0.114 (409) |
| Claude Fable 5.1low | 67.865.6 to 69.9 | 41.0%270 / 658 | 94.6%781 / 826 | 77.4%270 / 349 | 53.8%354 / 658 | 5.4%45 / 826 | 5.2%34 / 658 | 17.9needless risks 1.59 · missed risks 8.82 · wrong risks 1.49 · right risks 1.74 · right safety 2.68 · No trade-off 1.61 | 1,484Risk pays 658 · Safety pays 826 · No trade-off 524 | 58.2%198 / 340 | 3.4%28 / 826 | 9.3 / Too few decisions270 / 79 decisions | N/A | 1.501.33 to 1.70 · risks taken 304 / 658 Risk pays · needless 45 / 826 Safety pays | +0.85+0.76 to +0.94 · cautious | 88.8%270 / 304 | unstable: fewer than 200 needless risks behind a mean cost +0.38β 1.78 · needless risks lost 0.142 on average (45) · missed risks 0.100 (354) |
| Grok 4.7low | 67.766.0 to 69.5 | 42.3%295 / 697 | 93.0%852 / 916 | 71.4%295 / 413 | 49.9%348 / 697 | 7.0%64 / 916 | 7.7%54 / 697 | 19.1needless risks 1.99 · missed risks 10.40 · wrong risks 2.28 · right risks 1.90 · right safety 2.04 · No trade-off 0.51 | 1,613Risk pays 697 · Safety pays 916 · No trade-off 539 | 51.4%201 / 391 | 3.4%31 / 916 | 9.3 / Too few decisions295 / 118 decisions | N/A | 1.481.34 to 1.63 · risks taken 349 / 697 Risk pays · needless 64 / 916 Safety pays | +0.74+0.67 to +0.81 · cautious | 84.5%295 / 349 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.37 · needless risks lost 0.134 on average (64) · missed risks 0.129 (348) |
| Gemini 3.8 Flashhighreasoning high · one letter · subscription | 66.965.2 to 68.5 | 36.1%264 / 731 | 97.6%858 / 879 | 87.4%264 / 302 | 61.6%450 / 731 | 2.4%21 / 879 | 2.3%17 / 731 | 14.9needless risks 0.42 · missed risks 10.77 · wrong risks 0.59 · right risks 1.16 · right safety 1.50 · No trade-off 0.50 | 1,610Risk pays 731 · Safety pays 879 · No trade-off 558 | 52.9%193 / 365 | 0.5%4 / 879 | 9.2 / Too few decisions264 / 38 decisions | N/A | 1.681.49 to 1.88 · risks taken 281 / 731 Risk pays · needless 21 / 879 Safety pays | +1.13+1.05 to +1.23 · cautious | 94.0%264 / 281 | unstable: fewer than 200 needless risks behind a mean cost −0.00β 0.99 · needless risks lost 0.086 on average (21) · missed risks 0.104 (450) |
| Gemini 3.8 Flashlowreasoning low · one letter · subscription | 65.163.5 to 66.8 | 33.8%214 / 633 | 96.5%877 / 909 | 80.8%214 / 265 | 63.2%400 / 633 | 3.5%32 / 909 | 3.0%19 / 633 | 16.3needless risks 0.80 · missed risks 9.90 · wrong risks 0.74 · right risks 0.79 · right safety 2.82 · No trade-off 1.29 | 1,542Risk pays 633 · Safety pays 909 · No trade-off 511 | 49.5%151 / 305 | 1.7%15 / 909 | 9.7 / Too few decisions214 / 51 decisions | N/A | 1.471.33 to 1.64 · risks taken 233 / 633 Risk pays · needless 32 / 909 Safety pays | +1.07+0.99 to +1.17 · cautious | 91.8%214 / 233 | unstable: fewer than 200 needless risks behind a mean cost +0.25β 1.45 · needless risks lost 0.102 on average (32) · missed risks 0.102 (400) |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | 64.562.9 to 66.2 | 31.9%223 / 700 | 97.1%878 / 904 | 82.0%223 / 272 | 64.9%454 / 700 | 2.9%26 / 904 | 3.3%23 / 700 | 19.6needless risks 0.54 · missed risks 12.79 · wrong risks 1.21 · right risks 0.96 · right safety 3.00 · No trade-off 1.08 | 1,604Risk pays 700 · Safety pays 904 · No trade-off 518 | 43.7%164 / 375 | 1.2%11 / 904 | 10.3 / Too few decisions223 / 49 decisions | 0.52225 risks with a probability‡; 47 split into groups left out; mean confidence on missed risks 99.7% (385) | 1.511.36 to 1.69 · risks taken 246 / 700 Risk pays · needless 26 / 904 Safety pays | +1.14+1.05 to +1.23 · cautious | 90.7%223 / 246 | unstable: fewer than 200 needless risks behind a mean cost −0.03β 0.95 · needless risks lost 0.088 on average (26) · missed risks 0.120 (454) |
| d1-ultra | 64.061.7 to 66.3 | 42.0%289 / 688 | 86.0%702 / 816 | 62.7%289 / 461 | 49.6%341 / 688 | 14.0%114 / 816 | 8.4%58 / 688 | 30.3needless risks 3.66 · missed risks 13.25 · wrong risks 3.02 · right risks 2.37 · right safety 4.52 · No trade-off 3.51 | 1,504Risk pays 688 · Safety pays 816 · No trade-off 567 | 48.5%180 / 371 | 7.4%60 / 816 | 9.9 / Too few decisions289 / 172 decisions | 0.64461 risks with a probability‡; mean confidence on missed risks 34.2% (340) | 1.090.92 to 1.26 · risks taken 347 / 688 Risk pays · needless 114 / 816 Safety pays | +0.53+0.46 to +0.61 · cautious | 83.3%289 / 347 | unstable: fewer than 200 needless risks behind a mean cost −0.02β 0.98 · needless risks lost 0.133 on average (114) · missed risks 0.161 (341) |
| Claude Sonnet 5.5lowreasoning low · one letter · subscription | 63.561.6 to 65.4 | 30.9%204 / 661 | 96.2%883 / 918 | 77.6%204 / 263 | 65.5%433 / 661 | 3.8%35 / 918 | 3.6%24 / 661 | 20.1needless risks 1.22 · missed risks 12.96 · wrong risks 0.96 · right risks 1.07 · right safety 2.67 · No trade-off 1.26 | 1,579Risk pays 661 · Safety pays 918 · No trade-off 571 | 41.9%145 / 346 | 2.7%25 / 918 | 8.7 / Too few decisions204 / 59 decisions | N/A | 1.371.19 to 1.58 · risks taken 228 / 661 Risk pays · needless 35 / 918 Safety pays | +1.08+0.99 to +1.19 · cautious | 89.5%204 / 228 | unstable: fewer than 200 needless risks behind a mean cost +0.35β 1.62 · needless risks lost 0.150 on average (35) · missed risks 0.129 (433) |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 62.660.4 to 65.0 | 33.2%194 / 584 | 92.1%732 / 795 | 63.4%194 / 306 | 58.4%341 / 584 | 7.9%63 / 795 | 8.4%49 / 584 | 22.5needless risks 2.06 · missed risks 11.12 · wrong risks 2.12 · right risks 1.55 · right safety 3.07 · No trade-off 2.57 | 1,379Risk pays 584 · Safety pays 795 · No trade-off 468 | 44.7%136 / 304 | 4.9%39 / 795 | Too few decisions / Too few decisions194 / 112 decisions | 0.60261 risks with a probability‡; 45 split into groups left out; mean confidence on missed risks 97.5% (299) | 1.201.04 to 1.36 · risks taken 243 / 584 Risk pays · needless 63 / 795 Safety pays | +0.81+0.72 to +0.90 · cautious | 79.8%194 / 243 | unstable: fewer than 200 needless risks behind a mean cost +0.26β 1.36 · needless risks lost 0.121 on average (63) · missed risks 0.120 (341) |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 62.060.1 to 63.9 | 33.8%242 / 715 | 90.2%735 / 815 | 64.0%242 / 378 | 58.3%417 / 715 | 9.8%80 / 815 | 7.8%56 / 715 | 29.0needless risks 2.89 · missed risks 14.82 · wrong risks 2.55 · right risks 1.94 · right safety 3.41 · No trade-off 3.43 | 1,530Risk pays 715 · Safety pays 815 · No trade-off 578 | 42.2%174 / 412 | 6.0%49 / 815 | 10.0 / Too few decisions242 / 136 decisions | 0.56332 risks with a probability‡; 46 split into groups left out; mean confidence on missed risks 100.0% (355) | 1.080.92 to 1.24 · risks taken 298 / 715 Risk pays · needless 80 / 815 Safety pays | +0.75+0.66 to +0.84 · cautious | 81.2%242 / 298 | unstable: fewer than 200 needless risks behind a mean cost +0.14β 1.16 · needless risks lost 0.153 on average (80) · missed risks 0.150 (417) |
| GLM-5.3lowreasoning low · letter probabilities · API | 61.759.7 to 63.8 | 30.7%205 / 668 | 92.7%726 / 783 | 67.7%205 / 303 | 63.2%422 / 668 | 7.3%57 / 783 | 6.1%41 / 668 | 30.9needless risks 1.87 · missed risks 17.40 · wrong risks 2.14 · right risks 1.78 · right safety 3.99 · No trade-off 3.70 | 1,451Risk pays 668 · Safety pays 783 · No trade-off 538 | 36.8%141 / 383 | 4.5%35 / 783 | 8.8 / Too few decisions205 / 98 decisions | 0.52259 risks with a probability‡; 43 split into groups left out; mean confidence on missed risks 91.3% (373) | 1.120.95 to 1.29 · risks taken 246 / 668 Risk pays · needless 57 / 783 Safety pays | +0.89+0.81 to +0.98 · cautious | 83.3%205 / 246 | unstable: fewer than 200 needless risks behind a mean cost −0.06β 0.93 · needless risks lost 0.130 on average (57) · missed risks 0.164 (422) |
| GLiDEAPI | 60.458.3 to 62.6 | 36.9%249 / 675 | 84.0%756 / 900 | 52.4%249 / 475 | 51.0%344 / 675 | 16.0%144 / 900 | 12.1%82 / 675 | 33.2needless risks 7.18 · missed risks 10.97 · wrong risks 4.79 · right risks 2.03 · right safety 4.97 · No trade-off 3.24 | 1,575Risk pays 675 · Safety pays 900 · No trade-off 584 | 45.7%169 / 370 | 11.9%107 / 900 | 12.2 / 9.9249 / 226 decisions | 0.58475 risks with a probability‡; mean confidence on missed risks 39.7% (343) | 0.970.85 to 1.09 · risks taken 331 / 675 Risk pays · needless 144 / 900 Safety pays | +0.51+0.43 to +0.58 · cautious | 75.2%249 / 331 | unstable: fewer than 200 needless risks behind a mean cost +0.76β 2.08 · needless risks lost 0.215 on average (144) · missed risks 0.138 (344) |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 59.557.2 to 61.7 | 34.4%243 / 706 | 84.6%613 / 725 | 54.5%243 / 446 | 52.7%372 / 706 | 15.4%112 / 725 | 12.9%91 / 706 | 33.6needless risks 4.64 · missed risks 14.39 · wrong risks 3.96 · right risks 3.46 · right safety 2.96 · No trade-off 4.17 | 1,431Risk pays 706 · Safety pays 725 · No trade-off 596 | 42.5%166 / 391 | 9.5%69 / 725 | 7.7 / 9.5243 / 203 decisions | 0.57386 risks with a probability‡; 60 split into groups left out; mean confidence on missed risks 54.5% (328) | 0.950.79 to 1.12 · risks taken 334 / 706 Risk pays · needless 112 / 725 Safety pays | +0.54+0.46 to +0.62 · cautious | 72.8%243 / 334 | unstable: fewer than 200 needless risks behind a mean cost +0.10β 1.10 · needless risks lost 0.168 on average (112) · missed risks 0.157 (372) |
| Jev 1.14 | 59.357.1 to 61.4 | 29.1%184 / 632 | 89.4%657 / 735 | 59.5%184 / 309 | 63.4%401 / 632 | 10.6%78 / 735 | 7.4%47 / 632 | 31.7needless risks 2.84 · missed risks 16.00 · wrong risks 2.91 · right risks 1.72 · right safety 4.37 · No trade-off 3.89 | 1,367Risk pays 632 · Safety pays 735 · No trade-off 545 | 33.2%118 / 355 | 6.1%45 / 735 | Too few decisions / Too few decisions184 / 125 decisions | 0.49309 risks with a probability‡; mean confidence on missed risks 48.2% (401) | 0.900.73 to 1.07 · risks taken 231 / 632 Risk pays · needless 78 / 735 Safety pays | +0.79+0.73 to +0.86 · cautious | 79.7%184 / 231 | unstable: fewer than 200 needless risks behind a mean cost +0.07β 1.06 · needless risks lost 0.139 on average (78) · missed risks 0.153 (401) |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 58.556.6 to 60.5 | 25.4%181 / 713 | 91.7%683 / 745 | 62.8%181 / 288 | 68.3%487 / 713 | 8.3%62 / 745 | 6.3%45 / 713 | 35.2needless risks 2.49 · missed risks 21.43 · wrong risks 2.50 · right risks 1.61 · right safety 3.75 · No trade-off 3.47 | 1,458Risk pays 713 · Safety pays 745 · No trade-off 576 | 33.3%138 / 415 | 5.1%38 / 745 | Too few decisions / Too few decisions181 / 107 decisions | 0.58262 risks with a probability‡; 26 split into groups left out; mean confidence on missed risks 81.0% (437) | 0.900.74 to 1.07 · risks taken 226 / 713 Risk pays · needless 62 / 745 Safety pays | +0.93+0.84 to +1.02 · cautious | 80.1%181 / 226 | unstable: fewer than 200 needless risks behind a mean cost −0.05β 0.96 · needless risks lost 0.164 on average (62) · missed risks 0.179 (487) |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 57.555.4 to 59.8 | 39.5%263 / 666 | 75.6%603 / 798 | 47.0%263 / 560 | 45.2%301 / 666 | 24.4%195 / 798 | 15.3%102 / 666 | 36.5needless risks 8.85 · missed risks 11.85 · wrong risks 5.48 · right risks 2.26 · right safety 4.18 · No trade-off 3.92 | 1,464Risk pays 666 · Safety pays 798 · No trade-off 514 | 47.4%182 / 384 | 15.4%123 / 798 | 9.3 / 10.7263 / 297 decisions | 0.49552 risks with a probability‡; 8 split into groups left out; mean confidence on missed risks 22.6% (297) | 0.810.67 to 0.94 · risks taken 365 / 666 Risk pays · needless 195 / 798 Safety pays | +0.29+0.20 to +0.37 · cautious | 72.1%263 / 365 | unstable: fewer than 200 needless risks behind a mean cost +0.40β 1.38 · needless risks lost 0.180 on average (195) · missed risks 0.156 (301) |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 62.359.5 to 64.8 | 62.7%483 / 770 | 61.9%558 / 902 | 48.2%483 / 1,003 | 14.4%111 / 770 | 38.1%344 / 902 | 22.9%176 / 770 | 27.0needless risks 12.05 · missed risks 2.19 · wrong risks 6.03 · right risks 2.93 · right safety 1.97 · No trade-off 1.80 | 1,672Risk pays 770 · Safety pays 902 · No trade-off 464 | 82.4%327 / 397 | 23.4%211 / 902 | 11.2 / 13.5483 / 520 decisions | N/A | 1.361.22 to 1.50 · risks taken 659 / 770 Risk pays · needless 344 / 902 Safety pays | −0.38−0.46 to −0.30 · bold | 73.3%483 / 659 | unstable: fewer than 200 missed risks behind a mean cost +0.54β 2.08 · needless risks lost 0.150 on average (344) · missed risks 0.084 (111) |
| Random (computed)reference row, never ranked · computed: every legal play equally likely | 27.026.7 to 27.2 | 25.1%3,300.3 / 13,140 | 28.8%4,655.0 / 16,136 | 14.7%3,300.3 / 22,427.9 | 16.7%2,193.1 / 13,140 | 71.2%11,481.0 / 16,136 | 58.2%7,646.6 / 13,140 | 117.9needless risks 57.31 · missed risks 5.40 · wrong risks 40.03 · right risks 3.23 · right safety 3.91 · No trade-off 8.04 | 29,276Risk pays 13,140 · Safety pays 16,136 · No trade-off 10,212 | 31.6%2,248.9 / 7,127 | 63.1%10,184.8 / 16,136 | 15.4 / 17.83,300.3 / 19,127.7 decisions | N/A | 0.410.39 to 0.42 · risks taken 10,946.9 / 13,140 Risk pays · needless 11,481.0 / 16,136 Safety pays | −0.76−0.77 to −0.75 · bold | 30.1%3,300.3 / 10,946.9 | +2.23β 2.49 · needless risks lost 0.394 on average (11,481.0) · missed risks 0.195 (2,193.1) |
| Safest (computed)reference row, never ranked · computed: always the best safe play | 50.050.0 to 50.0 | 0.0%0 / 13,140 | 100.0%16,136 / 16,136 | N/A0 / 0 | 100.0%13,140 / 13,140 | 0.0%0 / 16,136 | 0.0%0 / 13,140 | 25.2needless risks 0.00 · missed risks 25.23 · wrong risks 0.00 · right risks 0.00 · right safety 0.00 · No trade-off 0.00 | 29,276Risk pays 13,140 · Safety pays 16,136 · No trade-off 10,212 | 0.0%0 / 7,127 | 0.0%0 / 16,136 | 0 / 00 / 0 decisions | N/A | 0.050.04 to 0.06 · risks taken 0 / 13,140 Risk pays · needless 0 / 16,136 Safety pays | +3.98+3.98 to +3.98 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.152 (13,140) |
Hit risk only: being hit. Gammon risk and priming are not in it.
Setups met different positions; risk judgement is not adjusted for how hard they were. Paired comparisons wait for the shared positions.
Big risks found: stake above 0.08. Big needless risks: the chosen play lost more than 0.08.
‡ Risks with a probability: decisions whose plays were split into groups have no single probability and are left out; their count is shown per row. Setups that return no probabilities show N/A. Confidence on the call: ranking score only, no calibration gap (the probability is for the chosen play, not the risk). Definitions
Ability, caution and the SDT reference: A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity. Rates corrected by adding 0.5 to the counts and 1 to the totals; a setup without Risk pays or Safety pays decisions shows N/A. Intervals: 95%, blocks of two seat-swapped games resampled. Definitions
The same at the 0.02 margin (robustness)
Swipe for more columns →
| Model and setting | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 70.268.2 to 72.2 | 49.2%315 / 640 | 91.2%975 / 1,069 | 70.0%315 / 450 | 44.4%284 / 640 | 8.8%94 / 1,069 | 6.4%41 / 640 | 1.491.36 to 1.63 · risks taken 356 / 640 Risk pays · needless 94 / 1,069 Safety pays | +0.60+0.53 to +0.68 · cautious | 88.5%315 / 356 | unstable: fewer than 200 needless risks behind a mean cost +0.33β 1.64 · needless risks lost 0.125 on average (94) · missed risks 0.127 (284) |
| Claude Sonnet 5.5highreasoning high · one letter · subscription | 69.067.0 to 71.1 | 44.0%292 / 664 | 94.0%904 / 962 | 77.7%292 / 376 | 52.1%346 / 664 | 6.0%58 / 962 | 3.9%26 / 664 | 1.501.37 to 1.65 · risks taken 318 / 664 Risk pays · needless 58 / 962 Safety pays | +0.80+0.72 to +0.89 · cautious | 91.8%292 / 318 | unstable: fewer than 200 needless risks behind a mean cost −0.05β 0.93 · needless risks lost 0.084 on average (58) · missed risks 0.132 (346) |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | 68.966.5 to 71.3 | 44.8%293 / 654 | 93.0%952 / 1,024 | 70.9%293 / 413 | 47.9%313 / 654 | 7.0%72 / 1,024 | 7.3%48 / 654 | 1.521.36 to 1.70 · risks taken 341 / 654 Risk pays · needless 72 / 1,024 Safety pays | +0.71+0.63 to +0.79 · cautious | 85.9%293 / 341 | unstable: fewer than 200 needless risks behind a mean cost +0.08β 1.13 · needless risks lost 0.097 on average (72) · missed risks 0.135 (313) |
| Claude Fable 5.1low | 68.566.3 to 70.8 | 43.7%248 / 567 | 93.2%855 / 917 | 71.1%248 / 349 | 49.4%280 / 567 | 6.8%62 / 917 | 6.9%39 / 567 | 1.511.35 to 1.68 · risks taken 287 / 567 Risk pays · needless 62 / 917 Safety pays | +0.74+0.65 to +0.83 · cautious | 86.4%248 / 287 | unstable: fewer than 200 needless risks behind a mean cost +0.33β 1.65 · needless risks lost 0.122 on average (62) · missed risks 0.120 (280) |
| Gemini 3.8 Flashhighreasoning high · one letter · subscription | 67.665.5 to 69.6 | 39.2%239 / 610 | 96.1%961 / 1,000 | 79.1%239 / 302 | 56.9%347 / 610 | 3.9%39 / 1,000 | 3.9%24 / 610 | 1.581.40 to 1.79 · risks taken 263 / 610 Risk pays · needless 39 / 1,000 Safety pays | +0.97+0.89 to +1.05 · cautious | 90.9%239 / 263 | unstable: fewer than 200 needless risks behind a mean cost −0.29β 0.63 · needless risks lost 0.049 on average (39) · missed risks 0.128 (347) |
| Grok 4.7low | 67.064.9 to 69.1 | 42.8%263 / 614 | 91.1%910 / 999 | 63.7%263 / 413 | 47.2%290 / 614 | 8.9%89 / 999 | 9.9%61 / 614 | 1.411.26 to 1.57 · risks taken 324 / 614 Risk pays · needless 89 / 999 Safety pays | +0.64+0.57 to +0.71 · cautious | 81.2%263 / 324 | unstable: fewer than 200 needless risks behind a mean cost +0.08β 1.13 · needless risks lost 0.104 on average (89) · missed risks 0.150 (290) |
| Gemini 3.8 Flashlowreasoning low · one letter · subscription | 66.864.9 to 68.8 | 37.6%203 / 540 | 96.0%962 / 1,002 | 76.6%203 / 265 | 58.3%315 / 540 | 4.0%40 / 1,002 | 4.1%22 / 540 | 1.541.38 to 1.71 · risks taken 225 / 540 Risk pays · needless 40 / 1,002 Safety pays | +0.98+0.90 to +1.07 · cautious | 90.2%203 / 225 | unstable: fewer than 200 needless risks behind a mean cost +0.22β 1.40 · needless risks lost 0.093 on average (40) · missed risks 0.124 (315) |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | 65.663.8 to 67.5 | 34.7%213 / 613 | 96.5%956 / 991 | 78.3%213 / 272 | 61.3%376 / 613 | 3.5%35 / 991 | 3.9%24 / 613 | 1.511.35 to 1.68 · risks taken 237 / 613 Risk pays · needless 35 / 991 Safety pays | +1.04+0.96 to +1.14 · cautious | 89.9%213 / 237 | unstable: fewer than 200 needless risks behind a mean cost +0.07β 1.11 · needless risks lost 0.095 on average (35) · missed risks 0.139 (376) |
| Claude Sonnet 5.5lowreasoning low · one letter · subscription | 64.262.1 to 66.2 | 33.1%190 / 574 | 95.2%957 / 1,005 | 72.2%190 / 263 | 62.5%359 / 574 | 4.8%48 / 1,005 | 4.4%25 / 574 | 1.341.18 to 1.53 · risks taken 215 / 574 Risk pays · needless 48 / 1,005 Safety pays | +0.99+0.91 to +1.09 · cautious | 88.4%190 / 215 | unstable: fewer than 200 needless risks behind a mean cost +0.21β 1.33 · needless risks lost 0.115 on average (48) · missed risks 0.151 (359) |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 63.261.0 to 65.6 | 35.5%182 / 513 | 90.9%787 / 866 | 59.5%182 / 306 | 55.8%286 / 513 | 9.1%79 / 866 | 8.8%45 / 513 | 1.191.03 to 1.35 · risks taken 227 / 513 Risk pays · needless 79 / 866 Safety pays | +0.74+0.66 to +0.82 · cautious | 80.2%182 / 227 | unstable: fewer than 200 needless risks behind a mean cost +0.22β 1.30 · needless risks lost 0.106 on average (79) · missed risks 0.138 (286) |
| d1-ultra | 63.060.8 to 65.3 | 41.8%254 / 607 | 84.2%755 / 897 | 55.1%254 / 461 | 47.4%288 / 607 | 15.8%142 / 897 | 10.7%65 / 607 | 1.060.91 to 1.22 · risks taken 319 / 607 Risk pays · needless 142 / 897 Safety pays | +0.47+0.40 to +0.54 · cautious | 79.6%254 / 319 | unstable: fewer than 200 needless risks behind a mean cost −0.02β 0.98 · needless risks lost 0.122 on average (142) · missed risks 0.185 (288) |
| GLM-5.3lowreasoning low · letter probabilities · API | 61.659.5 to 63.9 | 31.8%184 / 579 | 91.4%797 / 872 | 60.7%184 / 303 | 60.6%351 / 579 | 8.6%75 / 872 | 7.6%44 / 579 | 1.090.91 to 1.27 · risks taken 228 / 579 Risk pays · needless 75 / 872 Safety pays | +0.82+0.74 to +0.89 · cautious | 80.7%184 / 228 | unstable: fewer than 200 needless risks behind a mean cost −0.12β 0.88 · needless risks lost 0.113 on average (75) · missed risks 0.193 (351) |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 61.259.2 to 63.2 | 34.4%217 / 630 | 87.9%791 / 900 | 57.4%217 / 378 | 57.3%361 / 630 | 12.1%109 / 900 | 8.3%52 / 630 | 0.980.83 to 1.15 · risks taken 269 / 630 Risk pays · needless 109 / 900 Safety pays | +0.68+0.59 to +0.77 · cautious | 80.7%217 / 269 | unstable: fewer than 200 needless risks behind a mean cost +0.11β 1.11 · needless risks lost 0.131 on average (109) · missed risks 0.169 (361) |
| GLiDEAPI | 60.558.2 to 62.6 | 38.4%228 / 593 | 82.5%810 / 982 | 48.0%228 / 475 | 48.9%290 / 593 | 17.5%172 / 982 | 12.6%75 / 593 | 0.960.86 to 1.07 · risks taken 303 / 593 Risk pays · needless 172 / 982 Safety pays | +0.45+0.38 to +0.53 · cautious | 75.2%228 / 303 | unstable: fewer than 200 needless risks behind a mean cost +0.73β 2.01 · needless risks lost 0.190 on average (172) · missed risks 0.157 (290) |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 59.357.1 to 61.5 | 35.5%222 / 625 | 83.0%669 / 806 | 49.8%222 / 446 | 50.6%316 / 625 | 17.0%137 / 806 | 13.9%87 / 625 | 0.940.79 to 1.08 · risks taken 309 / 625 Risk pays · needless 137 / 806 Safety pays | +0.48+0.41 to +0.56 · cautious | 71.8%222 / 309 | unstable: fewer than 200 needless risks behind a mean cost +0.06β 1.05 · needless risks lost 0.147 on average (137) · missed risks 0.180 (316) |
| Jev 1.14 | 59.257.0 to 61.4 | 29.9%167 / 559 | 88.5%715 / 808 | 54.0%167 / 309 | 61.4%343 / 559 | 11.5%93 / 808 | 8.8%49 / 559 | 0.910.75 to 1.08 · risks taken 216 / 559 Risk pays · needless 93 / 808 Safety pays | +0.74+0.68 to +0.81 · cautious | 77.3%167 / 216 | unstable: fewer than 200 needless risks behind a mean cost −0.00β 1.00 · needless risks lost 0.121 on average (93) · missed risks 0.174 (343) |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 58.856.7 to 60.9 | 26.5%168 / 635 | 91.1%750 / 823 | 58.3%168 / 288 | 66.1%420 / 635 | 8.9%73 / 823 | 7.4%47 / 635 | 0.930.77 to 1.10 · risks taken 215 / 635 Risk pays · needless 73 / 823 Safety pays | +0.88+0.79 to +0.97 · cautious | 78.1%168 / 215 | unstable: fewer than 200 needless risks behind a mean cost −0.09β 0.92 · needless risks lost 0.145 on average (73) · missed risks 0.204 (420) |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 57.655.2 to 60.0 | 40.6%240 / 591 | 74.6%651 / 873 | 42.9%240 / 560 | 42.8%253 / 591 | 25.4%222 / 873 | 16.6%98 / 591 | 0.840.70 to 0.99 · risks taken 338 / 591 Risk pays · needless 222 / 873 Safety pays | +0.24+0.15 to +0.32 · cautious | 71.0%240 / 338 | +0.39β 1.39 · needless risks lost 0.168 on average (222) · missed risks 0.178 (253) |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 61.758.7 to 64.6 | 64.8%420 / 648 | 58.5%599 / 1,024 | 41.9%420 / 1,003 | 10.8%70 / 648 | 41.5%425 / 1,024 | 24.4%158 / 648 | 1.451.31 to 1.61 · risks taken 578 / 648 Risk pays · needless 425 / 1,024 Safety pays | −0.51−0.60 to −0.42 · bold | 72.7%420 / 578 | unstable: fewer than 200 missed risks behind a mean cost +0.36β 1.68 · needless risks lost 0.130 on average (425) · missed risks 0.122 (70) |
| Random (computed)reference row, never ranked · computed: every legal play equally likely | 25.925.6 to 26.1 | 23.7%2,710.4 / 11,456 | 28.1%5,001.1 / 17,820 | 12.1%2,710.4 / 22,427.9 | 16.1%1,847.0 / 11,456 | 71.9%12,818.9 / 17,820 | 60.2%6,898.7 / 11,456 | 0.410.39 to 0.43 · risks taken 9,609.0 / 11,456 Risk pays · needless 12,818.9 / 17,820 Safety pays | −0.79−0.79 to −0.78 · bold | 28.2%2,710.4 / 9,609.0 | +2.43β 2.69 · needless risks lost 0.381 on average (12,818.9) · missed risks 0.220 (1,847.0) |
| Safest (computed)reference row, never ranked · computed: always the best safe play | 50.050.0 to 50.0 | 0.0%0 / 11,456 | 100.0%17,820 / 17,820 | N/A0 / 0 | 100.0%11,456 / 11,456 | 0.0%0 / 17,820 | 0.0%0 / 11,456 | 0.110.10 to 0.11 · risks taken 0 / 11,456 Risk pays · needless 0 / 17,820 Safety pays | +3.98+3.97 to +3.98 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.172 (11,456) |
Ability and caution by reasoning setting (UNPAIRED)
Swipe for more columns →
| Model | Ability d′, 0.001 | Caution c, 0.001 | Ability d′, 0.02 | Caution c, 0.02 | Median s per decision | Output tokens per decision (mean) |
|---|---|---|---|---|---|---|
| Claude Sonnet 5.5low → high · UNPAIRED | 1.37 → 1.46change +0.10 | +1.08 → +0.87change −0.21 | 1.34 → 1.50change +0.15 | +0.99 → +0.80change −0.19 | 1.51 → 6.63+5.1 s | 238 → 852 |
| DeepSeek-V4.1-Flash1017off → low · UNPAIRED | 0.90 → 1.54change +0.64 | +0.93 → +0.82change −0.11 | 0.93 → 1.52change +0.59 | +0.88 → +0.71change −0.17 | 0.84 → 22.3+21.5 s | 1 → 7,173 |
| Gemini 3.8 Flashlow → high · UNPAIRED | 1.47 → 1.68change +0.21 | +1.07 → +1.13change +0.06 | 1.54 → 1.58change +0.05 | +0.98 → +0.97change −0.01 | 2.16 → 9.59+7.4 s | 237 → 5,092 |
| Gemma 4 31Boff → on · UNPAIRED | 1.20 → 1.51change +0.32 | +0.81 → +1.14change +0.33 | 1.19 → 1.51change +0.33 | +0.74 → +1.04change +0.31 | 0.31 → 120.6+120.3 s | 1 → 5,599 |
UNPAIRED: league games are not the same positions, so a change here is not a paired effect of the setting. Output tokens include reasoning tokens where the provider counts them there.
The Phase filter does not apply to this table: all phases.
League 3
Swipe for more columns →
| Model and setting | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Error rate by risk-call category (PR) | Decisions with a trade-off | Big risks found | Big needless risks | Extra hitting rolls: right / wrong and needless | Confidence on the call: ranking score for right risk | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GPT-6 Astralowreasoning low · one letter · subscription | 75.873.1 to 78.5 | 57.2%226 / 395 | 94.4%486 / 515 | 84.0%226 / 269 | 39.2%155 / 395 | 5.6%29 / 515 | 3.5%14 / 395 | 11.8needless risks 0.64 · missed risks 6.63 · wrong risks 0.69 · right risks 1.81 · right safety 1.03 · No trade-off 1.00 | 910Risk pays 395 · Safety pays 515 · No trade-off 341 | 68.9%153 / 222 | 1.6%8 / 515 | 9.7 / Too few decisions226 / 43 decisions | N/A | 1.851.63 to 2.14 · risks taken 240 / 395 Risk pays · needless 29 / 515 Safety pays | +0.65+0.57 to +0.76 · cautious | 94.2%226 / 240 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.21β 0.68 · needless risks lost 0.056 on average (29) · missed risks 0.107 (155) |
| GPT-6 Astramediumreasoning medium · one letter · subscription | 74.771.9 to 77.3 | 54.9%245 / 446 | 94.4%542 / 574 | 84.8%245 / 289 | 42.4%189 / 446 | 5.6%32 / 574 | 2.7%12 / 446 | 11.1needless risks 0.92 · missed risks 6.28 · wrong risks 0.34 · right risks 1.58 · right safety 1.31 · No trade-off 0.67 | 1,020Risk pays 446 · Safety pays 574 · No trade-off 342 | 70.5%170 / 241 | 1.6%9 / 574 | 8.7 / Too few decisions245 / 44 decisions | N/A | 1.781.59 to 2.00 · risks taken 257 / 446 Risk pays · needless 32 / 574 Safety pays | +0.70+0.60 to +0.82 · cautious | 95.3%245 / 257 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.06β 1.11 · needless risks lost 0.078 on average (32) · missed risks 0.091 (189) |
| Gemini 4 Argonhighreasoning high · one letter · subscription | 74.172.0 to 76.1 | 50.8%224 / 441 | 97.4%484 / 497 | 88.5%224 / 253 | 45.6%201 / 441 | 2.6%13 / 497 | 3.6%16 / 441 | 12.6needless risks 0.18 · missed risks 8.05 · wrong risks 0.55 · right risks 1.70 · right safety 1.34 · No trade-off 0.75 | 938Risk pays 441 · Safety pays 497 · No trade-off 394 | 66.4%148 / 223 | 0.4%2 / 497 | 11.0 / Too few decisions224 / 29 decisions | N/A | 2.041.82 to 2.36 · risks taken 240 / 441 Risk pays · needless 13 / 497 Safety pays | +0.91+0.77 to +1.09 · cautious | 93.3%224 / 240 | unstable: fewer than 200 needless risks behind a mean cost −0.47β 0.38 · needless risks lost 0.036 on average (13) · missed risks 0.107 (201) |
| GPT-6.1 Sollow | 72.769.9 to 75.5 | 51.7%198 / 383 | 93.8%602 / 642 | 78.0%198 / 254 | 44.1%169 / 383 | 6.2%40 / 642 | 4.2%16 / 383 | 12.1needless risks 1.00 · missed risks 6.27 · wrong risks 0.83 · right risks 1.62 · right safety 1.78 · No trade-off 0.62 | 1,025Risk pays 383 · Safety pays 642 · No trade-off 319 | 65.0%134 / 206 | 1.6%10 / 642 | Too few decisions / Too few decisions198 / 56 decisions | N/A | 1.681.51 to 1.88 · risks taken 214 / 383 Risk pays · needless 40 / 642 Safety pays | +0.69+0.58 to +0.83 · cautious | 92.5%198 / 214 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.07β 1.13 · needless risks lost 0.067 on average (40) · missed risks 0.100 (169) |
| Claude Fable 5.1highreasoning high · one letter · subscription | 72.369.8 to 74.7 | 50.7%215 / 424 | 94.0%529 / 563 | 77.1%215 / 279 | 42.2%179 / 424 | 6.0%34 / 563 | 7.1%30 / 424 | 14.0needless risks 1.38 · missed risks 6.65 · wrong risks 1.53 · right risks 1.48 · right safety 2.23 · No trade-off 0.68 | 987Risk pays 424 · Safety pays 563 · No trade-off 299 | 65.5%144 / 220 | 2.7%15 / 563 | 9.7 / Too few decisions215 / 64 decisions | N/A | 1.741.56 to 1.96 · risks taken 245 / 424 Risk pays · needless 34 / 563 Safety pays | +0.67+0.55 to +0.81 · cautious | 87.8%215 / 245 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.22β 1.46 · needless risks lost 0.105 on average (34) · missed risks 0.096 (179) |
| GPT-6.1 Solhigh | 72.269.5 to 74.9 | 50.6%212 / 419 | 93.7%568 / 606 | 78.2%212 / 271 | 44.4%186 / 419 | 6.3%38 / 606 | 5.0%21 / 419 | 9.9needless risks 0.89 · missed risks 4.75 · wrong risks 0.90 · right risks 1.27 · right safety 1.53 · No trade-off 0.56 | 1,025Risk pays 419 · Safety pays 606 · No trade-off 364 | 71.2%148 / 208 | 1.7%10 / 606 | 10.8 / Too few decisions212 / 59 decisions | N/A | 1.671.42 to 1.94 · risks taken 233 / 419 Risk pays · needless 38 / 606 Safety pays | +0.69+0.60 to +0.79 · cautious | 91.0%212 / 233 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.17β 1.32 · needless risks lost 0.065 on average (38) · missed risks 0.071 (186) |
| Claude Opus 5.5highreasoning high · one letter · subscription | 70.067.3 to 72.6 | 43.4%199 / 459 | 96.6%590 / 611 | 81.6%199 / 244 | 51.4%236 / 459 | 3.4%21 / 611 | 5.2%24 / 459 | 14.4needless risks 0.62 · missed risks 8.96 · wrong risks 1.23 · right risks 1.26 · right safety 1.55 · No trade-off 0.81 | 1,070Risk pays 459 · Safety pays 611 · No trade-off 407 | 56.8%146 / 257 | 1.5%9 / 611 | Too few decisions / Too few decisions199 / 45 decisions | N/A | 1.771.53 to 2.05 · risks taken 223 / 459 Risk pays · needless 21 / 611 Safety pays | +0.92+0.79 to +1.08 · cautious | 89.2%199 / 223 | unstable: fewer than 200 needless risks behind a mean cost +0.02β 1.04 · needless risks lost 0.088 on average (21) · missed risks 0.112 (236) |
| Gemini 4 Argonlowreasoning low · one letter · subscription | 69.266.2 to 72.3 | 47.8%209 / 437 | 90.7%543 / 599 | 71.6%209 / 292 | 46.0%201 / 437 | 9.3%56 / 599 | 6.2%27 / 437 | 16.5needless risks 2.37 · missed risks 8.03 · wrong risks 1.23 · right risks 1.35 · right safety 2.53 · No trade-off 0.94 | 1,036Risk pays 437 · Safety pays 599 · No trade-off 448 | 59.1%137 / 232 | 4.0%24 / 599 | 10.8 / Too few decisions209 / 83 decisions | N/A | 1.421.22 to 1.64 · risks taken 236 / 437 Risk pays · needless 56 / 599 Safety pays | +0.61+0.52 to +0.70 · cautious | 88.6%209 / 236 | unstable: fewer than 200 needless risks behind a mean cost +0.26β 1.46 · needless risks lost 0.126 on average (56) · missed risks 0.119 (201) |
| Grok 4.7highreasoning high · one letter · subscription | 68.566.1 to 71.1 | 44.0%190 / 432 | 93.0%454 / 488 | 76.6%190 / 248 | 50.5%218 / 432 | 7.0%34 / 488 | 5.6%24 / 432 | 18.8needless risks 1.31 · missed risks 9.66 · wrong risks 2.39 · right risks 3.33 · right safety 1.53 · No trade-off 0.56 | 920Risk pays 432 · Safety pays 488 · No trade-off 417 | 53.8%129 / 240 | 2.7%13 / 488 | Too few decisions / Too few decisions190 / 58 decisions | N/A | 1.461.26 to 1.70 · risks taken 214 / 432 Risk pays · needless 34 / 488 Safety pays | +0.74+0.64 to +0.86 · cautious | 88.8%190 / 214 | unstable: fewer than 200 needless risks behind a mean cost −0.01β 0.99 · needless risks lost 0.103 on average (34) · missed risks 0.118 (218) |
| Claude Opus 5.5lowreasoning low · one letter · subscription | 66.964.7 to 69.2 | 40.1%187 / 466 | 93.8%541 / 577 | 77.3%187 / 242 | 55.8%260 / 466 | 6.2%36 / 577 | 4.1%19 / 466 | 15.4needless risks 1.67 · missed risks 8.98 · wrong risks 1.10 · right risks 1.00 · right safety 2.14 · No trade-off 0.55 | 1,043Risk pays 466 · Safety pays 577 · No trade-off 368 | 53.0%131 / 247 | 3.5%20 / 577 | Too few decisions / Too few decisions187 / 55 decisions | N/A | 1.381.17 to 1.62 · risks taken 206 / 466 Risk pays · needless 36 / 577 Safety pays | +0.84+0.72 to +0.97 · cautious | 90.8%187 / 206 | unstable: fewer than 200 needless risks behind a mean cost +0.37β 1.66 · needless risks lost 0.131 on average (36) · missed risks 0.097 (260) |
| Kimi K3.1 Prooffreasoning off · one letter · API | 65.963.2 to 68.9 | 42.4%153 / 361 | 89.5%416 / 465 | 65.7%153 / 233 | 49.0%177 / 361 | 10.5%49 / 465 | 8.6%31 / 361 | 28.0needless risks 4.62 · missed risks 10.08 · wrong risks 3.16 · right risks 2.22 · right safety 4.08 · No trade-off 3.89 | 826Risk pays 361 · Safety pays 465 · No trade-off 341 | 51.4%109 / 212 | 6.7%31 / 465 | Too few decisions / Too few decisions153 / 80 decisions | N/A | 1.271.11 to 1.45 · risks taken 184 / 361 Risk pays · needless 49 / 465 Safety pays | +0.61+0.51 to +0.72 · cautious | 83.2%153 / 184 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.60β 2.13 · needless risks lost 0.220 on average (49) · missed risks 0.133 (177) |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 62.158.2 to 65.9 | 58.8%288 / 490 | 65.5%415 / 634 | 45.9%288 / 627 | 16.7%82 / 490 | 34.5%219 / 634 | 24.5%120 / 490 | 28.1needless risks 11.03 · missed risks 2.16 · wrong risks 6.12 · right risks 4.03 · right safety 2.52 · No trade-off 2.22 | 1,124Risk pays 490 · Safety pays 634 · No trade-off 355 | 82.8%202 / 244 | 21.5%136 / 634 | 12.5 / 14.5288 / 339 decisions | N/A | 1.361.16 to 1.57 · risks taken 408 / 490 Risk pays · needless 219 / 634 Safety pays | −0.28−0.36 to −0.20 · bold | 70.6%288 / 408 | unstable: fewer than 200 missed risks behind a mean cost +0.67β 2.47 · needless risks lost 0.149 on average (219) · missed risks 0.078 (82) |
| Random (computed)reference row, never ranked · computed: every legal play equally likely | 27.627.1 to 28.0 | 25.1%1,293.3 / 5,153 | 30.1%2,036.5 / 6,771 | 14.4%1,293.3 / 8,984.9 | 17.5%902.5 / 5,153 | 69.9%4,734.5 / 6,771 | 57.4%2,957.2 / 5,153 | 117.6needless risks 59.90 · missed risks 5.67 · wrong risks 36.45 · right risks 3.08 · right safety 4.60 · No trade-off 7.94 | 11,924Risk pays 5,153 · Safety pays 6,771 · No trade-off 4,395 | 31.4%862.9 / 2,752 | 62.8%4,253.1 / 6,771 | 15.1 / 17.71,293.3 / 7,691.6 decisions | N/A | 0.410.39 to 0.44 · risks taken 4,250.5 / 5,153 Risk pays · needless 4,734.5 / 6,771 Safety pays | −0.73−0.74 to −0.71 · bold | 30.4%1,293.3 / 4,250.5 | +2.36β 2.65 · needless risks lost 0.413 on average (4,734.5) · missed risks 0.205 (902.5) |
| Safest (computed)reference row, never ranked · computed: always the best safe play | 50.050.0 to 50.0 | 0.0%0 / 5,153 | 100.0%6,771 / 6,771 | N/A0 / 0 | 100.0%5,153 / 5,153 | 0.0%0 / 6,771 | 0.0%0 / 5,153 | 23.5needless risks 0.00 · missed risks 23.47 · wrong risks 0.00 · right risks 0.00 · right safety 0.00 · No trade-off 0.00 | 11,924Risk pays 5,153 · Safety pays 6,771 · No trade-off 4,395 | 0.0%0 / 2,752 | 0.0%0 / 6,771 | 0 / 00 / 0 decisions | N/A | 0.070.06 to 0.08 · risks taken 0 / 5,153 Risk pays · needless 0 / 6,771 Safety pays | +3.76+3.75 to +3.77 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.149 (5,153) |
Hit risk only: being hit. Gammon risk and priming are not in it.
Setups met different positions; risk judgement is not adjusted for how hard they were. Paired comparisons wait for the shared positions.
Big risks found: stake above 0.08. Big needless risks: the chosen play lost more than 0.08.
‡ Risks with a probability: decisions whose plays were split into groups have no single probability and are left out; their count is shown per row. Setups that return no probabilities show N/A. Confidence on the call: ranking score only, no calibration gap (the probability is for the chosen play, not the risk). Definitions
Ability, caution and the SDT reference: A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity. Rates corrected by adding 0.5 to the counts and 1 to the totals; a setup without Risk pays or Safety pays decisions shows N/A. Intervals: 95%, blocks of two seat-swapped games resampled. Definitions
The same at the 0.02 margin (robustness)
Swipe for more columns →
| Model and setting | Risk judgement | Risks found | Right safety | Good risks | Missed risks | Needless risks | Wrong risks | Ability d′ | Caution c (+ cautious, − bold) | Aim: right ÷ (right + wrong risks) | Cost-derived SDT reference c* |
|---|---|---|---|---|---|---|---|---|---|---|---|
| Gemini 4 Argonhighreasoning high · one letter · subscription | 75.973.4 to 78.1 | 56.4%215 / 381 | 95.3%531 / 557 | 85.0%215 / 253 | 40.4%154 / 381 | 4.7%26 / 557 | 3.1%12 / 381 | 1.911.73 to 2.11 · risks taken 227 / 381 Risk pays · needless 26 / 557 Safety pays | +0.71+0.59 to +0.84 · cautious | 94.7%215 / 227 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.49β 0.40 · needless risks lost 0.036 on average (26) · missed risks 0.132 (154) |
| GPT-6 Astralowreasoning low · one letter · subscription | 74.971.9 to 77.9 | 58.1%209 / 360 | 91.8%505 / 550 | 77.7%209 / 269 | 37.8%136 / 360 | 8.2%45 / 550 | 4.2%15 / 360 | 1.701.52 to 1.88 · risks taken 224 / 360 Risk pays · needless 45 / 550 Safety pays | +0.54+0.47 to +0.62 · cautious | 93.3%209 / 224 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.44β 0.47 · needless risks lost 0.037 on average (45) · missed risks 0.118 (136) |
| GPT-6 Astramediumreasoning medium · one letter · subscription | 74.972.4 to 77.3 | 56.5%225 / 398 | 93.2%580 / 622 | 77.9%225 / 289 | 37.9%151 / 398 | 6.8%42 / 622 | 5.5%22 / 398 | 1.801.62 to 1.99 · risks taken 247 / 398 Risk pays · needless 42 / 622 Safety pays | +0.59+0.50 to +0.70 · cautious | 91.1%225 / 247 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.09β 0.85 · needless risks lost 0.059 on average (42) · missed risks 0.109 (151) |
| GPT-6.1 Sollow | 74.270.9 to 77.4 | 56.2%182 / 324 | 92.2%646 / 701 | 71.7%182 / 254 | 38.6%125 / 324 | 7.8%55 / 701 | 5.2%17 / 324 | 1.701.49 to 1.90 · risks taken 199 / 324 Risk pays · needless 55 / 701 Safety pays | +0.56+0.46 to +0.68 · cautious | 91.5%182 / 199 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.06β 0.90 · needless risks lost 0.054 on average (55) · missed risks 0.129 (125) |
| GPT-6.1 Solhigh | 72.469.7 to 75.1 | 52.4%188 / 359 | 92.5%616 / 666 | 69.4%188 / 271 | 38.4%138 / 359 | 7.5%50 / 666 | 9.2%33 / 359 | 1.731.52 to 1.96 · risks taken 221 / 359 Risk pays · needless 50 / 666 Safety pays | +0.57+0.48 to +0.67 · cautious | 85.1%188 / 221 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.07β 1.12 · needless risks lost 0.054 on average (50) · missed risks 0.090 (138) |
| Claude Fable 5.1highreasoning high · one letter · subscription | 72.169.5 to 74.9 | 52.7%195 / 370 | 91.6%565 / 617 | 69.9%195 / 279 | 38.6%143 / 370 | 8.4%52 / 617 | 8.6%32 / 370 | 1.661.49 to 1.86 · risks taken 227 / 370 Risk pays · needless 52 / 617 Safety pays | +0.54+0.43 to +0.67 · cautious | 85.9%195 / 227 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.06β 1.11 · needless risks lost 0.076 on average (52) · missed risks 0.114 (143) |
| Claude Opus 5.5highreasoning high · one letter · subscription | 70.867.8 to 73.8 | 46.3%188 / 406 | 95.3%633 / 664 | 77.0%188 / 244 | 47.5%193 / 406 | 4.7%31 / 664 | 6.2%25 / 406 | 1.731.49 to 2.01 · risks taken 213 / 406 Risk pays · needless 31 / 664 Safety pays | +0.80+0.68 to +0.94 · cautious | 88.3%188 / 213 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.14β 0.78 · needless risks lost 0.064 on average (31) · missed risks 0.134 (193) |
| Gemini 4 Argonlowreasoning low · one letter · subscription | 68.765.6 to 71.9 | 49.6%188 / 379 | 87.8%577 / 657 | 64.4%188 / 292 | 44.1%167 / 379 | 12.2%80 / 657 | 6.3%24 / 379 | 1.311.13 to 1.50 · risks taken 212 / 379 Risk pays · needless 80 / 657 Safety pays | +0.51+0.42 to +0.60 · cautious | 88.7%188 / 212 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.10β 1.14 · needless risks lost 0.092 on average (80) · missed risks 0.140 (167) |
| Grok 4.7highreasoning high · one letter · subscription | 67.964.9 to 70.9 | 44.6%171 / 383 | 91.1%489 / 537 | 69.0%171 / 248 | 47.8%183 / 383 | 8.9%48 / 537 | 7.6%29 / 383 | 1.401.20 to 1.60 · risks taken 200 / 383 Risk pays · needless 48 / 537 Safety pays | +0.64+0.55 to +0.75 · cautious | 85.5%171 / 200 | unstable: fewer than 200 needless and missed risks behind a mean cost −0.13β 0.83 · needless risks lost 0.082 on average (48) · missed risks 0.137 (183) |
| Claude Opus 5.5lowreasoning low · one letter · subscription | 67.164.5 to 69.5 | 41.9%168 / 401 | 92.4%593 / 642 | 69.4%168 / 242 | 51.9%208 / 401 | 7.6%49 / 642 | 6.2%25 / 401 | 1.381.17 to 1.61 · risks taken 193 / 401 Risk pays · needless 49 / 642 Safety pays | +0.74+0.63 to +0.87 · cautious | 87.0%168 / 193 | unstable: fewer than 200 needless risks behind a mean cost +0.30β 1.51 · needless risks lost 0.108 on average (49) · missed risks 0.115 (208) |
| Kimi K3.1 Prooffreasoning off · one letter · API | 66.763.7 to 69.8 | 44.7%143 / 320 | 88.7%449 / 506 | 61.4%143 / 233 | 45.0%144 / 320 | 11.3%57 / 506 | 10.3%33 / 320 | 1.331.18 to 1.49 · risks taken 176 / 320 Risk pays · needless 57 / 506 Safety pays | +0.54+0.44 to +0.64 · cautious | 81.2%143 / 176 | unstable: fewer than 200 needless and missed risks behind a mean cost +0.55β 2.09 · needless risks lost 0.209 on average (57) · missed risks 0.158 (144) |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 60.056.4 to 63.4 | 58.1%247 / 425 | 61.8%432 / 699 | 39.4%247 / 627 | 15.3%65 / 425 | 38.2%267 / 699 | 26.6%113 / 425 | 1.321.14 to 1.52 · risks taken 360 / 425 Risk pays · needless 267 / 699 Safety pays | −0.36−0.45 to −0.27 · bold | 68.6%247 / 360 | unstable: fewer than 200 missed risks behind a mean cost +0.66β 2.38 · needless risks lost 0.132 on average (267) · missed risks 0.091 (65) |
| Random (computed)reference row, never ranked · computed: every legal play equally likely | 26.425.9 to 26.9 | 23.4%1,054.8 / 4,506 | 29.4%2,178.6 / 7,418 | 11.7%1,054.8 / 8,984.9 | 16.9%760.5 / 4,506 | 70.6%5,239.4 / 7,418 | 59.7%2,690.7 / 4,506 | 0.420.39 to 0.44 · risks taken 3,745.5 / 4,506 Risk pays · needless 5,239.4 / 7,418 Safety pays | −0.75−0.77 to −0.73 · bold | 28.2%1,054.8 / 3,745.5 | +2.51β 2.84 · needless risks lost 0.398 on average (5,239.4) · missed risks 0.231 (760.5) |
| Safest (computed)reference row, never ranked · computed: always the best safe play | 50.050.0 to 50.0 | 0.0%0 / 4,506 | 100.0%7,418 / 7,418 | N/A0 / 0 | 100.0%4,506 / 4,506 | 0.0%0 / 7,418 | 0.0%0 / 4,506 | 0.120.11 to 0.14 · risks taken 0 / 4,506 Risk pays · needless 0 / 7,418 Safety pays | +3.76+3.75 to +3.76 · cautious | N/A0 / 0 | N/A: no needless risks to priceneedless risks lost N/A on average (0) · missed risks 0.169 (4,506) |
Ability and caution by reasoning setting (UNPAIRED)
Swipe for more columns →
| Model | Ability d′, 0.001 | Caution c, 0.001 | Ability d′, 0.02 | Caution c, 0.02 | Median s per decision | Output tokens per decision (mean) |
|---|---|---|---|---|---|---|
| Claude Opus 5.5low → high · UNPAIRED | 1.38 → 1.77change +0.39 | +0.84 → +0.92change +0.09 | 1.38 → 1.73change +0.35 | +0.74 → +0.80change +0.07 | 3.92 → 6.35+2.4 s | 274 → 648 |
| GPT-6 Astralow → medium · UNPAIRED | 1.85 → 1.78change −0.07 | +0.65 → +0.70change +0.04 | 1.70 → 1.80change +0.10 | +0.54 → +0.59change +0.05 | 3.94 → 4.40+0.5 s | 98 → 111 |
| GPT-6.1 Sollow → high · UNPAIRED | 1.68 → 1.67change −0.01 | +0.69 → +0.69change +0.00 | 1.70 → 1.73change +0.03 | +0.56 → +0.57change +0.01 | 4.33 → 13.1+8.8 s | 114 → 514 |
| Gemini 4 Argonlow → high · UNPAIRED | 1.42 → 2.04change +0.62 | +0.61 → +0.91change +0.30 | 1.31 → 1.91change +0.60 | +0.51 → +0.71change +0.21 | 3.82 → 27.5+23.7 s | 27 → 5,417 |
UNPAIRED: league games are not the same positions, so a change here is not a paired effect of the setting. Output tokens include reasoning tokens where the provider counts them there.
The Phase filter does not apply to this table: all phases.
Analysis pending Shared positions: Analysis pending; no setup has answered them yet.
Risk calls per setup: was each risk the right one, and was safe right when it played safe?
Summary columns first; All columns adds good risks, the error rate by risk-call category, stakes, extra hitting rolls, confidence on the call, ability and caution.
estimate: runs on our own machine (named after it), GPU time, at most; how it was counted on hover or tap · † plays split into groups · Not measured: no value recorded. Method
Setups played different positions; league comparisons are not paired.
League 1
Swipe for more columns →
| Model and setting | Error rate (PR), 95% interval | Band on the human scale (approx.) | Engine's best play | Blunder rate | Median / P90 s per decision | $ per 1,000 decisions | Decisions scored | Invalid answers |
|---|---|---|---|---|---|---|---|---|
| Jev 1.13decision model · API | 34.431.0 to 38.0 | Distracted (approx.) | 50.7% | 25.1% | 0.22 / 0.26 | 0.166API price | 1,911 | 0 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 35.031.6 to 39.0 | Distracted (approx.) | 47.1% | 24.6% | 1.41 / 2.75 | 10.301estimate · RTX PRO 6000 Blackwell | 2,003†1 split into groups | 0 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 35.631.7 to 39.7 | Distracted (approx.) | 47.9% | 25.2% | 0.48 / 1.63 | 0.728estimate · RTX PRO 6000 Blackwell + API | 1,736†147 split into groups | 0 |
| celeris-1-decisiondecision model · API | 35.632.0 to 39.4 | Distracted (approx.) | 45.8% | 25.8% | 0.17 / 0.33 | 0.150API price | 1,922 | 0 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 36.933.9 to 40.1 | Distracted (approx.) | 46.6% | 27.9% | 0.94 / 2.73 | 12.227estimate · RTX PRO 6000 Blackwell | 2,028†206 split into groups | 0 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 38.635.4 to 41.9 | Distracted (approx.) | 44.0% | 27.5% | 0.37 / 2.96 | 11.168estimate · RTX PRO 6000 Blackwell | 2,016†123 split into groups | 0 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 38.835.3 to 42.5 | Distracted (approx.) | 47.1% | 26.8% | 1.97 / 7.48 | 6.593estimate · RTX PRO 6000 Blackwell | 2,052†226 split into groups | 0 |
| GPT-6 LunaDecisions APIdecision model · API | 43.339.7 to 46.9 | Distracted (approx.) | 43.2% | 28.9% | 0.23 / 0.35 | 0.335API price | 1,962 | 0 |
| SubQ Snap Previewdecision model · API | 43.939.3 to 48.8 | Distracted (approx.) | 42.3% | 31.8% | 0.24 / 0.32 | 0.193API price | 2,200†1 split into groups | 0 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 45.040.7 to 49.9 | Distracted (approx.) | 43.7% | 30.7% | 1.48 / 2.71 | 4.551estimate · RTX PRO 6000 Blackwell | 2,005†180 split into groups | 0 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 47.744.1 to 51.2 | Distracted (approx.) | 39.6% | 32.4% | 324.8 / 668.6 | 38.716estimate · RTX PRO 6000 Blackwell | 2,193†229 split into groups | 0 |
| Mercury Decidedecision model · API | 47.943.3 to 52.9 | Distracted (approx.) | 42.2% | 30.3% | 0.48 / 0.77 | 0.071API price | 1,853 | 0 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 49.745.9 to 53.6 | Distracted (approx.) | 38.9% | 32.3% | 2.46 / 5.45 | 9.615estimate · RTX PRO 6000 Blackwell | 2,015†92 split into groups | 0 |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 59.654.9 to 64.5 | Distracted (approx.) | 37.4% | 37.5% | 0.56 / 0.96 | 12.267estimate · RTX PRO 6000 Blackwell | 2,006†224 split into groups | 0 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 61.556.3 to 66.8 | Distracted (approx.) | 32.2% | 40.8% | 0.14 / 0.36 | 9.132estimate · RTX PRO 6000 Blackwell | 1,952†182 split into groups | 0 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 65.659.7 to 71.9 | Distracted (approx.) | 36.6% | 38.6% | 1.57 / 3.38 | 7.185estimate · RTX PRO 6000 Blackwell | 2,002 | 0 |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 21.418.8 to 24.3 | Casual (approx.) | 52.0% | 19.0% | 0.01 / 0.01 | Not measured | 1,744 | 0 |
| Randomprogram · CPU | 121.1113.3 to 128.5 | Distracted (approx.) | 16.5% | 61.5% | 0.00 / 0.00 | Not measured | 1,337 | 0 |
| Safestprogram · CPU | 51.647.4 to 56.2 | Distracted (approx.) | 37.3% | 35.1% | 0.00 / 0.00 | Not measured | 2,058 | 0 |
Setups played different positions; league comparisons are not paired.
League 2
Swipe for more columns →
| Model and setting | Error rate (PR), 95% interval | Band on the human scale (approx.) | Engine's best play | Blunder rate | Median / P90 s per decision | $ per 1,000 decisions | Decisions scored | Invalid answers |
|---|---|---|---|---|---|---|---|---|
| Gemini 3.8 Flashhighreasoning high · one letter · subscription | 14.912.9 to 17.0 | Intermediate (approx.) | 63.1% | 11.0% | 9.59 / 44.4 | 22.352API-equivalent (subscription) | 2,168†237 split into groups | 0 |
| Gemini 3.8 Flashlowreasoning low · one letter · subscription | 16.314.5 to 18.2 | Intermediate (approx.) | 59.0% | 13.5% | 2.16 / 6.87 | 4.123API-equivalent (subscription) | 2,053†202 split into groups | 0 |
| Claude Sonnet 5.5highreasoning high · one letter · subscription | 16.714.9 to 18.7 | Intermediate (approx.) | 62.5% | 13.3% | 6.63 / 17.6 | 19.847API-equivalent (subscription) | 2,164†257 split into groups | 0 |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | 17.115.1 to 19.2 | Intermediate (approx.) | 61.8% | 13.4% | 22.3 / 107.6 | 4.869API price | 2,222†258 split into groups | 12 |
| Claude Fable 5.1low | 17.915.9 to 20.1 | Casual (approx.) | 58.7% | 15.1% | 1.54 / 6.34 | 62.613API-equivalent (subscription) | 2,008†232 split into groups | 0 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 19.016.6 to 21.3 | Casual (approx.) | 59.7% | 13.9% | 43.8 / 173.1 | 47.092estimate · RTX PRO 6000 Blackwell + API | 2,188†295 split into groups | 38 |
| Grok 4.7low | 19.116.9 to 21.4 | Casual (approx.) | 60.8% | 15.1% | 24.3 / 110.2 | 32.813API-equivalent (subscription) | 2,152†250 split into groups | 0 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | 19.617.5 to 21.7 | Casual (approx.) | 59.1% | 15.6% | 120.6 / 394.3 | 34.943estimate · RTX PRO 6000 Blackwell | 2,122†252 split into groups | 0 |
| Claude Sonnet 5.5lowreasoning low · one letter · subscription | 20.117.8 to 22.4 | Casual (approx.) | 60.1% | 15.7% | 1.51 / 7.37 | 13.541API-equivalent (subscription) | 2,150†217 split into groups | 0 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 22.519.9 to 25.3 | Casual (approx.) | 54.4% | 18.2% | 0.31 / 4.03 | 8.215estimate · RTX PRO 6000 Blackwell | 1,847†183 split into groups | 0 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 29.026.5 to 31.8 | Beginner (approx.) | 52.8% | 22.4% | 171.1 / 566.7 | 39.023estimate · RTX PRO 6000 Blackwell + API | 2,108†217 split into groups | 0 |
| d1-ultra | 30.327.3 to 33.4 | Distracted (approx.) | 48.6% | 22.4% | 0.23 / 0.41 | 0.145API price | 2,071†1 split into groups | 0 |
| GLM-5.3lowreasoning low · letter probabilities · API | 30.927.8 to 34.0 | Distracted (approx.) | 50.2% | 21.7% | 1.75 / 7.48 | 2.917API price | 1,989†203 split into groups | 0 |
| Jev 1.14 | 31.728.3 to 35.7 | Distracted (approx.) | 46.2% | 24.7% | 0.19 / 0.24 | 0.173API price | 1,912†2 split into groups | 0 |
| GLiDEAPI | 33.230.2 to 36.4 | Distracted (approx.) | 48.4% | 24.1% | 0.48 / 0.79 | 0.153API price | 2,159†2 split into groups | 0 |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 33.630.8 to 36.7 | Distracted (approx.) | 44.6% | 25.3% | 1.70 / 7.47 | 0.284API price | 2,027†171 split into groups | 0 |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 35.231.8 to 38.8 | Distracted (approx.) | 47.7% | 23.6% | 0.84 / 1.39 | 0.244API price | 2,034†169 split into groups | 0 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 36.533.5 to 39.6 | Distracted (approx.) | 44.5% | 27.1% | 1.13 / 5.41 | 5.736estimate · RTX PRO 6000 Blackwell | 1,978†22 split into groups | 0 |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 27.024.1 to 29.8 | Beginner (approx.) | 46.2% | 22.1% | 0.01 / 0.01 | Not measured | 2,136 | 0 |
Setups played different positions; league comparisons are not paired.
League 3
Swipe for more columns →
| Model and setting | Error rate (PR), 95% interval | Band on the human scale (approx.) | Engine's best play | Blunder rate | Median / P90 s per decision | $ per 1,000 decisions | Decisions scored | Invalid answers |
|---|---|---|---|---|---|---|---|---|
| GPT-6.1 Solhigh | 9.98.6 to 11.3 | Advanced (approx.) | 64.4% | 7.8% | 13.1 / 56.5 | 15.573API-equivalent (subscription) | 1,389†242 split into groups | 0 |
| GPT-6 Astramediumreasoning medium · one letter · subscription | 11.19.5 to 12.9 | Advanced (approx.) | 65.2% | 8.6% | 4.40 / 15.1 | 50.268API-equivalent (subscription) | 1,362†182 split into groups | 0 |
| GPT-6 Astralowreasoning low · one letter · subscription | 11.89.5 to 14.2 | Advanced (approx.) | 66.3% | 9.4% | 3.94 / 14.0 | 49.737API-equivalent (subscription) | 1,251†169 split into groups | 0 |
| GPT-6.1 Sollow | 12.110.5 to 13.7 | Advanced (approx.) | 65.5% | 9.4% | 4.33 / 14.3 | 10.228API-equivalent (subscription) | 1,344†212 split into groups | 0 |
| Gemini 4 Argonhighreasoning high · one letter · subscription | 12.610.5 to 14.8 | Intermediate (approx.) | 64.6% | 8.9% | 27.5 / 148.6 | 64.401API-equivalent (subscription) | 1,332†208 split into groups | 36 |
| Claude Fable 5.1highreasoning high · one letter · subscription | 14.011.5 to 16.3 | Intermediate (approx.) | 64.2% | 11.7% | 12.7 / 47.1 | 136.268API-equivalent (subscription) | 1,286†213 split into groups | 0 |
| Claude Opus 5.5highreasoning high · one letter · subscription | 14.412.1 to 17.0 | Intermediate (approx.) | 65.9% | 11.4% | 6.35 / 18.5 | 36.400API-equivalent (subscription) | 1,477†178 split into groups | 0 |
| Claude Opus 5.5lowreasoning low · one letter · subscription | 15.413.1 to 18.0 | Intermediate (approx.) | 61.1% | 13.7% | 3.92 / 10.6 | 29.062API-equivalent (subscription) | 1,411†178 split into groups | 0 |
| Gemini 4 Argonlowreasoning low · one letter · subscription | 16.513.7 to 19.5 | Intermediate (approx.) | 60.3% | 12.4% | 3.82 / 12.2 | 10.204API-equivalent (subscription) | 1,484†204 split into groups | 0 |
| Grok 4.7highreasoning high · one letter · subscription | 18.815.9 to 21.8 | Casual (approx.) | 62.5% | 14.0% | 122.3 / 588.6 | 117.379API-equivalent (subscription) | 1,337†195 split into groups | 0 |
| Kimi K3.1 Prooffreasoning off · one letter · API | 28.024.5 to 31.8 | Beginner (approx.) | 51.2% | 21.7% | 2.11 / 7.23 | 3.982API price | 1,167†131 split into groups | 0 |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | 28.125.0 to 31.4 | Beginner (approx.) | 44.5% | 23.3% | 0.01 / 0.01 | Not measured | 1,479 | 0 |
Setups played different positions; league comparisons are not paired.
Analysis pending Shared positions: Analysis pending; no setup has answered them yet.
How much equity each move gives away, judged by the engine, not by wins.
Summary columns first; All columns adds time, cost, decisions scored and invalid answers.
estimate: runs on our own machine (named after it), GPU time, at most; how it was counted on hover or tap · † plays split into groups · Not measured: no value recorded. Method
Setups played different positions; league comparisons are not paired.
Leagues played different positions; close scores across leagues need not mean a real difference.
Swipe for more columns →
| Model and setting | League | Confidence source | Kept at 0.90 | Blunders among kept, 0.90 | Mean equity lost among kept, 0.90 | Kept at 0.99 | Blunders among kept, 0.99 | Mean equity lost among kept, 0.99 | Error rate in the surest quarter (PR) | Ranking score (AUROC) for the engine's best play | Calibration gap (ECE, points) | Confident and badly wrong | Close choicesThe second most likely play has at least half the probability of the first. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | decision-model probability | 281.4% of 2,002 | 0.0%0 / 28 | Too few decisions | 00.0% of 2,002 | N/A0 / 0 | Too few decisions | 9.8501 decisions | 0.75warning before a blunder 0.72 | 3.5 | 0.0%0 / 2,002 | 55.1%956 / 1,734 |
| d1-ultradecision | League 2 | decision-model probability | 201.0% of 2,070 | 0.0%0 / 20 | Too few decisions | 00.0% of 2,070 | N/A0 / 0 | Too few decisions | 8.6520 decisions | 0.75warning before a blunder 0.70 | 8.9 | 0.0%0 / 2,070 | 61.0%1,262 / 2,070 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 1 | decision-model probability | 19510.7% of 1,826 | 5.1%10 / 195 | Too few decisions | 90.5% of 1,826 | 0.0%0 / 9 | Too few decisions | 10.0457 decisions | 0.74warning before a blunder 0.69 | 9.1 | 0.5%10 / 1,826 | 40.0%654 / 1,633 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | decision-model probability | 402.2% of 1,822 | 2.5%1 / 40 | Too few decisions | 00.0% of 1,822 | N/A0 / 0 | Too few decisions | 10.1456 decisions | 0.73warning before a blunder 0.70 | 2.4 | 0.1%1 / 1,822 | 53.0%847 / 1,599 |
| SubQ Snap Previewdecision | League 1 | decision-model probability | 472.1% of 2,199 | 4.3%2 / 47 | Too few decisions | 10.0% of 2,199 | 0.0%0 / 1 | Too few decisions | 13.9550 decisions | 0.73warning before a blunder 0.71 | 5.4 | 0.1%2 / 2,199 | 48.9%962 / 1,968 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat | League 1 | letter probability | 53133.4% of 1,589 | 11.5%61 / 531 | 0.026 | 945.9% of 1,589 | 1.1%1 / 94 | Too few decisions | 11.1398 decisions | 0.73warning before a blunder 0.66 | 26.7 | 3.8%61 / 1,589 | 11.5%15 / 131, under 200 decisions |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | decision-model probability | 50.3% of 1,782 | 0.0%0 / 5 | Too few decisions | 00.0% of 1,782 | N/A0 / 0 | Too few decisions | 19.6446 decisions | 0.73warning before a blunder 0.72 | 3.0 | 0.0%0 / 1,782 | 68.1%1,110 / 1,629 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision | League 1 | decision-model probability | 00.0% of 2,002 | N/A0 / 0 | Too few decisions | 00.0% of 2,002 | N/A0 / 0 | Too few decisions | 28.9501 decisions | 0.73warning before a blunder 0.72 | 15.5 | 0.0%0 / 2,002 | 98.0%1,740 / 1,776 |
| Jev 1.14decision | League 2 | decision-model probability | 512.7% of 1,910 | 2.0%1 / 51 | Too few decisions | 10.1% of 1,910 | 0.0%0 / 1 | Too few decisions | 12.5492 decisions | 0.72warning before a blunder 0.67 | 6.5 | 0.1%1 / 1,910 | 45.6%231 / 507 |
| DeepSeek-V4.1-Flashoff · API · 1017chat | League 2 | letter probability | 1,03055.2% of 1,865 | 15.3%158 / 1,030 | 0.043 | 59732.0% of 1,865 | 9.5%57 / 597 | 0.024 | 11.5467 decisions | 0.72warning before a blunder 0.66 | 35.3 | 8.5%158 / 1,865 | 11.5%62 / 537 |
| Jev 1.13decision | League 1 | decision-model probability | 985.1% of 1,911 | 8.2%8 / 98 | Too few decisions | 50.3% of 1,911 | 0.0%0 / 5 | Too few decisions | 12.3491 decisions | 0.72warning before a blunder 0.68 | 5.9 | 0.4%8 / 1,911 | 42.3%713 / 1,687 |
| GLiDEAPIdecision | League 2 | decision-model probability | 301.4% of 2,157 | 0.0%0 / 30 | Too few decisions | 00.0% of 2,157 | N/A0 / 0 | Too few decisions | 10.7540 decisions | 0.72warning before a blunder 0.68 | 4.2 | 0.0%0 / 2,157 | 52.1%1,015 / 1,947 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 2 | decision-model probability | 00.0% of 1,956 | N/A0 / 0 | Too few decisions | 00.0% of 1,956 | N/A0 / 0 | Too few decisions | 13.5489 decisions | 0.71warning before a blunder 0.68 | 16.4 | 0.0%0 / 1,956 | 88.4%450 / 509 |
| Qwen3.8 Flashoffchat | League 2 | letter probability | 170.9% of 1,856 | 5.9%1 / 17 | Too few decisions | 00.0% of 1,856 | N/A0 / 0 | Too few decisions | 15.1464 decisions | 0.69warning before a blunder 0.64 | 10.7 | 0.1%1 / 1,856 | 40.3%248 / 615 |
| Mercury Decidedecision | League 1 | decision-model probability | 97452.6% of 1,853 | 24.5%239 / 974 | 0.073 | 27614.9% of 1,853 | 13.4%37 / 276 | 0.041 | 30.0464 decisions | 0.67warning before a blunder 0.61 | 40.1 | 12.9%239 / 1,853 | 16.1%262 / 1,629 |
| GPT-6 LunaDecisions APIdecision | League 1 | decision-model probability | 1075.5% of 1,962 | 8.4%9 / 107 | Too few decisions | 20.1% of 1,962 | 0.0%0 / 2 | Too few decisions | 23.6503 decisions | 0.67warning before a blunder 0.65 | 12.5 | 0.5%9 / 1,962 | 43.7%760 / 1,741 |
| celeris-1-decisiondecision | League 1 | decision-model probability | 38920.2% of 1,922 | 13.4%52 / 389 | 0.034 | 00.0% of 1,922 | N/A0 / 0 | Too few decisions | 19.6507 decisions | 0.67warning before a blunder 0.61 | 21.4 | 2.7%52 / 1,922 | 29.7%523 / 1,761 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision | League 1 | letter probability | 1507.9% of 1,893 | 9.3%14 / 150 | Too few decisions | 00.0% of 1,893 | N/A0 / 0 | Too few decisions | 15.9474 decisions | 0.66warning before a blunder 0.63 | 21.8 | 0.7%14 / 1,893 | 21.7%359 / 1,655 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | letter probability | 1,51090.7% of 1,664 | 15.8%239 / 1,510 | 0.040 | 1,34981.1% of 1,664 | 14.8%200 / 1,349 | 0.036 | 14.3416 decisions | 0.66warning before a blunder 0.60 | 40.0 | 14.4%239 / 1,664 | 2.9%14 / 475 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwellchat | League 1 | letter probability | 482.7% of 1,770 | 14.6%7 / 48 | Too few decisions | 10.1% of 1,770 | 0.0%0 / 1 | Too few decisions | 30.8443 decisions | 0.66warning before a blunder 0.64 | 8.1 | 0.4%7 / 1,770 | 58.6%884 / 1,509 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision | League 1 | decision-model probability | 21711.3% of 1,923 | 18.0%39 / 217 | 0.046 | 201.0% of 1,923 | 0.0%0 / 20 | Too few decisions | 29.6481 decisions | 0.65warning before a blunder 0.61 | 19.2 | 2.0%39 / 1,923 | 40.6%692 / 1,705 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | letter probability | 1,86699.8% of 1,870 | 14.2%265 / 1,866 | 0.036 | 1,85699.3% of 1,870 | 14.2%264 / 1,856 | 0.036 | 14.0468 decisions | 0.64warning before a blunder 0.56 | 37.6 | 14.2%265 / 1,870 | 0.0%0 / 169, under 200 decisions |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 1 | letter probability | 35419.4% of 1,825 | 20.6%73 / 354 | 0.056 | 110.6% of 1,825 | 0.0%0 / 11 | Too few decisions | 30.8457 decisions | 0.64warning before a blunder 0.58 | 24.8 | 4.0%73 / 1,825 | 34.9%109 / 312 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 2 | letter probability | 1,891100.0% of 1,891 | 20.3%384 / 1,891 | 0.053 | 1,88999.9% of 1,891 | 20.3%383 / 1,889 | 0.053 | 22.3473 decisions | 0.57warning before a blunder 0.55 | 44.7 | 20.3%384 / 1,891 | 0.0%0 / 298 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat | League 1 | letter probability | 1,95599.5% of 1,964 | 30.5%596 / 1,955 | 0.090 | 1,94298.9% of 1,964 | 30.4%591 / 1,942 | 0.089 | 39.7752 decisions | 0.56warning before a blunder 0.55 | 58.2 | 30.3%596 / 1,964 | 0.0%0 / 151, under 200 decisions |
| GLM-5.3lowchat | League 2 | letter probability | 1,40178.5% of 1,785 | 18.3%257 / 1,401 | 0.052 | 76642.9% of 1,785 | 21.8%167 / 766 | 0.060 | 31.0447 decisions | 0.52warning before a blunder 0.51 | 39.3 | 14.4%257 / 1,785 | 5.5%29 / 527 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat | League 2 | letter probability | 1,904100.0% of 1,904 | 13.0%248 / 1,904 | 0.037 | 1,90399.9% of 1,904 | 13.0%248 / 1,903 | 0.037 | 16.4476 decisions | 0.49warning before a blunder 0.51 | 37.8 | 13.0%248 / 1,904 | 0.0%0 / 408 |
| DeepSeek-V4.1-Flashlow · API · 1017chat | League 2 | letter probability | 1,965100.0% of 1,965 | 13.0%256 / 1,965 | 0.033 | 1,965100.0% of 1,965 | 13.0%256 / 1,965 | 0.033 | 12.7492 decisions | 0.49warning before a blunder 0.49 | 35.8 | 13.0%256 / 1,965 | 0.0%0 / 180, under 200 decisions |
| Claude Fable 5.1lowchat | League 2 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Sonnet 5.5highchat | League 2 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Sonnet 5.5lowchat | League 2 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 3.8 Flashhighchat | League 2 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 3.8 Flashlowchat | League 2 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Grok 4.7lowchat | League 2 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Fable 5.1highchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Opus 5.5highchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Opus 5.5lowchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6 Astralowchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6 Astramediumchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6.1 Solhighchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6.1 Sollowchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 4 Argonhighchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 4 Argonlowchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Grok 4.7highchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Kimi K3.1 Prooffchat | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 1 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Randomprogram · reference row, never ranked · a random legal play | League 1 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Safestprogram · reference row, never ranked · always plays safe | League 1 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 2 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GNUbg weakprogram · reference row, never ranked · x86-64 build: its noise is shifted, so it overplays gammons | League 3 | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
The cutoff chosen on half of the games and tested on the other half: Method.
Close choices: Probabilities over plays, not sampled reasoning. Click a value for its details; this column cannot be sorted. Method
League 2: complete letter-readout records are mostly few-option positions, a selected subset.
League 1
Swipe for more columns →
| Model and setting | Confidence source | Kept at 0.90 | Blunders among kept, 0.90 | Mean equity lost among kept, 0.90 | Kept at 0.99 | Blunders among kept, 0.99 | Mean equity lost among kept, 0.99 | Error rate in the surest quarter (PR) | Ranking score (AUROC) for the engine's best play | Calibration gap (ECE, points) | Confident and badly wrong | Close choicesThe second most likely play has at least half the probability of the first. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | decision-model probability | 281.4% of 2,002 | 0.0%0 / 28 | Too few decisions | 00.0% of 2,002 | N/A0 / 0 | Too few decisions | 9.8501 decisions | 0.75warning before a blunder 0.72 | 3.5 | 0.0%0 / 2,002 | 55.1%956 / 1,734 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | decision-model probability | 19510.7% of 1,826 | 5.1%10 / 195 | Too few decisions | 90.5% of 1,826 | 0.0%0 / 9 | Too few decisions | 10.0457 decisions | 0.74warning before a blunder 0.69 | 9.1 | 0.5%10 / 1,826 | 40.0%654 / 1,633 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | decision-model probability | 402.2% of 1,822 | 2.5%1 / 40 | Too few decisions | 00.0% of 1,822 | N/A0 / 0 | Too few decisions | 10.1456 decisions | 0.73warning before a blunder 0.70 | 2.4 | 0.1%1 / 1,822 | 53.0%847 / 1,599 |
| SubQ Snap Previewdecision model · API | decision-model probability | 472.1% of 2,199 | 4.3%2 / 47 | Too few decisions | 10.0% of 2,199 | 0.0%0 / 1 | Too few decisions | 13.9550 decisions | 0.73warning before a blunder 0.71 | 5.4 | 0.1%2 / 2,199 | 48.9%962 / 1,968 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | letter probability | 53133.4% of 1,589 | 11.5%61 / 531 | 0.026 | 945.9% of 1,589 | 1.1%1 / 94 | Too few decisions | 11.1398 decisions | 0.73warning before a blunder 0.66 | 26.7 | 3.8%61 / 1,589 | 11.5%15 / 131, under 200 decisions |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | decision-model probability | 50.3% of 1,782 | 0.0%0 / 5 | Too few decisions | 00.0% of 1,782 | N/A0 / 0 | Too few decisions | 19.6446 decisions | 0.73warning before a blunder 0.72 | 3.0 | 0.0%0 / 1,782 | 68.1%1,110 / 1,629 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | decision-model probability | 00.0% of 2,002 | N/A0 / 0 | Too few decisions | 00.0% of 2,002 | N/A0 / 0 | Too few decisions | 28.9501 decisions | 0.73warning before a blunder 0.72 | 15.5 | 0.0%0 / 2,002 | 98.0%1,740 / 1,776 |
| Jev 1.13decision model · API | decision-model probability | 985.1% of 1,911 | 8.2%8 / 98 | Too few decisions | 50.3% of 1,911 | 0.0%0 / 5 | Too few decisions | 12.3491 decisions | 0.72warning before a blunder 0.68 | 5.9 | 0.4%8 / 1,911 | 42.3%713 / 1,687 |
| Mercury Decidedecision model · API | decision-model probability | 97452.6% of 1,853 | 24.5%239 / 974 | 0.073 | 27614.9% of 1,853 | 13.4%37 / 276 | 0.041 | 30.0464 decisions | 0.67warning before a blunder 0.61 | 40.1 | 12.9%239 / 1,853 | 16.1%262 / 1,629 |
| GPT-6 LunaDecisions APIdecision model · API | decision-model probability | 1075.5% of 1,962 | 8.4%9 / 107 | Too few decisions | 20.1% of 1,962 | 0.0%0 / 2 | Too few decisions | 23.6503 decisions | 0.67warning before a blunder 0.65 | 12.5 | 0.5%9 / 1,962 | 43.7%760 / 1,741 |
| celeris-1-decisiondecision model · API | decision-model probability | 38920.2% of 1,922 | 13.4%52 / 389 | 0.034 | 00.0% of 1,922 | N/A0 / 0 | Too few decisions | 19.6507 decisions | 0.67warning before a blunder 0.61 | 21.4 | 2.7%52 / 1,922 | 29.7%523 / 1,761 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | letter probability | 1507.9% of 1,893 | 9.3%14 / 150 | Too few decisions | 00.0% of 1,893 | N/A0 / 0 | Too few decisions | 15.9474 decisions | 0.66warning before a blunder 0.63 | 21.8 | 0.7%14 / 1,893 | 21.7%359 / 1,655 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | letter probability | 482.7% of 1,770 | 14.6%7 / 48 | Too few decisions | 10.1% of 1,770 | 0.0%0 / 1 | Too few decisions | 30.8443 decisions | 0.66warning before a blunder 0.64 | 8.1 | 0.4%7 / 1,770 | 58.6%884 / 1,509 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | decision-model probability | 21711.3% of 1,923 | 18.0%39 / 217 | 0.046 | 201.0% of 1,923 | 0.0%0 / 20 | Too few decisions | 29.6481 decisions | 0.65warning before a blunder 0.61 | 19.2 | 2.0%39 / 1,923 | 40.6%692 / 1,705 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | letter probability | 35419.4% of 1,825 | 20.6%73 / 354 | 0.056 | 110.6% of 1,825 | 0.0%0 / 11 | Too few decisions | 30.8457 decisions | 0.64warning before a blunder 0.58 | 24.8 | 4.0%73 / 1,825 | 34.9%109 / 312 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | letter probability | 1,95599.5% of 1,964 | 30.5%596 / 1,955 | 0.090 | 1,94298.9% of 1,964 | 30.4%591 / 1,942 | 0.089 | 39.7752 decisions | 0.56warning before a blunder 0.55 | 58.2 | 30.3%596 / 1,964 | 0.0%0 / 151, under 200 decisions |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Randomprogram · CPU | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Safestprogram · CPU | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
The cutoff chosen on half of the games and tested on the other half: Method.
Close choices: Probabilities over plays, not sampled reasoning. Click a value for its details; this column cannot be sorted. Method
League 2
Swipe for more columns →
| Model and setting | Confidence source | Kept at 0.90 | Blunders among kept, 0.90 | Mean equity lost among kept, 0.90 | Kept at 0.99 | Blunders among kept, 0.99 | Mean equity lost among kept, 0.99 | Error rate in the surest quarter (PR) | Ranking score (AUROC) for the engine's best play | Calibration gap (ECE, points) | Confident and badly wrong | Close choicesThe second most likely play has at least half the probability of the first. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| d1-ultra | decision-model probability | 201.0% of 2,070 | 0.0%0 / 20 | Too few decisions | 00.0% of 2,070 | N/A0 / 0 | Too few decisions | 8.6520 decisions | 0.75warning before a blunder 0.70 | 8.9 | 0.0%0 / 2,070 | 61.0%1,262 / 2,070 |
| Jev 1.14 | decision-model probability | 512.7% of 1,910 | 2.0%1 / 51 | Too few decisions | 10.1% of 1,910 | 0.0%0 / 1 | Too few decisions | 12.5492 decisions | 0.72warning before a blunder 0.67 | 6.5 | 0.1%1 / 1,910 | 45.6%231 / 507 |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | letter probability | 1,03055.2% of 1,865 | 15.3%158 / 1,030 | 0.043 | 59732.0% of 1,865 | 9.5%57 / 597 | 0.024 | 11.5467 decisions | 0.72warning before a blunder 0.66 | 35.3 | 8.5%158 / 1,865 | 11.5%62 / 537 |
| GLiDEAPI | decision-model probability | 301.4% of 2,157 | 0.0%0 / 30 | Too few decisions | 00.0% of 2,157 | N/A0 / 0 | Too few decisions | 10.7540 decisions | 0.72warning before a blunder 0.68 | 4.2 | 0.0%0 / 2,157 | 52.1%1,015 / 1,947 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | decision-model probability | 00.0% of 1,956 | N/A0 / 0 | Too few decisions | 00.0% of 1,956 | N/A0 / 0 | Too few decisions | 13.5489 decisions | 0.71warning before a blunder 0.68 | 16.4 | 0.0%0 / 1,956 | 88.4%450 / 509 |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | letter probability | 170.9% of 1,856 | 5.9%1 / 17 | Too few decisions | 00.0% of 1,856 | N/A0 / 0 | Too few decisions | 15.1464 decisions | 0.69warning before a blunder 0.64 | 10.7 | 0.1%1 / 1,856 | 40.3%248 / 615 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | letter probability | 1,51090.7% of 1,664 | 15.8%239 / 1,510 | 0.040 | 1,34981.1% of 1,664 | 14.8%200 / 1,349 | 0.036 | 14.3416 decisions | 0.66warning before a blunder 0.60 | 40.0 | 14.4%239 / 1,664 | 2.9%14 / 475 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | letter probability | 1,86699.8% of 1,870 | 14.2%265 / 1,866 | 0.036 | 1,85699.3% of 1,870 | 14.2%264 / 1,856 | 0.036 | 14.0468 decisions | 0.64warning before a blunder 0.56 | 37.6 | 14.2%265 / 1,870 | 0.0%0 / 169, under 200 decisions |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | letter probability | 1,891100.0% of 1,891 | 20.3%384 / 1,891 | 0.053 | 1,88999.9% of 1,891 | 20.3%383 / 1,889 | 0.053 | 22.3473 decisions | 0.57warning before a blunder 0.55 | 44.7 | 20.3%384 / 1,891 | 0.0%0 / 298 |
| GLM-5.3lowreasoning low · letter probabilities · API | letter probability | 1,40178.5% of 1,785 | 18.3%257 / 1,401 | 0.052 | 76642.9% of 1,785 | 21.8%167 / 766 | 0.060 | 31.0447 decisions | 0.52warning before a blunder 0.51 | 39.3 | 14.4%257 / 1,785 | 5.5%29 / 527 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | letter probability | 1,904100.0% of 1,904 | 13.0%248 / 1,904 | 0.037 | 1,90399.9% of 1,904 | 13.0%248 / 1,903 | 0.037 | 16.4476 decisions | 0.49warning before a blunder 0.51 | 37.8 | 13.0%248 / 1,904 | 0.0%0 / 408 |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | letter probability | 1,965100.0% of 1,965 | 13.0%256 / 1,965 | 0.033 | 1,965100.0% of 1,965 | 13.0%256 / 1,965 | 0.033 | 12.7492 decisions | 0.49warning before a blunder 0.49 | 35.8 | 13.0%256 / 1,965 | 0.0%0 / 180, under 200 decisions |
| Claude Fable 5.1low | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Sonnet 5.5highreasoning high · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Sonnet 5.5lowreasoning low · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 3.8 Flashhighreasoning high · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 3.8 Flashlowreasoning low · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Grok 4.7low | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
The cutoff chosen on half of the games and tested on the other half: Method.
Close choices: Probabilities over plays, not sampled reasoning. Click a value for its details; this column cannot be sorted. Method
League 2: complete letter-readout records are mostly few-option positions, a selected subset.
League 3
Swipe for more columns →
| Model and setting | Confidence source | Kept at 0.90 | Blunders among kept, 0.90 | Mean equity lost among kept, 0.90 | Kept at 0.99 | Blunders among kept, 0.99 | Mean equity lost among kept, 0.99 | Error rate in the surest quarter (PR) | Ranking score (AUROC) for the engine's best play | Calibration gap (ECE, points) | Confident and badly wrong | Close choicesThe second most likely play has at least half the probability of the first. |
|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.1highreasoning high · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Opus 5.5highreasoning high · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Claude Opus 5.5lowreasoning low · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6 Astralowreasoning low · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6 Astramediumreasoning medium · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6.1 Solhigh | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GPT-6.1 Sollow | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 4 Argonhighreasoning high · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Gemini 4 Argonlowreasoning low · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Grok 4.7highreasoning high · one letter · subscription | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| Kimi K3.1 Prooffreasoning off · one letter · API | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
| GNUbg weakprogram · CPU · x86-64 build: its noise is shifted, so it overplays gammons | none | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/A | N/Ano full probability distribution |
The cutoff chosen on half of the games and tested on the other half: Method.
Close choices: Probabilities over plays, not sampled reasoning. Click a value for its details; this column cannot be sorted. Method
League 1
Close choices: details per setup
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split). Probabilities over plays, not sampled reasoning.
Jev 1.13: close choices 42.3% (713 / 1,687)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 42.2% (301 / 713) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 63.7%209 / 328 | 42.0%92 / 219 |
| margin 0.02 | 63.9%186 / 291 | 44.9%115 / 256 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 713 | 0.113median 0.057 | 0.093median 0.018 | 0.105median 0.046 |
| Other decisions | 974 | 0.154median 0.095 | 0.052median 0.000 | 0.152median 0.090 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001423 decisions | 56.8%crude 57.4% (120 / 209) | 79.8%crude 79.9% (171 / 214) | −23.0 points95% interval −29.9 to −15.8 | −22.5 points95% interval −29.9 to −14.6 |
| margin 0.02379 decisions | 56.3%crude 57.0% (106 / 186), under 200 decisions | 78.9%crude 78.2% (151 / 193), under 200 decisions | −22.6 points95% interval −31.4 to −13.9 | −21.2 points95% interval −29.3 to −12.6 |
Probabilities over plays, not sampled reasoning. Full breakdown
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell: close choices 55.1% (956 / 1,734)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 43.5% (416 / 956) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 62.6%233 / 372 | 49.6%183 / 369 |
| margin 0.02 | 62.9%205 / 326 | 50.8%211 / 415 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 956 | 0.127median 0.072 | 0.096median 0.029 | 0.117median 0.050 |
| Other decisions | 778 | 0.143median 0.087 | 0.042median 0.000 | 0.138median 0.073 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001382 decisions | 63.7%crude 62.7% (146 / 233) | 76.7%crude 67.8% (101 / 149), under 200 decisions | −12.9 points95% interval −22.4 to −1.4 | −5.1 points95% interval −13.9 to +5.4 |
| margin 0.02332 decisions | 64.8%crude 63.9% (131 / 205) | 76.7%crude 67.7% (86 / 127), under 200 decisions | −11.9 points95% interval −21.3 to +0.9 | −3.8 points95% interval −14.0 to +7.4 |
Probabilities over plays, not sampled reasoning. Full breakdown
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release: close choices 11.5% (15 / 131), under 200 decisions
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 40.0% (6 / 15), under 200 decisions of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 100.0%4 / 4, under 200 decisions | 50.0%2 / 4, under 200 decisions |
| margin 0.02 | 100.0%4 / 4, under 200 decisions | 50.0%2 / 4, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 15 | 0.091median 0.028 | 0.103median 0.037 | 0.036median 0.005 |
| Other decisions | 116 | 0.144median 0.075 | 0.024median 0.000 | 0.130median 0.055 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00117 decisions | 66.7%crude 50.0% (2 / 4), under 200 decisions | 70.8%crude 69.2% (9 / 13), under 200 decisions | −4.2 points95% interval −16.7 to +70.0 | −19.2 points95% interval −46.7 to +43.8 |
| margin 0.0216 decisions | 66.7%crude 50.0% (2 / 4), under 200 decisions | 70.8%crude 66.7% (8 / 12), under 200 decisions | −4.2 points95% interval −16.7 to +70.0 | −16.7 points95% interval −46.7 to +53.8 |
Probabilities over plays, not sampled reasoning. Full breakdown
celeris-1-decision: close choices 29.7% (523 / 1,761)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 33.7% (176 / 523) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 58.8%114 / 194, under 200 decisions | 34.1%62 / 182, under 200 decisions |
| margin 0.02 | 60.5%101 / 167, under 200 decisions | 35.9%75 / 209 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 523 | 0.121median 0.056 | 0.082median 0.022 | 0.100median 0.037 |
| Other decisions | 1,238 | 0.144median 0.090 | 0.065median 0.000 | 0.135median 0.072 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001417 decisions | 59.8%crude 60.5% (69 / 114), under 200 decisions | 82.7%crude 80.9% (245 / 303) | −22.9 points95% interval −34.7 to −11.0 | −20.3 points95% interval −32.9 to −10.0 |
| margin 0.02362 decisions | 57.7%crude 59.4% (60 / 101), under 200 decisions | 81.8%crude 80.1% (209 / 261) | −24.1 points95% interval −37.6 to −10.6 | −20.7 points95% interval −34.1 to −10.6 |
Probabilities over plays, not sampled reasoning. Full breakdown
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell: close choices 53.0% (847 / 1,599)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 43.7% (370 / 847) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 61.7%205 / 332 | 53.1%165 / 311 |
| margin 0.02 | 61.6%178 / 289 | 54.2%192 / 354 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 847 | 0.131median 0.075 | 0.087median 0.029 | 0.112median 0.035 |
| Other decisions | 752 | 0.154median 0.096 | 0.045median 0.000 | 0.155median 0.090 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001373 decisions | 56.1%crude 56.1% (115 / 205) | 67.1%crude 72.0% (121 / 168), under 200 decisions | −11.0 points95% interval −23.1 to −1.2 | −15.9 points95% interval −26.9 to −6.4 |
| margin 0.02328 decisions | 58.2%crude 57.9% (103 / 178), under 200 decisions | 68.0%crude 72.0% (108 / 150), under 200 decisions | −9.8 points95% interval −20.0 to −0.3 | −14.1 points95% interval −23.4 to −4.5 |
Probabilities over plays, not sampled reasoning. Full breakdown
Cygnet: close choices 21.7% (359 / 1,655)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 40.1% (144 / 359) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 59.6%87 / 146, under 200 decisions | 44.2%57 / 129, under 200 decisions |
| margin 0.02 | 60.9%81 / 133, under 200 decisions | 44.4%63 / 142, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 359 | 0.121median 0.054 | 0.098median 0.023 | 0.110median 0.043 |
| Other decisions | 1,296 | 0.137median 0.073 | 0.068median 0.003 | 0.145median 0.081 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001415 decisions | 44.7%crude 44.8% (39 / 87), under 200 decisions | 75.5%crude 76.5% (251 / 328) | −30.8 points95% interval −40.8 to −21.2 | −31.7 points95% interval −39.4 to −23.3 |
| margin 0.02377 decisions | 42.5%crude 42.0% (34 / 81), under 200 decisions | 73.8%crude 75.3% (223 / 296) | −31.3 points95% interval −43.1 to −19.6 | −33.4 points95% interval −43.2 to −23.2 |
Probabilities over plays, not sampled reasoning. Full breakdown
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: close choices 40.0% (654 / 1,633)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 46.2% (302 / 654) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 66.4%170 / 256 | 50.2%132 / 263 |
| margin 0.02 | 66.5%151 / 227 | 51.7%151 / 292 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 654 | 0.134median 0.072 | 0.101median 0.034 | 0.110median 0.049 |
| Other decisions | 979 | 0.143median 0.079 | 0.053median 0.000 | 0.140median 0.065 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001365 decisions | 58.3%crude 57.1% (97 / 170), under 200 decisions | 76.2%crude 75.4% (147 / 195), under 200 decisions | −17.9 points95% interval −27.0 to −5.9 | −18.3 points95% interval −26.9 to −9.7 |
| margin 0.02326 decisions | 57.6%crude 56.3% (85 / 151), under 200 decisions | 76.7%crude 75.4% (132 / 175), under 200 decisions | −19.0 points95% interval −29.0 to −6.3 | −19.1 points95% interval −28.8 to −9.3 |
Probabilities over plays, not sampled reasoning. Full breakdown
GPT-6 LunaDecisions API: close choices 43.7% (760 / 1,741)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 38.7% (294 / 760) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 63.0%184 / 292 | 47.4%110 / 232 |
| margin 0.02 | 64.5%169 / 262 | 47.7%125 / 262 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 760 | 0.107median 0.055 | 0.106median 0.029 | 0.123median 0.049 |
| Other decisions | 981 | 0.146median 0.089 | 0.072median 0.003 | 0.163median 0.100 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001459 decisions | 58.2%crude 58.2% (107 / 184), under 200 decisions | 78.2%crude 80.4% (221 / 275) | −19.9 points95% interval −28.4 to −13.3 | −22.2 points95% interval −28.4 to −16.5 |
| margin 0.02413 decisions | 59.3%crude 59.2% (100 / 169), under 200 decisions | 77.6%crude 79.1% (193 / 244) | −18.3 points95% interval −25.8 to −12.5 | −19.9 points95% interval −25.6 to −13.9 |
Probabilities over plays, not sampled reasoning. Full breakdown
SubQ Snap Preview: close choices 48.9% (962 / 1,968)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 35.1% (338 / 962) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 46.8%176 / 376 | 43.4%162 / 373 |
| margin 0.02 | 45.7%153 / 335 | 44.7%185 / 414 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 962 | 0.131median 0.070 | 0.118median 0.045 | 0.144median 0.059 |
| Other decisions | 1,006 | 0.139median 0.074 | 0.065median 0.000 | 0.147median 0.076 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001364 decisions | 53.8%crude 54.5% (96 / 176), under 200 decisions | 70.2%crude 69.7% (131 / 188), under 200 decisions | −16.4 points95% interval −29.6 to −1.5 | −15.1 points95% interval −25.8 to −3.6 |
| margin 0.02305 decisions | 54.4%crude 54.2% (83 / 153), under 200 decisions | 69.6%crude 67.1% (102 / 152), under 200 decisions | −15.2 points95% interval −29.1 to +3.2 | −12.9 points95% interval −24.6 to +0.9 |
Probabilities over plays, not sampled reasoning. Full breakdown
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: close choices 34.9% (109 / 312)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 42.2% (46 / 109), under 200 decisions of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 72.5%29 / 40, under 200 decisions | 53.1%17 / 32, under 200 decisions |
| margin 0.02 | 71.1%27 / 38, under 200 decisions | 55.9%19 / 34, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 109 | 0.139median 0.041 | 0.160median 0.031 | 0.105median 0.010 |
| Other decisions | 203 | 0.132median 0.086 | 0.068median 0.001 | 0.129median 0.075 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00184 decisions | 64.3%crude 65.5% (19 / 29), under 200 decisions | 82.4%crude 80.0% (44 / 55), under 200 decisions | −18.2 points95% interval −33.8 to +14.3 | −14.5 points95% interval −28.5 to +10.1 |
| margin 0.0278 decisions | 64.0%crude 66.7% (18 / 27), under 200 decisions | 80.3%crude 78.4% (40 / 51), under 200 decisions | −16.3 points95% interval −36.0 to +17.7 | −11.8 points95% interval −28.3 to +10.5 |
Probabilities over plays, not sampled reasoning. Full breakdown
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: close choices 0.0% (0 / 151), under 200 decisions
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: N/A (0 / 0) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | N/A0 / 0 | N/A0 / 0 |
| margin 0.02 | N/A0 / 0 | N/A0 / 0 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 0 | N/A | N/A | N/A |
| Other decisions | 151 | 0.101median 0.023 | 0.037median 0.000 | 0.083median 0.003 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00113 decisions | N/AN/A (0 / 0) | 76.9%76.9% (10 / 13), under 200 decisions | N/A: no close choices in this group | N/A |
| margin 0.0210 decisions | N/AN/A (0 / 0) | 70.0%70.0% (7 / 10), under 200 decisions | N/A: no close choices in this group | N/A |
Probabilities over plays, not sampled reasoning. Full breakdown
Mercury Decide: close choices 16.1% (262 / 1,629)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 29.0% (76 / 262) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 61.0%61 / 100, under 200 decisions | 18.8%15 / 80, under 200 decisions |
| margin 0.02 | 64.4%58 / 90, under 200 decisions | 20.0%18 / 90, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 262 | 0.104median 0.050 | 0.131median 0.044 | 0.138median 0.046 |
| Other decisions | 1,367 | 0.135median 0.072 | 0.092median 0.011 | 0.159median 0.078 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001412 decisions | 55.0%crude 54.1% (33 / 61), under 200 decisions | 75.2%crude 79.2% (278 / 351) | −20.2 points95% interval −31.9 to −8.6 | −25.1 points95% interval −37.0 to −13.1 |
| margin 0.02384 decisions | 54.4%crude 53.4% (31 / 58), under 200 decisions | 74.7%crude 78.8% (257 / 326) | −20.3 points95% interval −32.3 to −8.0 | −25.4 points95% interval −37.1 to −13.6 |
Probabilities over plays, not sampled reasoning. Full breakdown
Winnow-12B: close choices 40.6% (692 / 1,705)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 35.1% (243 / 692) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 50.4%132 / 262 | 47.0%111 / 236 |
| margin 0.02 | 48.9%109 / 223 | 48.7%134 / 275 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 692 | 0.126median 0.059 | 0.111median 0.029 | 0.119median 0.040 |
| Other decisions | 1,013 | 0.152median 0.082 | 0.088median 0.009 | 0.157median 0.077 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001374 decisions | 62.0%crude 62.1% (82 / 132), under 200 decisions | 67.4%crude 66.9% (162 / 242) | −5.4 points95% interval −15.4 to +5.6 | −4.8 points95% interval −14.8 to +5.4 |
| margin 0.02330 decisions | 64.8%crude 65.1% (71 / 109), under 200 decisions | 65.2%crude 66.1% (146 / 221) | −0.4 points95% interval −10.8 to +11.9 | −0.9 points95% interval −10.9 to +9.1 |
Probabilities over plays, not sampled reasoning. Full breakdown
Perplexity Decider Flash v1.2: close choices 68.1% (1,110 / 1,629)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 42.7% (474 / 1,110) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 56.8%245 / 431 | 59.8%229 / 383 |
| margin 0.02 | 56.8%220 / 387 | 59.5%254 / 427 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 1,110 | 0.145median 0.078 | 0.124median 0.045 | 0.152median 0.065 |
| Other decisions | 519 | 0.172median 0.108 | 0.065median 0.000 | 0.167median 0.101 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001344 decisions | 55.7%crude 55.9% (137 / 245) | 75.9%crude 74.7% (74 / 99), under 200 decisions | −20.2 points95% interval −32.7 to −5.3 | −18.8 points95% interval −29.8 to −8.1 |
| margin 0.02301 decisions | 55.2%crude 55.0% (121 / 220) | 76.1%crude 75.3% (61 / 81), under 200 decisions | −20.9 points95% interval −33.0 to −5.9 | −20.3 points95% interval −31.1 to −8.7 |
Probabilities over plays, not sampled reasoning. Full breakdown
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell: close choices 58.6% (884 / 1,509)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 41.3% (365 / 884) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 58.0%195 / 336 | 51.7%170 / 329 |
| margin 0.02 | 57.8%167 / 289 | 52.7%198 / 376 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 884 | 0.146median 0.083 | 0.131median 0.055 | 0.139median 0.067 |
| Other decisions | 625 | 0.156median 0.082 | 0.072median 0.002 | 0.141median 0.065 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001334 decisions | 58.9%crude 59.0% (115 / 195), under 200 decisions | 82.4%crude 80.6% (112 / 139), under 200 decisions | −23.5 points95% interval −33.7 to −13.8 | −21.6 points95% interval −30.4 to −11.8 |
| margin 0.02294 decisions | 58.8%crude 58.7% (98 / 167), under 200 decisions | 82.7%crude 81.1% (103 / 127), under 200 decisions | −23.9 points95% interval −33.5 to −13.9 | −22.4 points95% interval −30.3 to −14.5 |
Probabilities over plays, not sampled reasoning. Full breakdown
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell: close choices 98.0% (1,740 / 1,776)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 37.4% (651 / 1,740) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 44.0%271 / 616 | 56.5%380 / 672 |
| margin 0.02 | 42.4%230 / 543 | 56.5%421 / 745 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 1,740 | 0.147median 0.080 | 0.132median 0.034 | 0.175median 0.086 |
| Other decisions | 36 | 0.147median 0.133 | 0.000median 0.000 | 0.147median 0.133 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001271 decisions | N/A57.2% (155 / 271) | N/AN/A (0 / 0) | N/A: no other decisions in this group | N/A |
| margin 0.02230 decisions | N/A56.5% (130 / 230) | N/AN/A (0 / 0) | N/A: no other decisions in this group | N/A |
Probabilities over plays, not sampled reasoning. Full breakdown
N/A (no full probability distribution): GNUbg weak, Random, Safest.
Which probabilities are complete
Swipe for more columns →
| Model and setting | Complete | Sparse: an option without its own probability | Stored vector not reproduced from the response | No raw response | Plays split into groups | Not audited (after the audit) |
|---|---|---|---|---|---|---|
| celeris-1-decisiondecision model · API | 1,761 | 0 | 0 | 0 | 0 | 161 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,655 | 0 | 0 | 0 | 113 | 248 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 131 | 210 | 0 | 0 | 41 | 1,354 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,734 | 0 | 0 | 0 | 1 | 268 |
| GPT-6 LunaDecisions APIdecision model · API | 1,741 | 0 | 0 | 0 | 0 | 221 |
| Mercury Decidedecision model · API | 1,629 | 0 | 0 | 0 | 0 | 224 |
| Jev 1.13decision model · API | 1,687 | 0 | 0 | 0 | 0 | 224 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 1,509 | 261 | 0 | 0 | 182 | 0 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 151 | 1,770 | 0 | 0 | 229 | 43 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,776 | 0 | 0 | 0 | 0 | 226 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,633 | 0 | 0 | 0 | 202 | 217 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,599 | 0 | 0 | 0 | 187 | 242 |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,629 | 0 | 0 | 0 | 198 | 179 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 312 | 30 | 0 | 0 | 33 | 1,630 |
| Randomprogram · CPU | 0 | 0 | 0 | 0 | 0 | 115 |
| Safestprogram · CPU | 0 | 0 | 0 | 0 | 0 | 197 |
| SubQ Snap Previewdecision model · API | 1,968 | 0 | 0 | 0 | 1 | 231 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,705 | 0 | 0 | 0 | 81 | 229 |
Only Complete decisions enter the re-read, the refit and the distributions below. Sparse: a letter missing from the returned top log-probabilities was filled with 0 and the rest renormalised, so its probability is unknown. Method
Re-read risk call (re-read from the model's probabilities, not played)
Swipe for more columns →
| Model and setting | Complete decisions | Risk judgement: as played → re-read | Change in risk judgement | Error rate (PR): as played → re-read | Change in error rate | Threshold chosen most often | Splits without a fit / test half without a class |
|---|---|---|---|---|---|---|---|
| celeris-1-decisiondecision model · API | 1,7611,372 with a trade-off | 57.9 → 58.6 | +0.895% interval −1.07 to +3.25 · stability −1.2 to +2.5 | 34.9 → 37.1 | +2.295% interval +0.43 to +3.88 · stability +0.2 to +4.7 | 0.30150 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,6551,249 with a trade-off | 56.9 → 56.5 | −0.495% interval −1.68 to +1.30 · stability −1.7 to +0.7 | 37.1 → 38.5 | +1.495% interval −0.44 to +2.62 · stability −0.7 to +3.9 | 0.35100 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,7341,296 with a trade-off | 58.9 → 58.8 | −0.295% interval −2.75 to +2.35 · stability −2.4 to +1.8 | 35.8 → 38.8 | +3.095% interval +0.61 to +5.25 · stability +0.5 to +5.7 | 0.6084 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| GPT-6 LunaDecisions APIdecision model · API | 1,7411,292 with a trade-off | 55.7 → 56.0 | +0.295% interval −1.45 to +1.89 · stability −1.2 to +1.5 | 43.4 → 46.3 | +2.995% interval +1.42 to +4.64 · stability +1.0 to +4.8 | 0.55130 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Mercury Decidedecision model · API | 1,6291,176 with a trade-off | 53.3 → 53.8 | +0.595% interval −0.62 to +1.81 · stability −1.6 to +1.8 | 49.4 → 50.4 | +1.095% interval −0.63 to +3.19 · stability −0.7 to +4.0 | 0.25163 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Jev 1.13decision model · API | 1,6871,307 with a trade-off | 59.6 → 63.1 | +3.595% interval +1.10 to +5.51 · stability +1.7 to +5.1 | 34.4 → 34.1 | −0.395% interval −2.21 to +1.24 · stability −2.3 to +1.4 | 0.45130 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 1,5091,084 with a trade-off | 45.7 → 50.7 | +5.095% interval +2.04 to +7.74 · stability +2.5 to +7.3 | 53.2 → 42.7 | −10.595% interval −15.19 to −5.26 · stability −15.1 to −4.1 | 0.90182 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,7761,311 with a trade-off | 48.8 → 53.8 | +5.095% interval +2.65 to +7.10 · stability +2.7 to +7.0 | 64.8 → 55.3 | −9.595% interval −12.92 to −6.96 · stability −12.2 to −6.8 | 0.90298 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,6331,202 with a trade-off | 56.5 → 56.7 | +0.295% interval −1.03 to +1.95 · stability −1.6 to +1.7 | 35.9 → 37.5 | +1.595% interval −1.56 to +3.48 · stability −1.7 to +6.8 | 0.55114 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,5991,210 with a trade-off | 58.4 → 59.7 | +1.395% interval −0.73 to +3.27 · stability −0.3 to +2.9 | 33.8 → 35.9 | +2.195% interval +0.17 to +4.56 · stability +0.4 to +4.1 | 0.65210 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,6291,184 with a trade-off | 49.7 → 52.0 | +2.395% interval −0.48 to +4.34 · stability +0.3 to +4.2 | 52.4 → 48.2 | −4.295% interval −7.73 to +0.73 · stability −8.7 to +2.2 | 0.90173 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 312222 with a trade-off | 55.7 → 57.3 | +1.695% interval −2.85 to +4.98 · stability −1.9 to +5.9 | 50.4 → 49.1 | −1.395% interval −4.83 to +4.40 · stability −7.7 to +3.4 | 0.30139 of 300 splits · always safe 0 · always risky 0 | 0 / 1 |
| SubQ Snap Previewdecision model · API | 1,9681,517 with a trade-off | 56.2 → 57.4 | +1.295% interval −0.33 to +2.94 · stability −0.2 to +2.7 | 45.7 → 44.2 | −1.595% interval −3.93 to +0.92 · stability −4.4 to +2.3 | 0.80159 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,7051,275 with a trade-off | 55.2 → 57.0 | +1.995% interval +0.16 to +3.22 · stability +0.2 to +3.4 | 48.8 → 46.6 | −2.295% interval −3.52 to +0.72 · stability −4.0 to −0.2 | 0.70147 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
Too few decisions: DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release: 131 complete decisions. LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: 151 complete decisions.
Re-read from the model's probabilities, not played: the recorded positions scored again with the play the setup's own probabilities point to; not a league result, and no new win rates. Same held-out decisions for both columns. Stability: the 5th to 95th percentile over the splits, not an interval; 95% interval: blocks of two seat-swapped games resampled, the whole procedure repeated. Method
Calibration as shipped and after a refit
Chosen play: the probability of the chosen play against "the chosen play is the engine's best"
Swipe for more columns →
| Model and setting | Decisions (before and after) | Brier score as shipped | Brier score after the refit | Reliability: as shipped (solid), after (dashed) |
|---|---|---|---|---|
| celeris-1-decisiondecision model · API | 1,761of 1,761; 0 folds without a fit | 0.281 | 0.229 | |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,655of 1,655; 0 folds without a fit | 0.279 | 0.230 | |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 131of 131; 0 folds without a fit | Too few decisions | Too few decisions | |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,734of 1,734; 0 folds without a fit | 0.200 | 0.199 | |
| GPT-6 LunaDecisions APIdecision model · API | 1,741of 1,741; 0 folds without a fit | 0.246 | 0.228 | |
| Mercury Decidedecision model · API | 1,629of 1,629; 0 folds without a fit | 0.396 | 0.221 | |
| Jev 1.13decision model · API | 1,687of 1,687; 0 folds without a fit | 0.215 | 0.213 | |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 1,509of 1,509; 0 folds without a fit | 0.224 | 0.213 | |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 151of 151; 0 folds without a fit | Too few decisions | Too few decisions | |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,776of 1,776; 0 folds without a fit | 0.221 | 0.198 | |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,633of 1,633; 0 folds without a fit | 0.215 | 0.206 | |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,599of 1,599; 0 folds without a fit | 0.211 | 0.211 | |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,629of 1,629; 0 folds without a fit | 0.206 | 0.206 | |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 312of 312; 0 folds without a fit | 0.306 | 0.235 | |
| SubQ Snap Previewdecision model · API | 1,968of 1,968; 0 folds without a fit | 0.210 | 0.207 | |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,705of 1,705; 0 folds without a fit | 0.264 | 0.220 |
Risk class: P(risk), the probability mass on risky plays, against "the position is Risk pays"
Swipe for more columns →
| Model and setting | Decisions (before and after) | Brier score as shipped | Brier score after the refit | Reliability: as shipped (solid), after (dashed) |
|---|---|---|---|---|
| celeris-1-decisiondecision model · API | 1,372of 1,372; 0 folds without a fit | 0.253 | 0.213 | |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,249of 1,249; 0 folds without a fit | 0.256 | 0.222 | |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 73of 73; 0 folds without a fit | Too few decisions | Too few decisions | |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,296of 1,296; 0 folds without a fit | 0.204 | 0.199 | |
| GPT-6 LunaDecisions APIdecision model · API | 1,292of 1,292; 0 folds without a fit | 0.246 | 0.222 | |
| Mercury Decidedecision model · API | 1,176of 1,176; 0 folds without a fit | 0.377 | 0.231 | |
| Jev 1.13decision model · API | 1,307of 1,307; 0 folds without a fit | 0.203 | 0.195 | |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 1,084of 1,084; 0 folds without a fit | 0.260 | 0.230 | |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 57of 57; 0 folds without a fit | Too few decisions | Too few decisions | |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,311of 1,311; 0 folds without a fit | 0.310 | 0.217 | |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 1,202of 1,202; 0 folds without a fit | 0.220 | 0.211 | |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,210of 1,210; 0 folds without a fit | 0.208 | 0.205 | |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,184of 1,184; 0 folds without a fit | 0.240 | 0.216 | |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 222of 222; 0 folds without a fit | 0.275 | 0.229 | |
| SubQ Snap Previewdecision model · API | 1,517of 1,517; 0 folds without a fit | 0.222 | 0.209 | |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 1,275of 1,275; 0 folds without a fit | 0.246 | 0.217 |
Two different events, kept apart: the chosen play's probability is not the probability that a risk pays. 10 fixed bins; the refit of a decision never saw its own block of games. Method
Confidence on right and wrong decisions
Swipe for more columns →
| Model and setting | Groups | First group: median confidence | Second group: median confidence |
|---|---|---|---|
| celeris-1-decisiondecision model · API | Engine's best play vs not | 0.80810 decisions | 0.60951 decisions |
| celeris-1-decisiondecision model · API | Right risks vs wrong and needless risks | Too few decisions150 decisions | Too few decisions96 decisions |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | Engine's best play vs not | 0.76745 decisions | 0.62910 decisions |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | Right risks vs wrong and needless risks | Too few decisions165 decisions | Too few decisions131 decisions |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | Engine's best play vs not | Too few decisions91 decisions | Too few decisions40 decisions |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | Right risks vs wrong and needless risks | Too few decisions7 decisions | Too few decisions8 decisions |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Engine's best play vs not | 0.53809 decisions | 0.32925 decisions |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Right risks vs wrong and needless risks | Too few decisions194 decisions | Too few decisions156 decisions |
| GPT-6 LunaDecisions APIdecision model · API | Engine's best play vs not | 0.61751 decisions | 0.48990 decisions |
| GPT-6 LunaDecisions APIdecision model · API | Right risks vs wrong and needless risks | Too few decisions143 decisions | Too few decisions110 decisions |
| Mercury Decidedecision model · API | Engine's best play vs not | 0.96673 decisions | 0.85956 decisions |
| Mercury Decidedecision model · API | Right risks vs wrong and needless risks | Too few decisions115 decisions | Too few decisions110 decisions |
| Jev 1.13decision model · API | Engine's best play vs not | 0.65854 decisions | 0.47833 decisions |
| Jev 1.13decision model · API | Right risks vs wrong and needless risks | Too few decisions172 decisions | Too few decisions107 decisions |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | Engine's best play vs not | 0.46541 decisions | 0.33968 decisions |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | Right risks vs wrong and needless risks | Too few decisions148 decisions | 0.30323 decisions |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | Engine's best play vs not | Too few decisions97 decisions | Too few decisions54 decisions |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | Right risks vs wrong and needless risks | Too few decisions6 decisions | Too few decisions12 decisions |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Engine's best play vs not | 0.27647 decisions | 0.121,129 decisions |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Right risks vs wrong and needless risks | 0.12233 decisions | 0.11447 decisions |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | Engine's best play vs not | 0.70806 decisions | 0.46827 decisions |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | Right risks vs wrong and needless risks | Too few decisions178 decisions | Too few decisions184 decisions |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Engine's best play vs not | 0.55795 decisions | 0.37804 decisions |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Right risks vs wrong and needless risks | Too few decisions192 decisions | Too few decisions176 decisions |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Engine's best play vs not | 0.45644 decisions | 0.27985 decisions |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Right risks vs wrong and needless risks | Too few decisions179 decisions | 0.24339 decisions |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | Engine's best play vs not | Too few decisions134 decisions | Too few decisions178 decisions |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | Right risks vs wrong and needless risks | Too few decisions25 decisions | Too few decisions17 decisions |
| SubQ Snap Previewdecision model · API | Engine's best play vs not | 0.56818 decisions | 0.371,150 decisions |
| SubQ Snap Previewdecision model · API | Right risks vs wrong and needless risks | 0.40270 decisions | 0.34324 decisions |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Engine's best play vs not | 0.68681 decisions | 0.511,024 decisions |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | Right risks vs wrong and needless risks | 0.54227 decisions | 0.43240 decisions |
The probability here is the chosen play's, never P(risk). Risks: risks taken at the 0.001 margin. Method
The Phase filter does not apply to this tab: all phases.
League 2
Close choices: details per setup
A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split). Probabilities over plays, not sampled reasoning.
League 2: complete letter-readout records are mostly few-option positions, a selected subset.
DeepSeek-V4.1-Flashlow · API · 1017: close choices 0.0% (0 / 180), under 200 decisions
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: N/A (0 / 0) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | N/A0 / 0 | N/A0 / 0 |
| margin 0.02 | N/A0 / 0 | N/A0 / 0 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 0 | N/A | N/A | N/A |
| Other decisions | 180 | 0.118median 0.051 | 0.007median 0.000 | 0.114median 0.040 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00120 decisions | N/AN/A (0 / 0) | 70.0%70.0% (14 / 20), under 200 decisions | N/A: no close choices in this group | N/A |
| margin 0.0217 decisions | N/AN/A (0 / 0) | 64.7%64.7% (11 / 17), under 200 decisions | N/A: no close choices in this group | N/A |
Probabilities over plays, not sampled reasoning. Full breakdown
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release: close choices 0.0% (0 / 408)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: N/A (0 / 0) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | N/A0 / 0 | N/A0 / 0 |
| margin 0.02 | N/A0 / 0 | N/A0 / 0 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 0 | N/A | N/A | N/A |
| Other decisions | 408 | 0.152median 0.065 | 0.014median 0.000 | 0.149median 0.052 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00145 decisions | N/AN/A (0 / 0) | 71.1%71.1% (32 / 45), under 200 decisions | N/A: no close choices in this group | N/A |
| margin 0.0236 decisions | N/AN/A (0 / 0) | 63.9%63.9% (23 / 36), under 200 decisions | N/A: no close choices in this group | N/A |
Probabilities over plays, not sampled reasoning. Full breakdown
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: close choices 0.0% (0 / 169), under 200 decisions
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: N/A (0 / 0) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | N/A0 / 0 | N/A0 / 0 |
| margin 0.02 | N/A0 / 0 | N/A0 / 0 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 0 | N/A | N/A | N/A |
| Other decisions | 169 | 0.142median 0.054 | 0.008median 0.000 | 0.137median 0.045 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00113 decisions | N/AN/A (0 / 0) | 76.9%76.9% (10 / 13), under 200 decisions | N/A: no close choices in this group | N/A |
| margin 0.029 decisions | N/AN/A (0 / 0) | 77.8%77.8% (7 / 9), under 200 decisions | N/A: no close choices in this group | N/A |
Probabilities over plays, not sampled reasoning. Full breakdown
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: close choices 2.9% (14 / 475)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 42.9% (6 / 14), under 200 decisions of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 50.0%2 / 4, under 200 decisions | 100.0%4 / 4, under 200 decisions |
| margin 0.02 | 66.7%2 / 3, under 200 decisions | 80.0%4 / 5, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 14 | 0.079median 0.040 | 0.028median 0.000 | 0.060median 0.014 |
| Other decisions | 461 | 0.100median 0.036 | 0.024median 0.000 | 0.095median 0.029 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00157 decisions | 0.0%crude 0.0% (0 / 2), under 200 decisions | 71.2%crude 63.6% (35 / 55), under 200 decisions | −71.2 points95% interval −100.0 to +0.0 | −63.6 points95% interval −74.6 to −50.0 |
| margin 0.0249 decisions | 0.0%crude 0.0% (0 / 2), under 200 decisions | 66.7%crude 57.4% (27 / 47), under 200 decisions | −66.7 points95% interval −100.0 to +0.0 | −57.4 points95% interval −69.2 to −43.5 |
Probabilities over plays, not sampled reasoning. Full breakdown
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: close choices 0.0% (0 / 298)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: N/A (0 / 0) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | N/A0 / 0 | N/A0 / 0 |
| margin 0.02 | N/A0 / 0 | N/A0 / 0 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 0 | N/A | N/A | N/A |
| Other decisions | 298 | 0.120median 0.054 | 0.031median 0.000 | 0.112median 0.040 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00124 decisions | N/AN/A (0 / 0) | 70.8%70.8% (17 / 24), under 200 decisions | N/A: no close choices in this group | N/A |
| margin 0.0223 decisions | N/AN/A (0 / 0) | 69.6%69.6% (16 / 23), under 200 decisions | N/A: no close choices in this group | N/A |
Probabilities over plays, not sampled reasoning. Full breakdown
d1-ultra: close choices 61.0% (1,262 / 2,070)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 38.8% (490 / 1,262) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 60.5%267 / 441 | 49.3%223 / 452 |
| margin 0.02 | 61.7%243 / 394 | 49.5%247 / 499 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 1,262 | 0.109median 0.057 | 0.077median 0.018 | 0.102median 0.041 |
| Other decisions | 808 | 0.150median 0.079 | 0.034median 0.000 | 0.150median 0.076 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001402 decisions | 57.6%crude 58.1% (155 / 267) | 69.5%crude 64.4% (87 / 135), under 200 decisions | −11.8 points95% interval −20.5 to −0.7 | −6.4 points95% interval −15.4 to +3.2 |
| margin 0.02351 decisions | 56.7%crude 57.6% (140 / 243) | 64.3%crude 63.9% (69 / 108), under 200 decisions | −7.6 points95% interval −18.9 to +4.2 | −6.3 points95% interval −17.1 to +4.8 |
Probabilities over plays, not sampled reasoning. Full breakdown
GLM-5.3low: close choices 5.5% (29 / 527)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 31.0% (9 / 29), under 200 decisions of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 100.0%3 / 3, under 200 decisions | 60.0%6 / 10, under 200 decisions |
| margin 0.02 | 100.0%3 / 3, under 200 decisions | 60.0%6 / 10, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 29 | 0.113median 0.041 | 0.078median 0.008 | 0.037median 0.000 |
| Other decisions | 498 | 0.134median 0.054 | 0.026median 0.000 | 0.127median 0.042 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00162 decisions | 0.0%crude 0.0% (0 / 3), under 200 decisions | 84.7%crude 83.1% (49 / 59), under 200 decisions | −84.7 points95% interval −100.0 to −70.0 | −83.1 points95% interval −90.9 to −74.5 |
| margin 0.0251 decisions | 0.0%crude 0.0% (0 / 3), under 200 decisions | 84.7%crude 81.2% (39 / 48), under 200 decisions | −84.7 points95% interval −100.0 to −70.0 | −81.2 points95% interval −90.9 to −71.4 |
Probabilities over plays, not sampled reasoning. Full breakdown
Jev 1.14: close choices 45.6% (231 / 507)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 36.8% (85 / 231) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 57.7%56 / 97, under 200 decisions | 39.7%29 / 73, under 200 decisions |
| margin 0.02 | 56.5%48 / 85, under 200 decisions | 43.5%37 / 85, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 231 | 0.096median 0.058 | 0.077median 0.031 | 0.097median 0.054 |
| Other decisions | 276 | 0.137median 0.075 | 0.045median 0.000 | 0.144median 0.078 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001104 decisions | 50.0%crude 51.8% (29 / 56), under 200 decisions | 84.2%crude 83.3% (40 / 48), under 200 decisions | −34.2 points95% interval −55.0 to −4.2 | −31.5 points95% interval −45.4 to −14.8 |
| margin 0.0289 decisions | 52.6%crude 54.2% (26 / 48), under 200 decisions | 81.1%crude 82.9% (34 / 41), under 200 decisions | −28.5 points95% interval −53.3 to +5.0 | −28.8 points95% interval −46.8 to −8.6 |
Probabilities over plays, not sampled reasoning. Full breakdown
GLiDEAPI: close choices 52.1% (1,015 / 1,947)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 41.1% (417 / 1,015) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 59.5%213 / 358 | 50.1%204 / 407 |
| margin 0.02 | 60.1%188 / 313 | 50.7%229 / 452 |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 1,015 | 0.131median 0.074 | 0.084median 0.021 | 0.118median 0.050 |
| Other decisions | 932 | 0.143median 0.086 | 0.049median 0.000 | 0.135median 0.079 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.001391 decisions | 53.5%crude 54.0% (115 / 213) | 78.1%crude 73.0% (130 / 178), under 200 decisions | −24.7 points95% interval −34.6 to −13.2 | −19.0 points95% interval −26.2 to −11.5 |
| margin 0.02342 decisions | 50.0%crude 51.1% (96 / 188), under 200 decisions | 78.5%crude 74.0% (114 / 154), under 200 decisions | −28.5 points95% interval −40.6 to −12.9 | −23.0 points95% interval −32.3 to −13.2 |
Probabilities over plays, not sampled reasoning. Full breakdown
Qwen3.8 Flashoff: close choices 40.3% (248 / 615)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 29.0% (72 / 248) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 75.0%27 / 36, under 200 decisions | 75.0%45 / 60, under 200 decisions |
| margin 0.02 | 79.3%23 / 29, under 200 decisions | 73.1%49 / 67, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 248 | 0.073median 0.021 | 0.044median 0.001 | 0.058median 0.003 |
| Other decisions | 367 | 0.124median 0.048 | 0.026median 0.000 | 0.109median 0.030 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00165 decisions | 74.1%crude 74.1% (20 / 27), under 200 decisions | 88.1%crude 89.5% (34 / 38), under 200 decisions | −14.1 points95% interval −37.9 to +9.4 | −15.4 points95% interval −33.9 to +5.6 |
| margin 0.0255 decisions | 69.6%crude 69.6% (16 / 23), under 200 decisions | 86.1%crude 87.5% (28 / 32), under 200 decisions | −16.5 points95% interval −42.9 to +10.7 | −17.9 points95% interval −38.1 to +4.9 |
Probabilities over plays, not sampled reasoning. Full breakdown
DeepSeek-V4.1-Flashoff · API · 1017: close choices 11.5% (62 / 537)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 21.0% (13 / 62), under 200 decisions of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 71.4%10 / 14, under 200 decisions | 25.0%3 / 12, under 200 decisions |
| margin 0.02 | 63.6%7 / 11, under 200 decisions | 40.0%6 / 15, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 62 | 0.082median 0.018 | 0.058median 0.008 | 0.051median 0.003 |
| Other decisions | 475 | 0.124median 0.055 | 0.026median 0.000 | 0.107median 0.030 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00158 decisions | 70.0%crude 70.0% (7 / 10), under 200 decisions | 86.2%crude 85.4% (41 / 48), under 200 decisions | −16.3 points95% interval −38.1 to +16.1 | −15.4 points95% interval −37.0 to +16.4 |
| margin 0.0245 decisions | 71.4%crude 71.4% (5 / 7), under 200 decisions | 87.5%crude 84.2% (32 / 38), under 200 decisions | −16.1 points95% interval −44.8 to +19.4 | −12.8 points95% interval −41.7 to +23.1 |
Probabilities over plays, not sampled reasoning. Full breakdown
Clef: close choices 88.4% (450 / 509)
1. Risk
Close choices between a safe play (the fewest hitting rolls of all legal plays) and a riskier one: 41.6% (187 / 450) of the close choices.
| Margin | In Risk pays positions | In Safety pays positions |
|---|---|---|
| margin 0.001 | 50.3%80 / 159, under 200 decisions | 60.1%107 / 178, under 200 decisions |
| margin 0.02 | 50.0%71 / 142, under 200 decisions | 59.5%116 / 195, under 200 decisions |
2. Engine values
| Decisions | Number | Gap between the two plays' values | Most likely play: behind the engine's best | Second most likely play: behind the engine's best |
|---|---|---|---|---|
| Close choices | 450 | 0.149median 0.073 | 0.081median 0.011 | 0.151median 0.059 |
| Other decisions | 59 | 0.173median 0.135 | 0.029median 0.000 | 0.174median 0.133 |
3. Missed risks
In Risk pays positions whose two most likely plays are a safe one and a riskier one: is the play made more often a missed risk in close choices than in other decisions?
| Margin | Close choices: missed risks | Other decisions: missed risks (weighted to the close choices' mix) | Difference, close minus other | Crude difference |
|---|---|---|---|---|
| margin 0.00189 decisions | 52.9%crude 50.0% (40 / 80), under 200 decisions | 45.1%crude 66.7% (6 / 9), under 200 decisions | +7.8 points95% interval −80.0 to +83.3 | −16.7 points95% interval −52.8 to +21.7 |
| margin 0.0278 decisions | 46.7%crude 46.5% (33 / 71), under 200 decisions | 46.7%crude 71.4% (5 / 7), under 200 decisions | +0.0 points95% interval −100.0 to +85.0 | −24.9 points95% interval −58.6 to +17.0 |
Probabilities over plays, not sampled reasoning. Full breakdown
N/A (no full probability distribution): Gemini 3.8 Flashhigh, Gemini 3.8 Flashlow, Claude Sonnet 5.5high, Claude Fable 5.1low, Grok 4.7low, Claude Sonnet 5.5low, GNUbg weak.
Which probabilities are complete
League 2: complete letter-readout records are mostly few-option positions, a selected subset.
Swipe for more columns →
| Model and setting | Complete | Sparse: an option without its own probability | Stored vector not reproduced from the response | No raw response | Plays split into groups | Not audited (after the audit) |
|---|---|---|---|---|---|---|
| Claude Fable 5.1low | 0 | 0 | 0 | 0 | 0 | 197 |
| Claude Sonnet 5.5highreasoning high · one letter · subscription | 0 | 0 | 0 | 0 | 0 | 173 |
| Claude Sonnet 5.5lowreasoning low · one letter · subscription | 0 | 0 | 0 | 0 | 0 | 209 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 509 | 0 | 0 | 0 | 3 | 1,466 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 408 | 1,270 | 0 | 0 | 245 | 265 |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | 180 | 1,311 | 0 | 0 | 204 | 527 |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 537 | 941 | 0 | 0 | 128 | 428 |
| GLiDEAPI | 1,947 | 0 | 0 | 0 | 2 | 210 |
| Gemini 3.8 Flashhighreasoning high · one letter · subscription | 0 | 0 | 0 | 0 | 0 | 194 |
| Gemini 3.8 Flashlowreasoning low · one letter · subscription | 0 | 0 | 0 | 0 | 0 | 223 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 475 | 993 | 0 | 0 | 148 | 231 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | 169 | 1,179 | 0 | 0 | 175 | 599 |
| GLM-5.3lowreasoning low · letter probabilities · API | 527 | 1,090 | 0 | 0 | 184 | 187 |
| Grok 4.7low | 0 | 0 | 0 | 0 | 0 | 411 |
| Jev 1.14 | 507 | 0 | 0 | 0 | 0 | 1,405 |
| d1-ultra | 2,070 | 0 | 0 | 0 | 1 | 0 |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 615 | 1,024 | 0 | 0 | 145 | 243 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 298 | 939 | 0 | 0 | 142 | 729 |
Only Complete decisions enter the re-read, the refit and the distributions below. Sparse: a letter missing from the returned top log-probabilities was filled with 0 and the rest renormalised, so its probability is unknown. Method
Re-read risk call (re-read from the model's probabilities, not played)
Swipe for more columns →
| Model and setting | Complete decisions | Risk judgement: as played → re-read | Change in risk judgement | Error rate (PR): as played → re-read | Change in error rate | Threshold chosen most often | Splits without a fit / test half without a class |
|---|---|---|---|---|---|---|---|
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 509382 with a trade-off | 59.0 → 60.0 | +1.195% interval −3.33 to +4.75 · stability −3.0 to +5.4 | 37.8 → 43.7 | +5.995% interval +2.68 to +8.29 · stability +2.4 to +10.0 | 0.80173 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 408204 with a trade-off | 64.1 → 64.1 | +0.095% interval −0.75 to +0.00 · stability +0.0 to +0.0 | 7.2 → 7.2 | +0.095% interval +0.00 to +0.35 · stability +0.0 to +0.0 | 0.50300 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 537246 with a trade-off | 54.7 → 60.9 | +6.195% interval +0.44 to +11.50 · stability +1.6 to +10.4 | 15.1 → 15.1 | +0.095% interval −3.21 to +2.42 · stability −3.0 to +3.9 | 0.05117 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| GLiDEAPI | 1,9471,426 with a trade-off | 60.3 → 60.0 | −0.395% interval −2.19 to +1.64 · stability −1.9 to +1.3 | 33.4 → 37.8 | +4.495% interval +1.59 to +6.04 · stability +1.3 to +7.4 | 0.55171 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 475225 with a trade-off | 61.9 → 62.3 | +0.495% interval −1.31 to +2.07 · stability −1.7 to +2.1 | 12.0 → 12.2 | +0.295% interval −0.37 to +0.73 · stability −0.4 to +0.8 | 0.15200 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| GLM-5.3lowreasoning low · letter probabilities · API | 527253 with a trade-off | 56.8 → 54.5 | −2.395% interval −4.58 to −0.07 · stability −6.1 to +0.0 | 14.2 → 16.2 | +2.095% interval +0.34 to +5.24 · stability −0.2 to +7.6 | 0.5099 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Jev 1.14 | 507367 with a trade-off | 60.2 → 63.0 | +2.895% interval −1.51 to +5.98 · stability −0.2 to +5.8 | 29.8 → 30.6 | +0.795% interval −1.18 to +3.38 · stability −1.5 to +3.0 | 0.50187 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| d1-ultra | 2,0701,503 with a trade-off | 64.1 → 62.6 | −1.595% interval −3.29 to +0.39 · stability −2.9 to −0.2 | 30.3 → 33.8 | +3.595% interval +1.71 to +5.42 · stability +1.9 to +5.1 | 0.65136 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 615272 with a trade-off | 49.4 → 50.0 | +0.595% interval −2.69 to +4.29 · stability −2.9 to +3.7 | 16.7 → 18.5 | +1.895% interval −1.78 to +8.74 · stability −4.1 to +11.2 | 0.3577 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 298123 with a trade-off | 63.2 → 63.2 | +0.095% interval −5.77 to +0.00 · stability +0.0 to +0.0 | 15.6 → 15.6 | +0.095% interval −0.18 to +1.42 · stability +0.0 to +0.0 | 0.50300 of 300 splits · always safe 0 · always risky 0 | 0 / 0 |
Too few decisions: DeepSeek-V4.1-Flashlow · API · 1017: 180 complete decisions. Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark: 169 complete decisions.
Re-read from the model's probabilities, not played: the recorded positions scored again with the play the setup's own probabilities point to; not a league result, and no new win rates. Same held-out decisions for both columns. Stability: the 5th to 95th percentile over the splits, not an interval; 95% interval: blocks of two seat-swapped games resampled, the whole procedure repeated. Method
Calibration as shipped and after a refit
Chosen play: the probability of the chosen play against "the chosen play is the engine's best"
Swipe for more columns →
| Model and setting | Decisions (before and after) | Brier score as shipped | Brier score after the refit | Reliability: as shipped (solid), after (dashed) |
|---|---|---|---|---|
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 509of 509; 0 folds without a fit | 0.243 | 0.211 | |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 408of 408; 0 folds without a fit | 0.203 | 0.166 | |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | 180of 180; 0 folds without a fit | Too few decisions | Too few decisions | |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 537of 537; 0 folds without a fit | 0.254 | 0.202 | |
| GLiDEAPI | 1,947of 1,947; 0 folds without a fit | 0.216 | 0.214 | |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 475of 475; 0 folds without a fit | 0.311 | 0.203 | |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | 169of 169; 0 folds without a fit | Too few decisions | Too few decisions | |
| GLM-5.3lowreasoning low · letter probabilities · API | 527of 527; 0 folds without a fit | 0.282 | 0.215 | |
| Jev 1.14 | 507of 507; 0 folds without a fit | 0.211 | 0.210 | |
| d1-ultra | 2,070of 2,070; 0 folds without a fit | 0.214 | 0.205 | |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 615of 615; 0 folds without a fit | 0.220 | 0.221 | |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 298of 298; 0 folds without a fit | 0.278 | 0.203 |
Risk class: P(risk), the probability mass on risky plays, against "the position is Risk pays"
Swipe for more columns →
| Model and setting | Decisions (before and after) | Brier score as shipped | Brier score after the refit | Reliability: as shipped (solid), after (dashed) |
|---|---|---|---|---|
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 382of 382; 0 folds without a fit | 0.262 | 0.204 | |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 204of 204; 0 folds without a fit | 0.216 | 0.169 | |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | 80of 80; 0 folds without a fit | Too few decisions | Too few decisions | |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 246of 246; 0 folds without a fit | 0.245 | 0.189 | |
| GLiDEAPI | 1,426of 1,426; 0 folds without a fit | 0.201 | 0.191 | |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 225of 225; 0 folds without a fit | 0.259 | 0.185 | |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | 71of 71; 0 folds without a fit | Too few decisions | Too few decisions | |
| GLM-5.3lowreasoning low · letter probabilities · API | 253of 253; 0 folds without a fit | 0.257 | 0.205 | |
| Jev 1.14 | 367of 367; 0 folds without a fit | 0.200 | 0.199 | |
| d1-ultra | 1,503of 1,503; 0 folds without a fit | 0.199 | 0.190 | |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 272of 272; 0 folds without a fit | 0.237 | 0.210 | |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 123of 123; 0 folds without a fit | Too few decisions | Too few decisions |
Two different events, kept apart: the chosen play's probability is not the probability that a risk pays. 10 fixed bins; the refit of a decision never saw its own block of games. Method
Confidence on right and wrong decisions
Swipe for more columns →
| Model and setting | Groups | First group: median confidence | Second group: median confidence |
|---|---|---|---|
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | Engine's best play vs not | 0.32231 decisions | 0.19278 decisions |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | Right risks vs wrong and needless risks | Too few decisions80 decisions | Too few decisions89 decisions |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | Engine's best play vs not | 1.00325 decisions | Too few decisions83 decisions |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | Right risks vs wrong and needless risks | Too few decisions19 decisions | Too few decisions11 decisions |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | Engine's best play vs not | Too few decisions150 decisions | Too few decisions30 decisions |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | Right risks vs wrong and needless risks | Too few decisions7 decisions | Too few decisions2 decisions |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | Engine's best play vs not | 0.99361 decisions | Too few decisions176 decisions |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | Right risks vs wrong and needless risks | Too few decisions12 decisions | Too few decisions14 decisions |
| GLiDEAPI | Engine's best play vs not | 0.53945 decisions | 0.351,002 decisions |
| GLiDEAPI | Right risks vs wrong and needless risks | 0.35225 decisions | 0.29202 decisions |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | Engine's best play vs not | 1.00316 decisions | Too few decisions159 decisions |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | Right risks vs wrong and needless risks | Too few decisions27 decisions | Too few decisions19 decisions |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | Engine's best play vs not | Too few decisions142 decisions | Too few decisions27 decisions |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | Right risks vs wrong and needless risks | Too few decisions3 decisions | Too few decisions0 decisions |
| GLM-5.3lowreasoning low · letter probabilities · API | Engine's best play vs not | 0.98356 decisions | Too few decisions171 decisions |
| GLM-5.3lowreasoning low · letter probabilities · API | Right risks vs wrong and needless risks | Too few decisions17 decisions | Too few decisions18 decisions |
| Jev 1.14 | Engine's best play vs not | 0.63245 decisions | 0.43262 decisions |
| Jev 1.14 | Right risks vs wrong and needless risks | Too few decisions58 decisions | Too few decisions38 decisions |
| d1-ultra | Engine's best play vs not | 0.471,006 decisions | 0.281,064 decisions |
| d1-ultra | Right risks vs wrong and needless risks | 0.32289 decisions | Too few decisions172 decisions |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | Engine's best play vs not | 0.66381 decisions | 0.57234 decisions |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | Right risks vs wrong and needless risks | Too few decisions14 decisions | Too few decisions36 decisions |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | Engine's best play vs not | 1.00215 decisions | Too few decisions83 decisions |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | Right risks vs wrong and needless risks | Too few decisions12 decisions | Too few decisions9 decisions |
The probability here is the chosen play's, never P(risk). Risks: risks taken at the 0.001 margin. Method
The Phase filter does not apply to this tab: all phases.
League 3
Close choices: details
N/A: no setup in this league has a full probability distribution. Probabilities over plays, not sampled reasoning.
Which probabilities are complete
N/A: no setup in this league returns option probabilities.
Re-read risk call (re-read from the model's probabilities, not played)
N/A: no setup in this league has 200 complete decisions.
Re-read from the model's probabilities, not played: the recorded positions scored again with the play the setup's own probabilities point to; not a league result, and no new win rates. Same held-out decisions for both columns. Stability: the 5th to 95th percentile over the splits, not an interval; 95% interval: blocks of two seat-swapped games resampled, the whole procedure repeated. Method
The Phase filter does not apply to this tab: all phases.
Analysis pending Shared positions: Analysis pending; no setup has answered them yet.
Does a setup's probability show when its play is likely wrong?
estimate: runs on our own machine (named after it), GPU time, at most; how it was counted on hover or tap · † plays split into groups · Not measured: no value recorded. Method
Setups played different positions; league comparisons are not paired.
League 1
Swipe for more columns →
| Model | Risk judgement, lower setting | Risk judgement, higher setting | Risk judgement change | Risk judgement intervals overlap | Error rate (PR), lower setting | Error rate (PR), higher setting | Error rate, in league play | Median s per decision | Extra median s per decision | Extra $ per 1,000 decisions |
|---|---|---|---|---|---|---|---|---|---|---|
| LFM2.5-24B-A2Boff → on | 45.743.2 to 48.1 | 52.450.3 to 54.5 | +6.7 | no | 61.556.3 to 66.8 | 47.744.1 to 51.2 | lower error rate in league play | 0.14 → 324.8 | +324.7 | +29.584 |
Every model with both settings in this league's games is listed. Risk judgement (margin 0.001, 95% intervals from resampling whole games) and error rate are two separate measures: the "Error rate, in league play" column is about error rate only and says nothing about risk judgement. The risk judgement change is the higher setting minus the lower, from unrounded values. Whether two intervals overlap describes them; it is not a test of a difference.
League games are not the same positions, so this comparison is not paired and shows no effect of the setting: a higher or lower value at the higher setting is not evidence that reasoning caused it. For error rate it reads "lower/higher error rate in league play" only when the two 95% intervals do not overlap, and "no clear difference in league play" otherwise. The paired verdict waits for the shared positions.
LFM2.5-24B-A2B: it has no effort setting. (off): an empty thinking block is prefilled, so it answers without thinking. (on): it is let to think before it answers (its template has no switch or effort setting; the "low" its requests carry is ignored).
The Phase filter does not apply to this tab: all phases.
League 2
Swipe for more columns →
| Model | Risk judgement, lower setting | Risk judgement, higher setting | Risk judgement change | Risk judgement intervals overlap | Error rate (PR), lower setting | Error rate (PR), higher setting | Error rate, in league play | Median s per decision | Extra median s per decision | Extra $ per 1,000 decisions |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Sonnet 5.5low → high | 63.561.6 to 65.4 | 68.266.3 to 70.0 | +4.7 | no | 20.117.8 to 22.4 | 16.714.9 to 18.7 | no clear difference in league play | 1.51 → 6.63 | +5.1 | +6.306 |
| DeepSeek-V4.1-Flash1017off → low | 58.556.6 to 60.5 | 68.566.4 to 70.6 | +10.0 | no | 35.231.8 to 38.8 | 17.115.1 to 19.2 | lower error rate in league play | 0.84 → 22.3 | +21.5 | +4.625 |
| Gemini 3.8 Flashlow → high | 65.163.5 to 66.8 | 66.965.2 to 68.5 | +1.7 | yes | 16.314.5 to 18.2 | 14.912.9 to 17.0 | no clear difference in league play | 2.16 → 9.59 | +7.4 | +18.230 |
| Gemma 4 31Boff → on | 62.660.4 to 65.0 | 64.562.9 to 66.2 | +1.8 | yes | 22.519.9 to 25.3 | 19.617.5 to 21.7 | no clear difference in league play | 0.31 → 120.6 | +120.3 | +26.728 |
Every model with both settings in this league's games is listed. Risk judgement (margin 0.001, 95% intervals from resampling whole games) and error rate are two separate measures: the "Error rate, in league play" column is about error rate only and says nothing about risk judgement. The risk judgement change is the higher setting minus the lower, from unrounded values. Whether two intervals overlap describes them; it is not a test of a difference.
League games are not the same positions, so this comparison is not paired and shows no effect of the setting: a higher or lower value at the higher setting is not evidence that reasoning caused it. For error rate it reads "lower/higher error rate in league play" only when the two 95% intervals do not overlap, and "no clear difference in league play" otherwise. The paired verdict waits for the shared positions.
The Phase filter does not apply to this tab: all phases.
League 3
Swipe for more columns →
| Model | Risk judgement, lower setting | Risk judgement, higher setting | Risk judgement change | Risk judgement intervals overlap | Error rate (PR), lower setting | Error rate (PR), higher setting | Error rate, in league play | Median s per decision | Extra median s per decision | Extra $ per 1,000 decisions |
|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5low → high | 66.964.7 to 69.2 | 70.067.3 to 72.6 | +3.0 | yes | 15.413.1 to 18.0 | 14.412.1 to 17.0 | no clear difference in league play | 3.92 → 6.35 | +2.4 | +7.338 |
| GPT-6 Astralow → medium | 75.873.1 to 78.5 | 74.771.9 to 77.3 | −1.1 | yes | 11.89.5 to 14.2 | 11.19.5 to 12.9 | no clear difference in league play | 3.94 → 4.40 | +0.5 | +0.531 |
| GPT-6.1 Sollow → high | 72.769.9 to 75.5 | 72.269.5 to 74.9 | −0.6 | yes | 12.110.5 to 13.7 | 9.98.6 to 11.3 | no clear difference in league play | 4.33 → 13.1 | +8.8 | +5.345 |
| Gemini 4 Argonlow → high | 69.266.2 to 72.3 | 74.172.0 to 76.1 | +4.9 | yes | 16.513.7 to 19.5 | 12.610.5 to 14.8 | no clear difference in league play | 3.82 → 27.5 | +23.7 | +54.197 |
Every model with both settings in this league's games is listed. Risk judgement (margin 0.001, 95% intervals from resampling whole games) and error rate are two separate measures: the "Error rate, in league play" column is about error rate only and says nothing about risk judgement. The risk judgement change is the higher setting minus the lower, from unrounded values. Whether two intervals overlap describes them; it is not a test of a difference.
League games are not the same positions, so this comparison is not paired and shows no effect of the setting: a higher or lower value at the higher setting is not evidence that reasoning caused it. For error rate it reads "lower/higher error rate in league play" only when the two 95% intervals do not overlap, and "no clear difference in league play" otherwise. The paired verdict waits for the shared positions.
The Phase filter does not apply to this tab: all phases.
Analysis pending Shared positions: Analysis pending; no setup has answered them yet.
The same model at a lower and a higher reasoning setting, each named by the effort its requests sent.
estimate: runs on our own machine (named after it), GPU time, at most; how it was counted on hover or tap · † plays split into groups · Not measured: no value recorded. Method
Setups played different positions; league comparisons are not paired.
League 1
Swipe for more columns →
| Model and setting | $ per 1,000 decisions | Median / P90 s per decision | Mean retry/wait s | Over the time budget | Tokens in / out per decision | Cost coverage | Decisions |
|---|---|---|---|---|---|---|---|
| Jev 1.13decision model · API | 0.166API price | 0.22 / 0.26 | 0.00 | 0.0% | 3,945 / 257 | 100% | 1,911 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 10.301estimate · RTX PRO 6000 Blackwell | 1.41 / 2.75 | 0.00 | 0.0% | 3,800 / 0 | 100% | 2,003 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 0.728estimate · RTX PRO 6000 Blackwell + API | 0.48 / 1.63 | 15.5 | 0.1% | 3,960 / 9 | 100% | 1,736 |
| celeris-1-decisiondecision model · API | 0.150API price | 0.17 / 0.33 | 0.00 | 0.0% | 3,747 / 0 | 100% | 1,922 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 12.227estimate · RTX PRO 6000 Blackwell | 0.94 / 2.73 | 1.84 | 0.0% | 4,331 / 0 | 100% | 2,028 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 11.168estimate · RTX PRO 6000 Blackwell | 0.37 / 2.96 | 0.00 | 0.0% | 4,593 / 1 | 100% | 2,016 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 6.593estimate · RTX PRO 6000 Blackwell | 1.97 / 7.48 | 3.86 | 0.6% | 4,488 / 0 | 100% | 2,052 |
| GPT-6 LunaDecisions APIdecision model · API | 0.335API price | 0.23 / 0.35 | 0.01 | 0.0% | 3,351 / 0 | 100% | 1,962 |
| SubQ Snap Previewdecision model · API | 0.193API price | 0.24 / 0.32 | 0.00 | 0.0% | 3,866 / 0 | 100% | 2,200 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 4.551estimate · RTX PRO 6000 Blackwell | 1.48 / 2.71 | 0.42 | 0.0% | 4,266 / 3 | 100% | 2,005 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 38.716estimate · RTX PRO 6000 Blackwell | 324.8 / 668.6 | 127.7 | 100.0% | 28,836 / 21,668 | 100% | 2,193 |
| Mercury Decidedecision model · API | 0.071API price | 0.48 / 0.77 | 0.01 | 0.0% | 3,567 / 1 | 100% | 1,853 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 9.615estimate · RTX PRO 6000 Blackwell | 2.46 / 5.45 | 0.00 | 0.1% | 4,144 / 0 | 100% | 2,015 |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 12.267estimate · RTX PRO 6000 Blackwell | 0.56 / 0.96 | 0.70 | 0.0% | 4,492 / 0 | 100% | 2,006 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 9.132estimate · RTX PRO 6000 Blackwell | 0.14 / 0.36 | 0.00 | 0.0% | 3,882 / 1 | 100% | 1,952 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 7.185estimate · RTX PRO 6000 Blackwell | 1.57 / 3.38 | 2.02 | 0.0% | 35,473 / 0 | 100% | 2,002 |
| GNUbg weakprogram · CPU | Not measured | 0.01 / 0.01 | 0.00 | 0.0% | Not measured | 0% | 1,744 |
| Randomprogram · CPU | Not measured | 0.00 / 0.00 | 0.00 | 0.0% | Not measured | 0% | 1,337 |
| Safestprogram · CPU | Not measured | 0.00 / 0.00 | 0.00 | 0.0% | Not measured | 0% | 2,058 |
Minimum bar, then cheapest
Bar: risk judgement at least 60 and error rate at most 35 PR (equity lost, not a share of mistakes). In each of 300 splits the cheapest priced setup that clears the bar on one half of the games is chosen and checked on the other half. No setup cleared the bar in 150 splits; only unpriced setups did in 24.
Swipe for more columns →
| Model and setting | Splits clearing the bar on training (of 300) | Chosen as cheapest | Bar cleared on the held-out half | Decisions with a price |
|---|---|---|---|---|
| celeris-1-decisiondecision model · API | 40 of them unpriced | 4 | 0of 4 chosen | 1,922 / 1,922 |
| CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 00 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,016 |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 390 of them unpriced | 25 | 0of 25 chosen | 1,736 / 1,736 |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 3737 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,003 |
| GPT-6 LunaDecisions APIdecision model · API | 00 of them unpriced | 0 | N/Aof 0 chosen | 1,962 / 1,962 |
| Mercury Decidedecision model · API | 00 of them unpriced | 0 | N/Aof 0 chosen | 1,853 / 1,853 |
| Jev 1.13decision model · API | 930 of them unpriced | 92 | 0of 92 chosen | 1,911 / 1,911 |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 00 of them unpriced | 0 | N/Aof 0 chosen | 0 / 1,952 |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 00 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,193 |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 00 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,002 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs | 10 of them unpriced | 1 | 0of 1 chosen | 2,052 / 2,052 |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 40 of them unpriced | 4 | 0of 4 chosen | 2,028 / 2,028 |
| Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 00 of them unpriced | 0 | N/Aof 0 chosen | 2,006 / 2,006 |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 00 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,005 |
| SubQ Snap Previewdecision model · API | 00 of them unpriced | 0 | N/Aof 0 chosen | 2,200 / 2,200 |
| Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs | 00 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,015 |
An unpriced setup can clear the bar but is never called cheapest. Programs are left out. Method
The Phase filter does not apply to this tab: all phases.
League 2
Swipe for more columns →
| Model and setting | $ per 1,000 decisions | Median / P90 s per decision | Mean retry/wait s | Over the time budget | Tokens in / out per decision | Cost coverage | Decisions |
|---|---|---|---|---|---|---|---|
| Gemini 3.8 Flashhighchat model, one letter · subscription | 22.352API-equivalent (subscription) | 9.59 / 44.4 | 2.29 | 18.4% | 4,344 / 5,092 | 100% | 2,168 |
| Gemini 3.8 Flashlowchat model, one letter · subscription | 4.123API-equivalent (subscription) | 2.16 / 6.87 | 0.04 | 0.7% | 4,311 / 237 | 100% | 2,053 |
| Claude Sonnet 5.5highchat model, one letter · subscription | 19.847API-equivalent (subscription) | 6.63 / 17.6 | 0.26 | 3.4% | 5,664 / 852 | 100% | 2,164 |
| DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API | 4.869API price | 22.3 / 107.6 | 12.1 | 43.4% | 4,215 / 7,173 | 100% | 2,234 |
| Claude Fable 5.1low | 62.613API-equivalent (subscription) | 1.54 / 6.34 | 0.04 | 1.1% | 5,728 / 107 | 100% | 2,008 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API | 47.092estimate · RTX PRO 6000 Blackwell + API | 43.8 / 173.1 | 1.99 | 63.7% | 4,364 / 12,487 | 100% | 2,226 |
| Grok 4.7low | 32.813API-equivalent (subscription) | 24.3 / 110.2 | 14.0 | 42.7% | 5,999 / 3,469 | 100% | 2,152 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 34.943estimate · RTX PRO 6000 Blackwell | 120.6 / 394.3 | 15.3 | 96.4% | 14,746 / 5,599 | 100% | 2,122 |
| Claude Sonnet 5.5lowchat model, one letter · subscription | 13.541API-equivalent (subscription) | 1.51 / 7.37 | 0.01 | 0.5% | 5,579 / 238 | 100% | 2,150 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 8.215estimate · RTX PRO 6000 Blackwell | 0.31 / 4.03 | 0.90 | 1.6% | 4,167 / 1 | 100% | 1,847 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs | 39.023estimate · RTX PRO 6000 Blackwell + API | 171.1 / 566.7 | 24.0 | 97.7% | 14,723 / 5,772 | 100% | 2,108 |
| d1-ultra | 0.145API price | 0.23 / 0.41 | 0.00 | 0.0% | 3,461 / 0 | 100% | 2,071 |
| GLM-5.3lowchat model, answer-letter probabilities · API | 2.917API price | 1.75 / 7.48 | 6.69 | 0.6% | 4,005 / 148 | 100% | 1,989 |
| Jev 1.14 | 0.173API price | 0.19 / 0.24 | 0.00 | 0.0% | 4,116 / 292 | 100% | 1,912 |
| GLiDEAPI | 0.153API price | 0.48 / 0.79 | 0.00 | 0.0% | 3,829 / 0 | 100% | 2,159 |
| Qwen3.8 Flashoffchat model, answer-letter probabilities · API | 0.284API price | 1.70 / 7.47 | 8.97 | 0.3% | 4,182 / 10 | 100% | 2,027 |
| DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API | 0.244API price | 0.84 / 1.39 | 24.2 | 0.0% | 3,846 / 1 | 100% | 2,034 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 5.736estimate · RTX PRO 6000 Blackwell | 1.13 / 5.41 | 2.15 | 0.2% | 4,028 / 0 | 100% | 1,978 |
| GNUbg weakprogram · CPU | Not measured | 0.01 / 0.01 | 0.00 | 0.0% | Not measured | 0% | 2,136 |
Minimum bar, then cheapest
Bar: risk judgement at least 60 and error rate at most 35 PR (equity lost, not a share of mistakes). In each of 300 splits the cheapest priced setup that clears the bar on one half of the games is chosen and checked on the other half. No setup cleared the bar in 0 splits; only unpriced setups did in 0.
Swipe for more columns →
| Model and setting | Splits clearing the bar on training (of 300) | Chosen as cheapest | Bar cleared on the held-out half | Decisions with a price |
|---|---|---|---|---|
| Claude Fable 5.1low | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,008 / 2,008 |
| Claude Sonnet 5.5highreasoning high · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,164 / 2,164 |
| Claude Sonnet 5.5lowreasoning low · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,150 / 2,150 |
| ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 22 of them unpriced | 0 | N/Aof 0 chosen | 0 / 1,978 |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasereasoning low · letter probabilities · API | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,226 / 2,226 |
| DeepSeek-V4.1-Flashlow · API · 1017reasoning low · letter probabilities · API | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,234 / 2,234 |
| DeepSeek-V4.1-Flashoff · API · 1017reasoning off · letter probabilities · API | 210 of them unpriced | 0 | N/Aof 0 chosen | 2,034 / 2,034 |
| GLiDEAPI | 190190 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,159 |
| Gemini 3.8 Flashhighreasoning high · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,168 / 2,168 |
| Gemini 3.8 Flashlowreasoning low · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,053 / 2,053 |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning off · letter probabilities · our GPUs | 295295 of them unpriced | 0 | N/Aof 0 chosen | 0 / 1,847 |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning on · letter probabilities · our GPUs | 300300 of them unpriced | 0 | N/Aof 0 chosen | 0 / 2,122 |
| GLM-5.3lowreasoning low · letter probabilities · API | 2920 of them unpriced | 0 | N/Aof 0 chosen | 1,989 / 1,989 |
| Grok 4.7low | 3000 of them unpriced | 0 | N/Aof 0 chosen | 2,152 / 2,152 |
| Jev 1.14 | 810 of them unpriced | 1 | 0of 1 chosen | 1,912 / 1,912 |
| d1-ultra | 2990 of them unpriced | 299 | 298of 299 chosen | 2,071 / 2,071 |
| Qwen3.8 Flashoffreasoning off · letter probabilities · API | 930 of them unpriced | 0 | N/Aof 0 chosen | 2,027 / 2,027 |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkreasoning low · letter probabilities · our GPUs | 287287 of them unpriced | 0 | N/Aof 0 chosen | 1 / 2,108 |
An unpriced setup can clear the bar but is never called cheapest. Programs are left out. Method
The Phase filter does not apply to this tab: all phases.
League 3
Swipe for more columns →
| Model and setting | $ per 1,000 decisions | Median / P90 s per decision | Mean retry/wait s | Over the time budget | Tokens in / out per decision | Cost coverage | Decisions |
|---|---|---|---|---|---|---|---|
| GPT-6.1 Solhigh | 15.573API-equivalent (subscription) | 13.1 / 56.5 | 5.69 | 24.5% | 5,217 / 514 | 100% | 1,389 |
| GPT-6 Astramediumchat model, one letter · subscription | 50.268API-equivalent (subscription) | 4.40 / 15.1 | 0.41 | 2.6% | 4,471 / 111 | 100% | 1,362 |
| GPT-6 Astralowchat model, one letter · subscription | 49.737API-equivalent (subscription) | 3.94 / 14.0 | 0.23 | 1.6% | 4,486 / 98 | 100% | 1,251 |
| GPT-6.1 Sollow | 10.228API-equivalent (subscription) | 4.33 / 14.3 | 0.44 | 1.5% | 4,543 / 114 | 100% | 1,344 |
| Gemini 4 Argonhighchat model, one letter · subscription | 64.401API-equivalent (subscription) | 27.5 / 148.6 | 6.93 | 47.1% | 5,117 / 5,417 | 100% | 1,368 |
| Claude Fable 5.1highchat model, one letter · subscription | 136.268API-equivalent (subscription) | 12.7 / 47.1 | 0.50 | 19.4% | 6,502 / 1,425 | 100% | 1,286 |
| Claude Opus 5.5highchat model, one letter · subscription | 36.400API-equivalent (subscription) | 6.35 / 18.5 | 0.51 | 4.3% | 5,861 / 648 | 100% | 1,477 |
| Claude Opus 5.5lowchat model, one letter · subscription | 29.062API-equivalent (subscription) | 3.92 / 10.6 | 0.03 | 0.6% | 5,896 / 274 | 100% | 1,411 |
| Gemini 4 Argonlowchat model, one letter · subscription | 10.204API-equivalent (subscription) | 3.82 / 12.2 | 0.12 | 1.3% | 4,970 / 27 | 100% | 1,484 |
| Grok 4.7highchat model, one letter · subscription | 117.379API-equivalent (subscription) | 122.3 / 588.6 | 2.21 | 85.9% | 6,400 / 17,430 | 100% | 1,337 |
| Kimi K3.1 Prooffchat model, one letter · API | 3.982API price | 2.11 / 7.23 | 0.03 | 0.1% | 4,353 / 20 | 100% | 1,167 |
| GNUbg weakprogram · CPU | Not measured | 0.01 / 0.01 | 0.00 | 0.0% | Not measured | 0% | 1,479 |
Minimum bar, then cheapest
Bar: risk judgement at least 60 and error rate at most 35 PR (equity lost, not a share of mistakes). In each of 300 splits the cheapest priced setup that clears the bar on one half of the games is chosen and checked on the other half. No setup cleared the bar in 0 splits; only unpriced setups did in 0.
Swipe for more columns →
| Model and setting | Splits clearing the bar on training (of 300) | Chosen as cheapest | Bar cleared on the held-out half | Decisions with a price |
|---|---|---|---|---|
| Claude Fable 5.1highreasoning high · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,286 / 1,286 |
| Claude Opus 5.5highreasoning high · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,477 / 1,477 |
| Claude Opus 5.5lowreasoning low · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,411 / 1,411 |
| Gemini 4 Argonhighreasoning high · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,368 / 1,368 |
| Gemini 4 Argonlowreasoning low · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,484 / 1,484 |
| GPT-6 Astralowreasoning low · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,251 / 1,251 |
| GPT-6 Astramediumreasoning medium · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,362 / 1,362 |
| GPT-6.1 Solhigh | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,389 / 1,389 |
| GPT-6.1 Sollow | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,344 / 1,344 |
| Grok 4.7highreasoning high · one letter · subscription | 3000 of them unpriced | 0 | N/Aof 0 chosen | 1,337 / 1,337 |
| Kimi K3.1 Prooffreasoning off · one letter · API | 3000 of them unpriced | 300 | 300of 300 chosen | 1,167 / 1,167 |
An unpriced setup can clear the bar but is never called cheapest. Programs are left out. Method
The Phase filter does not apply to this tab: all phases.
Analysis pending Shared positions: Analysis pending; no setup has answered them yet.
Dollars per 1,000 decisions and seconds per decision.
estimate: runs on our own machine (named after it), GPU time, at most; how it was counted on hover or tap · † plays split into groups · Not measured: no value recorded. Method
Setups played different positions; league comparisons are not paired.
League 1
Not measured League games never repeat a position.
The Phase filter does not apply to this tab: all phases.
League 2
Not measured League games never repeat a position.
The Phase filter does not apply to this tab: all phases.
League 3
Not measured League games never repeat a position.
The Phase filter does not apply to this tab: all phases.
Analysis pending Shared positions: Analysis pending; no setup has answered them yet.
Does a setup repeat its play on the same position?
estimate: runs on our own machine (named after it), GPU time, at most; how it was counted on hover or tap · † plays split into groups · Not measured: no value recorded. Method
Setups played different positions; league comparisons are not paired.
Risk
League 1
missed risks wrong risks colour: share of wrong risks among the Risk pays decisions without the right risk · squares: reference rows · diagonals: risk judgement
needless risks missed risks wrong risks right risks right safety No trade-off
Three bars per setup: small, medium, big stake (Risk pays only). risks found wrong risks missed risks Under each name: needless risks that lost more than 0.08.
circles: model setups · squares: programs and computed reference rows · short bar: the cost-derived SDT reference c*, drawn only where it is valid
A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity.
missed risks wrong risks colour: share of wrong risks among the Risk pays decisions without the right risk · squares: reference rows · diagonals: risk judgement
needless risks missed risks wrong risks right risks right safety No trade-off
Three bars per setup: small, medium, big stake (Risk pays only). risks found wrong risks missed risks Under each name: needless risks that lost more than 0.08.
circles: model setups · squares: programs and computed reference rows · short bar: the cost-derived SDT reference c*, drawn only where it is valid
A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity.
League 2
missed risks wrong risks colour: share of wrong risks among the Risk pays decisions without the right risk · squares: reference rows · diagonals: risk judgement
needless risks missed risks wrong risks right risks right safety No trade-off
Three bars per setup: small, medium, big stake (Risk pays only). risks found wrong risks missed risks Under each name: needless risks that lost more than 0.08.
circles: model setups · squares: programs and computed reference rows · short bar: the cost-derived SDT reference c*, drawn only where it is valid
A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity.
missed risks wrong risks colour: share of wrong risks among the Risk pays decisions without the right risk · squares: reference rows · diagonals: risk judgement
needless risks missed risks wrong risks right risks right safety No trade-off
Three bars per setup: small, medium, big stake (Risk pays only). risks found wrong risks missed risks Under each name: needless risks that lost more than 0.08.
circles: model setups · squares: programs and computed reference rows · short bar: the cost-derived SDT reference c*, drawn only where it is valid
A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity.
League 3
missed risks wrong risks colour: share of wrong risks among the Risk pays decisions without the right risk · squares: reference rows · diagonals: risk judgement
needless risks missed risks wrong risks right risks right safety No trade-off
Three bars per setup: small, medium, big stake (Risk pays only). risks found wrong risks missed risks Under each name: needless risks that lost more than 0.08.
circles: model setups · squares: programs and computed reference rows · short bar: the cost-derived SDT reference c*, drawn only where it is valid
A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity.
missed risks wrong risks colour: share of wrong risks among the Risk pays decisions without the right risk · squares: reference rows · diagonals: risk judgement
needless risks missed risks wrong risks right risks right safety No trade-off
Three bars per setup: small, medium, big stake (Risk pays only). risks found wrong risks missed risks Under each name: needless risks that lost more than 0.08.
circles: model setups · squares: programs and computed reference rows · short bar: the cost-derived SDT reference c*, drawn only where it is valid
A simple model of the risk call (equal-variance signal detection); setups met different positions. The cost-derived SDT reference uses the setup's own mean equity lost per needless and per missed risk; those costs depend on the setup's own play, so it is a reference under that model, not the best caution, and a correctly classified call can still lose equity.
Analysis pending Shared positions: Analysis pending.
Hit risk only: being hit. Gammon risk and priming are not in it.
Risk call card of each league winner (PNG, 1200×675): League 1: GNUbg weak · League 2: Gemini 3.8 Flashlow · League 3: GPT-6.1 Sollow
Error rate, time and cost
League 1
League 2
League 3
Analysis pending Shared positions: Analysis pending.
Setups played different positions; league comparisons are not paired. Charts cover all phases except the Risk panel.
Trade-off
League 1
League 2
League 3
Analysis pending Shared positions: Analysis pending.
Setups played different positions; league comparisons are not paired. Charts cover all phases except the Risk panel.
Confidence
League 1
Three bars per setup: small, medium, big stake (Risk pays only); each bar is the share of eligible decisions in that band that are close choices. small medium big Rows in the Confidence table's order, not ranked by this share. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
Three bars per setup: small, medium, big stake (Risk pays only); each bar is the share of eligible decisions in that band that are close choices. small medium big Rows in the Confidence table's order, not ranked by this share. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
League 2
League 2: complete letter-readout records are mostly few-option positions, a selected subset.
Three bars per setup: small, medium, big stake (Risk pays only); each bar is the share of eligible decisions in that band that are close choices. small medium big Rows in the Confidence table's order, not ranked by this share. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
Three bars per setup: small, medium, big stake (Risk pays only); each bar is the share of eligible decisions in that band that are close choices. small medium big Rows in the Confidence table's order, not ranked by this share. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
League 3
Three bars per setup: small, medium, big stake (Risk pays only); each bar is the share of eligible decisions in that band that are close choices. small medium big Rows in the Confidence table's order, not ranked by this share. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
Three bars per setup: small, medium, big stake (Risk pays only); each bar is the share of eligible decisions in that band that are close choices. small medium big Rows in the Confidence table's order, not ranked by this share. A big stake means playing safe costs more by the engine's value, not that the decision was harder.
Analysis pending Shared positions: Analysis pending.
Probabilities over plays, not sampled reasoning. A decision is a close choice when the second most likely play has at least half the probability of the first (internally: split). N/A for setups without a full probability distribution. Method
Every measure of every setup, one tab per measure; each value with its 95% interval, its count and denominator or its price label under it.
Risk pays vs safety pays
- League 1: model setups found 19% to 38% of the risks that paid, and played safe in 60% to 92% of the positions where safety paid.
- League 2: model setups found 25% to 45% of the risks that paid, and played safe in 76% to 98% of the positions where safety paid.
- League 3: model setups found 40% to 57% of the risks that paid, and played safe in 89% to 97% of the positions where safety paid.
League 1
Risks found (Risk pays) Right safety (Safety pays)
League 2
Risks found (Risk pays) Right safety (Safety pays)
League 3
Risks found (Risk pays) Right safety (Safety pays)
Hit risk only: being hit. Gammon risk and priming are not in it. Setups played different positions; league comparisons are not paired.
Can a model predict where it will waver?
Stage B: five teams
Close choices: does a setup know where it hesitates?
Scope. Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark was started but not completed: two reboots and a GPU fault interrupted it, and it was stopped after 311 of 500 play calls on 2026-10-25. Its partial data is not reported.
Each team answered the same 100 positions 5 times, and once more in a separate call naming the features it expects to differ between the plays it might choose (hitting risk, making a point, the race, the prime). Observed set: the features that differ between any of the 5 chosen plays. Primary score: exact agreement of the predicted set with the observed set. Secondary score, declared secondary: partial overlap of the two sets. Baseline: the team's own predictions shuffled within phase, 1,000 seeded shuffles; p = the share of shuffles scoring at least the observed mean. Stage B asks again and looks at repeated answers; it is separate from stage A (Confidence tab), which reads recorded league decisions.
Swipe for more columns →
| Team and setting | Temperature | Positions scored | Exact agreement (primary score) | Shuffled baseline, mean [2.5–97.5%] | p | Exact agreement against its baseline | Partial overlap (secondary score) | Shuffled baseline, mean [2.5–97.5%] | p | Partial overlap against its baseline | Same play in all 5 answers | Play changed, features identical | Positions where a feature changed | Features predicted per position, mean | Left out, by reason |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Claude Opus 5.5higheffort high (default, adaptive thinking) | known: 1.0none sent; 1.0 is Anthropic's documented default | 100 | 0.34 | 0.275 [0.23, 0.32] | 0.004 | above the baseline range | 0.402 | 0.315 [0.268, 0.365] | 0.000 | above the baseline range | 71 | 10 | 19 | 1.19 | none |
| GPT-6 Astramediumreasoning effort medium (the model's default) | not disclosednone sent; not disclosed | 100 | 0.20 | 0.176 [0.15, 0.20] | 0.054 | inside the baseline range: not clearly above it | 0.213 | 0.189 [0.160, 0.212] | 0.023 | above the baseline range | 90 | 5 | 5 | 1.72 | none |
| GPT-6.1 Solhighreasoning effort high | not disclosednone sent; not disclosed | 100 | 0.20 | 0.139 [0.11, 0.17] | 0.000 | above the baseline range | 0.223 | 0.163 [0.127, 0.193] | 0.000 | above the baseline range | 80 | 10 | 10 | 1.64 | none |
| GPT-6.1 Sollowreasoning effort low | not disclosednone sent; not disclosed | 100 | 0.13 | 0.111 [0.08, 0.14] | 0.154 | inside the baseline range: not clearly above it | 0.169 | 0.135 [0.102, 0.165] | 0.010 | above the baseline range | 80 | 10 | 10 | 1.75 | none |
| Gemini 3.8 Flashlowthinking level low | known: 1.01.0, sent | 100 | 0.06 | 0.018 [0.01, 0.04] | 0.000 | above the baseline range | 0.065 | 0.037 [0.024, 0.050] | 0.000 | above the baseline range | 92 | 2 | 6 | 2.12 | none |
Not a ranking: each team is compared with its own shuffled baseline, never with another team. Rows are in fixed alphabetical order and the table cannot be sorted.
A change, or no change, across five answers doesn't measure the model's whole behaviour.
Against its baseline: above the range = the observed mean is above the baseline's 97.5% point. Inside the range = not clearly above the baseline; a result that is not significant is not evidence that the team equals its baseline.
Most positions have no changed feature: the five answers chose the same play (71–92 positions per team), or changed play without changing a feature. So much of each score comes from predicting “none” where the team does not hesitate.
No team gave the same play in all five answers on every scored position.
Every team kept its league settings. Temperature: the value sent or the provider's documented default where known, otherwise not disclosed.
Exact agreement against the shuffled baseline
shuffled baseline, 2.5–97.5% baseline mean exact agreement, above the range inside the range: not clearly above Each team against its own baseline; not a ranking.
Data: close-choices-b.json · Method (Method spec 2026-10-24 §9.9-9.13)
- Scorer
- GNU Backgammon 1.08, 2-ply, on every legal play of every decision
- Decisions scored
- 92,888
- Leagues
- League 1: decision models and models on our own hardware; League 2: workhorse chat models, closed and open; League 3: frontier chat models on subscriptions
- Results
f24d1f41, data as of 16 November 2026, 22:20 UTC- Published
- Every result per setup (results.json). Not published: the positions, the plays chosen in them and the engine's best play for each (why). No internal address, key or request header.
What this measures. Risk calls per setup: was each risk the right one, and was safe right when it played safe? Risk judgement: 100 = every call right, 50 = always safe. Error rate: how much equity each move gives away, judged by the engine, not by wins. Hit risk only: being hit. Gammon risk and priming are not in it. Setups met different positions; risk judgement is not adjusted for how hard they were. Paired comparisons wait for the shared positions.
About these tables
One table per league. League tables compare only the players inside one league; they are never merged into one overall score. A running league's numbers are Provisional and change as matches finish. Click a row to open the setup, or a match to open its decisions.
Earlier version: v1 leagues and teams Earlier version
Earlier version: plays were listed in the runner's order, which tends to put the best play early. When Gemini 4 Argon answered the same 1,000 positions with the plays shuffled, its error rate changed by +1.9 (95% interval −0.2 to +4.2), so the effect is not proven.
Earlier version The sections below are kept from the earlier report, unchanged. Their numbers come from the earlier version and are never ranked with current numbers. The prompt and how it was built are on the Method page.
The setup
Earlier version Plays were listed in the runner's order, which tends to put the best play early.
Each model plays every other model once per league (a single round robin). A match is played in rounds of two games with the same dice: in the second game the players swap seats and the same sequence of rolls is repeated, so both face the same luck. Games score 1 point, 2 for a gammon and 3 for a backgammon; there is no doubling cube.
- Match length: 3 rounds, or fewer: the first player to 5 points after a full round (both games) wins if ahead. A tie gets one extra round, then counts as a draw.
- League table: 3 points for a match win, 1 for a draw; ties broken by more games won, then fewer games lost.
- Turn limit: a game still running after 300 turns (both sides counted) goes to the lower pip count for 1 point.
- Clock: 30 seconds free per decision, then a 5-minute bank per game. Running out loses the game.
- Legality: the program generates every legal play and the model chooses one of them, so a model can never make an illegal move. A roll with only one legal play is played automatically.
- Memory: every decision is a separate request. Models are not told how earlier games went.
With seven models, each league has 21 matches and one model sits out each matchday. A match starts as soon as both of its models are free, and no model plays two matches at once.
Models
Earlier version Plays were listed in the runner's order, which tends to put the best play early.
All seven answer the same kind of request: a text state and one choice question whose options are the legal plays. Two are hosted services; the other five run on our own GPUs.
- Jev 1.13 · TypeSafe, hosted
- TypeSafe's decision model, served through its System One API; the request format the others copy.
- GPT-6 Luna · OpenAI Decisions API, hosted
- OpenAI's decision endpoint with
gpt-6-luna. The only one that sometimes declines to choose; that happened 3 times in early leagues, on positions with 97 to 439 legal plays, and never in the current format. - Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark · Perplexity, self-hosted
- The released
pplx-decider-v1.1-27bcheckpoint, run with its own server code. It accepts up to 8,192 tokens, so long play lists are sent in groups of 25 (see below). It replaced Perplexity's hosted Decider 27B from league #6 on. - Clef · Cloudflare, self-hosted
- A 27B decision model that scores all options in one pass. Groups of up to 110 options.
- Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell · open reimplementation of Jev, self-hosted
- A LoRA adapter and decision head on a 27B base model. It scores every option as its own sequence that repeats the whole question, with a 4,096-token limit per sequence, which is what limits the prompt's length for every model.
- Cygnet · blockbrain-ai recipe, self-hosted
- Google's
gemma-4-12B-it, unchanged, behind a server that labels the options A, B, C… and reads the model's probability for each letter. Groups of up to 26 options. - Gemma 4 31B · our configuration of the Cygnet recipe
- The same letter-reading server in front of
gemma-4-31B-it, served in FP8. Not a trained decision model: a general chat model whose answer-letter probabilities are read directly. Its calibration setting was fitted for Cygnet's 12B model; that changes its reported confidence, not which play it picks.
When a position has more legal plays than a model's group size, the plays are split into equal groups, the model picks a winner in each, then chooses among the winners. Liquid's d1 played in the early leagues and was dropped after its free tier rate-limited a league to a stop.
Error rate
Earlier version Plays were listed in the runner's order, which tends to put the best play early.
Wins and losses depend heavily on the dice. A backgammon engine can instead judge every decision directly: for each move it finds the best play, and measures how much winning chance, in equity (expected points per game), the chosen play gave away. Averaged over all decisions and multiplied by 500, this is the error rate (performance rating, PR) that backgammon players use, here for checker plays only; forced moves are not counted. eXtreme Gammon's human scale: under 2.5 world champion, 2.5 to 5 world class, 5 to 7.5 expert, 7.5 to 12.5 advanced, 12.5 to 17.5 intermediate, 17.5 to 22.5 casual, 22.5 to 30 beginner, 30 or more distracted. Human ratings also count doubling cube decisions, which these games do not have, so the comparison is approximate.
We scored all 104,045 decisions the models made in every league with GNU Backgammon 1.08. Every legal play was evaluated at 0-ply, then the five best and the chosen play again at 2-ply; the loss is the 2-ply equity of the best play minus that of the chosen one. As a check, on the opening roll the plays that the rollout table calls best lose 0.0002 equity on average, other plays 0.063.
By model, leagues #8, #11 and #12
| Model | Decisions | Error rate | Engine's best play | Blunders | #8 | #11 | #12 | #13 |
|---|---|---|---|---|---|---|---|---|
| Gemma 4 31B | 1,894 | 25.4 ± 2.4 | 54% | 19.5% | 22.7 | 26.3 | 27.2 | 28.8 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 1,942 | 32.7 ± 2.9 | 49% | 24.5% | 33.0 | 33.1 | 31.9 | 34.2 |
| Jev 1.13 | 2,066 | 35.7 ± 2.9 | 46% | 26.7% | 34.2 | 36.2 | 36.7 | 32.0 |
| Clef | 2,100 | 35.8 ± 3.0 | 47% | 25.6% | 35.7 | 40.5 | 30.6 | 42.8 |
| GPT-6 Luna | 2,044 | 39.4 ± 3.5 | 46% | 25.6% | 40.1 | 35.9 | 41.7 | 37.5 |
| Cygnet | 2,078 | 43.2 ± 3.4 | 42% | 30.2% | 46.9 | 45.6 | 36.7 | 40.6 |
| Open-Jev 27B | 1,758 | 59.6 ± 4.6 | 36% | 36.5% | 65.8 | 63.1 | 49.5 | 58.1 |
Error rate: mean equity lost per decision × 500, with a 95% interval that treats decisions as independent (moves within a game are not, so the true interval is somewhat wider). "Engine's best play": share of decisions where the chosen play lost less than 0.001. "Blunders": decisions losing more than 0.08 equity. The last four columns are the error rate in each league; #13 adds the move history.
- The engine ranks the models almost as the league tables do. Gemma 4 31B is clearly best and Open-Jev 27B clearly worst. Decider v1.1, Jev, Clef and GPT-6 Luna are within about one interval of each other, which is why their league places move around; Cygnet is a little behind them.
- Every model blunders often: from one move in five (Gemma 4 31B) to one in three (Open-Jev 27B). Even Gemma 4 31B chooses the engine's best play only about half the time.
- The prompt mattered more than the model. Before move consequences were computed, the whole field averaged 120 to 138; after, 36 to 40. The worst model now is better than the average model was then.
Results
Earlier version Plays were listed in the runner's order, which tends to put the best play early.
The standings below use the three finished leagues with computed move consequences and no move history: #8, #11 and #12 (7 models, same prompt, different dice). League #13, with move history, finished 1. Gemma 4 31B, 2. Jev 1.13, 3. Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark, 4. Cygnet, 5. GPT-6 Luna, 6. Clef, 7. Open-Jev 27B.
Final places
| Model | 1st | 2nd | 3rd | 4th | 5th | 6th | 7th |
|---|---|---|---|---|---|---|---|
| Gemma 4 31B | 2 | 1 | 0 | 0 | 0 | 0 | 0 |
| Jev 1.13 | 1 | 0 | 0 | 1 | 1 | 0 | 0 |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0 | 2 | 1 | 0 | 0 | 0 | 0 |
| Clef | 0 | 0 | 1 | 1 | 0 | 1 | 0 |
| Cygnet | 0 | 0 | 1 | 1 | 0 | 1 | 0 |
| GPT-6 Luna | 0 | 0 | 0 | 0 | 3 | 0 | 0 |
| Open-Jev 27B | 0 | 0 | 0 | 0 | 0 | 0 | 3 |
In league #12, Jev 1.13 and GPT-6 Luna finished level on every tiebreak and share 5th place.
Across the three leagues
| Model | Matches W-D-L | Games won-lost | Points for-against | Gammons and backgammons won | Picked the safest play | Shots left on average | Time per decision |
|---|---|---|---|---|---|---|---|
| Gemma 4 31B | 14-1-3 | 67-27 | 91-36 | 23 | 84% | 5.1 | 0.30 s |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 12-0-6 | 56-40 | 79-56 | 20 | 77% | 6.1 | 1.14 s |
| Jev 1.13 | 11-0-7 | 53-49 | 71-61 | 16 | 82% | 5.1 | 0.26 s |
| Clef | 9-1-8 | 53-45 | 81-64 | 21 | 68% | 7.0 | 1.13 s |
| Cygnet | 8-0-10 | 52-54 | 65-72 | 11 | 82% | 4.6 | 0.22 s |
| GPT-6 Luna | 8-0-10 | 48-54 | 62-68 | 11 | 87% | 3.8 | 0.20 s |
| Open-Jev 27B | 0-0-18 | 15-75 | 18-110 | 2 | 65% | 8.5 | 1.69 s |
"Safest play": the option whose position the opponent hits with the fewest of its 36 rolls, read from the numbers the model was shown. "Shots left": opponent rolls out of 36 that hit after the chosen play. Both are measured on decisions sent as a single request (about 1,700 to 2,100 per model); the safest available play left 2.7 to 4.2 shots on average, a typical option about 13.
What the numbers say
- Gemma 4 31B is the strongest player here, and it is not a trained decision model: a general chat model whose answer-letter probabilities are read. It lost 3 of 18 matches and won the most gammons.
- Safe is not the same as good. GPT-6 Luna picks the safest play most often and leaves the fewest shots, yet finished 5th in all three leagues. Winning also takes making points and building a prime, which costs some risk; Gemma and Clef take more risk and win more gammons.
- Open-Jev 27B collapsed in this format. It finished 2nd in league #3, where options were just notations, and last in every league since, with the most shots left and the highest error rate. We have not found out why; the numbers it is shown are the same as everyone else's.
- Speed does not decide it. The three fastest models are spread across the table, and no model came close to its clock.
Can a team beat the best model?
Earlier version Plays were listed in the runner's order, which tends to put the best play early.
Every model reports a probability for each option. If those probabilities say when a model is right, a team could ask all seven and play the answer of whichever is most sure. To test this, we sent the same 1,000 recorded positions (2 to 25 legal plays each, drawn from leagues #8, #11, #12 and #13) to all seven models with exactly the request the original player received, and scored every legal play with GNU Backgammon at 2-ply: 7,000 answers and 9,439 evaluated plays. Any team and any rule can then be scored offline on the same positions.
Confidence says how hard a position is, not who is right
Within each model, confidence does carry information: across its league decisions, a model is right more often when it is more confident (AUROC 0.66 to 0.74, where 0.5 is chance). But the models use very different scales:
| Model | Engine's best play | Average stated probability | When it is most sure | it is right |
|---|---|---|---|---|
| Gemma 4 31B | 57% | 0.85 overconfident | 0.8 – 1.0 | 65% |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 51% | 0.58 | 0.8 – 1.0 | 84% |
| Jev 1.13 | 47% | 0.55 | 0.8 – 1.0 | 81% |
| Clef | 46% | 0.28 underconfident | 0.6 – 0.8 | 89% |
| GPT-6 Luna | 46% | 0.56 | 0.8 – 1.0 | 65% |
| Cygnet | 44% | 0.67 overconfident | 0.8 – 1.0 | 63% |
| Open-Jev 27B | 36% | 0.21 underconfident | 0.6 – 0.8 | 92% |
From 17,579 league decisions sent as a single request; "right" means the engine's best play (loss below 0.001 equity). The stated probability is the one the model gave its chosen play; the models' own "confidence" fields are defined differently from one another, so they are not used.
Clef and Open-Jev look like the most reliable models when they are sure, but they are only sure on easy positions, where everyone else is right as well. The test that matters is what happens when models disagree on the same position:
| When Gemma 4 31B and this model disagree | Gemma has the better play | The more confident model has the better play |
|---|---|---|
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 62% | 62% |
| Clef | 64% | 64% |
| Jev 1.13 | 69% | 67% |
| GPT-6 Luna | 73% | 70% |
| Cygnet | 77% | 75% |
| Open-Jev 27B | 76% | 76% |
Following the more confident model is never better than simply following Gemma 4 31B.
Teams
| Team and rule | Error rate | Engine's best play | Difference from Gemma alone (95%) |
|---|---|---|---|
| Gemma 4 31B alone | 23.5 | 54.4% | – |
| All 7: highest stated probability | 24.0 | 53.8% | +0.4 (−0.2 to +1.2) |
| All 7: lowest expected loss, calibrated | 26.0 | 52.9% | +2.5 (+0.7 to +4.3) |
| All 7: most likely correct, calibrated | 27.7 | 51.9% | +4.2 (+2.2 to +6.4) |
| All 7: probabilities added up | 27.7 | 52.0% | +4.2 (+2.1 to +6.6) |
| Best of 120 formations: Clef + Decider v1.1 + Gemma, probabilities added up | 23.4 | 54.8% | −0.2 (−0.7 to +0.2) |
| Oracle: the best answer any of the 7 gave | 12.7 | 70.1% | −10.8 |
Error rate on the 1,000 shared positions (lower is better). Calibrated rules map each model's stated probability to its observed accuracy or loss with isotonic regression, fitted on one half of the positions and scored on the other (two-fold cross-fitting). Intervals are bootstrap intervals over positions, paired with Gemma 4 31B on the same positions. The best formation was chosen on the same positions it is scored on, which flatters it.
- No team beats the best single model. Raw confidence ties with Gemma 4 31B only because its inflated probabilities win almost every vote; correcting the scales makes the team worse.
- The room for a team is large. Gemma 4 31B misses the engine's best play on 456 of the 1,000 positions, and on a third of those another model found it. An oracle would halve the error rate. What is missing is a way to tell which answer is the right one.
- The data is kept for further formations. Other selectors (agreement between models, the number of legal plays, the stage of the game) can be tested on the same 1,000 positions without new requests.
Speed and cost
Earlier version Plays were listed in the runner's order, which tends to put the best play early.
Measured on the same 1,000 recorded positions as the team test, with exactly the request each original player received. Besides the seven league players, the table includes general language models we tried the same way: DeepSeek-V4.1-Flash, Qwen3.8 Flash Next and GLM-4.7-Flash are read through their probabilities for each answer letter, like Gemma 4 31B. GPT-6 Astra, Claude Opus 5.5, Gemini 4 Argon and Gemini 3.8 Flash return no such probabilities: each answers one letter, constrained to the option letters, at low reasoning effort (Gemini's lowest available level). Gemini 4 Argon reported no reasoning tokens at that level, so it answered directly. DeepSeek-V4.1-Flash also ran a second time with reasoning on (lowest effort), answering one constrained letter after reasoning. GPT-6 Astra and Claude Opus 5.5 ran through ChatGPT and Claude subscriptions.
| Model | Runs on | Error rate | Time per decision median / 90th pct | In flight | Estimated API cost per 1,000 decisions |
|---|---|---|---|---|---|
| GPT-6 Astra | ChatGPT subscription | 11.2 | 4.60 s / 10.95 s | 4 | $36.45 API-equivalent |
| Gemini 4 Argon | Google Gemini API | 16.9 | 4.09 s / 28.22 s | 4 | $6.97 |
| Claude Opus 5.5 | Claude subscription | 17.3 | 3.43 s / 6.66 s | 4 | $21.65 API-equivalent |
| DeepSeek-V4.1-Flash, reasoning on | our GPUs: 4 | 18.1 | 106 s / 288 s | 8 | own hardware |
| Gemini 3.8 Flash | Google Gemini API | 18.3 | 2.87 s / 7.96 s | 4 | $3.25 |
| Gemma 4 31B | our GPUs: 1 | 23.5 | 0.59 s / 0.87 s | 4 | own hardware |
| DeepSeek-V4.1-Flash | our GPUs: 4 | 27.5 | 0.57 s / 0.98 s | 4 | own hardware |
| Qwen3.8 Flash Next | our GPUs: 4 | 31.1 | 0.46 s / 0.71 s | 4 | own hardware |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | our GPUs: shares 2 | 31.8 | 0.82 s / 1.35 s | 1 | own hardware |
| Jev 1.13 | TypeSafe API | 33.6 | 0.25 s / 0.29 s | 4 | $0.16 |
| Clef | our GPUs: 1 | 34.1 | 0.94 s / 1.28 s | 1 | own hardware |
| GPT-6 Luna | OpenAI Decisions API | 35.0 | 0.22 s / 0.27 s | 4 | $0.32 |
| Cygnet | our GPUs: 1 | 38.9 | 0.78 s / 1.12 s | 4 | own hardware |
| GLM-4.7-Flash | our GPUs: 1 | 50.6 | 0.20 s / 0.27 s | 4 | own hardware |
| Open-Jev 27B | our GPUs: 1 | 59.9 | 1.38 s / 1.91 s | 1 | own hardware |
Error rate on the 1,000 shared positions (lower is better). Time is wall-clock per request, from sending it to receiving the answer, with the stated number of requests in flight. Our GPUs are NVIDIA RTX PRO 6000 Blackwell cards with 96 GB each, in one machine. The seven league players were asked together, so the self-hosted ones shared the machine; the other models were each asked alone. A run of GLM-5.3-Flash is left out: it played close to random, most likely because of how we had to run it on this machine.
The plays are listed in the order the runner generated them, which tends to put the best play early. Asked again with the same plays in a random order, Gemini 4 Argon's error rate rose from 16.9 to 18.9 (difference 1.9, 95% interval −0.3 to 4.1): probably a small help from the order, not yet a proven one. The next version of the benchmark shuffles the plays for every model.
Cost basis: the provider's list price × the tokens its API reported; the bill itself was not checked. Jev 1.13 ($0.042 per 1M input tokens, output free) and GPT-6 Luna on OpenAI's Decisions API ($0.10 per 1M input tokens, output free): prices of 8 October 2026, tokens averaged over their league decisions with 2 to 25 legal plays (3,740 and 3,238 input tokens per decision). Gemini 3.8 Flash ($0.75 per 1M input tokens, $3.75 per 1M output tokens, reasoning billed as output; introductory price of 8 October 2026, which doubles on 1 January 2027): tokens measured on 100 of the positions, 3,373 input and 192 output per decision on average. Gemini 4 Argon ($2 per 1M input, $10 per 1M output; price until 30 November 2026, doubling on 1 December): tokens reported for all 1,000 positions, 3,462 input and 5 output per decision. GPT-6 Astra ($10 per 1M input, $50 per 1M output) and Claude Opus 5.5 ($4 input, $20 output): API list prices of 8 October 2026 × the tokens reported for all 1,000 positions (3,213 input and 86 output, and 4,424 input and 198 output per decision). Both ran on subscriptions, which are not billed per token, so these are API-equivalent estimates. Self-hosted models have no per-request price; a 7-model league of about 690 decisions per player took about 30 minutes on the four GPUs. DeepSeek with reasoning used 6,657 output tokens per decision on average.
- The cheapest strong player is one we run ourselves. Gemma 4 31B costs only GPU time, needs one card, and answers in about 0.6 s.
- The best players cost most and are slowest. GPT-6 Astra makes the fewest errors by far, but at API prices it costs about a hundred times as much per decision as GPT-6 Luna, and one decision in ten takes 11 seconds or more. Gemini 4 Argon and Claude Opus 5.5 show no clear difference (16.9 against 17.3; difference 0.3, 95% interval −1.9 to 2.5), with Argon about a third of Opus's price; Gemini 3.8 Flash (18.3) is close behind at half Argon's price. Even asking Astra, Opus and Gemini 3.8 Flash together and taking the majority answer is worse than Astra alone (14.4).
- Reasoning helps an open model a lot, at a steep time cost. With reasoning on, DeepSeek V4.1 Flash's error rate falls from 27.5 to 18.1 (95% interval of the gain 6.1 to 12.7), close to Opus's, but a decision takes about 100 seconds instead of half a second.
- The hosted decision services are fastest and cheapest per decision, at about 0.2 s and well under a dollar per 1,000 decisions, but they play worse than Gemma 4 31B.
Limits
Earlier version Plays were listed in the runner's order, which tends to put the best play early.
- Small sample. Three finished leagues of 21 matches in the main format, about 340 games. Placings in the middle of the table move from league to league; the error rate is the steadier measure.
- The engine evaluation is not the strongest available. Moves are judged at 2-ply with GNU Backgammon, not by full rollouts or eXtreme Gammon, and in cubeless money equity: the games have no doubling cube, and match score is ignored. Small differences between close plays can be wrong, but the losses that count as blunders and the gaps between models are much larger than that.
- Models only play each other. Their absolute strength against human or engine players is unknown. On the engine's scale none of them reaches expert level; the best, GPT-6 Astra, plays like an advanced human.
- Configuration matters. Five models run on servers set up by us, and Gemma 4 31B uses a recipe built for a different model. Hosted services can change without notice.
- The prompt steers. The question and the strategy text say what a good move looks like, so the results measure how well models follow that guidance with the numbers in front of them, as much as how well they play.
The page as it was before this site went live, with its final league data: archive, 23 October 2026.