Preliminary / partial
Clear risk: when one play is clearly better
Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.
The scan: which decisions have a clear risk call
Swipe for more columns →
| League 1 | League 2 | League 3 | Total | |
|---|---|---|---|---|
| league decisions | 36,995 | 39,538 | 16,355 | 92,888 |
| skip: invalid answer (fallback) | 0 | 50 | 36 | 86 |
| skip: forced move (1 play) | 0 | 0 | 0 | 0 |
| skip: 2 legal plays | 3,661 | 3,819 | 1,385 | 8,865 |
| skip: no contact (race or bear-off) | 5,184 | 5,677 | 2,499 | 13,360 |
| skip: safe bin empty | 10,828 | 10,095 | 4,070 | 24,993 |
| skip: unsafe bin empty | 1,937 | 2,589 | 1,227 | 5,753 |
| scanned (both bins non-empty) | 15,385 | 17,308 | 7,138 | 39,831 |
| Risk wins (gap ≥ +0.04) | 4,498 | 4,304 | 1,645 | 10,447 |
| Safety wins (gap ≤ −0.04) | 8,186 | 9,960 | 4,247 | 22,393 |
| not flagged (|gap| < 0.04) | 2,701 | 3,044 | 1,246 | 6,991 |
| flagged | 12,684 | 14,264 | 5,892 | 32,840 |
Swipe for more columns →
| league | group | flagged | kept | unstable: direction flips | unstable: |gap3| < 0.04 | unstable: |gap3 − gap2| > 0.02 |
|---|---|---|---|---|---|---|
| 1 | Risk wins | 4,498 | 3,353 | 5 | 167 | 973 |
| 2 | Risk wins | 4,304 | 3,337 | 2 | 141 | 824 |
| 3 | Risk wins | 1,645 | 1,307 | 1 | 65 | 272 |
| all | Risk wins | 10,447 | 7,997 | 8 | 373 | 2,069 |
| 1 | Safety wins | 8,186 | 6,223 | 3 | 176 | 1,784 |
| 2 | Safety wins | 9,960 | 7,704 | 4 | 187 | 2,065 |
| 3 | Safety wins | 4,247 | 3,275 | 2 | 65 | 905 |
| all | Safety wins | 22,393 | 17,202 | 9 | 428 | 4,754 |
| all | both | 32,840 | 25,199 | 17 | 801 | 6,823 |
Preliminary / partial
How this part was done and checked *
- A decision is scanned when the mover has both a safe play (0 hitting rolls for the opponent) and an unsafe play (12 or more hitting rolls), there is contact, and there are more than 2 legal plays. Invalid answers (fallback) are skipped.
- Gap = 2-ply cubeless equity of the best unsafe play minus that of the best safe play (GNU Backgammon). Risk wins: gap of +0.04 or more. Safety wins: gap of -0.04 or less. Both are 'flagged'.
- 3-ply stability check (it replaced rollouts for the leagues): a flagged decision is kept only if the gap is 0.04 or more at both 2-ply and 3-ply, has the same sign at both, and the two gaps differ by 0.02 or less. It is a stability signal, not a rollout.
- Kept: 25,199 of 32,840 flagged decisions (7,997 Risk wins, 17,202 Safety wins).
- The method was written down and committed before any computation; the 3-ply rule was committed before any 3-ply value was computed.
Missed and needless risks per setup
Swipe for more columns →
| setup | Risk wins kept | missed risks (played safe) | in-between (Risk wins) | Safety wins kept | needless risks (played unsafe) | in-between (Safety wins) | bias = missed − needless | thread: risks found |
|---|---|---|---|---|---|---|---|---|
| celeris-1-decision | 168 | 81.5% (137/168) | 13.1% (22/168) | 408 | 1.0% (4/408) | 4.2% (17/408) | +80.6 pp | 23.9% (163/683) |
| Cygnet | 240 | 75.8% (182/240) | 13.8% (33/240) | 297 | 3.4% (10/297) | 6.4% (19/297) | +72.5 pp | 27.3% (205/750) |
| Perplexity Decider Flash v1.2 | 160 | 51.9% (83/160) | 15.6% (25/160) | 339 | 13.9% (47/339) | 10.0% (34/339) | +38.0 pp | 31.3% (223/712) |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 175 | 67.4% (118/175) | 16.6% (29/175) | 369 | 7.3% (27/369) | 10.6% (39/369) | +60.1 pp | 33.1% (233/703) |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 145 | 66.9% (97/145) | 16.6% (24/145) | 361 | 4.4% (16/361) | 10.8% (39/361) | +62.5 pp | 33.4% (244/731) |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 194 | 62.9% (122/194) | 26.3% (51/194) | 299 | 1.3% (4/299) | 10.4% (31/299) | +61.5 pp | 29.1% (192/659) |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 139 | 63.3% (88/139) | 17.3% (24/139) | 417 | 6.5% (27/417) | 7.0% (29/417) | +56.8 pp | 33.0% (219/664) |
| GNUbg weak | 104 | 3.8% (4/104) | 16.3% (17/104) | 312 | 15.1% (47/312) | 24.0% (75/312) | -11.2 pp | 66.6% (410/616) |
| GPT-6 LunaDecisions API | 197 | 80.2% (158/197) | 14.2% (28/197) | 330 | 2.1% (7/330) | 8.5% (28/330) | +78.1 pp | 21.6% (160/740) |
| Jev 1.13 | 185 | 77.3% (143/185) | 13.5% (25/185) | 354 | 0.8% (3/354) | 4.5% (16/354) | +76.4 pp | 27.0% (190/705) |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 178 | 64.6% (115/178) | 9.0% (16/178) | 278 | 10.4% (29/278) | 13.3% (37/278) | +54.2 pp | 28.8% (206/715) |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 185 | 55.7% (103/185) | 13.5% (25/185) | 356 | 12.1% (43/356) | 9.3% (33/356) | +43.6 pp | 31.5% (248/788) |
| Mercury Decide | 217 | 77.4% (168/217) | 16.1% (35/217) | 307 | 1.3% (4/307) | 10.1% (31/307) | +76.1 pp | 18.7% (129/690) |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 137 | 50.4% (69/137) | 16.8% (23/137) | 306 | 20.3% (62/306) | 11.4% (35/306) | +30.1 pp | 37.3% (264/708) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 231 | 74.9% (173/231) | 20.8% (48/231) | 331 | 1.8% (6/331) | 4.8% (16/331) | +73.1 pp | 23.5% (178/756) |
| Random | 80 | 23.8% (19/80) | 17.5% (14/80) | 131 | 65.6% (86/131) | 12.2% (16/131) | -41.9 pp | 24.8% (112/452) |
| Safest | 233 | 100.0% (233/233) | 0.0% (0/233) | 329 | 0.0% (0/329) | 0.0% (0/329) | +100.0 pp | 0.0% (0/785) |
| SubQ Snap Preview | 180 | 52.8% (95/180) | 25.6% (46/180) | 388 | 7.2% (28/388) | 12.6% (49/388) | +45.6 pp | 37.9% (297/783) |
| Winnow-12B | 205 | 61.5% (126/205) | 21.0% (43/205) | 311 | 7.4% (23/311) | 8.7% (27/311) | +54.1 pp | 33.7% (263/780) |
| Safest program (reference) | 3353 | 100.0% (3353/3353) | 0.0% (0/3353) | 6223 | 0.0% (0/6223) | 0.0% (0/6223) | +100.0 pp | |
| Random legal play (expected) | 3353 | 16.3% (547.1/3353) | 11.5% (384.9/3353) | 6223 | 59.4% (3694.0/6223) | 17.2% (1068.4/6223) | -43.0 pp |
Swipe for more columns →
| setup | Risk wins kept | missed risks (played safe) | in-between (Risk wins) | Safety wins kept | needless risks (played unsafe) | in-between (Safety wins) | bias = missed − needless | thread: risks found |
|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.1low | 165 | 44.2% (73/165) | 31.5% (52/165) | 385 | 2.1% (8/385) | 5.7% (22/385) | +42.2 pp | 41.0% (270/658) |
| Claude Sonnet 5.5high | 194 | 53.1% (103/194) | 26.8% (52/194) | 428 | 0.5% (2/428) | 7.2% (31/428) | +52.6 pp | 41.8% (308/737) |
| Claude Sonnet 5.5low | 170 | 61.8% (105/170) | 27.1% (46/170) | 466 | 0.6% (3/466) | 3.2% (15/466) | +61.1 pp | 30.9% (204/661) |
| Clef | 182 | 54.9% (100/182) | 16.5% (30/182) | 340 | 10.6% (36/340) | 14.1% (48/340) | +44.4 pp | 39.5% (263/666) |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 187 | 48.1% (90/187) | 25.1% (47/187) | 472 | 1.9% (9/472) | 9.5% (45/472) | +46.2 pp | 45.0% (338/751) |
| DeepSeek-V4.1-Flashoff · API · 1017 | 194 | 72.2% (140/194) | 19.1% (37/194) | 395 | 0.8% (3/395) | 11.4% (45/395) | +71.4 pp | 25.4% (181/713) |
| DeepSeek-V4.1-Flashlow · API · 1017 | 185 | 49.7% (92/185) | 30.8% (57/185) | 476 | 2.1% (10/476) | 9.7% (46/476) | +47.6 pp | 42.5% (321/755) |
| GLiDEAPI | 156 | 50.6% (79/156) | 21.8% (34/156) | 410 | 4.4% (18/410) | 11.2% (46/410) | +46.3 pp | 36.9% (249/675) |
| GLM-5.3low | 193 | 73.6% (142/193) | 19.7% (38/193) | 365 | 0.3% (1/365) | 9.0% (33/365) | +73.3 pp | 30.7% (205/668) |
| GNUbg weak | 121 | 7.4% (9/121) | 20.7% (25/121) | 409 | 21.5% (88/409) | 26.4% (108/409) | -14.1 pp | 62.7% (483/770) |
| Gemini 3.8 Flashhigh | 170 | 53.5% (91/170) | 27.1% (46/170) | 454 | 0.2% (1/454) | 4.0% (18/454) | +53.3 pp | 36.1% (264/731) |
| Gemini 3.8 Flashlow | 151 | 57.6% (87/151) | 19.9% (30/151) | 421 | 1.4% (6/421) | 2.6% (11/421) | +56.2 pp | 33.8% (214/633) |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 133 | 60.9% (81/133) | 22.6% (30/133) | 371 | 1.6% (6/371) | 3.2% (12/371) | +59.3 pp | 33.2% (194/584) |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 190 | 57.4% (109/190) | 21.6% (41/190) | 435 | 0.0% (0/435) | 3.9% (17/435) | +57.4 pp | 31.9% (223/700) |
| Grok 4.7low | 182 | 46.2% (84/182) | 34.6% (63/182) | 414 | 0.0% (0/414) | 9.2% (38/414) | +46.2 pp | 42.3% (295/697) |
| Jev 1.14 | 187 | 74.3% (139/187) | 17.1% (32/187) | 347 | 2.9% (10/347) | 7.2% (25/347) | +71.4 pp | 29.1% (184/632) |
| d1-ultra | 171 | 49.1% (84/171) | 29.2% (50/171) | 414 | 4.3% (18/414) | 10.6% (44/414) | +44.8 pp | 42.0% (289/688) |
| Qwen3.8 Flashoff | 218 | 55.0% (120/218) | 25.2% (55/218) | 334 | 3.3% (11/334) | 13.8% (46/334) | +51.8 pp | 34.4% (243/706) |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 188 | 62.2% (117/188) | 21.8% (41/188) | 368 | 3.3% (12/368) | 5.2% (19/368) | +59.0 pp | 33.8% (242/715) |
| Safest program (reference) | 3337 | 100.0% (3337/3337) | 0.0% (0/3337) | 7704 | 0.0% (0/7704) | 0.0% (0/7704) | +100.0 pp | |
| Random legal play (expected) | 3337 | 16.9% (563.3/3337) | 11.3% (376.8/3337) | 7704 | 57.2% (4408.4/7704) | 18.2% (1403.6/7704) | -40.3 pp |
Swipe for more columns →
| setup | Risk wins kept | missed risks (played safe) | in-between (Risk wins) | Safety wins kept | needless risks (played unsafe) | in-between (Safety wins) | bias = missed − needless | thread: risks found |
|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.1high | 102 | 35.3% (36/102) | 32.4% (33/102) | 295 | 2.4% (7/295) | 8.1% (24/295) | +32.9 pp | 50.7% (215/424) |
| Claude Opus 5.5low | 119 | 47.1% (56/119) | 26.1% (31/119) | 289 | 1.7% (5/289) | 8.3% (24/289) | +45.3 pp | 40.1% (187/466) |
| Claude Opus 5.5high | 117 | 48.7% (57/117) | 29.9% (35/117) | 260 | 1.5% (4/260) | 5.4% (14/260) | +47.2 pp | 43.4% (199/459) |
| GNUbg weak | 111 | 10.8% (12/111) | 13.5% (15/111) | 245 | 14.7% (36/245) | 22.9% (56/245) | -3.9 pp | 58.8% (288/490) |
| GPT-6 Astralow | 105 | 39.0% (41/105) | 30.5% (32/105) | 261 | 1.1% (3/261) | 11.1% (29/261) | +37.9 pp | 57.2% (226/395) |
| GPT-6 Astramedium | 117 | 35.0% (41/117) | 38.5% (45/117) | 280 | 0.7% (2/280) | 8.9% (25/280) | +34.3 pp | 54.9% (245/446) |
| GPT-6.1 Solhigh | 88 | 31.8% (28/88) | 27.3% (24/88) | 334 | 1.8% (6/334) | 10.8% (36/334) | +30.0 pp | 50.6% (212/419) |
| GPT-6.1 Sollow | 108 | 32.4% (35/108) | 32.4% (35/108) | 302 | 1.0% (3/302) | 10.3% (31/302) | +31.4 pp | 51.7% (198/383) |
| Gemini 4 Argonlow | 115 | 54.8% (63/115) | 20.9% (24/115) | 297 | 2.4% (7/297) | 6.4% (19/297) | +52.4 pp | 47.8% (209/437) |
| Gemini 4 Argonhigh | 101 | 42.6% (43/101) | 20.8% (21/101) | 237 | 0.4% (1/237) | 7.2% (17/237) | +42.2 pp | 50.8% (224/441) |
| Grok 4.7high | 139 | 52.5% (73/139) | 26.6% (37/139) | 250 | 1.6% (4/250) | 9.6% (24/250) | +50.9 pp | 44.0% (190/432) |
| Kimi K3.1 Prooff | 85 | 45.9% (39/85) | 25.9% (22/85) | 225 | 2.2% (5/225) | 10.2% (23/225) | +43.7 pp | 42.4% (153/361) |
| Safest program (reference) | 1307 | 100.0% (1307/1307) | 0.0% (0/1307) | 3275 | 0.0% (0/3275) | 0.0% (0/3275) | +100.0 pp | |
| Random legal play (expected) | 1307 | 18.9% (247.3/1307) | 10.5% (137.4/1307) | 3275 | 54.6% (1789.7/3275) | 18.6% (609.9/3275) | -35.7 pp |
Preliminary / partial
How this part was done and checked *
- Missed risks: Risk wins played safe (0 hitting rolls). Needless risks: Safety wins played unsafe (12 or more). In-between: 1 to 11 hitting rolls. Bias = missed minus needless, in percentage points; positive means too cautious. Each share is shown with its count; no group has under 30 positions.
- Reference rows use all of the league's kept decisions: the Safest program always plays a 0-roll play; Random legal play gives each decision's expected shares (fraction of legal plays in each bin).
- Not paired: setups met different positions, so the rows compare only loosely.
- Thread: risks found is a different, broader measure: the site's published Risks found at margin 0.001, from the deployed results a12573f2, computed on all of a setup's Risk pays decisions, a much larger set than the kept clear-risk positions. Risks found counts a right risk, missed risks here count playing safe.
Checks on the scan
Swipe for more columns →
| best play safe | best play in between | best play unsafe | kept decisions | |
|---|---|---|---|---|
| Risk wins, all leagues | 0 | 22.4% (1,793) | 77.6% (6,204) | 7,997 |
| Risk wins, L1 / L2 / L3 | 0 / 0 / 0 | 23.8% (799) / 21.4% (715) / 21.3% (279) | 2,554 / 2,622 / 1,028 | 3,353 / 3,337 / 1,307 |
| Risk wins, band 0.04–0.08 / 0.08–0.15 / 0.15+ | 0 | 29.1% (677) / 19.2% (426) / 20.0% (690) | 1,646 / 1,792 / 2,766 | 2,323 / 2,218 / 3,456 |
| Safety wins, all leagues | 84.0% (14,443) | 16.0% (2,759) | 0 | 17,202 |
| Safety wins, L1 / L2 / L3 | 5,185 / 6,459 / 2,799 | 16.7% (1,038) / 16.2% (1,245) / 14.5% (476) | 0 | 6,223 / 7,704 / 3,275 |
| Safety wins, band 0.04–0.08 / 0.08–0.15 / 0.15+ | 1,721 / 3,396 / 9,326 | 23.4% (527) / 18.6% (777) / 13.5% (1,455) | 0 | 2,248 / 4,173 / 10,781 |
Swipe for more columns →
| band | setups (pooled): missed | needless | bias | median setup bias | random legal play, L1 / L2 / L3: missed · needless |
|---|---|---|---|---|---|
| 0.04–0.08 | 59.1% (1,250/2,114) | 6.0% (123/2,060) | +53.2 pp | +50.7 pp | 16.6 / 17.4 / 18.6% · 66.0 / 63.2 / 61.7% |
| 0.08–0.15 | 65.4% (1,340/2,048) | 4.7% (179/3,778) | +60.7 pp | +60.0 pp | 17.7 / 17.8 / 20.8% · 65.8 / 64.0 / 62.5% |
| 0.15+ | 54.5% (1,735/3,186) | 2.4% (239/9,938) | +52.1 pp | +47.1 pp | 15.3 / 15.9 / 17.7% · 55.2 / 53.4 / 50.6% |
Swipe for more columns →
| setup (league) | 0.04–0.08 | 0.08–0.15 | 0.15+ |
|---|---|---|---|
| celeris-1-decision (L1) | 33/41 · 0/53 · +80.5 | 41/50 · 0/96 · +82.0 | 63/77 · 4/259 · +80.3 |
| Jev 1.14 (L2) | 37/60 · 1/45 · +59.4 | 49/62 · 4/88 · +74.5 | 53/65 · 5/214 · +79.2 |
| d1-ultra (L2) | 16/37 · 6/57 · +32.7 | 32/65 · 6/90 · +42.6 | 36/69 · 6/267 · +49.9 |
| Claude Sonnet 5.5high (L2) | 28/52 · 1/53 · +52.0 | 36/60 · 0/102 · +60.0 | 39/82 · 1/273 · +47.2 |
| GPT-6.1 Solhigh (L3) | 9/28 (few) · 0/38 · +32.1 | 14/29 (few) · 3/68 · +43.9 | 5/31 · 3/228 · +14.8 |
Swipe for more columns →
| Risk wins: chose safe | in between | unsafe | took the right risk | Safety wins: chose safe | in between | unsafe | |
|---|---|---|---|---|---|---|---|
| setups (pooled) | 0.205 (4,325) | 0.073 (1,617) | 0.095 (1,406) | 53.3% (749/1,406) | 0.035 (13,928) | 0.084 (1,307) | 0.280 (541) |
| GNUbg weak, all leagues | 53.5% (136/254) | ||||||
| Random, L1 | 6.4% (3/47) | ||||||
| celeris-1-decision | 0.222 (137) | 0.055 (22) | 0.055 (9) | 5/9 (few) | 0.052 (387) | 0.093 (17) | 0.304 (4) |
| Jev 1.14 | 0.185 (139) | 0.094 (32) | 0.074 (16) | 37.5% (6/16) (few) | 0.041 (312) | 0.082 (25) | 0.251 (10) |
| d1-ultra | 0.241 (84) | 0.070 (50) | 0.088 (37) | 48.6% (18/37) | 0.042 (352) | 0.072 (44) | 0.243 (18) |
| Claude Sonnet 5.5high | 0.173 (103) | 0.031 (52) | 0.064 (39) | 59.0% (23/39) | 0.018 (395) | 0.043 (31) | 0.150 (2) |
| GPT-6.1 Solhigh | 0.122 (28) | 0.038 (24) | 0.032 (36) | 69.4% (25/36) | 0.012 (292) | 0.013 (36) | 0.191 (6) |
Swipe for more columns →
| setup | 0.04-0.08: missed | 0.04-0.08: needless | 0.04-0.08: bias | 0.08-0.15: missed | 0.08-0.15: needless | 0.08-0.15: bias | 0.15+: missed | 0.15+: needless | 0.15+: bias |
|---|---|---|---|---|---|---|---|---|---|
| celeris-1-decision | 80.5% (33/41) | 0.0% (0/53) | +80.5 pp | 82.0% (41/50) | 0.0% (0/96) | +82.0 pp | 81.8% (63/77) | 1.5% (4/259) | +80.3 pp |
| Cygnet | 82.2% (60/73) | 0.0% (0/35) | +82.2 pp | 75.5% (40/53) | 8.7% (6/69) | +66.8 pp | 71.9% (82/114) | 2.1% (4/193) | +69.9 pp |
| Perplexity Decider Flash v1.2 | 62.2% (28/45) | 15.0% (9/60) | +47.2 pp | 51.7% (15/29) (few) | 18.3% (17/93) | +33.4 pp | 46.5% (40/86) | 11.3% (21/186) | +35.2 pp |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 59.6% (31/52) | 11.4% (4/35) | +48.2 pp | 78.7% (37/47) | 6.8% (6/88) | +71.9 pp | 65.8% (50/76) | 6.9% (17/246) | +58.9 pp |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 58.8% (20/34) | 8.3% (4/48) | +50.5 pp | 70.3% (26/37) | 4.4% (4/91) | +65.9 pp | 68.9% (51/74) | 3.6% (8/222) | +65.3 pp |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 69.4% (34/49) | 0.0% (0/41) | +69.4 pp | 70.8% (34/48) | 1.4% (1/71) | +69.4 pp | 55.7% (54/97) | 1.6% (3/187) | +54.1 pp |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 63.3% (31/49) | 13.0% (7/54) | +50.3 pp | 69.0% (20/29) (few) | 8.0% (7/88) | +61.0 pp | 60.7% (37/61) | 4.7% (13/275) | +55.9 pp |
| GNUbg weak | 10.0% (4/40) | 28.6% (10/35) | -18.6 pp | 0.0% (0/25) (few) | 31.8% (27/85) | -31.8 pp | 0.0% (0/39) | 5.2% (10/192) | -5.2 pp |
| GPT-6 LunaDecisions API | 80.6% (54/67) | 4.3% (2/46) | +76.2 pp | 90.5% (38/42) | 1.2% (1/85) | +89.3 pp | 75.0% (66/88) | 2.0% (4/199) | +73.0 pp |
| Jev 1.13 | 78.6% (33/42) | 0.0% (0/43) | +78.6 pp | 80.4% (45/56) | 0.0% (0/88) | +80.4 pp | 74.7% (65/87) | 1.3% (3/223) | +73.4 pp |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 78.4% (40/51) | 22.0% (9/41) | +56.5 pp | 69.4% (34/49) | 8.9% (7/79) | +60.5 pp | 52.6% (41/78) | 8.2% (13/158) | +44.3 pp |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 65.2% (43/66) | 33.3% (14/42) | +31.8 pp | 61.0% (25/41) | 12.5% (12/96) | +48.5 pp | 44.9% (35/78) | 7.8% (17/218) | +37.1 pp |
| Mercury Decide | 61.3% (38/62) | 2.2% (1/45) | +59.1 pp | 86.8% (46/53) | 1.4% (1/74) | +85.4 pp | 82.4% (84/102) | 1.1% (2/188) | +81.3 pp |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 33.3% (11/33) | 26.3% (10/38) | +7.0 pp | 53.7% (22/41) | 19.4% (18/93) | +34.3 pp | 57.1% (36/63) | 19.4% (34/175) | +37.7 pp |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 60.9% (28/46) | 3.6% (2/55) | +57.2 pp | 78.7% (59/75) | 2.5% (2/80) | +76.2 pp | 78.2% (86/110) | 1.0% (2/196) | +77.2 pp |
| Random | 13.0% (3/23) (few) | 78.3% (18/23) (few) | -65.2 pp | 25.0% (4/16) (few) | 76.1% (35/46) | -51.1 pp | 29.3% (12/41) | 53.2% (33/62) | -24.0 pp |
| Safest | 100.0% (70/70) | 0.0% (0/48) | +100.0 pp | 100.0% (72/72) | 0.0% (0/103) | +100.0 pp | 100.0% (91/91) | 0.0% (0/178) | +100.0 pp |
| SubQ Snap Preview | 47.3% (26/55) | 17.6% (9/51) | +29.6 pp | 61.4% (27/44) | 10.0% (9/90) | +51.4 pp | 51.9% (42/81) | 4.0% (10/247) | +47.8 pp |
| Winnow-12B | 58.7% (27/46) | 12.2% (6/49) | +46.5 pp | 69.7% (46/66) | 6.5% (5/77) | +63.2 pp | 57.0% (53/93) | 6.5% (12/185) | +50.5 pp |
| Random legal play (expected) | 16.6% (156.6/944) | 66.0% (555.4/842) | -49.4 pp | 17.7% (154.9/873) | 65.8% (1047.0/1592) | -48.0 pp | 15.3% (235.6/1536) | 55.2% (2091.6/3789) | -39.9 pp |
Swipe for more columns →
| setup | 0.04-0.08: missed | 0.04-0.08: needless | 0.04-0.08: bias | 0.08-0.15: missed | 0.08-0.15: needless | 0.08-0.15: bias | 0.15+: missed | 0.15+: needless | 0.15+: bias |
|---|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.1low | 50.0% (26/52) | 5.1% (2/39) | +44.9 pp | 69.8% (30/43) | 5.6% (5/90) | +64.2 pp | 24.3% (17/70) | 0.4% (1/256) | +23.9 pp |
| Claude Sonnet 5.5high | 53.8% (28/52) | 1.9% (1/53) | +52.0 pp | 60.0% (36/60) | 0.0% (0/102) | +60.0 pp | 47.6% (39/82) | 0.4% (1/273) | +47.2 pp |
| Claude Sonnet 5.5low | 63.0% (34/54) | 0.0% (0/61) | +63.0 pp | 68.5% (37/54) | 3.1% (3/96) | +65.4 pp | 54.8% (34/62) | 0.0% (0/309) | +54.8 pp |
| Clef | 56.9% (29/51) | 18.9% (10/53) | +38.0 pp | 63.3% (31/49) | 14.4% (15/104) | +48.8 pp | 48.8% (40/82) | 6.0% (11/183) | +42.8 pp |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 56.5% (35/62) | 3.0% (2/66) | +53.4 pp | 64.6% (31/48) | 5.1% (6/118) | +59.5 pp | 31.2% (24/77) | 0.3% (1/288) | +30.8 pp |
| DeepSeek-V4.1-Flashoff · API · 1017 | 66.0% (33/50) | 1.8% (1/56) | +64.2 pp | 76.4% (42/55) | 2.0% (2/99) | +74.3 pp | 73.0% (65/89) | 0.0% (0/240) | +73.0 pp |
| DeepSeek-V4.1-Flashlow · API · 1017 | 55.2% (32/58) | 7.7% (5/65) | +47.5 pp | 59.6% (28/47) | 2.6% (3/114) | +56.9 pp | 40.0% (32/80) | 0.7% (2/297) | +39.3 pp |
| GLiDEAPI | 65.8% (25/38) | 0.0% (0/73) | +65.8 pp | 51.9% (27/52) | 6.2% (6/96) | +45.7 pp | 40.9% (27/66) | 5.0% (12/241) | +35.9 pp |
| GLM-5.3low | 75.5% (40/53) | 0.0% (0/52) | +75.5 pp | 79.1% (34/43) | 1.1% (1/92) | +78.0 pp | 70.1% (68/97) | 0.0% (0/221) | +70.1 pp |
| GNUbg weak | 11.1% (4/36) | 40.7% (22/54) | -29.6 pp | 6.2% (2/32) | 37.1% (36/97) | -30.9 pp | 5.7% (3/53) | 11.6% (30/258) | -6.0 pp |
| Gemini 3.8 Flashhigh | 56.9% (29/51) | 0.0% (0/60) | +56.9 pp | 62.2% (28/45) | 0.0% (0/106) | +62.2 pp | 45.9% (34/74) | 0.3% (1/288) | +45.6 pp |
| Gemini 3.8 Flashlow | 54.5% (30/55) | 4.5% (2/44) | +50.0 pp | 68.8% (22/32) | 2.0% (2/99) | +66.7 pp | 54.7% (35/64) | 0.7% (2/278) | +54.0 pp |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 53.3% (16/30) | 1.9% (1/53) | +51.4 pp | 80.0% (36/45) | 1.2% (1/86) | +78.8 pp | 50.0% (29/58) | 1.7% (4/232) | +48.3 pp |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 59.1% (39/66) | 0.0% (0/53) | +59.1 pp | 68.5% (37/54) | 0.0% (0/96) | +68.5 pp | 47.1% (33/70) | 0.0% (0/286) | +47.1 pp |
| Grok 4.7low | 45.6% (26/57) | 0.0% (0/39) | +45.6 pp | 47.7% (21/44) | 0.0% (0/97) | +47.7 pp | 45.7% (37/81) | 0.0% (0/278) | +45.7 pp |
| Jev 1.14 | 61.7% (37/60) | 2.2% (1/45) | +59.4 pp | 79.0% (49/62) | 4.5% (4/88) | +74.5 pp | 81.5% (53/65) | 2.3% (5/214) | +79.2 pp |
| d1-ultra | 43.2% (16/37) | 10.5% (6/57) | +32.7 pp | 49.2% (32/65) | 6.7% (6/90) | +42.6 pp | 52.2% (36/69) | 2.2% (6/267) | +49.9 pp |
| Qwen3.8 Flashoff | 52.5% (32/61) | 1.8% (1/57) | +50.7 pp | 59.0% (36/61) | 5.1% (4/79) | +54.0 pp | 54.2% (52/96) | 3.0% (6/198) | +51.1 pp |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 63.8% (37/58) | 6.5% (3/46) | +57.3 pp | 70.6% (36/51) | 4.8% (4/83) | +65.8 pp | 55.7% (44/79) | 2.1% (5/239) | +53.6 pp |
| Random legal play (expected) | 17.4% (171.0/981) | 63.2% (648.6/1026) | -45.8 pp | 17.8% (167.8/942) | 63.9% (1171.5/1832) | -46.1 pp | 15.9% (224.5/1414) | 53.4% (2588.3/4846) | -37.5 pp |
Swipe for more columns →
| setup | 0.04-0.08: missed | 0.04-0.08: needless | 0.04-0.08: bias | 0.08-0.15: missed | 0.08-0.15: needless | 0.08-0.15: bias | 0.15+: missed | 0.15+: needless | 0.15+: bias |
|---|---|---|---|---|---|---|---|---|---|
| Claude Fable 5.1high | 41.2% (14/34) | 2.5% (1/40) | +38.7 pp | 46.4% (13/28) (few) | 2.7% (2/75) | +43.8 pp | 22.5% (9/40) | 2.2% (4/180) | +20.3 pp |
| Claude Opus 5.5low | 62.5% (25/40) | 3.3% (1/30) | +59.2 pp | 43.9% (18/41) | 2.6% (2/78) | +41.3 pp | 34.2% (13/38) | 1.1% (2/181) | +33.1 pp |
| Claude Opus 5.5high | 53.1% (17/32) | 4.8% (1/21) (few) | +48.4 pp | 52.6% (20/38) | 4.3% (3/69) | +48.3 pp | 42.6% (20/47) | 0.0% (0/170) | +42.6 pp |
| GNUbg weak | 17.5% (7/40) | 35.7% (10/28) (few) | -18.2 pp | 12.0% (3/25) (few) | 23.4% (15/64) | -11.4 pp | 4.3% (2/46) | 7.2% (11/153) | -2.8 pp |
| GPT-6 Astralow | 31.2% (10/32) | 0.0% (0/29) (few) | +31.2 pp | 58.1% (18/31) | 6.5% (3/46) | +51.5 pp | 31.0% (13/42) | 0.0% (0/186) | +31.0 pp |
| GPT-6 Astramedium | 41.9% (13/31) | 5.4% (2/37) | +36.5 pp | 44.4% (16/36) | 0.0% (0/51) | +44.4 pp | 24.0% (12/50) | 0.0% (0/192) | +24.0 pp |
| GPT-6.1 Solhigh | 32.1% (9/28) (few) | 0.0% (0/38) | +32.1 pp | 48.3% (14/29) (few) | 4.4% (3/68) | +43.9 pp | 16.1% (5/31) | 1.3% (3/228) | +14.8 pp |
| GPT-6.1 Sollow | 30.6% (11/36) | 8.0% (2/25) (few) | +22.6 pp | 43.8% (14/32) | 1.5% (1/67) | +42.3 pp | 25.0% (10/40) | 0.0% (0/210) | +25.0 pp |
| Gemini 4 Argonlow | 69.7% (23/33) | 3.0% (1/33) | +66.7 pp | 60.0% (27/45) | 5.1% (3/59) | +54.9 pp | 35.1% (13/37) | 1.5% (3/205) | +33.7 pp |
| Gemini 4 Argonhigh | 44.1% (15/34) | 3.2% (1/31) | +40.9 pp | 40.6% (13/32) | 0.0% (0/67) | +40.6 pp | 42.9% (15/35) | 0.0% (0/139) | +42.9 pp |
| Grok 4.7high | 61.5% (24/39) | 2.4% (1/41) | +59.1 pp | 61.5% (24/39) | 5.8% (3/52) | +55.8 pp | 41.0% (25/61) | 0.0% (0/157) | +41.0 pp |
| Kimi K3.1 Prooff | 42.1% (8/19) (few) | 3.7% (1/27) (few) | +38.4 pp | 55.6% (15/27) (few) | 1.9% (1/53) | +53.7 pp | 41.0% (16/39) | 2.1% (3/145) | +39.0 pp |
| Random legal play (expected) | 18.6% (74.1/398) | 61.7% (234.6/380) | -43.1 pp | 20.8% (83.9/403) | 62.5% (468.3/749) | -41.7 pp | 17.7% (89.4/506) | 50.6% (1086.8/2146) | -33.0 pp |
Swipe for more columns →
| setup | Risk wins: safe | Risk wins: in between | Risk wins: unsafe | Risk wins: took the right risk | Safety wins: safe | Safety wins: in between | Safety wins: unsafe |
|---|---|---|---|---|---|---|---|
| celeris-1-decision | 0.222 (137) | 0.055 (22) | 0.055 (9) | 55.6% (5/9) | 0.052 (387) | 0.093 (17) | 0.304 (4) |
| Cygnet | 0.217 (182) | 0.064 (33) | 0.123 (25) | 36.0% (9/25) | 0.038 (268) | 0.140 (19) | 0.216 (10) |
| Perplexity Decider Flash v1.2 | 0.229 (83) | 0.170 (25) | 0.254 (52) | 17.3% (9/52) | 0.074 (258) | 0.130 (34) | 0.333 (47) |
| Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0.223 (118) | 0.084 (29) | 0.110 (28) | 50.0% (14/28) | 0.039 (303) | 0.124 (39) | 0.381 (27) |
| Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell | 0.216 (97) | 0.065 (24) | 0.114 (24) | 37.5% (9/24) | 0.051 (306) | 0.081 (39) | 0.283 (16) |
| DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release | 0.206 (122) | 0.111 (51) | 0.117 (21) | 57.1% (12/21) | 0.050 (264) | 0.092 (31) | 0.294 (4) |
| GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell | 0.254 (88) | 0.069 (24) | 0.097 (27) | 48.1% (13/27) | 0.058 (361) | 0.075 (29) | 0.244 (27) |
| GNUbg weak | 0.058 (4) | 0.020 (17) | 0.058 (83) | 53.0% (44/83) | 0.013 (190) | 0.042 (75) | 0.192 (47) |
| GPT-6 LunaDecisions API | 0.218 (158) | 0.105 (28) | 0.122 (11) | 27.3% (3/11) | 0.044 (295) | 0.138 (28) | 0.171 (7) |
| Jev 1.13 | 0.220 (143) | 0.068 (25) | 0.193 (17) | 47.1% (8/17) | 0.040 (335) | 0.031 (16) | 0.190 (3) |
| LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell | 0.192 (115) | 0.083 (16) | 0.208 (47) | 36.2% (17/47) | 0.065 (212) | 0.164 (37) | 0.330 (29) |
| LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0.202 (103) | 0.087 (25) | 0.197 (57) | 35.1% (20/57) | 0.052 (280) | 0.134 (33) | 0.342 (43) |
| Mercury Decide | 0.273 (168) | 0.102 (35) | 0.173 (14) | 28.6% (4/14) | 0.053 (272) | 0.113 (31) | 0.227 (4) |
| Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell | 0.265 (69) | 0.075 (23) | 0.220 (45) | 28.9% (13/45) | 0.080 (209) | 0.158 (35) | 0.378 (62) |
| Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0.250 (173) | 0.134 (48) | 0.088 (10) | 40.0% (4/10) | 0.056 (309) | 0.062 (16) | 0.172 (6) |
| Random | 0.429 (19) | 0.339 (14) | 0.283 (47) | 6.4% (3/47) | 0.106 (29) | 0.168 (16) | 0.438 (86) |
| Safest | 0.227 (233) | – (0) | – (0) | – (0) | 0.082 (329) | – (0) | – (0) |
| SubQ Snap Preview | 0.295 (95) | 0.151 (46) | 0.165 (39) | 38.5% (15/39) | 0.052 (311) | 0.129 (49) | 0.293 (28) |
| Winnow-12B | 0.250 (126) | 0.110 (43) | 0.120 (36) | 44.4% (16/36) | 0.078 (261) | 0.109 (27) | 0.255 (23) |
Swipe for more columns →
| setup | Risk wins: safe | Risk wins: in between | Risk wins: unsafe | Risk wins: took the right risk | Safety wins: safe | Safety wins: in between | Safety wins: unsafe |
|---|---|---|---|---|---|---|---|
| Claude Fable 5.1low | 0.148 (73) | 0.056 (52) | 0.047 (40) | 62.5% (25/40) | 0.028 (355) | 0.074 (22) | 0.146 (8) |
| Claude Sonnet 5.5high | 0.173 (103) | 0.031 (52) | 0.064 (39) | 59.0% (23/39) | 0.018 (395) | 0.043 (31) | 0.150 (2) |
| Claude Sonnet 5.5low | 0.180 (105) | 0.036 (46) | 0.074 (19) | 52.6% (10/19) | 0.028 (448) | 0.053 (15) | 0.115 (3) |
| Clef | 0.191 (100) | 0.086 (30) | 0.115 (52) | 38.5% (20/52) | 0.044 (256) | 0.095 (48) | 0.189 (36) |
| DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release | 0.140 (90) | 0.085 (47) | 0.065 (50) | 58.0% (29/50) | 0.024 (418) | 0.045 (45) | 0.151 (9) |
| DeepSeek-V4.1-Flashoff · API · 1017 | 0.246 (140) | 0.099 (37) | 0.140 (17) | 58.8% (10/17) | 0.036 (347) | 0.075 (45) | 0.263 (3) |
| DeepSeek-V4.1-Flashlow · API · 1017 | 0.165 (92) | 0.059 (57) | 0.040 (36) | 75.0% (27/36) | 0.018 (420) | 0.035 (46) | 0.217 (10) |
| GLiDEAPI | 0.171 (79) | 0.123 (34) | 0.063 (43) | 55.8% (24/43) | 0.043 (346) | 0.127 (46) | 0.277 (18) |
| GLM-5.3low | 0.218 (142) | 0.066 (38) | 0.109 (13) | 53.8% (7/13) | 0.033 (331) | 0.057 (33) | 0.108 (1) |
| GNUbg weak | 0.199 (9) | 0.052 (25) | 0.060 (87) | 58.6% (51/87) | 0.022 (213) | 0.066 (108) | 0.207 (88) |
| Gemini 3.8 Flashhigh | 0.163 (91) | 0.044 (46) | 0.048 (33) | 60.6% (20/33) | 0.012 (435) | 0.011 (18) | 0.794 (1) |
| Gemini 3.8 Flashlow | 0.172 (87) | 0.014 (30) | 0.032 (34) | 70.6% (24/34) | 0.022 (404) | 0.048 (11) | 0.209 (6) |
| Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0.177 (81) | 0.047 (30) | 0.098 (22) | 59.1% (13/22) | 0.029 (353) | 0.080 (12) | 0.246 (6) |
| Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0.163 (109) | 0.038 (41) | 0.056 (40) | 62.5% (25/40) | 0.029 (418) | 0.064 (17) | – (0) |
| Grok 4.7low | 0.188 (84) | 0.061 (63) | 0.052 (35) | 60.0% (21/35) | 0.017 (376) | 0.100 (38) | – (0) |
| Jev 1.14 | 0.185 (139) | 0.094 (32) | 0.074 (16) | 37.5% (6/16) | 0.041 (312) | 0.082 (25) | 0.251 (10) |
| d1-ultra | 0.241 (84) | 0.070 (50) | 0.088 (37) | 48.6% (18/37) | 0.042 (352) | 0.072 (44) | 0.243 (18) |
| Qwen3.8 Flashoff | 0.223 (120) | 0.113 (55) | 0.126 (43) | 27.9% (12/43) | 0.038 (277) | 0.119 (46) | 0.340 (11) |
| Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark | 0.218 (117) | 0.061 (41) | 0.074 (30) | 60.0% (18/30) | 0.040 (337) | 0.076 (19) | 0.203 (12) |
Swipe for more columns →
| setup | Risk wins: safe | Risk wins: in between | Risk wins: unsafe | Risk wins: took the right risk | Safety wins: safe | Safety wins: in between | Safety wins: unsafe |
|---|---|---|---|---|---|---|---|
| Claude Fable 5.1high | 0.131 (36) | 0.019 (33) | 0.059 (33) | 60.6% (20/33) | 0.016 (264) | 0.043 (24) | 0.224 (7) |
| Claude Opus 5.5low | 0.143 (56) | 0.034 (31) | 0.043 (32) | 75.0% (24/32) | 0.022 (260) | 0.056 (24) | 0.199 (5) |
| Claude Opus 5.5high | 0.171 (57) | 0.041 (35) | 0.033 (25) | 68.0% (17/25) | 0.010 (242) | 0.017 (14) | 0.135 (4) |
| GNUbg weak | 0.163 (12) | 0.023 (15) | 0.074 (84) | 48.8% (41/84) | 0.021 (153) | 0.102 (56) | 0.191 (36) |
| GPT-6 Astralow | 0.157 (41) | 0.027 (32) | 0.026 (32) | 75.0% (24/32) | 0.010 (229) | 0.009 (29) | 0.160 (3) |
| GPT-6 Astramedium | 0.129 (41) | 0.049 (45) | 0.023 (31) | 74.2% (23/31) | 0.012 (253) | 0.022 (25) | 0.081 (2) |
| GPT-6.1 Solhigh | 0.122 (28) | 0.038 (24) | 0.032 (36) | 69.4% (25/36) | 0.012 (292) | 0.013 (36) | 0.191 (6) |
| GPT-6.1 Sollow | 0.163 (35) | 0.032 (35) | 0.027 (38) | 65.8% (25/38) | 0.012 (268) | 0.044 (31) | 0.072 (3) |
| Gemini 4 Argonlow | 0.148 (63) | 0.048 (24) | 0.035 (28) | 71.4% (20/28) | 0.021 (271) | 0.052 (19) | 0.286 (7) |
| Gemini 4 Argonhigh | 0.181 (43) | 0.039 (21) | 0.009 (37) | 83.8% (31/37) | 0.018 (219) | 0.029 (17) | 0.045 (1) |
| Grok 4.7high | 0.162 (73) | 0.100 (37) | 0.072 (29) | 72.4% (21/29) | 0.013 (222) | 0.072 (24) | 0.105 (4) |
| Kimi K3.1 Prooff | 0.195 (39) | 0.110 (22) | 0.048 (24) | 66.7% (16/24) | 0.035 (197) | 0.152 (23) | 0.487 (5) |
Preliminary / partial
How this part was done and checked *
- The best legal play is in between (1 to 11 hitting rolls) in 22% of Risk wins and 16% of Safety wins: in about one decision in five or six, neither of the two plays the scan compares is the best play.
- Bands are the absolute 3-ply gap: 0.04-0.08, 0.08-0.15 and 0.15+ (lower edge included). Setups (pooled) = the 45 setups other than the three programs, Random, Safest and GNUbg weak; each decision counts once.
- Missed risks do not fall as the gap grows (59%, 65%, 55%); needless risks do (6.0%, 4.7%, 2.4%).
- Equity lost = 2-ply equity of the best legal play minus that of the chosen play, mean (count). Took the right risk: a Risk win played unsafe, within 0.02 of the best unsafe play.
- Not paired: the five Part-2 models played in different leagues.
Rollout check of a 200-position sample
Swipe for more columns →
| group | 0.04–0.08 | 0.08–0.15 | 0.15+ | all | not confirmed: why |
|---|---|---|---|---|---|
| Risk wins | 79% (27/34) | 100% (33/33) | 100% (33/33) | 93% (93/100) | gap < 0.04: 7 |
| Safety wins | 88% (30/34) | 100% (33/33) | 100% (33/33) | 96% (96/100) | gap < 0.04: 4 |
| both | 84% (57/68) | 100% (66/66) | 100% (66/66) | 94.5% (189/200) | gap < 0.04: 11 |
Swipe for more columns →
| positions | Risk wins | Safety wins | all | not confirmed: why |
|---|---|---|---|---|
| back game: in the 200 | 67% (2/3) | 100% (5/5) | 88% (7/8) | gap < 0.04: 1 (sample 13) |
| back game: oversample (30) | 100% (8/8) | 100% (22/22) | 100% (30/30) | – |
| back game: both | 91% (10/11) | 100% (29/29) | 98% (39/40) | gap < 0.04: 1 |
| prime-vs-prime: in the 200 | 100% (6/6) | 100% (3/3) | 100% (9/9) | – |
| prime-vs-prime: oversample (30) | 100% (2/2) | 100% (28/28) | 100% (30/30) | – |
| prime-vs-prime: both | 100% (8/8) | 100% (32/32) | 100% (40/40) | – |
| neither, in the 200 | 93% (85/91) | 96% (88/92) | 95% (173/183) | gap < 0.04: 10 |
Preliminary / partial
How this part was done and checked *
- Sample: 200 kept positions drawn with a fixed seed, 100 Risk wins and 100 Safety wins, about a third per gap band; Bray position excluded. Oversample: 30 back-game and 30 prime-vs-prime kept positions not in the 200 (fixed seed), reported separately and not used in Part 2.
- Settings: GNU Backgammon 1.08.003, 1,296 games per play, variance reduction, quasi-random dice, same seed for both plays, no truncation, cubeless money play with gammons and backgammons counted (1/2/3 points), Jacoby off, 2-ply checker play (cubeless, prune, no noise). Each play is rolled out from the position after it, opponent on roll.
- Confirmed = same direction as the scan, rollout gap 0.04 or more, and more than 3 standard errors of the gap. The rule, the settings and the oversample were committed before the first rollout.
- 189 of 200 confirmed (94.5%): 93 of 100 Risk wins, 96 of 100 Safety wins. No position flipped; all 11 misses are in the 0.04-0.08 band with a smaller gap in the same direction. Back game 39 of 40, prime-vs-prime 40 of 40 (sample and oversample together); most of those are large-gap Safety wins already kept by the 3-ply check, so this says nothing about the positions the check dropped.
Part 2: asking the models
Swipe for more columns →
| model | version | Risk wins right | Safety wins right | missed (Risk wins → safe) | needless (Safety wins → unsafe) | bias | league: missed · needless · bias (all legal plays) | share of A answers | Q1 not answered |
|---|---|---|---|---|---|---|---|---|---|
| celeris-1-decision | with numbers | 6% (6/93) | 98% (94/96) | 94% (87/93) | 2% (2/96) | +91 pp | 82% (137/168) · 1% (4/408) · +81 pp | 47% (88/189) | 0 |
| celeris-1-decision | board only | 75% (69/92) | 61% (57/93) | 25% (23/92) | 39% (36/93) | -14 pp | 48% (88/185) | 4 | |
| Jev 1.14 | with numbers | 4% (4/91) | 97% (93/96) | 96% (87/91) | 3% (3/96) | +92 pp | 74% (139/187) · 3% (10/347) · +71 pp | 50% (94/187) | 2 |
| Jev 1.14 | board only | 67% (62/93) | 45% (41/92) | 33% (31/93) | 55% (51/92) | -22 pp | 47% (87/185) | 4 | |
| d1-ultra | with numbers | 19% (18/93) | 94% (90/96) | 81% (75/93) | 6% (6/96) | +74 pp | 49% (84/171) · 4% (18/414) · +45 pp | 51% (96/189) | 0 |
| d1-ultra | board only | 68% (63/93) | 47% (41/87) | 32% (30/93) | 53% (46/87) | -21 pp | 62% (112/180) | 9 | |
| Claude Sonnet 5.5high | with numbers | 32% (30/93) | 98% (94/96) | 68% (63/93) | 2% (2/96) | +66 pp | 53% (103/194) · 0% (2/428) · +53 pp | 50% (94/189) | 0 |
| Claude Sonnet 5.5high | board only | 52% (48/93) | 94% (90/96) | 48% (45/93) | 6% (6/96) | +42 pp | 54% (102/189) | 0 | |
| GPT-6.1 Solhigh | with numbers | 44% (41/93) | 98% (94/96) | 56% (52/93) | 2% (2/96) | +54 pp | 32% (28/88) · 2% (6/334) · +30 pp | 50% (95/189) | 0 |
| GPT-6.1 Solhigh | board only | 77% (72/93) | 81% (78/96) | 23% (21/93) | 19% (18/96) | +4 pp | 46% (86/189) | 0 | |
| Kimi K3.1 Prooff | with numbers | 24% (22/93) | 94% (90/96) | 76% (71/93) | 6% (6/96) | +70 pp | 46% (39/85) · 2% (5/225) · +44 pp | 49% (92/189) | 0 |
| Kimi K3.1 Prooff | board only | 70% (65/93) | 45% (43/96) | 30% (28/93) | 55% (53/96) | -25 pp | 65% (122/189) | 0 | |
| GLM-5.3low | with numbers | 14% (13/93) | 92% (88/96) | 86% (80/93) | 8% (8/96) | +78 pp | 74% (142/193) · 0% (1/365) · +73 pp | 59% (111/189) | 0 |
| GLM-5.3low | board only | 51% (47/93) | 69% (66/96) | 49% (46/93) | 31% (30/96) | +18 pp | 65% (123/189) | 0 | |
| Claude Opus 5.5high | with numbers | 40% (37/93) | 96% (92/96) | 60% (56/93) | 4% (4/96) | +56 pp | 49% (57/117) · 2% (4/260) · +47 pp | 48% (91/189) | 0 |
| Claude Opus 5.5high | board only | 49% (46/93) | 96% (92/96) | 51% (47/93) | 4% (4/96) | +46 pp | 48% (90/189) | 0 |
Swipe for more columns →
| model | right | random baseline |
|---|---|---|
| celeris-1-decision | 92% (55/60) | 25% (4 options, one right) |
| Jev 1.14 | 62% (37/60) | 25% (4 options, one right) |
| d1-ultra | 92% (55/60) | 25% (4 options, one right) |
| Claude Sonnet 5.5high | 100% (60/60) | 25% (4 options, one right) |
| GPT-6.1 Solhigh | 100% (60/60) | 25% (4 options, one right) |
| Kimi K3.1 Prooff | 92% (55/60) | 25% (4 options, one right) |
| GLM-5.3low | 95% (57/60) | 25% (4 options, one right) |
| Claude Opus 5.5high | 100% (60/60) | 25% (4 options, one right) |
Swipe for more columns →
| model | version | all pairs | safe-vs-unsafe pair |
|---|---|---|---|
| celeris-1-decision | with numbers | 10% (87/834) | 10% (18/189) |
| celeris-1-decision | board only | 6% (46/834) | 5% (10/189) |
| Jev 1.14 | with numbers | 17% (138/834) | 17% (32/189) |
| Jev 1.14 | board only | 7% (59/834) | 11% (21/189) |
| d1-ultra | with numbers | 19% (157/834) | 17% (33/189) |
| d1-ultra | board only | 6% (49/834) | 5% (9/189) |
| Claude Sonnet 5.5high | with numbers | 81% (674/834) | 62% (118/189) |
| Claude Sonnet 5.5high | board only | 82% (682/834) | 68% (129/189) |
| GPT-6.1 Solhigh | with numbers | 89% (742/834) | 72% (137/189) |
| GPT-6.1 Solhigh | board only | 88% (736/834) | 79% (150/189) |
| Kimi K3.1 Prooff | with numbers | 75% (622/834) | 55% (104/189) |
| Kimi K3.1 Prooff | board only | 55% (458/834) | 54% (103/189) |
| GLM-5.3low | with numbers | 73% (610/834) | 55% (104/189) |
| GLM-5.3low | board only | 66% (552/834) | 53% (100/189) |
| Claude Opus 5.5high | with numbers | 82% (681/834) | 63% (120/189) |
| Claude Opus 5.5high | board only | 83% (691/834) | 70% (132/189) |
| random one-to-one mapping (chat-style) | 50% | 50% | |
| random play per value, independently (decision-style) | 10% | 11% |
Swipe for more columns →
| model | version | Risk wins with a safe-vs-unsafe Q3 pair | sees the value, plays safe (Q3 pair right, Q1 safe) | sees the value, takes the risk (Q3 right, Q1 unsafe) | doesn't see the value (Q3 pair wrong) | doesn't know the game (Q2 and Q3 wrong; of Q3-wrong items with a Q2 item) |
|---|---|---|---|---|---|---|
| celeris-1-decision | with numbers | 93 | 2 | 0 | 91 | 1 of 38 |
| celeris-1-decision | board only | 93 | 1 | 2 | 90 | 1 of 37 |
| Jev 1.14 | with numbers | 93 | 1 | 0 | 91 | 11 of 36 |
| Jev 1.14 | board only | 93 | 5 | 8 | 80 | 10 of 30 |
| d1-ultra | with numbers | 93 | 0 | 0 | 93 | 1 of 38 |
| d1-ultra | board only | 93 | 0 | 4 | 89 | 1 of 36 |
| Claude Sonnet 5.5high | with numbers | 93 | 6 | 17 | 70 | 0 of 26 |
| Claude Sonnet 5.5high | board only | 93 | 6 | 36 | 51 | 0 of 17 |
| GPT-6.1 Solhigh | with numbers | 93 | 16 | 30 | 47 | 0 of 19 |
| GPT-6.1 Solhigh | board only | 93 | 12 | 57 | 24 | 0 of 15 |
| Kimi K3.1 Prooff | with numbers | 93 | 10 | 9 | 74 | 0 of 31 |
| Kimi K3.1 Prooff | board only | 93 | 14 | 49 | 30 | 0 of 14 |
| GLM-5.3low | with numbers | 93 | 7 | 6 | 80 | 1 of 31 |
| GLM-5.3low | board only | 93 | 10 | 24 | 59 | 1 of 21 |
| Claude Opus 5.5high | with numbers | 93 | 4 | 22 | 67 | 0 of 24 |
| Claude Opus 5.5high | board only | 93 | 7 | 33 | 53 | 0 of 18 |
Swipe for more columns →
| model | expected requests | recorded answer rows | answer rows with a request error | usable answers | item-level flags (see notes) | exact ties at the top (no answer) | input tokens | output tokens | reasoning tokens | cost |
|---|---|---|---|---|---|---|---|---|---|---|
| celeris-1-decision | 1,782 | 1,782 | 0 | 1,756 | 333 Q3 items with two values on one play | 26 | 4,759,594 | 0 | 0 | $0.19 |
| Jev 1.14 | 1,782 | 1,782 | 0 | 1,741 | 315 Q3 items with two values on one play | 41 | 5,186,004 | 140,677 | 0 | $0.22 |
| d1-ultra | 1,782 | 1,782 | 0 | 1,683 | 287 Q3 items with two values on one play | 99 | 4,364,915 | 0 | 0 | $0.18 |
| Claude Sonnet 5.5high | 816 | 816 | 0 | 816 | 0 | 0 | 2,845,140 | 506,104 | 492,088 | $10.75 API-equiv. |
| GPT-6.1 Solhigh | 816 | 816 | 1 | 814 | failed or missing 1 | 0 | 2,027,910 | 447,317 | 429,686 | $8.53 API-equiv. |
| Kimi K3.1 Prooff | 816 | 816 | 0 | 816 | 0 | 0 | 2,091,754 | 16,300 | 0 | $1.39 API-equiv. |
| GLM-5.3low | 816 | 816 | 0 | 809 | invalid 7 | 0 | 1,990,210 | 787,791 | 607,727 | $4.30 API-equiv. |
| Claude Opus 5.5high | 816 | 816 | 0 | 816 | 0 | 0 | 2,845,140 | 393,825 | 379,806 | $19.26 API-equiv. |
Preliminary / partial
How this part was done and checked *
- Two or four plays shown, not league play.
- Positions: the 189 rollout-confirmed sample positions (93 Risk wins, 96 Safety wins). Each question is its own request with no shared context. Not paired across models beyond these shared items, and the league comparison is unpaired (the models played in different leagues on different positions).
- Q1: best safe vs best unsafe play, 'Which play is better?'; half the right answers are A (fixed seed). Q2 (board only, 60 items where exactly one option is true of the unsafe play B and not of the safe play A): what does play B do that play A does not; random baseline 25%. Q3: 3 or 4 plays (best safe, best unsafe, best in-between if any, one clear blunder at least 0.10 below the others at 3-ply if any) and their 3-ply values (2 decimals, labelled W-Z); chat models match each value to a play (each play once), decision models get one question per value.
- Two versions: with numbers (the league prompt with the computed consequences of each play) and board only (notation only, no pip count or other computed number).
- Models at league settings; temperature not sent (provider default) for all. Decision models: the top-probability option is the answer; an exact tie at the top is no answer. Q3 scoring: only pairs 0.04 or more apart at 3-ply; a decision model that puts two values on one play gets the pairs of that play wrong; a chat answer that is not one-to-one counts every pair wrong.
- All models asked so far are complete. GPT-6.1 Solhigh here is separate from the Codex effort sweep, whose GPT-6.1 Sol (max) row is a different run. Added after the first publication (Zeki / Ümit, 30 Oct): Kimi K3.1 Prooff and GLM-5.3low via OpenRouter and Claude Opus 5.5high on the Claude subscription, at league settings, same items and frozen scoring. GLM-5.3low (v3 setup name GLM-5.3 (on)) stopped at its $3.00 cost cap with 558 of 816 answers and was resumed with a $4.60 total cap, asking only the missing items on the same OpenRouter route (Fireworks), model id and reasoning setting.
- Q3 random baselines were added AFTER the first score output: first score 2026-10-30T06:25:29.387Z, baseline edit 2026-10-30T06:25:46.496Z, baseline output 2026-10-30T06:25:50.444Z. Not pre-registered and not exact: a simulation with seed 2026103005, 200 draws per item (4000 // 20), rounded to whole percent. Scoring was not changed.
- The cross-table's 'doesn't know the game' uses the board-only Q2 answer for both versions (Q2 was asked board only).
- Denominators: expected = items sent per model; recorded = answer rows stored; usable = recorded minus rows with a request error, invalid answers and exact ties. 'Item-level flags' for decision models count Q3 items (position and version) where two values went to one play, not pairs and not failed requests. Answer-row error counts are rows whose request failed after retries, not attempts.
- Cost: hosted decision models at list price (the cash cap was $3; the stored list-price total is about $0.59); OpenRouter models at the cost OpenRouter reports per response (cap $3.00 per model, raised to $4.60 for GLM-5.3low); subscription models shown as API-equivalent, not money paid.
Status
Preliminary / partial
Preliminary / partial. Two or four plays shown, not league play. Data: clear-risk.json · Method
The scan, its checks and the rollout check are as of results 1082e024 (league games up to then); they are not recomputed with later league games.
- Hand-check position lists: not published (position-to-choice records stay private).
Models are too cautious: the clear-risk scan
In the clear-risk scan (25,199 league decisions where the engine's call is clear and stable), every one of the 45 model setups missed more risks than it took needlessly: its bias (missed minus needless) runs from +30.0 pp (GPT-6.1 Solhigh, League 3) to +80.6 pp (celeris-1-decision, League 1). The programs are not like that: GNUbg weak's bias is negative in all three leagues (-11.2 pp, -14.1 pp, -3.9 pp), and so is Random's (-41.9 pp); Safest, which never takes a risk, is +100.0 pp by construction.
Bias = missed risks (a clear Risk win played safe) minus needless risks (a clear Safety win played unsafe), in percentage points; right of 0 = too cautious program (reference row)
Not paired: setups met different positions, so the rows compare only loosely. More caveats: Method