Preliminary / partial

Clear risk: when one play is clearly better

Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.

The scan: which decisions have a clear risk call

Swipe for more columns →

Steps 2-3: league decisions, skipped by the first reason that applies
League 1League 2League 3Total
league decisions36,99539,53816,35592,888
skip: invalid answer (fallback)0503686
skip: forced move (1 play)0000
skip: 2 legal plays3,6613,8191,3858,865
skip: no contact (race or bear-off)5,1845,6772,49913,360
skip: safe bin empty10,82810,0954,07024,993
skip: unsafe bin empty1,9372,5891,2275,753
scanned (both bins non-empty)15,38517,3087,13839,831
Risk wins (gap ≥ +0.04)4,4984,3041,64510,447
Safety wins (gap ≤ −0.04)8,1869,9604,24722,393
not flagged (|gap| < 0.04)2,7013,0441,2466,991
flagged12,68414,2645,89232,840

Swipe for more columns →

Step 4: kept and unstable decisions per league and group
leaguegroupflaggedkeptunstable: direction flipsunstable: |gap3| < 0.04unstable: |gap3 − gap2| > 0.02
1Risk wins4,4983,3535167973
2Risk wins4,3043,3372141824
3Risk wins1,6451,307165272
allRisk wins10,4477,99783732,069
1Safety wins8,1866,22331761,784
2Safety wins9,9607,70441872,065
3Safety wins4,2473,275265905
allSafety wins22,39317,20294284,754
allboth32,84025,199178016,823

Preliminary / partial

How this part was done and checked *
  • A decision is scanned when the mover has both a safe play (0 hitting rolls for the opponent) and an unsafe play (12 or more hitting rolls), there is contact, and there are more than 2 legal plays. Invalid answers (fallback) are skipped.
  • Gap = 2-ply cubeless equity of the best unsafe play minus that of the best safe play (GNU Backgammon). Risk wins: gap of +0.04 or more. Safety wins: gap of -0.04 or less. Both are 'flagged'.
  • 3-ply stability check (it replaced rollouts for the leagues): a flagged decision is kept only if the gap is 0.04 or more at both 2-ply and 3-ply, has the same sign at both, and the two gaps differ by 0.02 or less. It is a stability signal, not a rollout.
  • Kept: 25,199 of 32,840 flagged decisions (7,997 Risk wins, 17,202 Safety wins).
  • The method was written down and committed before any computation; the 3-ply rule was committed before any 3-ply value was computed.

Missed and needless risks per setup

Swipe for more columns →

League 1: per setup, kept decisions only
setupRisk wins keptmissed risks (played safe)in-between (Risk wins)Safety wins keptneedless risks (played unsafe)in-between (Safety wins)bias = missed − needlessthread: risks found
celeris-1-decision16881.5% (137/168)13.1% (22/168)4081.0% (4/408)4.2% (17/408)+80.6 pp23.9% (163/683)
Cygnet24075.8% (182/240)13.8% (33/240)2973.4% (10/297)6.4% (19/297)+72.5 pp27.3% (205/750)
Perplexity Decider Flash v1.216051.9% (83/160)15.6% (25/160)33913.9% (47/339)10.0% (34/339)+38.0 pp31.3% (223/712)
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark17567.4% (118/175)16.6% (29/175)3697.3% (27/369)10.6% (39/369)+60.1 pp33.1% (233/703)
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell14566.9% (97/145)16.6% (24/145)3614.4% (16/361)10.8% (39/361)+62.5 pp33.4% (244/731)
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release19462.9% (122/194)26.3% (51/194)2991.3% (4/299)10.4% (31/299)+61.5 pp29.1% (192/659)
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell13963.3% (88/139)17.3% (24/139)4176.5% (27/417)7.0% (29/417)+56.8 pp33.0% (219/664)
GNUbg weak1043.8% (4/104)16.3% (17/104)31215.1% (47/312)24.0% (75/312)-11.2 pp66.6% (410/616)
GPT-6 LunaDecisions API19780.2% (158/197)14.2% (28/197)3302.1% (7/330)8.5% (28/330)+78.1 pp21.6% (160/740)
Jev 1.1318577.3% (143/185)13.5% (25/185)3540.8% (3/354)4.5% (16/354)+76.4 pp27.0% (190/705)
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell17864.6% (115/178)9.0% (16/178)27810.4% (29/278)13.3% (37/278)+54.2 pp28.8% (206/715)
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark18555.7% (103/185)13.5% (25/185)35612.1% (43/356)9.3% (33/356)+43.6 pp31.5% (248/788)
Mercury Decide21777.4% (168/217)16.1% (35/217)3071.3% (4/307)10.1% (31/307)+76.1 pp18.7% (129/690)
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell13750.4% (69/137)16.8% (23/137)30620.3% (62/306)11.4% (35/306)+30.1 pp37.3% (264/708)
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark23174.9% (173/231)20.8% (48/231)3311.8% (6/331)4.8% (16/331)+73.1 pp23.5% (178/756)
Random8023.8% (19/80)17.5% (14/80)13165.6% (86/131)12.2% (16/131)-41.9 pp24.8% (112/452)
Safest233100.0% (233/233)0.0% (0/233)3290.0% (0/329)0.0% (0/329)+100.0 pp0.0% (0/785)
SubQ Snap Preview18052.8% (95/180)25.6% (46/180)3887.2% (28/388)12.6% (49/388)+45.6 pp37.9% (297/783)
Winnow-12B20561.5% (126/205)21.0% (43/205)3117.4% (23/311)8.7% (27/311)+54.1 pp33.7% (263/780)
Safest program (reference)3353100.0% (3353/3353)0.0% (0/3353)62230.0% (0/6223)0.0% (0/6223)+100.0 pp
Random legal play (expected)335316.3% (547.1/3353)11.5% (384.9/3353)622359.4% (3694.0/6223)17.2% (1068.4/6223)-43.0 pp

Swipe for more columns →

League 2: per setup, kept decisions only
setupRisk wins keptmissed risks (played safe)in-between (Risk wins)Safety wins keptneedless risks (played unsafe)in-between (Safety wins)bias = missed − needlessthread: risks found
Claude Fable 5.1low16544.2% (73/165)31.5% (52/165)3852.1% (8/385)5.7% (22/385)+42.2 pp41.0% (270/658)
Claude Sonnet 5.5high19453.1% (103/194)26.8% (52/194)4280.5% (2/428)7.2% (31/428)+52.6 pp41.8% (308/737)
Claude Sonnet 5.5low17061.8% (105/170)27.1% (46/170)4660.6% (3/466)3.2% (15/466)+61.1 pp30.9% (204/661)
Clef18254.9% (100/182)16.5% (30/182)34010.6% (36/340)14.1% (48/340)+44.4 pp39.5% (263/666)
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release18748.1% (90/187)25.1% (47/187)4721.9% (9/472)9.5% (45/472)+46.2 pp45.0% (338/751)
DeepSeek-V4.1-Flashoff · API · 101719472.2% (140/194)19.1% (37/194)3950.8% (3/395)11.4% (45/395)+71.4 pp25.4% (181/713)
DeepSeek-V4.1-Flashlow · API · 101718549.7% (92/185)30.8% (57/185)4762.1% (10/476)9.7% (46/476)+47.6 pp42.5% (321/755)
GLiDEAPI15650.6% (79/156)21.8% (34/156)4104.4% (18/410)11.2% (46/410)+46.3 pp36.9% (249/675)
GLM-5.3low19373.6% (142/193)19.7% (38/193)3650.3% (1/365)9.0% (33/365)+73.3 pp30.7% (205/668)
GNUbg weak1217.4% (9/121)20.7% (25/121)40921.5% (88/409)26.4% (108/409)-14.1 pp62.7% (483/770)
Gemini 3.8 Flashhigh17053.5% (91/170)27.1% (46/170)4540.2% (1/454)4.0% (18/454)+53.3 pp36.1% (264/731)
Gemini 3.8 Flashlow15157.6% (87/151)19.9% (30/151)4211.4% (6/421)2.6% (11/421)+56.2 pp33.8% (214/633)
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark13360.9% (81/133)22.6% (30/133)3711.6% (6/371)3.2% (12/371)+59.3 pp33.2% (194/584)
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark19057.4% (109/190)21.6% (41/190)4350.0% (0/435)3.9% (17/435)+57.4 pp31.9% (223/700)
Grok 4.7low18246.2% (84/182)34.6% (63/182)4140.0% (0/414)9.2% (38/414)+46.2 pp42.3% (295/697)
Jev 1.1418774.3% (139/187)17.1% (32/187)3472.9% (10/347)7.2% (25/347)+71.4 pp29.1% (184/632)
d1-ultra17149.1% (84/171)29.2% (50/171)4144.3% (18/414)10.6% (44/414)+44.8 pp42.0% (289/688)
Qwen3.8 Flashoff21855.0% (120/218)25.2% (55/218)3343.3% (11/334)13.8% (46/334)+51.8 pp34.4% (243/706)
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark18862.2% (117/188)21.8% (41/188)3683.3% (12/368)5.2% (19/368)+59.0 pp33.8% (242/715)
Safest program (reference)3337100.0% (3337/3337)0.0% (0/3337)77040.0% (0/7704)0.0% (0/7704)+100.0 pp
Random legal play (expected)333716.9% (563.3/3337)11.3% (376.8/3337)770457.2% (4408.4/7704)18.2% (1403.6/7704)-40.3 pp

Swipe for more columns →

League 3: per setup, kept decisions only
setupRisk wins keptmissed risks (played safe)in-between (Risk wins)Safety wins keptneedless risks (played unsafe)in-between (Safety wins)bias = missed − needlessthread: risks found
Claude Fable 5.1high10235.3% (36/102)32.4% (33/102)2952.4% (7/295)8.1% (24/295)+32.9 pp50.7% (215/424)
Claude Opus 5.5low11947.1% (56/119)26.1% (31/119)2891.7% (5/289)8.3% (24/289)+45.3 pp40.1% (187/466)
Claude Opus 5.5high11748.7% (57/117)29.9% (35/117)2601.5% (4/260)5.4% (14/260)+47.2 pp43.4% (199/459)
GNUbg weak11110.8% (12/111)13.5% (15/111)24514.7% (36/245)22.9% (56/245)-3.9 pp58.8% (288/490)
GPT-6 Astralow10539.0% (41/105)30.5% (32/105)2611.1% (3/261)11.1% (29/261)+37.9 pp57.2% (226/395)
GPT-6 Astramedium11735.0% (41/117)38.5% (45/117)2800.7% (2/280)8.9% (25/280)+34.3 pp54.9% (245/446)
GPT-6.1 Solhigh8831.8% (28/88)27.3% (24/88)3341.8% (6/334)10.8% (36/334)+30.0 pp50.6% (212/419)
GPT-6.1 Sollow10832.4% (35/108)32.4% (35/108)3021.0% (3/302)10.3% (31/302)+31.4 pp51.7% (198/383)
Gemini 4 Argonlow11554.8% (63/115)20.9% (24/115)2972.4% (7/297)6.4% (19/297)+52.4 pp47.8% (209/437)
Gemini 4 Argonhigh10142.6% (43/101)20.8% (21/101)2370.4% (1/237)7.2% (17/237)+42.2 pp50.8% (224/441)
Grok 4.7high13952.5% (73/139)26.6% (37/139)2501.6% (4/250)9.6% (24/250)+50.9 pp44.0% (190/432)
Kimi K3.1 Prooff8545.9% (39/85)25.9% (22/85)2252.2% (5/225)10.2% (23/225)+43.7 pp42.4% (153/361)
Safest program (reference)1307100.0% (1307/1307)0.0% (0/1307)32750.0% (0/3275)0.0% (0/3275)+100.0 pp
Random legal play (expected)130718.9% (247.3/1307)10.5% (137.4/1307)327554.6% (1789.7/3275)18.6% (609.9/3275)-35.7 pp

Preliminary / partial

How this part was done and checked *
  • Missed risks: Risk wins played safe (0 hitting rolls). Needless risks: Safety wins played unsafe (12 or more). In-between: 1 to 11 hitting rolls. Bias = missed minus needless, in percentage points; positive means too cautious. Each share is shown with its count; no group has under 30 positions.
  • Reference rows use all of the league's kept decisions: the Safest program always plays a 0-roll play; Random legal play gives each decision's expected shares (fraction of legal plays in each bin).
  • Not paired: setups met different positions, so the rows compare only loosely.
  • Thread: risks found is a different, broader measure: the site's published Risks found at margin 0.001, from the deployed results a12573f2, computed on all of a setup's Risk pays decisions, a much larger set than the kept clear-risk positions. Risks found counts a right risk, missed risks here count playing safe.

Checks on the scan

Swipe for more columns →

Check 4: where the best legal play is (2-ply)
best play safebest play in betweenbest play unsafekept decisions
Risk wins, all leagues022.4% (1,793)77.6% (6,204)7,997
Risk wins, L1 / L2 / L30 / 0 / 023.8% (799) / 21.4% (715) / 21.3% (279)2,554 / 2,622 / 1,0283,353 / 3,337 / 1,307
Risk wins, band 0.04–0.08 / 0.08–0.15 / 0.15+029.1% (677) / 19.2% (426) / 20.0% (690)1,646 / 1,792 / 2,7662,323 / 2,218 / 3,456
Safety wins, all leagues84.0% (14,443)16.0% (2,759)017,202
Safety wins, L1 / L2 / L35,185 / 6,459 / 2,79916.7% (1,038) / 16.2% (1,245) / 14.5% (476)06,223 / 7,704 / 3,275
Safety wins, band 0.04–0.08 / 0.08–0.15 / 0.15+1,721 / 3,396 / 9,32623.4% (527) / 18.6% (777) / 13.5% (1,455)02,248 / 4,173 / 10,781

Swipe for more columns →

Check 2: missed and needless per 3-ply gap band (setups pooled; random legal play in italics in the source)
bandsetups (pooled): missedneedlessbiasmedian setup biasrandom legal play, L1 / L2 / L3: missed · needless
0.04–0.0859.1% (1,250/2,114)6.0% (123/2,060)+53.2 pp+50.7 pp16.6 / 17.4 / 18.6% · 66.0 / 63.2 / 61.7%
0.08–0.1565.4% (1,340/2,048)4.7% (179/3,778)+60.7 pp+60.0 pp17.7 / 17.8 / 20.8% · 65.8 / 64.0 / 62.5%
0.15+54.5% (1,735/3,186)2.4% (239/9,938)+52.1 pp+47.1 pp15.3 / 15.9 / 17.7% · 55.2 / 53.4 / 50.6%

Swipe for more columns →

Check 2: the five Part-2 models (missed n/N · needless n/N · bias)
setup (league)0.04–0.080.08–0.150.15+
celeris-1-decision (L1)33/41 · 0/53 · +80.541/50 · 0/96 · +82.063/77 · 4/259 · +80.3
Jev 1.14 (L2)37/60 · 1/45 · +59.449/62 · 4/88 · +74.553/65 · 5/214 · +79.2
d1-ultra (L2)16/37 · 6/57 · +32.732/65 · 6/90 · +42.636/69 · 6/267 · +49.9
Claude Sonnet 5.5high (L2)28/52 · 1/53 · +52.036/60 · 0/102 · +60.039/82 · 1/273 · +47.2
GPT-6.1 Solhigh (L3)9/28 (few) · 0/38 · +32.114/29 (few) · 3/68 · +43.95/31 · 3/228 · +14.8

Swipe for more columns →

Check 3: equity lost per bin, and took the right risk
Risk wins: chose safein betweenunsafetook the right riskSafety wins: chose safein betweenunsafe
setups (pooled)0.205 (4,325)0.073 (1,617)0.095 (1,406)53.3% (749/1,406)0.035 (13,928)0.084 (1,307)0.280 (541)
GNUbg weak, all leagues53.5% (136/254)
Random, L16.4% (3/47)
celeris-1-decision0.222 (137)0.055 (22)0.055 (9)5/9 (few)0.052 (387)0.093 (17)0.304 (4)
Jev 1.140.185 (139)0.094 (32)0.074 (16)37.5% (6/16) (few)0.041 (312)0.082 (25)0.251 (10)
d1-ultra0.241 (84)0.070 (50)0.088 (37)48.6% (18/37)0.042 (352)0.072 (44)0.243 (18)
Claude Sonnet 5.5high0.173 (103)0.031 (52)0.064 (39)59.0% (23/39)0.018 (395)0.043 (31)0.150 (2)
GPT-6.1 Solhigh0.122 (28)0.038 (24)0.032 (36)69.4% (25/36)0.012 (292)0.013 (36)0.191 (6)

Swipe for more columns →

Check 2 per setup, League 1 (missed · needless · bias per band)
setup0.04-0.08: missed0.04-0.08: needless0.04-0.08: bias0.08-0.15: missed0.08-0.15: needless0.08-0.15: bias0.15+: missed0.15+: needless0.15+: bias
celeris-1-decision80.5% (33/41)0.0% (0/53)+80.5 pp82.0% (41/50)0.0% (0/96)+82.0 pp81.8% (63/77)1.5% (4/259)+80.3 pp
Cygnet82.2% (60/73)0.0% (0/35)+82.2 pp75.5% (40/53)8.7% (6/69)+66.8 pp71.9% (82/114)2.1% (4/193)+69.9 pp
Perplexity Decider Flash v1.262.2% (28/45)15.0% (9/60)+47.2 pp51.7% (15/29) (few)18.3% (17/93)+33.4 pp46.5% (40/86)11.3% (21/186)+35.2 pp
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark59.6% (31/52)11.4% (4/35)+48.2 pp78.7% (37/47)6.8% (6/88)+71.9 pp65.8% (50/76)6.9% (17/246)+58.9 pp
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell58.8% (20/34)8.3% (4/48)+50.5 pp70.3% (26/37)4.4% (4/91)+65.9 pp68.9% (51/74)3.6% (8/222)+65.3 pp
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release69.4% (34/49)0.0% (0/41)+69.4 pp70.8% (34/48)1.4% (1/71)+69.4 pp55.7% (54/97)1.6% (3/187)+54.1 pp
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell63.3% (31/49)13.0% (7/54)+50.3 pp69.0% (20/29) (few)8.0% (7/88)+61.0 pp60.7% (37/61)4.7% (13/275)+55.9 pp
GNUbg weak10.0% (4/40)28.6% (10/35)-18.6 pp0.0% (0/25) (few)31.8% (27/85)-31.8 pp0.0% (0/39)5.2% (10/192)-5.2 pp
GPT-6 LunaDecisions API80.6% (54/67)4.3% (2/46)+76.2 pp90.5% (38/42)1.2% (1/85)+89.3 pp75.0% (66/88)2.0% (4/199)+73.0 pp
Jev 1.1378.6% (33/42)0.0% (0/43)+78.6 pp80.4% (45/56)0.0% (0/88)+80.4 pp74.7% (65/87)1.3% (3/223)+73.4 pp
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell78.4% (40/51)22.0% (9/41)+56.5 pp69.4% (34/49)8.9% (7/79)+60.5 pp52.6% (41/78)8.2% (13/158)+44.3 pp
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark65.2% (43/66)33.3% (14/42)+31.8 pp61.0% (25/41)12.5% (12/96)+48.5 pp44.9% (35/78)7.8% (17/218)+37.1 pp
Mercury Decide61.3% (38/62)2.2% (1/45)+59.1 pp86.8% (46/53)1.4% (1/74)+85.4 pp82.4% (84/102)1.1% (2/188)+81.3 pp
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell33.3% (11/33)26.3% (10/38)+7.0 pp53.7% (22/41)19.4% (18/93)+34.3 pp57.1% (36/63)19.4% (34/175)+37.7 pp
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark60.9% (28/46)3.6% (2/55)+57.2 pp78.7% (59/75)2.5% (2/80)+76.2 pp78.2% (86/110)1.0% (2/196)+77.2 pp
Random13.0% (3/23) (few)78.3% (18/23) (few)-65.2 pp25.0% (4/16) (few)76.1% (35/46)-51.1 pp29.3% (12/41)53.2% (33/62)-24.0 pp
Safest100.0% (70/70)0.0% (0/48)+100.0 pp100.0% (72/72)0.0% (0/103)+100.0 pp100.0% (91/91)0.0% (0/178)+100.0 pp
SubQ Snap Preview47.3% (26/55)17.6% (9/51)+29.6 pp61.4% (27/44)10.0% (9/90)+51.4 pp51.9% (42/81)4.0% (10/247)+47.8 pp
Winnow-12B58.7% (27/46)12.2% (6/49)+46.5 pp69.7% (46/66)6.5% (5/77)+63.2 pp57.0% (53/93)6.5% (12/185)+50.5 pp
Random legal play (expected)16.6% (156.6/944)66.0% (555.4/842)-49.4 pp17.7% (154.9/873)65.8% (1047.0/1592)-48.0 pp15.3% (235.6/1536)55.2% (2091.6/3789)-39.9 pp

Swipe for more columns →

Check 2 per setup, League 2 (missed · needless · bias per band)
setup0.04-0.08: missed0.04-0.08: needless0.04-0.08: bias0.08-0.15: missed0.08-0.15: needless0.08-0.15: bias0.15+: missed0.15+: needless0.15+: bias
Claude Fable 5.1low50.0% (26/52)5.1% (2/39)+44.9 pp69.8% (30/43)5.6% (5/90)+64.2 pp24.3% (17/70)0.4% (1/256)+23.9 pp
Claude Sonnet 5.5high53.8% (28/52)1.9% (1/53)+52.0 pp60.0% (36/60)0.0% (0/102)+60.0 pp47.6% (39/82)0.4% (1/273)+47.2 pp
Claude Sonnet 5.5low63.0% (34/54)0.0% (0/61)+63.0 pp68.5% (37/54)3.1% (3/96)+65.4 pp54.8% (34/62)0.0% (0/309)+54.8 pp
Clef56.9% (29/51)18.9% (10/53)+38.0 pp63.3% (31/49)14.4% (15/104)+48.8 pp48.8% (40/82)6.0% (11/183)+42.8 pp
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release56.5% (35/62)3.0% (2/66)+53.4 pp64.6% (31/48)5.1% (6/118)+59.5 pp31.2% (24/77)0.3% (1/288)+30.8 pp
DeepSeek-V4.1-Flashoff · API · 101766.0% (33/50)1.8% (1/56)+64.2 pp76.4% (42/55)2.0% (2/99)+74.3 pp73.0% (65/89)0.0% (0/240)+73.0 pp
DeepSeek-V4.1-Flashlow · API · 101755.2% (32/58)7.7% (5/65)+47.5 pp59.6% (28/47)2.6% (3/114)+56.9 pp40.0% (32/80)0.7% (2/297)+39.3 pp
GLiDEAPI65.8% (25/38)0.0% (0/73)+65.8 pp51.9% (27/52)6.2% (6/96)+45.7 pp40.9% (27/66)5.0% (12/241)+35.9 pp
GLM-5.3low75.5% (40/53)0.0% (0/52)+75.5 pp79.1% (34/43)1.1% (1/92)+78.0 pp70.1% (68/97)0.0% (0/221)+70.1 pp
GNUbg weak11.1% (4/36)40.7% (22/54)-29.6 pp6.2% (2/32)37.1% (36/97)-30.9 pp5.7% (3/53)11.6% (30/258)-6.0 pp
Gemini 3.8 Flashhigh56.9% (29/51)0.0% (0/60)+56.9 pp62.2% (28/45)0.0% (0/106)+62.2 pp45.9% (34/74)0.3% (1/288)+45.6 pp
Gemini 3.8 Flashlow54.5% (30/55)4.5% (2/44)+50.0 pp68.8% (22/32)2.0% (2/99)+66.7 pp54.7% (35/64)0.7% (2/278)+54.0 pp
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark53.3% (16/30)1.9% (1/53)+51.4 pp80.0% (36/45)1.2% (1/86)+78.8 pp50.0% (29/58)1.7% (4/232)+48.3 pp
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark59.1% (39/66)0.0% (0/53)+59.1 pp68.5% (37/54)0.0% (0/96)+68.5 pp47.1% (33/70)0.0% (0/286)+47.1 pp
Grok 4.7low45.6% (26/57)0.0% (0/39)+45.6 pp47.7% (21/44)0.0% (0/97)+47.7 pp45.7% (37/81)0.0% (0/278)+45.7 pp
Jev 1.1461.7% (37/60)2.2% (1/45)+59.4 pp79.0% (49/62)4.5% (4/88)+74.5 pp81.5% (53/65)2.3% (5/214)+79.2 pp
d1-ultra43.2% (16/37)10.5% (6/57)+32.7 pp49.2% (32/65)6.7% (6/90)+42.6 pp52.2% (36/69)2.2% (6/267)+49.9 pp
Qwen3.8 Flashoff52.5% (32/61)1.8% (1/57)+50.7 pp59.0% (36/61)5.1% (4/79)+54.0 pp54.2% (52/96)3.0% (6/198)+51.1 pp
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark63.8% (37/58)6.5% (3/46)+57.3 pp70.6% (36/51)4.8% (4/83)+65.8 pp55.7% (44/79)2.1% (5/239)+53.6 pp
Random legal play (expected)17.4% (171.0/981)63.2% (648.6/1026)-45.8 pp17.8% (167.8/942)63.9% (1171.5/1832)-46.1 pp15.9% (224.5/1414)53.4% (2588.3/4846)-37.5 pp

Swipe for more columns →

Check 2 per setup, League 3 (missed · needless · bias per band)
setup0.04-0.08: missed0.04-0.08: needless0.04-0.08: bias0.08-0.15: missed0.08-0.15: needless0.08-0.15: bias0.15+: missed0.15+: needless0.15+: bias
Claude Fable 5.1high41.2% (14/34)2.5% (1/40)+38.7 pp46.4% (13/28) (few)2.7% (2/75)+43.8 pp22.5% (9/40)2.2% (4/180)+20.3 pp
Claude Opus 5.5low62.5% (25/40)3.3% (1/30)+59.2 pp43.9% (18/41)2.6% (2/78)+41.3 pp34.2% (13/38)1.1% (2/181)+33.1 pp
Claude Opus 5.5high53.1% (17/32)4.8% (1/21) (few)+48.4 pp52.6% (20/38)4.3% (3/69)+48.3 pp42.6% (20/47)0.0% (0/170)+42.6 pp
GNUbg weak17.5% (7/40)35.7% (10/28) (few)-18.2 pp12.0% (3/25) (few)23.4% (15/64)-11.4 pp4.3% (2/46)7.2% (11/153)-2.8 pp
GPT-6 Astralow31.2% (10/32)0.0% (0/29) (few)+31.2 pp58.1% (18/31)6.5% (3/46)+51.5 pp31.0% (13/42)0.0% (0/186)+31.0 pp
GPT-6 Astramedium41.9% (13/31)5.4% (2/37)+36.5 pp44.4% (16/36)0.0% (0/51)+44.4 pp24.0% (12/50)0.0% (0/192)+24.0 pp
GPT-6.1 Solhigh32.1% (9/28) (few)0.0% (0/38)+32.1 pp48.3% (14/29) (few)4.4% (3/68)+43.9 pp16.1% (5/31)1.3% (3/228)+14.8 pp
GPT-6.1 Sollow30.6% (11/36)8.0% (2/25) (few)+22.6 pp43.8% (14/32)1.5% (1/67)+42.3 pp25.0% (10/40)0.0% (0/210)+25.0 pp
Gemini 4 Argonlow69.7% (23/33)3.0% (1/33)+66.7 pp60.0% (27/45)5.1% (3/59)+54.9 pp35.1% (13/37)1.5% (3/205)+33.7 pp
Gemini 4 Argonhigh44.1% (15/34)3.2% (1/31)+40.9 pp40.6% (13/32)0.0% (0/67)+40.6 pp42.9% (15/35)0.0% (0/139)+42.9 pp
Grok 4.7high61.5% (24/39)2.4% (1/41)+59.1 pp61.5% (24/39)5.8% (3/52)+55.8 pp41.0% (25/61)0.0% (0/157)+41.0 pp
Kimi K3.1 Prooff42.1% (8/19) (few)3.7% (1/27) (few)+38.4 pp55.6% (15/27) (few)1.9% (1/53)+53.7 pp41.0% (16/39)2.1% (3/145)+39.0 pp
Random legal play (expected)18.6% (74.1/398)61.7% (234.6/380)-43.1 pp20.8% (83.9/403)62.5% (468.3/749)-41.7 pp17.7% (89.4/506)50.6% (1086.8/2146)-33.0 pp

Swipe for more columns →

Check 3 per setup, League 1 (mean equity lost (n); took the right risk)
setupRisk wins: safeRisk wins: in betweenRisk wins: unsafeRisk wins: took the right riskSafety wins: safeSafety wins: in betweenSafety wins: unsafe
celeris-1-decision0.222 (137)0.055 (22)0.055 (9)55.6% (5/9)0.052 (387)0.093 (17)0.304 (4)
Cygnet0.217 (182)0.064 (33)0.123 (25)36.0% (9/25)0.038 (268)0.140 (19)0.216 (10)
Perplexity Decider Flash v1.20.229 (83)0.170 (25)0.254 (52)17.3% (9/52)0.074 (258)0.130 (34)0.333 (47)
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark0.223 (118)0.084 (29)0.110 (28)50.0% (14/28)0.039 (303)0.124 (39)0.381 (27)
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell0.216 (97)0.065 (24)0.114 (24)37.5% (9/24)0.051 (306)0.081 (39)0.283 (16)
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release0.206 (122)0.111 (51)0.117 (21)57.1% (12/21)0.050 (264)0.092 (31)0.294 (4)
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell0.254 (88)0.069 (24)0.097 (27)48.1% (13/27)0.058 (361)0.075 (29)0.244 (27)
GNUbg weak0.058 (4)0.020 (17)0.058 (83)53.0% (44/83)0.013 (190)0.042 (75)0.192 (47)
GPT-6 LunaDecisions API0.218 (158)0.105 (28)0.122 (11)27.3% (3/11)0.044 (295)0.138 (28)0.171 (7)
Jev 1.130.220 (143)0.068 (25)0.193 (17)47.1% (8/17)0.040 (335)0.031 (16)0.190 (3)
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell0.192 (115)0.083 (16)0.208 (47)36.2% (17/47)0.065 (212)0.164 (37)0.330 (29)
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark0.202 (103)0.087 (25)0.197 (57)35.1% (20/57)0.052 (280)0.134 (33)0.342 (43)
Mercury Decide0.273 (168)0.102 (35)0.173 (14)28.6% (4/14)0.053 (272)0.113 (31)0.227 (4)
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell0.265 (69)0.075 (23)0.220 (45)28.9% (13/45)0.080 (209)0.158 (35)0.378 (62)
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark0.250 (173)0.134 (48)0.088 (10)40.0% (4/10)0.056 (309)0.062 (16)0.172 (6)
Random0.429 (19)0.339 (14)0.283 (47)6.4% (3/47)0.106 (29)0.168 (16)0.438 (86)
Safest0.227 (233)– (0)– (0)– (0)0.082 (329)– (0)– (0)
SubQ Snap Preview0.295 (95)0.151 (46)0.165 (39)38.5% (15/39)0.052 (311)0.129 (49)0.293 (28)
Winnow-12B0.250 (126)0.110 (43)0.120 (36)44.4% (16/36)0.078 (261)0.109 (27)0.255 (23)

Swipe for more columns →

Check 3 per setup, League 2 (mean equity lost (n); took the right risk)
setupRisk wins: safeRisk wins: in betweenRisk wins: unsafeRisk wins: took the right riskSafety wins: safeSafety wins: in betweenSafety wins: unsafe
Claude Fable 5.1low0.148 (73)0.056 (52)0.047 (40)62.5% (25/40)0.028 (355)0.074 (22)0.146 (8)
Claude Sonnet 5.5high0.173 (103)0.031 (52)0.064 (39)59.0% (23/39)0.018 (395)0.043 (31)0.150 (2)
Claude Sonnet 5.5low0.180 (105)0.036 (46)0.074 (19)52.6% (10/19)0.028 (448)0.053 (15)0.115 (3)
Clef0.191 (100)0.086 (30)0.115 (52)38.5% (20/52)0.044 (256)0.095 (48)0.189 (36)
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release0.140 (90)0.085 (47)0.065 (50)58.0% (29/50)0.024 (418)0.045 (45)0.151 (9)
DeepSeek-V4.1-Flashoff · API · 10170.246 (140)0.099 (37)0.140 (17)58.8% (10/17)0.036 (347)0.075 (45)0.263 (3)
DeepSeek-V4.1-Flashlow · API · 10170.165 (92)0.059 (57)0.040 (36)75.0% (27/36)0.018 (420)0.035 (46)0.217 (10)
GLiDEAPI0.171 (79)0.123 (34)0.063 (43)55.8% (24/43)0.043 (346)0.127 (46)0.277 (18)
GLM-5.3low0.218 (142)0.066 (38)0.109 (13)53.8% (7/13)0.033 (331)0.057 (33)0.108 (1)
GNUbg weak0.199 (9)0.052 (25)0.060 (87)58.6% (51/87)0.022 (213)0.066 (108)0.207 (88)
Gemini 3.8 Flashhigh0.163 (91)0.044 (46)0.048 (33)60.6% (20/33)0.012 (435)0.011 (18)0.794 (1)
Gemini 3.8 Flashlow0.172 (87)0.014 (30)0.032 (34)70.6% (24/34)0.022 (404)0.048 (11)0.209 (6)
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark0.177 (81)0.047 (30)0.098 (22)59.1% (13/22)0.029 (353)0.080 (12)0.246 (6)
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark0.163 (109)0.038 (41)0.056 (40)62.5% (25/40)0.029 (418)0.064 (17)– (0)
Grok 4.7low0.188 (84)0.061 (63)0.052 (35)60.0% (21/35)0.017 (376)0.100 (38)– (0)
Jev 1.140.185 (139)0.094 (32)0.074 (16)37.5% (6/16)0.041 (312)0.082 (25)0.251 (10)
d1-ultra0.241 (84)0.070 (50)0.088 (37)48.6% (18/37)0.042 (352)0.072 (44)0.243 (18)
Qwen3.8 Flashoff0.223 (120)0.113 (55)0.126 (43)27.9% (12/43)0.038 (277)0.119 (46)0.340 (11)
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark0.218 (117)0.061 (41)0.074 (30)60.0% (18/30)0.040 (337)0.076 (19)0.203 (12)

Swipe for more columns →

Check 3 per setup, League 3 (mean equity lost (n); took the right risk)
setupRisk wins: safeRisk wins: in betweenRisk wins: unsafeRisk wins: took the right riskSafety wins: safeSafety wins: in betweenSafety wins: unsafe
Claude Fable 5.1high0.131 (36)0.019 (33)0.059 (33)60.6% (20/33)0.016 (264)0.043 (24)0.224 (7)
Claude Opus 5.5low0.143 (56)0.034 (31)0.043 (32)75.0% (24/32)0.022 (260)0.056 (24)0.199 (5)
Claude Opus 5.5high0.171 (57)0.041 (35)0.033 (25)68.0% (17/25)0.010 (242)0.017 (14)0.135 (4)
GNUbg weak0.163 (12)0.023 (15)0.074 (84)48.8% (41/84)0.021 (153)0.102 (56)0.191 (36)
GPT-6 Astralow0.157 (41)0.027 (32)0.026 (32)75.0% (24/32)0.010 (229)0.009 (29)0.160 (3)
GPT-6 Astramedium0.129 (41)0.049 (45)0.023 (31)74.2% (23/31)0.012 (253)0.022 (25)0.081 (2)
GPT-6.1 Solhigh0.122 (28)0.038 (24)0.032 (36)69.4% (25/36)0.012 (292)0.013 (36)0.191 (6)
GPT-6.1 Sollow0.163 (35)0.032 (35)0.027 (38)65.8% (25/38)0.012 (268)0.044 (31)0.072 (3)
Gemini 4 Argonlow0.148 (63)0.048 (24)0.035 (28)71.4% (20/28)0.021 (271)0.052 (19)0.286 (7)
Gemini 4 Argonhigh0.181 (43)0.039 (21)0.009 (37)83.8% (31/37)0.018 (219)0.029 (17)0.045 (1)
Grok 4.7high0.162 (73)0.100 (37)0.072 (29)72.4% (21/29)0.013 (222)0.072 (24)0.105 (4)
Kimi K3.1 Prooff0.195 (39)0.110 (22)0.048 (24)66.7% (16/24)0.035 (197)0.152 (23)0.487 (5)

Preliminary / partial

How this part was done and checked *
  • The best legal play is in between (1 to 11 hitting rolls) in 22% of Risk wins and 16% of Safety wins: in about one decision in five or six, neither of the two plays the scan compares is the best play.
  • Bands are the absolute 3-ply gap: 0.04-0.08, 0.08-0.15 and 0.15+ (lower edge included). Setups (pooled) = the 45 setups other than the three programs, Random, Safest and GNUbg weak; each decision counts once.
  • Missed risks do not fall as the gap grows (59%, 65%, 55%); needless risks do (6.0%, 4.7%, 2.4%).
  • Equity lost = 2-ply equity of the best legal play minus that of the chosen play, mean (count). Took the right risk: a Risk win played unsafe, within 0.02 of the best unsafe play.
  • Not paired: the five Part-2 models played in different leagues.

Rollout check of a 200-position sample

Swipe for more columns →

Agreement per group and band (the 200)
group0.04–0.080.08–0.150.15+allnot confirmed: why
Risk wins79% (27/34)100% (33/33)100% (33/33)93% (93/100)gap < 0.04: 7
Safety wins88% (30/34)100% (33/33)100% (33/33)96% (96/100)gap < 0.04: 4
both84% (57/68)100% (66/66)100% (66/66)94.5% (189/200)gap < 0.04: 11

Swipe for more columns →

Back game and prime-vs-prime
positionsRisk winsSafety winsallnot confirmed: why
back game: in the 20067% (2/3)100% (5/5)88% (7/8)gap < 0.04: 1 (sample 13)
back game: oversample (30)100% (8/8)100% (22/22)100% (30/30)–
back game: both91% (10/11)100% (29/29)98% (39/40)gap < 0.04: 1
prime-vs-prime: in the 200100% (6/6)100% (3/3)100% (9/9)–
prime-vs-prime: oversample (30)100% (2/2)100% (28/28)100% (30/30)–
prime-vs-prime: both100% (8/8)100% (32/32)100% (40/40)–
neither, in the 20093% (85/91)96% (88/92)95% (173/183)gap < 0.04: 10

Preliminary / partial

How this part was done and checked *
  • Sample: 200 kept positions drawn with a fixed seed, 100 Risk wins and 100 Safety wins, about a third per gap band; Bray position excluded. Oversample: 30 back-game and 30 prime-vs-prime kept positions not in the 200 (fixed seed), reported separately and not used in Part 2.
  • Settings: GNU Backgammon 1.08.003, 1,296 games per play, variance reduction, quasi-random dice, same seed for both plays, no truncation, cubeless money play with gammons and backgammons counted (1/2/3 points), Jacoby off, 2-ply checker play (cubeless, prune, no noise). Each play is rolled out from the position after it, opponent on roll.
  • Confirmed = same direction as the scan, rollout gap 0.04 or more, and more than 3 standard errors of the gap. The rule, the settings and the oversample were committed before the first rollout.
  • 189 of 200 confirmed (94.5%): 93 of 100 Risk wins, 96 of 100 Safety wins. No position flipped; all 11 misses are in the 0.04-0.08 band with a smaller gap in the same direction. Back game 39 of 40, prime-vs-prime 40 of 40 (sample and oversample together); most of those are large-gap Safety wins already kept by the 3-ply check, so this says nothing about the positions the check dropped.

Part 2: asking the models

Swipe for more columns →

Q1: share right per group, missed and needless, bias (two plays shown, not league play)
modelversionRisk wins rightSafety wins rightmissed (Risk wins → safe)needless (Safety wins → unsafe)biasleague: missed · needless · bias (all legal plays)share of A answersQ1 not answered
celeris-1-decisionwith numbers6% (6/93)98% (94/96)94% (87/93)2% (2/96)+91 pp82% (137/168) · 1% (4/408) · +81 pp47% (88/189)0
celeris-1-decisionboard only75% (69/92)61% (57/93)25% (23/92)39% (36/93)-14 pp48% (88/185)4
Jev 1.14with numbers4% (4/91)97% (93/96)96% (87/91)3% (3/96)+92 pp74% (139/187) · 3% (10/347) · +71 pp50% (94/187)2
Jev 1.14board only67% (62/93)45% (41/92)33% (31/93)55% (51/92)-22 pp47% (87/185)4
d1-ultrawith numbers19% (18/93)94% (90/96)81% (75/93)6% (6/96)+74 pp49% (84/171) · 4% (18/414) · +45 pp51% (96/189)0
d1-ultraboard only68% (63/93)47% (41/87)32% (30/93)53% (46/87)-21 pp62% (112/180)9
Claude Sonnet 5.5highwith numbers32% (30/93)98% (94/96)68% (63/93)2% (2/96)+66 pp53% (103/194) · 0% (2/428) · +53 pp50% (94/189)0
Claude Sonnet 5.5highboard only52% (48/93)94% (90/96)48% (45/93)6% (6/96)+42 pp54% (102/189)0
GPT-6.1 Solhighwith numbers44% (41/93)98% (94/96)56% (52/93)2% (2/96)+54 pp32% (28/88) · 2% (6/334) · +30 pp50% (95/189)0
GPT-6.1 Solhighboard only77% (72/93)81% (78/96)23% (21/93)19% (18/96)+4 pp46% (86/189)0
Kimi K3.1 Prooffwith numbers24% (22/93)94% (90/96)76% (71/93)6% (6/96)+70 pp46% (39/85) · 2% (5/225) · +44 pp49% (92/189)0
Kimi K3.1 Prooffboard only70% (65/93)45% (43/96)30% (28/93)55% (53/96)-25 pp65% (122/189)0
GLM-5.3lowwith numbers14% (13/93)92% (88/96)86% (80/93)8% (8/96)+78 pp74% (142/193) · 0% (1/365) · +73 pp59% (111/189)0
GLM-5.3lowboard only51% (47/93)69% (66/96)49% (46/93)31% (30/96)+18 pp65% (123/189)0
Claude Opus 5.5highwith numbers40% (37/93)96% (92/96)60% (56/93)4% (4/96)+56 pp49% (57/117) · 2% (4/260) · +47 pp48% (91/189)0
Claude Opus 5.5highboard only49% (46/93)96% (92/96)51% (47/93)4% (4/96)+46 pp48% (90/189)0

Swipe for more columns →

Q2 (board only): share right vs random
modelrightrandom baseline
celeris-1-decision92% (55/60)25% (4 options, one right)
Jev 1.1462% (37/60)25% (4 options, one right)
d1-ultra92% (55/60)25% (4 options, one right)
Claude Sonnet 5.5high100% (60/60)25% (4 options, one right)
GPT-6.1 Solhigh100% (60/60)25% (4 options, one right)
Kimi K3.1 Prooff92% (55/60)25% (4 options, one right)
GLM-5.3low95% (57/60)25% (4 options, one right)
Claude Opus 5.5high100% (60/60)25% (4 options, one right)

Swipe for more columns →

Q3: share of pairs in the right order (pairs 0.04 or more apart at 3-ply)
modelversionall pairssafe-vs-unsafe pair
celeris-1-decisionwith numbers10% (87/834)10% (18/189)
celeris-1-decisionboard only6% (46/834)5% (10/189)
Jev 1.14with numbers17% (138/834)17% (32/189)
Jev 1.14board only7% (59/834)11% (21/189)
d1-ultrawith numbers19% (157/834)17% (33/189)
d1-ultraboard only6% (49/834)5% (9/189)
Claude Sonnet 5.5highwith numbers81% (674/834)62% (118/189)
Claude Sonnet 5.5highboard only82% (682/834)68% (129/189)
GPT-6.1 Solhighwith numbers89% (742/834)72% (137/189)
GPT-6.1 Solhighboard only88% (736/834)79% (150/189)
Kimi K3.1 Prooffwith numbers75% (622/834)55% (104/189)
Kimi K3.1 Prooffboard only55% (458/834)54% (103/189)
GLM-5.3lowwith numbers73% (610/834)55% (104/189)
GLM-5.3lowboard only66% (552/834)53% (100/189)
Claude Opus 5.5highwith numbers82% (681/834)63% (120/189)
Claude Opus 5.5highboard only83% (691/834)70% (132/189)
random one-to-one mapping (chat-style)50%50%
random play per value, independently (decision-style)10%11%

Swipe for more columns →

Risk wins cross-table (counts)
modelversionRisk wins with a safe-vs-unsafe Q3 pairsees the value, plays safe (Q3 pair right, Q1 safe)sees the value, takes the risk (Q3 right, Q1 unsafe)doesn't see the value (Q3 pair wrong)doesn't know the game (Q2 and Q3 wrong; of Q3-wrong items with a Q2 item)
celeris-1-decisionwith numbers9320911 of 38
celeris-1-decisionboard only9312901 of 37
Jev 1.14with numbers93109111 of 36
Jev 1.14board only93588010 of 30
d1-ultrawith numbers9300931 of 38
d1-ultraboard only9304891 of 36
Claude Sonnet 5.5highwith numbers93617700 of 26
Claude Sonnet 5.5highboard only93636510 of 17
GPT-6.1 Solhighwith numbers931630470 of 19
GPT-6.1 Solhighboard only931257240 of 15
Kimi K3.1 Prooffwith numbers93109740 of 31
Kimi K3.1 Prooffboard only931449300 of 14
GLM-5.3lowwith numbers9376801 of 31
GLM-5.3lowboard only931024591 of 21
Claude Opus 5.5highwith numbers93422670 of 24
Claude Opus 5.5highboard only93733530 of 18

Swipe for more columns →

Denominators, failures and tokens per model (list-price cost for the hosted decision models, provider-reported cost for the OpenRouter models, API-equivalent for subscription models: not money paid; none of these is billed spending checked against an invoice)
modelexpected requestsrecorded answer rowsanswer rows with a request errorusable answersitem-level flags (see notes)exact ties at the top (no answer)input tokensoutput tokensreasoning tokenscost
celeris-1-decision1,7821,78201,756333 Q3 items with two values on one play264,759,59400$0.19
Jev 1.141,7821,78201,741315 Q3 items with two values on one play415,186,004140,6770$0.22
d1-ultra1,7821,78201,683287 Q3 items with two values on one play994,364,91500$0.18
Claude Sonnet 5.5high8168160816002,845,140506,104492,088$10.75 API-equiv.
GPT-6.1 Solhigh8168161814failed or missing 102,027,910447,317429,686$8.53 API-equiv.
Kimi K3.1 Prooff8168160816002,091,75416,3000$1.39 API-equiv.
GLM-5.3low8168160809invalid 701,990,210787,791607,727$4.30 API-equiv.
Claude Opus 5.5high8168160816002,845,140393,825379,806$19.26 API-equiv.

Preliminary / partial

How this part was done and checked *
  • Two or four plays shown, not league play.
  • Positions: the 189 rollout-confirmed sample positions (93 Risk wins, 96 Safety wins). Each question is its own request with no shared context. Not paired across models beyond these shared items, and the league comparison is unpaired (the models played in different leagues on different positions).
  • Q1: best safe vs best unsafe play, 'Which play is better?'; half the right answers are A (fixed seed). Q2 (board only, 60 items where exactly one option is true of the unsafe play B and not of the safe play A): what does play B do that play A does not; random baseline 25%. Q3: 3 or 4 plays (best safe, best unsafe, best in-between if any, one clear blunder at least 0.10 below the others at 3-ply if any) and their 3-ply values (2 decimals, labelled W-Z); chat models match each value to a play (each play once), decision models get one question per value.
  • Two versions: with numbers (the league prompt with the computed consequences of each play) and board only (notation only, no pip count or other computed number).
  • Models at league settings; temperature not sent (provider default) for all. Decision models: the top-probability option is the answer; an exact tie at the top is no answer. Q3 scoring: only pairs 0.04 or more apart at 3-ply; a decision model that puts two values on one play gets the pairs of that play wrong; a chat answer that is not one-to-one counts every pair wrong.
  • All models asked so far are complete. GPT-6.1 Solhigh here is separate from the Codex effort sweep, whose GPT-6.1 Sol (max) row is a different run. Added after the first publication (Zeki / Ümit, 30 Oct): Kimi K3.1 Prooff and GLM-5.3low via OpenRouter and Claude Opus 5.5high on the Claude subscription, at league settings, same items and frozen scoring. GLM-5.3low (v3 setup name GLM-5.3 (on)) stopped at its $3.00 cost cap with 558 of 816 answers and was resumed with a $4.60 total cap, asking only the missing items on the same OpenRouter route (Fireworks), model id and reasoning setting.
  • Q3 random baselines were added AFTER the first score output: first score 2026-10-30T06:25:29.387Z, baseline edit 2026-10-30T06:25:46.496Z, baseline output 2026-10-30T06:25:50.444Z. Not pre-registered and not exact: a simulation with seed 2026103005, 200 draws per item (4000 // 20), rounded to whole percent. Scoring was not changed.
  • The cross-table's 'doesn't know the game' uses the board-only Q2 answer for both versions (Q2 was asked board only).
  • Denominators: expected = items sent per model; recorded = answer rows stored; usable = recorded minus rows with a request error, invalid answers and exact ties. 'Item-level flags' for decision models count Q3 items (position and version) where two values went to one play, not pairs and not failed requests. Answer-row error counts are rows whose request failed after retries, not attempts.
  • Cost: hosted decision models at list price (the cash cap was $3; the stored list-price total is about $0.59); OpenRouter models at the cost OpenRouter reports per response (cap $3.00 per model, raised to $4.60 for GLM-5.3low); subscription models shown as API-equivalent, not money paid.

Status

Preliminary / partial

Preliminary / partial. Two or four plays shown, not league play. Data: clear-risk.json · Method

The scan, its checks and the rollout check are as of results 1082e024 (league games up to then); they are not recomputed with later league games.

Models are too cautious: the clear-risk scan

In the clear-risk scan (25,199 league decisions where the engine's call is clear and stable), every one of the 45 model setups missed more risks than it took needlessly: its bias (missed minus needless) runs from +30.0 pp (GPT-6.1 Solhigh, League 3) to +80.6 pp (celeris-1-decision, League 1). The programs are not like that: GNUbg weak's bias is negative in all three leagues (-11.2 pp, -14.1 pp, -3.9 pp), and so is Random's (-41.9 pp); Safest, which never takes a risk, is +100.0 pp by construction.

  1. celeris-1-decision · League 1+80.6 pp
  2. GPT-6 LunaDecisions API · League 1+78.1 pp
  3. Jev 1.13 · League 1+76.4 pp
  4. Mercury Decide · League 1+76.1 pp
  5. GLM-5.3low · League 2+73.3 pp
  6. Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark · League 1+73.1 pp
  7. Cygnet · League 1+72.5 pp
  8. DeepSeek-V4.1-Flashoff · API · 1017 · League 2+71.4 pp
  9. Jev 1.14 · League 2+71.4 pp
  10. Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwell · League 1+62.5 pp
  11. DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release · League 1+61.5 pp
  12. Claude Sonnet 5.5low · League 2+61.1 pp
  13. Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark · League 1+60.1 pp
  14. Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark · League 2+59.3 pp
  15. Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark · League 2+59.0 pp
  16. Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark · League 2+57.4 pp
  17. GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwell · League 1+56.8 pp
  18. Gemini 3.8 Flashlow · League 2+56.2 pp
  19. LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell · League 1+54.2 pp
  20. Winnow-12B · League 1+54.1 pp
  21. Gemini 3.8 Flashhigh · League 2+53.3 pp
  22. Claude Sonnet 5.5high · League 2+52.6 pp
  23. Gemini 4 Argonlow · League 3+52.4 pp
  24. Qwen3.8 Flashoff · League 2+51.8 pp
  25. Grok 4.7high · League 3+50.9 pp
  26. DeepSeek-V4.1-Flashlow · API · 1017 · League 2+47.6 pp
  27. Claude Opus 5.5high · League 3+47.2 pp
  28. GLiDEAPI · League 2+46.3 pp
  29. DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first release · League 2+46.2 pp
  30. Grok 4.7low · League 2+46.2 pp
  31. SubQ Snap Preview · League 1+45.6 pp
  32. Claude Opus 5.5low · League 3+45.3 pp
  33. d1-ultra · League 2+44.8 pp
  34. Clef · League 2+44.4 pp
  35. Kimi K3.1 Prooff · League 3+43.7 pp
  36. LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark · League 1+43.6 pp
  37. Claude Fable 5.1low · League 2+42.2 pp
  38. Gemini 4 Argonhigh · League 3+42.2 pp
  39. Perplexity Decider Flash v1.2 · League 1+38.0 pp
  40. GPT-6 Astralow · League 3+37.9 pp
  41. GPT-6 Astramedium · League 3+34.3 pp
  42. Claude Fable 5.1high · League 3+32.9 pp
  43. GPT-6.1 Sollow · League 3+31.4 pp
  44. Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwell · League 1+30.1 pp
  45. GPT-6.1 Solhigh · League 3+30.0 pp
  46. Safest · League 1+100.0 pp
  47. GNUbg weak · League 3-3.9 pp
  48. GNUbg weak · League 1-11.2 pp
  49. GNUbg weak · League 2-14.1 pp
  50. Random · League 1-41.9 pp

Bias = missed risks (a clear Risk win played safe) minus needless risks (a clear Safety win played unsafe), in percentage points; right of 0 = too cautious program (reference row)

Not paired: setups met different positions, so the rows compare only loosely. More caveats: Method