Phases

Where do models fail?

Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.

Shared positions

Analysis pendingwaits for the shared-position answers

League 1

FinishedPublished

Risk judgement per phase

Swipe for more columns →

Risk judgement (margin 0.001) per game phase; N/A where a phase has no Risk pays or no Safety pays decisions
Model and settingOpeningContactHolding gameRaceBear-off
GNUbg weakprogram · CPU57.3219 with a trade-off69.11,174 with a trade-off75.011 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Jev 1.13decision model · API66.1278 with a trade-off57.51,166 with a trade-off60.332 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs71.1285 with a trade-off55.11,130 with a trade-off52.574 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release63.0255 with a trade-off58.51,055 with a trade-off46.923 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
celeris-1-decisiondecision model · API66.3283 with a trade-off55.41,156 with a trade-off56.161 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs72.2286 with a trade-off54.01,230 with a trade-off51.042 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs61.8298 with a trade-off55.91,203 with a trade-off65.338 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs63.6302 with a trade-off54.01,181 with a trade-off56.465 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GPT-6 LunaDecisions APIdecision model · API58.1293 with a trade-off55.01,129 with a trade-off60.027 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
SubQ Snap Previewdecision model · API68.4305 with a trade-off52.71,301 with a trade-off50.573 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark65.3310 with a trade-off55.11,154 with a trade-off66.724 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark59.1296 with a trade-off51.31,321 with a trade-off44.853 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Mercury Decidedecision model · API60.4288 with a trade-off51.51,036 with a trade-off43.828 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs66.5298 with a trade-off52.61,175 with a trade-off41.655 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Safestprogram · CPU50.0317 with a trade-off50.01,202 with a trade-off50.015 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs54.1290 with a trade-off48.41,175 with a trade-off40.723 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell49.1282 with a trade-off44.61,147 with a trade-off51.936 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs65.8295 with a trade-off42.91,130 with a trade-off52.544 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Randomprogram · CPU19.1181 with a trade-off26.1735 with a trade-off22.720 with a trade-offN/A0 with a trade-offN/A0 with a trade-off

Hit risk only: being hit. Gammon risk and priming are not in it.

Game phase

Swipe for more columns →

Error rate (PR) per game phase; the small number is the decisions in the cell
Model and settingOpeningContactHolding gameRaceBear-off
GNUbg weakprogram · CPU35.222122.81,254Too few decisions14Too few decisions184Too few decisions71
Jev 1.13decision model · API35.027940.41,281Too few decisions3811.1227Too few decisions86
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs25.628844.11,220Too few decisions8314.8286Too few decisions126
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release33.825543.41,141Too few decisions29Too few decisions179Too few decisions132
celeris-1-decisiondecision model · API28.228541.41,220Too few decisions6820.4233Too few decisions116
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs26.128746.11,304Too few decisions4718.5274Too few decisions116
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs39.129844.31,302Too few decisions4422.0246Too few decisions126
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs38.030446.11,277Too few decisions7520.8251Too few decisions145
GPT-6 LunaDecisions APIdecision model · API40.129350.91,272Too few decisions3019.0231Too few decisions136
SubQ Snap Previewdecision model · API35.030753.61,375Too few decisions8220.8295Too few decisions141
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark35.731057.21,259Too few decisions2818.3273Too few decisions135
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark42.129756.01,436Too few decisions5516.2281Too few decisions124
Mercury Decidedecision model · API44.628856.91,188Too few decisions3024.0247Too few decisions100
Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs37.829861.01,283Too few decisions6720.2265Too few decisions102
Safestprogram · CPU53.831754.51,341Too few decisions1734.6248Too few decisions135
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs50.929171.11,283Too few decisions2534.3283Too few decisions124
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell56.028273.31,254Too few decisions4426.5268Too few decisions104
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs35.429687.41,217Too few decisions4729.5312Too few decisions130
Randomprogram · CPUToo few decisions183148.5836Too few decisions2243.6257Too few decisions39

Strategy tag (contact and holding positions after the opening)

Swipe for more columns →

Error rate (PR) per strategy tag; the small number is the decisions in the cell
Model and settingBlitzBack gameHolding gamePrimingLow-anchorOther
GNUbg weakprogram · CPU17.3474Too few decisions115Too few decisions1427.325824.8253Too few decisions154
Jev 1.13decision model · API31.6255Too few decisions60Too few decisions3840.137947.9481Too few decisions106
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs40.1225Too few decisions23Too few decisions8339.739949.034348.6230
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release26.6245Too few decisions16Too few decisions2949.040854.0370Too few decisions102
celeris-1-decisiondecision model · API28.1213Too few decisions29Too few decisions6835.133551.6471Too few decisions172
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs42.5308Too few decisions23Too few decisions4747.735949.539642.7218
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs40.2230Too few decisions44Too few decisions4440.638254.7498Too few decisions148
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs52.6227Too few decisions16Too few decisions7544.245546.1397Too few decisions182
GPT-6 LunaDecisions APIdecision model · APIToo few decisions184Too few decisions11Too few decisions3046.745660.8517Too few decisions104
SubQ Snap Previewdecision model · API42.8298Too few decisions37Too few decisions8252.740554.042876.0207
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkToo few decisions199Too few decisions5Too few decisions2854.242369.8517Too few decisions115
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark49.8302Too few decisions117Too few decisions5551.433865.544555.4234
Mercury Decidedecision model · APIToo few decisions142Too few decisions11Too few decisions3053.847562.4477Too few decisions83
Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs49.7244Too few decisions27Too few decisions6762.736360.3455Too few decisions194
Safestprogram · CPUToo few decisions165Too few decisions14Too few decisions1738.431670.3691Too few decisions155
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs54.7238Too few decisions44Too few decisions2566.934375.545887.5200
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell66.7230Too few decisions87Too few decisions4462.627975.543792.6221
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUsToo few decisions186Too few decisions11Too few decisions4797.628681.9506101.1228
Randomprogram · CPU120.5228Too few decisions65Too few decisions22Too few decisions94148.8272Too few decisions177

Number of legal plays

Swipe for more columns →

Error rate (PR) by number of legal plays; the small number is the decisions in the cell
Model and setting2–56–1011–1516–2526+
GNUbg weakprogram · CPU9.545217.632023.224930.735829.1365
Jev 1.13decision model · API15.266541.740740.732845.9340Too few decisions171
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs12.658232.647543.133243.736668.4248
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release11.554843.340250.431143.8313Too few decisions162
celeris-1-decisiondecision model · API17.060133.945838.529045.436071.6213
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs20.567634.550041.929847.734871.2206
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs21.471942.448149.235147.5334Too few decisions131
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs19.563436.947754.933345.038262.6226
GPT-6 LunaDecisions APIdecision model · API19.669555.545152.934554.8311Too few decisions160
SubQ Snap Previewdecision model · API15.464443.146948.438356.641584.2289
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark22.570453.744951.231062.5351Too few decisions191
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark26.266345.645956.237862.344869.8245
Mercury Decidedecision model · API26.765544.245761.330867.0294Too few decisions139
Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs28.564252.646455.035266.034372.6214
Safestprogram · CPU29.178251.956965.434775.6230Too few decisions130
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs28.161054.747770.533779.7358107.4224
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell42.860254.946563.633875.0347105.6200
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs33.067366.744466.431185.7349128.4225
Randomprogram · CPU60.2278116.0238123.2209132.8307167.4305

Equity gap between the two best plays

Swipe for more columns →

Error rate (PR) by the gap between the two best plays; the small number is the decisions in the cell
Model and settingUnder 0.020.02 to 0.08Over 0.08
GNUbg weakprogram · CPU14.873825.454727.3459
Jev 1.13decision model · API20.270235.959949.4610
GLiDEv1.2 · NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs21.886040.159050.0553
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first release22.164337.554649.5547
celeris-1-decisiondecision model · API19.469539.361450.5613
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs24.674239.563448.5652
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs25.074337.965655.5617
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkdecision model · our GPUs22.975537.564658.5651
GPT-6 LunaDecisions APIdecision model · API24.973347.661861.0611
SubQ Snap Previewdecision model · API28.886341.468966.5648
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark29.075443.762465.7627
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark31.685250.173667.4605
Mercury Decidedecision model · API25.071146.357878.5564
Winnow-12BNVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs26.379952.662578.4591
Safestprogram · CPU32.276243.764982.6647
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs39.678258.760385.7621
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 Blackwell35.477161.061997.7562
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 Blackwelldecision model · our GPUs41.182966.960699.9567
Randomprogram · CPU59.4546126.0395201.3396

League 2

FinishedPublished

Risk judgement per phase

Swipe for more columns →

Risk judgement (margin 0.001) per game phase; N/A where a phase has no Risk pays or no Safety pays decisions
Model and settingOpeningContactHolding gameRaceBear-off
Gemini 3.8 Flashhighchat model, one letter · subscription80.5307 with a trade-off61.61,251 with a trade-off65.552 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Gemini 3.8 Flashlowchat model, one letter · subscription76.6300 with a trade-off60.21,171 with a trade-off62.471 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Claude Sonnet 5.5highchat model, one letter · subscription77.3299 with a trade-off65.01,281 with a trade-off70.846 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API80.8318 with a trade-off64.31,300 with a trade-off65.660 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Claude Fable 5.1low78.3289 with a trade-off63.91,156 with a trade-off64.839 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API79.5292 with a trade-off65.91,344 with a trade-off47.573 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Grok 4.7low77.3295 with a trade-off64.11,212 with a trade-off66.3106 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs74.7295 with a trade-off60.51,249 with a trade-off59.660 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Claude Sonnet 5.5lowchat model, one letter · subscription74.8312 with a trade-off59.41,221 with a trade-off62.246 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs69.7272 with a trade-off59.81,029 with a trade-off56.378 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GNUbg weakprogram · CPU53.5266 with a trade-off63.51,381 with a trade-off93.825 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs66.8285 with a trade-off60.31,170 with a trade-off57.775 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
d1-ultra70.1294 with a trade-off60.51,135 with a trade-off56.975 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GLM-5.3lowchat model, answer-letter probabilities · API71.2296 with a trade-off58.41,110 with a trade-off58.345 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Jev 1.1468.2272 with a trade-off56.51,060 with a trade-off45.035 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GLiDEAPI70.2293 with a trade-off56.81,216 with a trade-off58.466 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Qwen3.8 Flashoffchat model, answer-letter probabilities · API65.2298 with a trade-off57.21,110 with a trade-off46.223 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API62.1304 with a trade-off57.21,129 with a trade-off47.625 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark72.0278 with a trade-off53.71,174 with a trade-off61.112 with a trade-offN/A0 with a trade-offN/A0 with a trade-off

Hit risk only: being hit. Gammon risk and priming are not in it.

Game phase

Swipe for more columns →

Error rate (PR) per game phase; the small number is the decisions in the cell
Model and settingOpeningContactHolding gameRaceBear-off
Gemini 3.8 Flashhighchat model, one letter · subscription12.030820.21,363Too few decisions542.0244Too few decisions199
Gemini 3.8 Flashlowchat model, one letter · subscription13.230220.41,279Too few decisions836.8218Too few decisions171
Claude Sonnet 5.5highchat model, one letter · subscription13.730021.41,417Too few decisions563.9234Too few decisions157
DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API15.731821.61,439Too few decisions664.7238Too few decisions161
Claude Fable 5.1low13.929022.91,285Too few decisions545.9216Too few decisions163
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API16.929623.21,434Too few decisions862.3231Too few decisions141
Grok 4.7low17.629725.41,320Too few decisions1212.9217Too few decisions197
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs19.029824.31,367Too few decisions656.0228Too few decisions164
Claude Sonnet 5.5lowchat model, one letter · subscription17.531326.21,340Too few decisions566.0251Too few decisions190
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs18.827327.31,121Too few decisions8715.9224Too few decisions142
GNUbg weakprogram · CPU36.327130.31,502Too few decisions255.0247Too few decisions91
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs32.928834.21,305Too few decisions8215.5246Too few decisions187
d1-ultra27.729436.01,263Too few decisions9315.0263Too few decisions158
GLM-5.3lowchat model, answer-letter probabilities · API29.029736.41,257Too few decisions5021.3254Too few decisions131
Jev 1.1437.627235.11,237Too few decisions4016.1220Too few decisions143
GLiDEAPI25.029742.61,333Too few decisions7514.1291Too few decisions163
Qwen3.8 Flashoffchat model, answer-letter probabilities · API36.829837.11,337Too few decisions2720.6249Too few decisions116
DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API39.630541.51,299Too few decisions2818.7270Too few decisions132
ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark32.128041.81,361Too few decisions1526.7206Too few decisions116

Strategy tag (contact and holding positions after the opening)

Swipe for more columns →

Error rate (PR) per strategy tag; the small number is the decisions in the cell
Model and settingBlitzBack gameHolding gamePrimingLow-anchorOther
Gemini 3.8 Flashhighchat model, one letter · subscriptionToo few decisions195Too few decisions77Too few decisions5417.154622.1423Too few decisions122
Gemini 3.8 Flashlowchat model, one letter · subscription16.6268Too few decisions62Too few decisions8318.440825.033822.7203
Claude Sonnet 5.5highchat model, one letter · subscription21.1263Too few decisions51Too few decisions5622.057820.6374Too few decisions151
DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API18.2309Too few decisions77Too few decisions6620.754828.7337Too few decisions168
Claude Fable 5.1low17.0261Too few decisions78Too few decisions5423.350827.3325Too few decisions113
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API19.1334Too few decisions108Too few decisions8623.050427.7335Too few decisions153
Grok 4.7low23.4359Too few decisions52Too few decisions12127.847928.7281Too few decisions149
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs22.5296Too few decisions75Too few decisions6519.941726.8432Too few decisions147
Claude Sonnet 5.5lowchat model, one letter · subscription21.3297Too few decisions37Too few decisions5625.849429.2392Too few decisions120
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs26.2203Too few decisions59Too few decisions8728.437031.3343Too few decisions146
GNUbg weakprogram · CPU21.6597Too few decisions85Too few decisions2534.535435.326641.1200
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs34.1293Too few decisions48Too few decisions8230.440535.5381Too few decisions178
d1-ultra25.7266Too few decisions62Too few decisions9338.351841.8305Too few decisions112
GLM-5.3lowchat model, answer-letter probabilities · API27.5224Too few decisions31Too few decisions5033.556546.2349Too few decisions88
Jev 1.1426.5210Too few decisions56Too few decisions4033.347039.4371Too few decisions130
GLiDEAPI40.8286Too few decisions55Too few decisions7541.944148.4369Too few decisions182
Qwen3.8 Flashoffchat model, answer-letter probabilities · API39.3209Too few decisions54Too few decisions2734.364040.9362Too few decisions72
DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API34.1240Too few decisions23Too few decisions2842.950647.0423Too few decisions107
ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark44.5296Too few decisions84Too few decisions1540.251643.1341Too few decisions124

Number of legal plays

Swipe for more columns →

Error rate (PR) by number of legal plays; the small number is the decisions in the cell
Model and setting2–56–1011–1516–2526+
Gemini 3.8 Flashhighchat model, one letter · subscription7.066514.548318.636921.838520.6266
Gemini 3.8 Flashlowchat model, one letter · subscription7.764118.342716.734520.941828.3222
Claude Sonnet 5.5highchat model, one letter · subscription7.565418.546121.037321.440123.2275
DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API7.767918.343423.738020.344224.2287
Claude Fable 5.1low11.163117.144422.232722.536224.5244
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API9.059919.745221.639226.344023.5305
Grok 4.7low6.864421.144224.036025.443628.5270
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs8.865922.245321.133126.939928.5280
Claude Sonnet 5.5lowchat model, one letter · subscription9.867823.852424.435124.436329.0234
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs8.559321.641631.132430.6318Too few decisions196
GNUbg weakprogram · CPU14.446223.438826.429834.042034.7568
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs17.464826.146935.636433.439950.0228
d1-ultra10.561629.541938.831441.140447.9318
GLM-5.3lowchat model, answer-letter probabilities · API14.464533.846934.532843.133549.3212
Jev 1.1414.763933.039744.829037.133550.9251
GLiDEAPI17.465529.548640.933941.840655.0273
Qwen3.8 Flashoffchat model, answer-letter probabilities · API15.870236.544245.533743.3358Too few decisions188
DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API15.769240.845647.837345.4325Too few decisions188
ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark22.067041.243037.531942.534661.8213

Equity gap between the two best plays

Swipe for more columns →

Error rate (PR) by the gap between the two best plays; the small number is the decisions in the cell
Model and settingUnder 0.020.02 to 0.08Over 0.08
Gemini 3.8 Flashhighchat model, one letter · subscription8.790417.167722.1587
Gemini 3.8 Flashlowchat model, one letter · subscription10.578418.566321.5606
Claude Sonnet 5.5highchat model, one letter · subscription11.082117.866122.5682
DeepSeek-V4.1-Flashlow · API · 1017chat model, answer-letter probabilities · API10.186720.572322.9632
Claude Fable 5.1low12.979120.864221.5575
DeepSeek-V4.1-Flashlow · API + NVIDIA RTX PRO 6000 Blackwell · first releasechat model, answer-letter probabilities · API12.886321.067825.2647
Grok 4.7low11.490222.566627.2584
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs15.280722.069822.6617
Claude Sonnet 5.5lowchat model, one letter · subscription13.582620.669728.5627
Gemma 4 31Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs17.575622.558329.9508
GNUbg weakprogram · CPU17.295931.662538.8552
Qwen3.8-27Blow · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Sparkchat model, answer-letter probabilities · our GPUs17.984230.465142.9615
d1-ultra20.584233.166441.8565
GLM-5.3lowchat model, answer-letter probabilities · API19.179031.259245.9607
Jev 1.1419.678033.062048.6512
GLiDEAPI21.089335.463548.1631
Qwen3.8 Flashoffchat model, answer-letter probabilities · API18.285430.865862.6515
DeepSeek-V4.1-Flashoff · API · 1017chat model, answer-letter probabilities · API20.979532.160856.3631
ClefNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark24.279237.561752.7569

League 3

FinishedPublished

Risk judgement per phase

Swipe for more columns →

Risk judgement (margin 0.001) per game phase; N/A where a phase has no Risk pays or no Safety pays decisions
Model and settingOpeningContactHolding gameRaceBear-off
GPT-6.1 Solhigh82.4183 with a trade-off66.6765 with a trade-off72.377 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GPT-6 Astramediumchat model, one letter · subscription83.6183 with a trade-off71.1785 with a trade-off50.052 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GPT-6 Astralowchat model, one letter · subscription87.6177 with a trade-off71.9693 with a trade-off63.640 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GPT-6.1 Sollow80.1194 with a trade-off69.1767 with a trade-off65.464 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Gemini 4 Argonhighchat model, one letter · subscription82.1188 with a trade-off70.4708 with a trade-off48.642 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Claude Fable 5.1highchat model, one letter · subscription90.1174 with a trade-off66.1752 with a trade-off58.661 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Claude Opus 5.5highchat model, one letter · subscription80.4204 with a trade-off66.0813 with a trade-off55.653 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Claude Opus 5.5lowchat model, one letter · subscription76.6199 with a trade-off63.4781 with a trade-off56.263 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Gemini 4 Argonlowchat model, one letter · subscription79.1205 with a trade-off64.8762 with a trade-off61.869 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Grok 4.7highchat model, one letter · subscription80.2201 with a trade-off64.3635 with a trade-off52.484 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
Kimi K3.1 Prooffchat model, one letter · API76.6166 with a trade-off61.5615 with a trade-off54.545 with a trade-offN/A0 with a trade-offN/A0 with a trade-off
GNUbg weakprogram · CPU52.5178 with a trade-off64.2915 with a trade-off47.031 with a trade-offN/A0 with a trade-offN/A0 with a trade-off

Hit risk only: being hit. Gammon risk and priming are not in it.

Game phase

Swipe for more columns →

Error rate (PR) per game phase; the small number is the decisions in the cell
Model and settingOpeningContactHolding gameRaceBear-off
GPT-6.1 SolhighToo few decisions18313.6824Too few decisions85Too few decisions150Too few decisions147
GPT-6 Astramediumchat model, one letter · subscriptionToo few decisions18414.1887Too few decisions62Too few decisions137Too few decisions92
GPT-6 Astralowchat model, one letter · subscriptionToo few decisions17915.6799Too few decisions49Too few decisions136Too few decisions88
GPT-6.1 SollowToo few decisions19616.2826Too few decisions71Too few decisions128Too few decisions123
Gemini 4 Argonhighchat model, one letter · subscriptionToo few decisions19117.2814Too few decisions54Too few decisions153Too few decisions120
Claude Fable 5.1highchat model, one letter · subscriptionToo few decisions17618.7840Too few decisions74Too few decisions116Too few decisions80
Claude Opus 5.5highchat model, one letter · subscription13.520418.6925Too few decisions70Too few decisions137Too few decisions141
Claude Opus 5.5lowchat model, one letter · subscriptionToo few decisions19920.6856Too few decisions73Too few decisions161Too few decisions122
Gemini 4 Argonlowchat model, one letter · subscription10.420823.4856Too few decisions80Too few decisions184Too few decisions156
Grok 4.7highchat model, one letter · subscription17.620127.5724Too few decisions101Too few decisions161Too few decisions150
Kimi K3.1 Prooffchat model, one letter · APIToo few decisions16836.6717Too few decisions53Too few decisions158Too few decisions71
GNUbg weakprogram · CPUToo few decisions18031.61,026Too few decisions37Too few decisions160Too few decisions76

Strategy tag (contact and holding positions after the opening)

Swipe for more columns →

Error rate (PR) per strategy tag; the small number is the decisions in the cell
Model and settingBlitzBack gameHolding gamePrimingLow-anchorOther
GPT-6.1 Solhigh11.0223Too few decisions49Too few decisions8513.6300Too few decisions148Too few decisions104
GPT-6 Astramediumchat model, one letter · subscription12.7239Too few decisions86Too few decisions6213.8311Too few decisions186Too few decisions65
GPT-6 Astralowchat model, one letter · subscriptionToo few decisions192Too few decisions57Too few decisions4915.9366Too few decisions136Too few decisions48
GPT-6.1 SollowToo few decisions190Too few decisions60Too few decisions7117.7326Too few decisions152Too few decisions98
Gemini 4 Argonhighchat model, one letter · subscriptionToo few decisions163Too few decisions58Too few decisions5417.5342Too few decisions145Too few decisions106
Claude Fable 5.1highchat model, one letter · subscription20.9208Too few decisions154Too few decisions7415.9286Too few decisions110Too few decisions82
Claude Opus 5.5highchat model, one letter · subscription15.3201Too few decisions90Too few decisions7014.9341Too few decisions159Too few decisions134
Claude Opus 5.5lowchat model, one letter · subscriptionToo few decisions170Too few decisions81Too few decisions7318.5315Too few decisions181Too few decisions109
Gemini 4 Argonlowchat model, one letter · subscriptionToo few decisions173Too few decisions62Too few decisions8025.6377Too few decisions122Too few decisions122
Grok 4.7highchat model, one letter · subscriptionToo few decisions144Too few decisions14Too few decisions10126.0341Too few decisions148Too few decisions77
Kimi K3.1 Prooffchat model, one letter · APIToo few decisions177Too few decisions36Too few decisions5328.4245Too few decisions167Too few decisions92
GNUbg weakprogram · CPU20.6291Too few decisions142Too few decisions3733.5250Too few decisions171Too few decisions172

Number of legal plays

Swipe for more columns →

Error rate (PR) by number of legal plays; the small number is the decisions in the cell
Model and setting2–56–1011–1516–2526+
GPT-6.1 Solhigh5.63429.025610.221512.031113.6265
GPT-6 Astramediumchat model, one letter · subscription5.83818.727113.823012.3289Too few decisions191
GPT-6 Astralowchat model, one letter · subscription5.03339.527112.521016.3255Too few decisions182
GPT-6.1 Sollow6.936511.027413.822313.425718.9225
Gemini 4 Argonhighchat model, one letter · subscription4.039511.925717.120617.227319.3201
Claude Fable 5.1highchat model, one letter · subscription7.838315.5234Too few decisions19918.024616.9224
Claude Opus 5.5highchat model, one letter · subscription6.844415.532712.325924.8260Too few decisions187
Claude Opus 5.5lowchat model, one letter · subscription7.740918.129718.123517.7277Too few decisions193
Gemini 4 Argonlowchat model, one letter · subscription6.140616.531716.823825.130823.2215
Grok 4.7highchat model, one letter · subscription4.335118.828326.424727.124824.2208
Kimi K3.1 Prooffchat model, one letter · API15.934627.9246Too few decisions19337.1237Too few decisions145
GNUbg weakprogram · CPU10.036225.628532.822533.631043.1297

Equity gap between the two best plays

Swipe for more columns →

Error rate (PR) by the gap between the two best plays; the small number is the decisions in the cell
Model and settingUnder 0.020.02 to 0.08Over 0.08
GPT-6.1 Solhigh6.462414.240611.1359
GPT-6 Astramediumchat model, one letter · subscription7.956911.842715.3366
GPT-6 Astralowchat model, one letter · subscription7.652615.338814.4337
GPT-6.1 Sollow8.257113.140917.1364
Gemini 4 Argonhighchat model, one letter · subscription8.359415.439416.8344
Claude Fable 5.1highchat model, one letter · subscription8.851919.541515.0352
Claude Opus 5.5highchat model, one letter · subscription11.656115.646416.7452
Claude Opus 5.5lowchat model, one letter · subscription10.761618.940319.3392
Gemini 4 Argonlowchat model, one letter · subscription8.867119.645526.8358
Grok 4.7highchat model, one letter · subscription8.858817.842338.1326
Kimi K3.1 Prooffchat model, one letter · API19.151533.032737.2325
GNUbg weakprogram · CPU18.464234.544736.8390

Earlier version: the v1 set

Earlier version

Game phase

Swipe for more columns →

Error rate (PR) per game phase; the small number is the decisions in the cell
Model and settingOpeningContactHolding gameRaceBear-off
GPT-6 Astrachat model, one letter · subscriptionToo few decisions14315.1641Too few decisions20Too few decisions136Too few decisions60
Gemini 4 Argonchat model, one letter · APIToo few decisions14322.4641Too few decisions20Too few decisions136Too few decisions60
Claude Opus 5.5chat model, one letter · subscriptionToo few decisions14323.0641Too few decisions20Too few decisions136Too few decisions60
DeepSeek-V4.1-Flashthinking · first releasechat model, one letter · our GPUsToo few decisions14323.4641Too few decisions20Too few decisions136Too few decisions60
Gemini 3.8 Flashchat model, one letter · APIToo few decisions14323.6641Too few decisions20Too few decisions136Too few decisions60
Gemini 4 Argonshuffledchat model, one letter · API · shuffled play orderToo few decisions14325.7641Too few decisions20Too few decisions136Too few decisions60
Gemma 4 31Bchat model, answer-letter probabilities · our GPUsToo few decisions14329.0641Too few decisions20Too few decisions136Too few decisions60
DeepSeek-V4.1-Flashfirst releasechat model, answer-letter probabilities · our GPUsToo few decisions14333.7641Too few decisions20Too few decisions136Too few decisions60
Qwen3.8 Flash Nextchat model, answer-letter probabilities · our GPUsToo few decisions14337.9641Too few decisions20Too few decisions136Too few decisions60
Perplexity Decider 27B v1.1decision model · our GPUsToo few decisions14341.2641Too few decisions20Too few decisions136Too few decisions60
Jev 1.13decision model · APIToo few decisions14339.2641Too few decisions20Too few decisions136Too few decisions60
Clefdecision model · our GPUsToo few decisions14342.8641Too few decisions20Too few decisions136Too few decisions60
GPT-6 LunaDecisions APIdecision model · APIToo few decisions14340.5641Too few decisions20Too few decisions136Too few decisions60
Cygnetdecision model · our GPUsToo few decisions14345.5641Too few decisions20Too few decisions136Too few decisions60
GLM-4.7-Flashchat model, answer-letter probabilities · our GPUsToo few decisions14361.3641Too few decisions20Too few decisions136Too few decisions60
Open-Jev 27Bdecision model · our GPUsToo few decisions14382.0641Too few decisions20Too few decisions136Too few decisions60
GLM-5.3-Flash NVFP4chat model, answer-letter probabilitiesToo few decisions143106.4641Too few decisions20Too few decisions136Too few decisions60

Strategy tag (contact and holding positions after the opening)

Swipe for more columns →

Error rate (PR) per strategy tag; the small number is the decisions in the cell
Model and settingBlitzBack gameHolding gamePrimingLow-anchorOther
GPT-6 Astrachat model, one letter · subscriptionToo few decisions143Too few decisions30Too few decisions2014.7235Too few decisions168Too few decisions65
Gemini 4 Argonchat model, one letter · APIToo few decisions143Too few decisions30Too few decisions2020.1235Too few decisions168Too few decisions65
Claude Opus 5.5chat model, one letter · subscriptionToo few decisions143Too few decisions30Too few decisions2019.6235Too few decisions168Too few decisions65
DeepSeek-V4.1-Flashthinking · first releasechat model, one letter · our GPUsToo few decisions143Too few decisions30Too few decisions2021.8235Too few decisions168Too few decisions65
Gemini 3.8 Flashchat model, one letter · APIToo few decisions143Too few decisions30Too few decisions2018.6235Too few decisions168Too few decisions65
Gemini 4 Argonshuffledchat model, one letter · API · shuffled play orderToo few decisions143Too few decisions30Too few decisions2023.6235Too few decisions168Too few decisions65
Gemma 4 31Bchat model, answer-letter probabilities · our GPUsToo few decisions143Too few decisions30Too few decisions2028.1235Too few decisions168Too few decisions65
DeepSeek-V4.1-Flashfirst releasechat model, answer-letter probabilities · our GPUsToo few decisions143Too few decisions30Too few decisions2033.6235Too few decisions168Too few decisions65
Qwen3.8 Flash Nextchat model, answer-letter probabilities · our GPUsToo few decisions143Too few decisions30Too few decisions2035.0235Too few decisions168Too few decisions65
Perplexity Decider 27B v1.1decision model · our GPUsToo few decisions143Too few decisions30Too few decisions2041.0235Too few decisions168Too few decisions65
Jev 1.13decision model · APIToo few decisions143Too few decisions30Too few decisions2040.6235Too few decisions168Too few decisions65
Clefdecision model · our GPUsToo few decisions143Too few decisions30Too few decisions2043.3235Too few decisions168Too few decisions65
GPT-6 LunaDecisions APIdecision model · APIToo few decisions143Too few decisions30Too few decisions2037.6235Too few decisions168Too few decisions65
Cygnetdecision model · our GPUsToo few decisions143Too few decisions30Too few decisions2045.7235Too few decisions168Too few decisions65
GLM-4.7-Flashchat model, answer-letter probabilities · our GPUsToo few decisions143Too few decisions30Too few decisions2062.4235Too few decisions168Too few decisions65
Open-Jev 27Bdecision model · our GPUsToo few decisions143Too few decisions30Too few decisions2080.1235Too few decisions168Too few decisions65
GLM-5.3-Flash NVFP4chat model, answer-letter probabilitiesToo few decisions143Too few decisions30Too few decisions20106.5235Too few decisions168Too few decisions65

Number of legal plays

Swipe for more columns →

Error rate (PR) by number of legal plays; the small number is the decisions in the cell
Model and setting2–56–1011–1516–2526+
GPT-6 Astrachat model, one letter · subscription8.038510.8234Too few decisions16812.9213Too few decisions0
Gemini 4 Argonchat model, one letter · API7.838517.9234Too few decisions16824.1213Too few decisions0
Claude Opus 5.5chat model, one letter · subscription10.538518.0234Too few decisions16820.3213Too few decisions0
DeepSeek-V4.1-Flashthinking · first releasechat model, one letter · our GPUs10.838518.5234Too few decisions16824.3213Too few decisions0
Gemini 3.8 Flashchat model, one letter · API11.538518.5234Too few decisions16821.8213Too few decisions0
Gemini 4 Argonshuffledchat model, one letter · API · shuffled play order8.638520.4234Too few decisions16826.5213Too few decisions0
Gemma 4 31Bchat model, answer-letter probabilities · our GPUs16.438523.7234Too few decisions16832.4213Too few decisions0
DeepSeek-V4.1-Flashfirst releasechat model, answer-letter probabilities · our GPUs14.538531.5234Too few decisions16837.0213Too few decisions0
Qwen3.8 Flash Nextchat model, answer-letter probabilities · our GPUs17.438536.7234Too few decisions16841.8213Too few decisions0
Perplexity Decider 27B v1.1decision model · our GPUs19.038535.5234Too few decisions16844.5213Too few decisions0
Jev 1.13decision model · API20.738535.6234Too few decisions16841.4213Too few decisions0
Clefdecision model · our GPUs21.038536.8234Too few decisions16843.8213Too few decisions0
GPT-6 LunaDecisions APIdecision model · API18.138541.9234Too few decisions16848.5213Too few decisions0
Cygnetdecision model · our GPUs25.938541.5234Too few decisions16847.6213Too few decisions0
GLM-4.7-Flashchat model, answer-letter probabilities · our GPUs41.838549.6234Too few decisions16865.6213Too few decisions0
Open-Jev 27Bdecision model · our GPUs35.138567.0234Too few decisions16885.5213Too few decisions0
GLM-5.3-Flash NVFP4chat model, answer-letter probabilities28.538588.4234Too few decisions168139.3213Too few decisions0

Equity gap between the two best plays

Swipe for more columns →

Error rate (PR) by the gap between the two best plays; the small number is the decisions in the cell
Model and settingUnder 0.020.02 to 0.08Over 0.08
GPT-6 Astrachat model, one letter · subscription6.540913.629115.3300
Gemini 4 Argonchat model, one letter · API9.740917.729126.0300
Claude Opus 5.5chat model, one letter · subscription10.840918.929124.4300
DeepSeek-V4.1-Flashthinking · first releasechat model, one letter · our GPUs11.240921.329124.5300
Gemini 3.8 Flashchat model, one letter · API11.040922.429124.1300
Gemini 4 Argonshuffledchat model, one letter · API · shuffled play order12.440921.129125.6300
Gemma 4 31Bchat model, answer-letter probabilities · our GPUs14.140924.529135.5300
DeepSeek-V4.1-Flashfirst releasechat model, answer-letter probabilities · our GPUs17.540928.429140.1300
Qwen3.8 Flash Nextchat model, answer-letter probabilities · our GPUs20.840929.029147.2300
Perplexity Decider 27B v1.1decision model · our GPUs20.440931.229147.9300
Jev 1.13decision model · API22.940935.029146.9300
Clefdecision model · our GPUs23.540934.829147.7300
GPT-6 LunaDecisions APIdecision model · API23.440933.429152.3300
Cygnetdecision model · our GPUs24.640939.529157.8300
GLM-4.7-Flashchat model, answer-letter probabilities · our GPUs27.940949.829182.2300
Open-Jev 27Bdecision model · our GPUs37.740966.629183.5300
GLM-5.3-Flash NVFP4chat model, answer-letter probabilities55.140995.4291117.1300

Earlier version: plays were listed in the runner's order, which tends to put the best play early. When Gemini 4 Argon answered the same 1,000 positions with the plays shuffled, its error rate changed by +1.9 (95% interval −0.2 to +4.2), so the effect is not proven.

About these tables

Error rate by game phase, strategy tag, number of legal plays and the equity gap between the two best plays. Each league on its own; the shared-position version waits for the answers. A cell with fewer decisions than the minimum shows the status instead of a value.