Experiments

Nackgammon

Results f24d1f41, updated 16 November 2026, 21:24 UTC; scored by GNU Backgammon 1.08, 2-ply.

Nackgammon paired test

Does a model still play well when the starting position changes?

Class:
Class: by the maker's total parameters, or by list price when the size is undisclosed *

Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3.

Swipe for more columns →

The primary rule (an invalid answer scores the position's Random error), final look: positive drop = worse on Nackgammon. Grey rows: the GNUbg weak and Random yardsticks.
Setup (group)PairsStandard PRNackgammon PRDrop (Nackgammon − standard)97.5% intervalLabel
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group50026.3327.93+1.60[−5.33, +8.30]unclear
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read50025.7328.80+3.07[−3.84, +9.81]unclear
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read50025.3031.47+6.17[−2.06, +13.95]unclear
Jev 1.13Closed · EfficientLeague 1 group50024.6727.44+2.77[−5.20, +10.44]unclear
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group50029.5635.44+5.87[−4.27, +15.07]unclear
celeris-1-decisionClosed · EfficientLeague 1 group50026.8630.93+4.08[−4.58, +12.64]unclear
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group50031.2732.08+0.81[−7.98, +9.24]unclear
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group50028.4628.66+0.20[−7.64, +7.43]unclear
SubQ Snap PreviewClosed · EfficientLeague 1 group50031.2532.87+1.62[−7.07, +10.07]unclear
Mercury DecideClosed · EfficientLeague 1 group50031.3434.08+2.73[−5.94, +10.85]unclear
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group50031.9334.61+2.68[−6.26, +11.16]unclear
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group50031.7229.03−2.68[−11.03, +5.25]unclear
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group50040.2348.01+7.78[−2.16, +18.39]unclear
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group50047.0743.99−3.08[−13.33, +6.76]unclear
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group50011.149.12−2.03[−5.57, +1.38]unclear
GPT-6.1 SollowClosed · WorkhorseLeague 2 group50012.2110.75−1.46[−5.01, +2.05]unclear
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group50016.4515.30−1.15[−5.89, +3.32]unclear
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group50016.7115.41−1.30[−5.48, +2.90]unclear
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group50017.9716.84−1.13[−6.19, +4.39]unclear
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group50023.3529.29+5.94[−2.48, +13.87]unclear
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group50026.8931.65+4.76[−3.56, +12.53]unclear
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group50037.7842.54+4.77[−4.14, +13.49]unclear
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group50046.4450.87+4.43[−6.53, +15.38]unclear
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read5009.6410.80+1.16[−2.17, +4.40]kept
GPT-6 AstramediumClosed · FlagshipLeague 3 group50010.868.88−1.99[−5.52, +1.47]unclear
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read50015.6816.53+0.85[−3.74, +5.35]unclear
GPT-6.1 Terrano group (exploratory)50013.1816.22+3.04[−0.94, +7.39]unclear
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)50014.7117.42+2.71[−2.06, +7.62]unclear
GNUbg weakyardstick50025.3520.78−4.57[−10.32, +1.34]unclear
Randomyardstick50085.1681.43−3.73[−11.43, +3.74]unclear

Labels, from the final 97.5% interval (Method §10, outcome label (c)): kept = the interval lies within ±5 PR; dropped = the data file's label “worse”: the interval lies above 0 (lower end > 0) and the setup is not kept; better = the interval lies below 0 and the setup is not kept; unclear = anything else.

Unclear means the test can’t tell, not that the scores are equal. 4 setups were added after the results were read; the primary results are shown with and without them. *

Random play's own drop is −4.6 PR on the main 200 pairs: the Nackgammon positions were slightly easier for random play, so a drop mixes how a model copes with the new start with how hard the positions are; the test cannot separate the two.

One setting per model: a model asked at several settings shows the one with its highest stored risk judgement in league play, on an exact tie the first in league order (chosen from league results, never from this test); All settings shows every setup asked (2 more rows). Decision / Chat is the model type; after each name is its model class (the Class row filters every table and the chart) and under it the paired test's group, the setups from one original league, so a chat model can sit in the League 1 group. Every table further down shows every setup.

Late entries / exploratory

Added after the results were known; exploratory; not in any group comparison (§10.14(i)). Late entry, exploratory (Method §10.14(i), 2026-11-07): added after this test's results had been computed and read; no group, no primary comparison, M unchanged; same rules and scoring as every setup.

Swipe for more columns →

Late entries, the primary rule, final look; Random on the same pairs as the grey row.
SetupPairsStandard PRNackgammon PRChange (Nackgammon − standard)97.5% intervalLabel
SubQ Snap 1.1late entry, exploratory · its answers equal SubQ Snap Preview's answers of 27 October on all 1,000 positions (the same letters and probability vectors)50031.2532.87+1.62[−7.07, +10.07]unclear
Drex Max v1.1late entry, exploratory50028.4029.09+0.69[−6.03, +7.48]unclear
Drex 2.0late entry, exploratory50029.3931.35+1.95[−6.39, +9.95]unclear
Elephant-1late entry, exploratory50025.2333.89+8.66[+0.42, +16.02]dropped
Mammoth-1late entry, exploratory50022.9724.81+1.84[−5.72, +8.72]unclear
Drex Maxlate entry, exploratory50026.2127.92+1.71[−6.67, +9.37]unclear
Drex 2.0locallate entry, exploratory50029.1930.79+1.60[−6.90, +9.75]unclear
Randomyardstick50085.1681.43−3.73[−11.43, +3.74]unclear

Labels, from the final 97.5% interval (Method §10, outcome label (c)): kept = the interval lies within ±5 PR; dropped = the data file's label “worse”: the interval lies above 0 (lower end > 0) and the setup is not kept; better = the interval lies below 0 and the setup is not kept; unclear = anything else.

REAX 1.3: not run (pending approval).

Data: nack-paired-late.json

Swipe for more columns →

Per setup: standard PR, Nackgammon PR and the drop (Nackgammon − standard; positive = worse on Nackgammon)
Setup (group)Standard PR, main 200 pairsNackgammon PR, main 200 pairsDrop, main 200 pairsFirst look 97.5% intervalRandom's drop, main 200 pairsExtended to 500 pairsPairs, final lookStandard PR, finalNackgammon PR, finalDrop, finalFinal 97.5% intervalRandom's drop, same pairs as the final lookLabel (final interval)Positions without a valid choice, main 200 pairs (standard / Nackgammon)Positions without a valid choice, final pairs (standard / Nackgammon)
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group22.7424.91+2.18[−7.28, +11.93]−4.56yes50026.3327.93+1.60[−5.33, +8.30]−3.73unclear0 / 00 / 0
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read22.9323.38+0.45[−8.03, +8.77]−4.56yes50025.7328.80+3.07[−3.84, +9.81]−3.73unclear0 / 00 / 0
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read24.2528.27+4.01[−11.37, +17.17]−4.56yes50025.3031.47+6.17[−2.06, +13.95]−3.73unclear0 / 00 / 0
Jev 1.13Closed · EfficientLeague 1 group26.5324.17−2.37[−17.72, +9.97]−4.56yes50024.6727.44+2.77[−5.20, +10.44]−3.73unclear0 / 00 / 0
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group28.2734.11+5.84[−10.57, +20.46]−4.56yes50029.5635.44+5.87[−4.27, +15.07]−3.73unclear0 / 00 / 0
celeris-1-decisionClosed · EfficientLeague 1 group29.7128.47−1.24[−18.27, +14.59]−4.56yes50026.8630.93+4.08[−4.58, +12.64]−3.73unclear0 / 00 / 0
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group30.4228.64−1.78[−17.30, +11.94]−4.56yes50031.2732.08+0.81[−7.98, +9.24]−3.73unclear0 / 00 / 0
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group31.3324.48−6.85[−22.36, +5.43]−4.56yes50028.4628.66+0.20[−7.64, +7.43]−3.73unclear0 / 00 / 0
SubQ Snap PreviewClosed · EfficientLeague 1 group31.9129.66−2.25[−18.09, +11.88]−4.56yes50031.2532.87+1.62[−7.07, +10.07]−3.73unclear0 / 00 / 0
Mercury DecideClosed · EfficientLeague 1 group32.7428.46−4.28[−20.11, +8.75]−4.56yes50031.3434.08+2.73[−5.94, +10.85]−3.73unclear0 / 00 / 0
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group34.3333.43−0.89[−17.96, +14.97]−4.56yes50031.9334.61+2.68[−6.26, +11.16]−3.73unclear0 / 00 / 0
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group34.6822.80−11.87[−27.93, +1.03]−4.56yes50031.7229.03−2.68[−11.03, +5.25]−3.73unclear0 / 00 / 0
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group39.7742.86+3.09[−12.09, +20.11]−4.56yes50040.2348.01+7.78[−2.16, +18.39]−3.73unclear0 / 00 / 0
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group52.0840.96−11.12[−29.52, +6.79]−4.56yes50047.0743.99−3.08[−13.33, +6.76]−3.73unclear0 / 00 / 0
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group9.888.43−1.45[−5.57, +2.68]−4.56yes50011.149.12−2.03[−5.57, +1.38]−3.73unclear0 / 00 / 0
GPT-6.1 SollowClosed · WorkhorseLeague 2 group11.159.90−1.26[−5.70, +3.31]−4.56yes50012.2110.75−1.46[−5.01, +2.05]−3.73unclear0 / 00 / 0
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group13.9614.50+0.54[−5.04, +6.02]−4.56yes50016.4515.30−1.15[−5.89, +3.32]−3.73unclear0 / 00 / 0
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group15.9211.81−4.11[−9.72, +1.04]−4.56yes50016.7115.41−1.30[−5.48, +2.90]−3.73unclear0 / 00 / 0
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group18.3513.38−4.97[−11.81, +1.78]−4.56yes50017.9716.84−1.13[−6.19, +4.39]−3.73unclear0 / 00 / 0
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group23.9127.90+3.98[−11.72, +17.96]−4.56yes50023.3529.29+5.94[−2.48, +13.87]−3.73unclear0 / 00 / 0
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group28.1328.43+0.30[−14.48, +12.12]−4.56yes50026.8931.65+4.76[−3.56, +12.53]−3.73unclear0 / 00 / 0
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group38.2037.55−0.65[−15.17, +14.42]−4.56yes50037.7842.54+4.77[−4.14, +13.49]−3.73unclear22 / 1655 / 52
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group50.7553.98+3.23[−16.93, +22.07]−4.56yes50046.4450.87+4.43[−6.53, +15.38]−3.73unclear0 / 00 / 0
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read9.139.51+0.38[−4.64, +5.07]−4.56yes5009.6410.80+1.16[−2.17, +4.40]−3.73kept0 / 00 / 0
GPT-6 AstramediumClosed · FlagshipLeague 3 group9.667.30−2.37[−6.69, +1.77]−4.56yes50010.868.88−1.99[−5.52, +1.47]−3.73unclear0 / 00 / 0
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read15.2014.53−0.67[−6.58, +5.07]−4.56yes50015.6816.53+0.85[−3.74, +5.35]−3.73unclear6 / 916 / 18
GPT-6.1 Terrano group (exploratory)13.5214.40+0.89[−5.73, +8.59]−4.56yes50013.1816.22+3.04[−0.94, +7.39]−3.73unclear0 / 00 / 0
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)13.7915.71+1.92[−4.33, +8.42]−4.56yes50014.7117.42+2.71[−2.06, +7.62]−3.73unclear0 / 00 / 0
GNUbg weakyardstick27.6924.13−3.56[−13.92, +7.29]−4.56yes50025.3520.78−4.57[−10.32, +1.34]−3.73unclear0 / 00 / 0
Randomyardstick87.4582.89−4.56[−18.04, +7.86]−4.56no50085.1681.43−3.73[−11.43, +3.74]−3.73unclear0 / 00 / 0

LFM2.5-24B-A2Bon: 107 positions recorded without a valid choice (timeouts or missing choices), 38 in the main 200 pairs and 69 in the reserve; on its final 500 pairs 55 standard / 52 Nackgammon; Grok 4.7high: 34 positions recorded without a valid choice (timeouts or missing choices), 15 in the main 200 pairs and 19 in the reserve; on its final 500 pairs 16 standard / 18 Nackgammon. They are not answers: each is scored by the primary rule (the position's Random error), with both sensitivity checks below. Every other asked setup gave a valid choice on every recorded position.

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Additions. Added under §10.14(g) after this test's results had been computed and read: d1-ultra and GLiDE (League 1 group); Claude Fable 5.1high and Grok 4.7high (League 3 group). They follow the Cup roster, not the results. The primary comparisons are shown both ways: with the additions (the primary result, §10.14(b)) and with the original members only (22 setups, §10.14(d)).

Nackgammon has the rules of backgammon; only the starting position differs. Each setup was asked 200 Nackgammon positions and 200 standard positions matched to them by phase, position class, number of legal plays and the gap between the two best plays. Error rate (PR) = 500 × mean GNUbg 2-ply equity loss. Drop = Nackgammon PR − standard PR on the same pairs; positive = worse on Nackgammon. A setup whose first-look 97.5% interval was wide and contained 0 was asked the 300 reserve pairs too (500 pairs). Every recorded position is scored: an invalid, timed-out or missing answer scores the position's Random error, with the sensitivity checks below. Label from the final 97.5% interval: kept (within ±5 PR), dropped (the data's “worse”), better or unclear. “Unclear” means the interval is too wide to decide; it is not evidence of equivalence.

A larger drop is consistent with relying on familiar positions but is not proof: Nackgammon positions also differ in structure. “Kept” is a valid result.

The paired test's standard-side prompt differs from the league prompt (no opening table; the starting setup on its own line), so the standard error rates here are not compared with league error rates.

Random's own drop: −4.56 PR on the main 200 pairs, −3.73 on all 500 (Random scores slightly better on the Nackgammon positions). It is shown next to every result. Random is computed exactly from GNUbg's 2-ply values (mean loss over all legal plays): it is never asked and is not extended by the stopping rule; it is reported on the main 200 pairs and on all 500.

No group (exploratory), in no group comparison: GPT-6.1 Terra (a setup that had played in no league before this test, §10.14(b)); Gemini 4 Argonlow, added by Can's decision of 2026-10-28 instead of Gemini 4 Argonhigh, not because of the Cup roster (§10.14(h)).

Not a ranking: rows are in fixed order (group, then standard PR) and no table can be sorted.

Standard against Nackgammon error rate

00101020203030404050506060707080809090100100Standard PR, main 200 pairsNackgammon PR, main 200 pairsGLiDE (v1.2 · NVIDIA RTX PRO 6000 Blackwell) (Open · 27B · League 1 group): standard PR 22.74, Nackgammon PR 24.91GLiDE (API) (Closed · Efficient · League 1 group): standard PR 22.93, Nackgammon PR 23.38d1-ultra (Closed · Efficient · League 1 group): standard PR 24.25, Nackgammon PR 28.27Jev 1.13 (Closed · Efficient · League 1 group): standard PR 26.53, Nackgammon PR 24.17GPT-6 Luna (Closed · Efficient · League 1 group): standard PR 28.27, Nackgammon PR 34.11celeris-1-decision (Closed · Efficient · League 1 group): standard PR 29.71, Nackgammon PR 28.47Perplexity Decider 27B v1.1 (NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark) (Open · 27B · League 1 group): standard PR 30.42, Nackgammon PR 28.64Cygnet (NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark) (Open · 12B · League 1 group): standard PR 31.33, Nackgammon PR 24.48SubQ Snap Preview (Closed · Efficient · League 1 group): standard PR 31.91, Nackgammon PR 29.66Mercury Decide (Closed · Efficient · League 1 group): standard PR 32.74, Nackgammon PR 28.46Winnow-12B (NVIDIA RTX PRO 6000 Blackwell) (Open · 12B · League 1 group): standard PR 34.33, Nackgammon PR 33.43Perplexity Decider 27B v1.2 (NVIDIA RTX PRO 6000 Blackwell) (Open · 27B · League 1 group): standard PR 34.68, Nackgammon PR 22.80Perplexity Decider Flash v1.2 (NVIDIA RTX PRO 6000 Blackwell) (Open · 4.0B · League 1 group): standard PR 39.77, Nackgammon PR 42.86Open-Jev-27B-v1.1 (NVIDIA RTX PRO 6000 Blackwell) (Open · 27B · League 1 group): standard PR 52.08, Nackgammon PR 40.96GPT-6.1 Sol (high) (Closed · Workhorse · League 2 group): standard PR 9.88, Nackgammon PR 8.43GPT-6.1 Sol (low) (Closed · Workhorse · League 2 group): standard PR 11.15, Nackgammon PR 9.90Claude Sonnet 5.5 (low) (Closed · Workhorse · League 2 group): standard PR 13.96, Nackgammon PR 14.50Gemini 3.8 Flash (low) (Closed · Workhorse · League 2 group): standard PR 15.92, Nackgammon PR 11.81Gemma 4 31B (on · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark) (Open · 31B · League 2 group): standard PR 18.35, Nackgammon PR 13.38DeepSeek-V4.1-Flash (off · API + NVIDIA RTX PRO 6000 Blackwell · first release) (Open · 552B MoE · 16B active · League 2 group): standard PR 23.91, Nackgammon PR 27.90Qwen3.8-27B (off · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark) (Open · 27B · League 2 group): standard PR 28.13, Nackgammon PR 28.43LFM2.5-24B-A2B (on · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX Spark) (Open · 24B MoE · 2.5B active · League 2 group): standard PR 38.20, Nackgammon PR 37.55LFM2.5-24B-A2B (off · NVIDIA RTX PRO 6000 Blackwell) (Open · 24B MoE · 2.5B active · League 2 group): standard PR 50.75, Nackgammon PR 53.98Claude Fable 5.1 (high) (Closed · Flagship · League 3 group): standard PR 9.13, Nackgammon PR 9.51GPT-6 Astra (medium) (Closed · Flagship · League 3 group): standard PR 9.66, Nackgammon PR 7.30Grok 4.7 (high) (Closed · Workhorse · League 3 group): standard PR 15.20, Nackgammon PR 14.53GPT-6.1 Terra (no group (exploratory)): standard PR 13.52, Nackgammon PR 14.40Gemini 4 Argon (low) (Closed · Flagship · no group (exploratory)): standard PR 13.79, Nackgammon PR 15.71GNUbg weak (yardstick): standard PR 27.69, Nackgammon PR 24.13GNUbg weakRandom (yardstick): standard PR 87.45, Nackgammon PR 82.89Random

OpenAIGoogleAnthropicxAIDeepSeekLiquid AIAlibabaInceptionFastinoPerplexitySubquadraticCelerisEldanRingTypeSafeZefan CaiBlockbrain Labs GNUbg weak, Random (programs) Above the dashed diagonal: higher error on Nackgammon.

Primary comparisons

Swipe for more columns →

With the §10.14(g) additions (primary result). Primary rule: an invalid or missing answer scores the position's Random error, 99% intervals (M = 5)
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group31.5529.61−1.9323.3622.88−0.49−1.44[−8.85, +4.96]−4.56
League 1 group − League 3 group31.5529.61−1.9311.3310.45−0.89−1.04[−14.19, +10.26]−4.56
League 2 group − League 3 group23.3622.88−0.4911.3310.45−0.89+0.40[−6.68, +6.96]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff38.2037.55−0.6550.7553.98+3.23−3.88[−26.98, +21.76]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Swipe for more columns →

With the §10.14(g) additions (primary result). Sensitivity (i): an invalid or missing answer scores the worst legal play's loss
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group31.5529.61−1.9324.9023.70−1.20−0.73[−8.24, +5.88]−4.56
League 1 group − League 3 group31.5529.61−1.9311.4510.64−0.81−1.12[−14.14, +10.21]−4.56
League 2 group − League 3 group24.9023.70−1.2011.4510.64−0.81−0.39[−7.62, +6.32]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff52.0644.97−7.1050.7553.98+3.23−10.33[−38.92, +19.74]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Swipe for more columns →

With the §10.14(g) additions (primary result). Sensitivity (ii): the pair is left out for that setup
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group31.5529.61−1.9322.5222.36−0.16−1.77[−9.22, +4.92]−4.56
League 1 group − League 3 group31.5529.61−1.9311.5810.64−0.94−0.99[−14.18, +10.32]−4.56
League 2 group − League 3 group22.5222.36−0.1611.5810.64−0.94+0.78[−6.32, +7.42]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff30.5932.91+2.3250.7553.98+3.23−0.91[−24.13, +25.20]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Swipe for more columns →

With the §10.14(g) additions (primary result). Sensitivity (iii): a pair with an answer cut off at the output limit is left out for that setup
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group31.5529.61−1.9323.3622.88−0.49−1.44[−8.85, +4.96]−4.56
League 1 group − League 3 group31.5529.61−1.9311.3310.45−0.89−1.04[−14.19, +10.26]−4.56
League 2 group − League 3 group23.3622.88−0.4911.3310.45−0.89+0.40[−6.68, +6.96]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff38.2037.55−0.6550.7553.98+3.23−3.88[−26.98, +21.76]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Each difference is A's drop − B's drop (a group's drop = the mean of its setups' drops), main 200 pairs, paired bootstrap. Five comparisons, so each interval is at 99% (Bonferroni, 1 − 0.05/5). Both sides' standard and Nackgammon PR are shown next to every difference. Shown both ways: with the §10.14(g) additions (the primary result) and with the original members only.

Original members only

Swipe for more columns →

Original members only. Primary rule: an invalid or missing answer scores the position's Random error, 99% intervals (M = 5)
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group32.8830.25−2.6323.3622.88−0.49−2.14[−10.07, +4.88]−4.56
League 1 group − League 3 group32.8830.25−2.639.667.30−2.37−0.26[−14.64, +12.04]−4.56
League 2 group − League 3 group23.3622.88−0.499.667.30−2.37+1.88[−5.73, +9.33]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff38.2037.55−0.6550.7553.98+3.23−3.88[−26.98, +21.76]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

Original members only: the League 3 group has 1 setup (GPT-6 Astramedium); comparisons 2 and 3 rest on it.

Swipe for more columns →

Original members only. Sensitivity (i): an invalid or missing answer scores the worst legal play's loss
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group32.8830.25−2.6324.9023.70−1.20−1.43[−9.56, +5.66]−4.56
League 1 group − League 3 group32.8830.25−2.639.667.30−2.37−0.26[−14.64, +12.04]−4.56
League 2 group − League 3 group24.9023.70−1.209.667.30−2.37+1.17[−6.49, +8.72]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff52.0644.97−7.1050.7553.98+3.23−10.33[−38.92, +19.74]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

Original members only: the League 3 group has 1 setup (GPT-6 Astramedium); comparisons 2 and 3 rest on it.

Swipe for more columns →

Original members only. Sensitivity (ii): the pair is left out for that setup
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group32.8830.25−2.6322.5222.36−0.16−2.47[−10.57, +4.72]−4.56
League 1 group − League 3 group32.8830.25−2.639.667.30−2.37−0.26[−14.64, +12.04]−4.56
League 2 group − League 3 group22.5222.36−0.169.667.30−2.37+2.21[−5.45, +9.55]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff30.5932.91+2.3250.7553.98+3.23−0.91[−24.13, +25.20]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

Original members only: the League 3 group has 1 setup (GPT-6 Astramedium); comparisons 2 and 3 rest on it.

Swipe for more columns →

Original members only. Sensitivity (iii): a pair with an answer cut off at the output limit is left out for that setup
Comparison (A − B)A: standard PRA: Nackgammon PRA: dropB: standard PRB: Nackgammon PRB: dropDifference of drops99% intervalRandom's drop, same pairs
League 1 group − League 2 group32.8830.25−2.6323.3622.88−0.49−2.14[−10.07, +4.88]−4.56
League 1 group − League 3 group32.8830.25−2.639.667.30−2.37−0.26[−14.64, +12.04]−4.56
League 2 group − League 3 group23.3622.88−0.499.667.30−2.37+1.88[−5.73, +9.33]−4.56
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff38.2037.55−0.6550.7553.98+3.23−3.88[−26.98, +21.76]−4.56
GPT-6.1 Solhigh − GPT-6.1 Sollow9.888.43−1.4511.159.90−1.26−0.19[−3.00, +2.63]−4.56

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

Original members only: the League 3 group has 1 setup (GPT-6 Astramedium); comparisons 2 and 3 rest on it.

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Invalid answers, output limit and sensitivity

Swipe for more columns →

Invalid answers, output-limit flags and the three sensitivity checks, main 200 pairs
Setup (group)Invalid answers, standard / Nackgammon (main 200 pairs)Drop, primary rule97.5% intervalDrop, invalid = worst play97.5% intervalPairs, invalid pairs left outDrop, invalid pairs left out97.5% intervalPairs, cut-off pairs left outDrop, cut-off pairs left out97.5% intervalCut off at the output limit, main 200 pairs (standard / Nackgammon)Cut off at the output limit, all recorded positions (standard / Nackgammon)Past 32,768 output tokens, no limit sent, main 200 pairs (standard / Nackgammon)Past 32,768 output tokens, no limit sent, all recorded positions (standard / Nackgammon)
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group0 / 0+2.18[−7.28, +11.93]+2.18[−7.28, +11.93]200+2.18[−7.28, +11.93]200+2.18[−7.28, +11.93]0 / 00 / 00 / 00 / 0
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read0 / 0+0.45[−8.03, +8.77]+0.45[−8.03, +8.77]200+0.45[−8.03, +8.77]200+0.45[−8.03, +8.77]0 / 00 / 00 / 00 / 0
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read0 / 0+4.01[−11.37, +17.17]+4.01[−11.37, +17.17]200+4.01[−11.37, +17.17]200+4.01[−11.37, +17.17]0 / 00 / 00 / 00 / 0
Jev 1.13Closed · EfficientLeague 1 group0 / 0−2.37[−17.72, +9.97]−2.37[−17.72, +9.97]200−2.37[−17.72, +9.97]200−2.37[−17.72, +9.97]0 / 00 / 00 / 00 / 0
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group0 / 0+5.84[−10.57, +20.46]+5.84[−10.57, +20.46]200+5.84[−10.57, +20.46]200+5.84[−10.57, +20.46]0 / 00 / 00 / 00 / 0
celeris-1-decisionClosed · EfficientLeague 1 group0 / 0−1.24[−18.27, +14.59]−1.24[−18.27, +14.59]200−1.24[−18.27, +14.59]200−1.24[−18.27, +14.59]0 / 00 / 00 / 00 / 0
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group0 / 0−1.78[−17.30, +11.94]−1.78[−17.30, +11.94]200−1.78[−17.30, +11.94]200−1.78[−17.30, +11.94]0 / 00 / 00 / 00 / 0
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group0 / 0−6.85[−22.36, +5.43]−6.85[−22.36, +5.43]200−6.85[−22.36, +5.43]200−6.85[−22.36, +5.43]0 / 00 / 00 / 00 / 0
SubQ Snap PreviewClosed · EfficientLeague 1 group0 / 0−2.25[−18.09, +11.88]−2.25[−18.09, +11.88]200−2.25[−18.09, +11.88]200−2.25[−18.09, +11.88]0 / 00 / 00 / 00 / 0
Mercury DecideClosed · EfficientLeague 1 group0 / 0−4.28[−20.11, +8.75]−4.28[−20.11, +8.75]200−4.28[−20.11, +8.75]200−4.28[−20.11, +8.75]0 / 00 / 00 / 00 / 0
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group0 / 0−0.89[−17.96, +14.97]−0.89[−17.96, +14.97]200−0.89[−17.96, +14.97]200−0.89[−17.96, +14.97]0 / 00 / 00 / 00 / 0
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group0 / 0−11.87[−27.93, +1.03]−11.87[−27.93, +1.03]200−11.87[−27.93, +1.03]200−11.87[−27.93, +1.03]0 / 00 / 00 / 00 / 0
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group0 / 0+3.09[−12.09, +20.11]+3.09[−12.09, +20.11]200+3.09[−12.09, +20.11]200+3.09[−12.09, +20.11]0 / 00 / 00 / 00 / 0
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group0 / 0−11.12[−29.52, +6.79]−11.12[−29.52, +6.79]200−11.12[−29.52, +6.79]200−11.12[−29.52, +6.79]0 / 00 / 00 / 00 / 0
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group0 / 0−1.45[−5.57, +2.68]−1.45[−5.57, +2.68]200−1.45[−5.57, +2.68]200−1.45[−5.57, +2.68]0 / 00 / 00 / 00 / 0
GPT-6.1 SollowClosed · WorkhorseLeague 2 group0 / 0−1.26[−5.70, +3.31]−1.26[−5.70, +3.31]200−1.26[−5.70, +3.31]200−1.26[−5.70, +3.31]0 / 00 / 00 / 00 / 0
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group0 / 0+0.54[−5.04, +6.02]+0.54[−5.04, +6.02]200+0.54[−5.04, +6.02]200+0.54[−5.04, +6.02]0 / 00 / 00 / 00 / 0
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group0 / 0−4.11[−9.72, +1.04]−4.11[−9.72, +1.04]200−4.11[−9.72, +1.04]200−4.11[−9.72, +1.04]0 / 00 / 00 / 00 / 0
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group0 / 0−4.97[−11.81, +1.78]−4.97[−11.81, +1.78]200−4.97[−11.81, +1.78]200−4.97[−11.81, +1.78]0 / 00 / 00 / 00 / 0
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group0 / 0+3.98[−11.72, +17.96]+3.98[−11.72, +17.96]200+3.98[−11.72, +17.96]200+3.98[−11.72, +17.96]0 / 00 / 00 / 00 / 0
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group0 / 0+0.30[−14.48, +12.12]+0.30[−14.48, +12.12]200+0.30[−14.48, +12.12]200+0.30[−14.48, +12.12]0 / 00 / 00 / 00 / 0
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group22 / 16−0.65[−15.17, +14.42]−7.10[−27.35, +13.34]166+2.32[−12.28, +18.07]200−0.65[−15.17, +14.42]0 / 00 / 00 / 00 / 0
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group0 / 0+3.23[−16.93, +22.07]+3.23[−16.93, +22.07]200+3.23[−16.93, +22.07]200+3.23[−16.93, +22.07]0 / 00 / 00 / 00 / 0
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read0 / 0+0.38[−4.64, +5.07]+0.38[−4.64, +5.07]200+0.38[−4.64, +5.07]200+0.38[−4.64, +5.07]0 / 00 / 00 / 00 / 0
GPT-6 AstramediumClosed · FlagshipLeague 3 group0 / 0−2.37[−6.69, +1.77]−2.37[−6.69, +1.77]200−2.37[−6.69, +1.77]200−2.37[−6.69, +1.77]0 / 00 / 00 / 00 / 0
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read6 / 9−0.67[−6.58, +5.07]−0.46[−6.52, +5.59]186−0.84[−7.17, +5.33]200−0.67[−6.58, +5.07]0 / 00 / 05 / 27 / 10
GPT-6.1 Terrano group (exploratory)0 / 0+0.89[−5.73, +8.59]+0.89[−5.73, +8.59]200+0.89[−5.73, +8.59]200+0.89[−5.73, +8.59]0 / 00 / 00 / 00 / 0
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)0 / 0+1.92[−4.33, +8.42]+1.92[−4.33, +8.42]200+1.92[−4.33, +8.42]200+1.92[−4.33, +8.42]0 / 00 / 00 / 00 / 0
GNUbg weakyardstick0 / 0−3.56[−13.92, +7.29]−3.56[−13.92, +7.29]200−3.56[−13.92, +7.29]200−3.56[−13.92, +7.29]0 / 00 / 00 / 00 / 0
Randomyardstick0 / 0−4.56[−18.04, +7.86]−4.56[−18.04, +7.86]200−4.56[−18.04, +7.86]200−4.56[−18.04, +7.86]––––

Output limit (§10.14(h)): an answer cut off at 32,768 output tokens counts as invalid, with its own sensitivity check (cut-off pairs left out). No recorded answer was cut off at the limit. Past 32,768 output tokens from a run that sent no limit (flagged; the answers keep their scores): Grok 4.7high, 17 of its recorded positions (7 standard / 10 Nackgammon), 7 in the main 200 pairs.

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

By phase (descriptive)

Swipe for more columns →

By phase, main 200 pairs (descriptive, 95% intervals); Random's row is in the table
Setup (group)opening: standard / Nackgammon PRopening: drop [95%]contact: standard / Nackgammon PRcontact: drop [95%]holding: standard / Nackgammon PRholding: drop [95%]race: standard / Nackgammon PRrace: drop [95%]bear-off: standard / Nackgammon PRbear-off: drop [95%]Opening drop − other phases' drop [95%]
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group17.73 / 26.99+9.25 [−2.30, +20.66]39.49 / 29.34−10.16 [−28.79, +7.76]26.14 / 43.33+17.19 [−11.01, +52.43]7.15 / 10.53+3.38 [−5.64, +14.63]6.03 / 1.62−4.41 [−9.76, −0.42]+10.11 [−5.64, +25.57]
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read18.05 / 26.07+8.02 [−2.60, +18.11]40.73 / 31.12−9.62 [−28.23, +8.37]25.89 / 31.67+5.78 [−15.22, +30.11]7.15 / 10.16+3.01 [−5.95, +14.23]4.14 / 1.62−2.53 [−6.40, −0.04]+10.81 [−3.54, +24.91]
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read18.68 / 28.66+9.98 [−1.30, +21.21]32.73 / 28.98−3.75 [−19.32, +12.24]48.27 / 55.33+7.06 [−72.46, +68.47]7.54 / 14.15+6.61 [−4.67, +24.16]5.16 / 7.24+2.08 [−3.98, +11.87]+8.52 [−11.10, +29.64]
Jev 1.13Closed · EfficientLeague 1 group20.21 / 23.69+3.49 [−7.33, +14.00]31.78 / 28.13−3.65 [−21.53, +13.76]63.62 / 42.43−21.19 [−97.45, +35.95]6.34 / 10.79+4.46 [−4.55, +15.49]4.82 / 7.24+2.42 [−3.09, +11.88]+8.36 [−10.22, +29.77]
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group16.69 / 35.63+18.94 [+6.05, +31.44]40.34 / 30.85−9.49 [−31.25, +11.04]60.56 / 64.87+4.31 [−74.85, +63.66]9.65 / 25.92+16.28 [−5.65, +51.13]7.01 / 9.57+2.57 [−6.03, +13.91]+18.72 [−2.72, +42.58]
celeris-1-decisionClosed · EfficientLeague 1 group33.69 / 24.63−9.07 [−26.13, +6.22]33.67 / 28.66−5.01 [−22.42, +12.21]51.28 / 59.76+8.48 [−74.23, +83.54]7.12 / 19.44+12.32 [−7.03, +45.44]7.35 / 8.75+1.40 [−9.03, +13.58]−11.19 [−36.43, +14.27]
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group19.69 / 37.37+17.68 [+4.29, +31.27]43.35 / 29.99−13.36 [−30.51, +3.94]65.89 / 48.33−17.56 [−95.10, +44.28]6.19 / 8.59+2.40 [−5.96, +13.24]6.83 / 0.85−5.98 [−14.04, −0.23]+27.80 [+7.23, +50.59]
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group28.33 / 27.23−1.10 [−15.92, +13.25]38.19 / 31.50−6.69 [−24.72, +11.29]66.50 / 31.26−35.24 [−109.32, +15.97]5.97 / 13.21+7.24 [−1.28, +18.07]5.20 / 4.13−1.06 [−5.91, +4.81]+8.22 [−12.67, +31.39]
SubQ Snap PreviewClosed · EfficientLeague 1 group30.16 / 37.55+7.39 [−6.51, +21.06]42.13 / 38.64−3.49 [−23.46, +18.42]60.08 / 42.23−17.85 [−94.94, +42.28]5.27 / 2.75−2.51 [−8.14, +2.98]4.44 / 1.02−3.41 [−7.89, −0.15]+13.77 [−7.21, +37.11]
Mercury DecideClosed · EfficientLeague 1 group26.72 / 30.21+3.49 [−10.13, +15.91]46.91 / 29.39−17.52 [−35.81, −0.62]60.18 / 47.77−12.41 [−88.53, +41.14]7.74 / 25.88+18.14 [−4.66, +52.55]5.22 / 1.42−3.80 [−8.36, −0.66]+11.10 [−9.50, +33.39]
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group24.97 / 32.92+7.95 [−6.25, +21.71]45.25 / 33.14−12.11 [−31.78, +7.01]73.23 / 76.68+3.45 [−84.49, +79.88]11.29 / 12.97+1.68 [−7.12, +10.17]6.94 / 3.95−2.99 [−9.39, +3.76]+12.63 [−10.31, +37.00]
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group27.28 / 27.14−0.14 [−13.61, +12.94]50.09 / 31.86−18.23 [−39.05, +1.11]66.47 / 26.41−40.06 [−113.99, +16.88]7.57 / 8.51+0.94 [−9.24, +12.63]4.39 / 0.63−3.76 [−8.54, −0.46]+16.76 [−4.00, +39.66]
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group32.27 / 33.12+0.86 [−15.82, +18.38]69.47 / 58.06−11.41 [−40.76, +18.75]46.43 / 75.18+28.75 [−24.00, +99.67]10.73 / 24.51+13.78 [−1.66, +32.85]7.52 / 9.30+1.79 [−6.82, +11.56]−3.19 [−29.19, +21.46]
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group45.60 / 29.70−15.90 [−34.91, +3.03]88.55 / 60.66−27.89 [−63.56, +5.21]63.23 / 81.96+18.72 [−49.40, +90.12]11.32 / 10.76−0.56 [−13.57, +12.64]7.48 / 1.74−5.75 [−13.02, −0.57]−6.83 [−35.83, +21.69]
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group12.39 / 13.21+0.82 [−6.57, +8.28]13.67 / 11.24−2.43 [−9.76, +5.00]13.41 / 7.15−6.26 [−19.56, +3.87]0.43 / 0.18−0.24 [−1.01, +0.38]0.00 / 0.00+0.00 [+0.00, +0.00]+3.24 [−5.08, +12.07]
GPT-6.1 SollowClosed · WorkhorseLeague 2 group12.75 / 16.48+3.73 [−4.12, +12.06]16.46 / 12.74−3.71 [−12.27, +4.62]13.70 / 7.37−6.33 [−19.68, +3.53]1.10 / 0.18−0.91 [−2.57, +0.28]1.59 / 0.00−1.59 [−4.77, +0.00]+7.12 [−1.90, +16.65]
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group14.50 / 18.95+4.45 [−3.80, +13.62]24.43 / 24.37−0.07 [−12.53, +11.73]13.68 / 7.60−6.07 [−19.26, +3.64]1.85 / 2.86+1.01 [−3.88, +7.07]0.00 / 0.07+0.07 [+0.00, +0.21]+5.58 [−4.32, +16.34]
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group14.94 / 16.01+1.07 [−5.51, +7.26]28.74 / 16.38−12.35 [−24.99, −0.09]14.24 / 13.35−0.88 [−14.62, +10.17]3.31 / 0.64−2.67 [−6.94, +0.62]2.15 / 0.07−2.08 [−6.01, +0.00]+7.40 [−1.46, +16.42]
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group19.52 / 19.53+0.01 [−9.76, +10.08]32.82 / 19.17−13.65 [−29.63, +2.24]13.15 / 11.62−1.54 [−15.22, +9.50]3.36 / 0.18−3.17 [−6.88, −0.32]2.02 / 0.03−1.99 [−5.93, +0.06]+7.11 [−5.19, +19.68]
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group13.39 / 27.97+14.58 [+3.87, +25.35]38.21 / 27.57−10.64 [−31.09, +9.00]47.28 / 62.50+15.21 [−64.76, +76.60]6.31 / 13.97+7.66 [−3.66, +24.98]4.43 / 0.94−3.49 [−8.35, −0.14]+15.13 [−4.89, +37.40]
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group19.21 / 32.36+13.15 [−0.07, +25.87]38.40 / 29.03−9.37 [−23.98, +4.78]61.66 / 49.21−12.45 [−88.96, +41.03]7.24 / 11.56+4.32 [−4.51, +15.35]5.56 / 9.54+3.97 [−4.31, +15.03]+18.36 [−1.24, +39.95]
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group33.94 / 37.10+3.16 [−13.56, +21.60]64.84 / 39.90−24.94 [−54.30, +5.53]42.82 / 72.20+29.38 [−14.29, +77.60]11.24 / 19.24+8.00 [−7.23, +27.95]5.92 / 9.74+3.82 [−4.51, +14.94]+5.44 [−18.48, +29.98]
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group40.54 / 56.04+15.51 [−2.57, +33.81]65.65 / 65.61−0.04 [−28.23, +29.22]88.29 / 73.95−14.33 [−107.31, +69.03]24.63 / 26.07+1.45 [−14.37, +21.51]20.60 / 25.09+4.49 [−6.96, +17.06]+17.54 [−10.83, +47.11]
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read10.73 / 14.81+4.08 [−3.84, +12.23]12.57 / 14.10+1.53 [−8.40, +11.25]14.04 / 5.40−8.63 [−21.56, +1.07]0.25 / 0.18−0.07 [−0.76, +0.55]0.00 / 0.00+0.00 [+0.00, +0.00]+5.29 [−4.00, +15.02]
GPT-6 AstramediumClosed · FlagshipLeague 3 group8.84 / 12.14+3.30 [−3.03, +9.70]16.23 / 9.73−6.50 [−14.88, +1.78]13.57 / 4.68−8.88 [−22.08, +1.05]0.83 / 0.18−0.64 [−2.22, +0.37]0.03 / 0.07+0.04 [+0.00, +0.13]+8.10 [+0.51, +16.05]
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read12.73 / 22.83+10.09 [+1.10, +19.52]28.56 / 17.57−10.99 [−21.81, −0.91]16.84 / 14.39−2.45 [−20.92, +17.77]2.30 / 0.93−1.37 [−5.37, +1.51]0.00 / 1.11+1.11 [+0.00, +3.18]+15.37 [+4.49, +26.73]
GPT-6.1 Terrano group (exploratory)13.35 / 20.77+7.42 [+1.12, +13.95]23.74 / 17.13−6.61 [−18.78, +5.67]14.00 / 19.69+5.69 [−18.06, +40.13]0.56 / 0.64+0.08 [−0.99, +1.35]1.76 / 0.00−1.76 [−5.12, +0.00]+9.34 [−1.66, +19.61]
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)15.22 / 18.13+2.91 [−7.20, +12.64]23.00 / 17.18−5.83 [−16.79, +4.71]13.58 / 31.67+18.09 [+0.69, +42.23]2.09 / 2.60+0.50 [−4.03, +6.55]0.23 / 0.34+0.11 [−0.48, +0.81]+1.42 [−10.49, +13.27]
GNUbg weakyardstick38.78 / 31.32−7.46 [−21.30, +6.50]39.90 / 34.06−5.85 [−32.05, +20.17]22.63 / 26.00+3.37 [−12.48, +21.09]0.39 / 3.40+3.01 [−0.03, +7.82]5.15 / 1.53−3.61 [−10.52, +0.66]−5.57 [−23.29, +11.90]
Randomyardstick95.63 / 84.31−11.32 [−23.34, +0.42]125.31 / 115.99−9.32 [−27.33, +8.75]113.57 / 121.18+7.61 [−57.23, +63.11]19.17 / 22.57+3.40 [−7.66, +16.86]13.92 / 14.46+0.54 [−5.63, +6.89]−9.66 [−28.68, +9.80]

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

By game-plan tag (descriptive)

Swipe for more columns →

By game-plan tag, main 200 pairs (descriptive; the back-game row is too thin to read)
Setup (group)(none: bear-off): standard PR · n · Nackgammon PR · n(none: opening): standard PR · n · Nackgammon PR · n(none: race): standard PR · n · Nackgammon PR · nback game: standard PR · n · Nackgammon PR · nblitz: standard PR · n · Nackgammon PR · nholding: standard PR · n · Nackgammon PR · nlow-anchor: standard PR · n · Nackgammon PR · nother: standard PR · n · Nackgammon PR · npriming: standard PR · n · Nackgammon PR · n
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group6.03 · 25 · 1.62 · 2517.73 · 60 · 26.99 · 607.15 · 25 · 10.53 · 2522.94 · 2 · 44.17 · 1048.46 · 23 · 8.80 · 1726.14 · 30 · 43.33 · 3039.62 · 8 · 31.03 · 725.00 · 10 · 29.68 · 1137.78 · 17 · 41.69 · 15
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read4.14 · 25 · 1.62 · 2518.05 · 60 · 26.07 · 607.15 · 25 · 10.16 · 2522.94 · 2 · 44.17 · 1049.93 · 23 · 12.95 · 1725.89 · 30 · 31.67 · 3039.62 · 8 · 31.03 · 729.05 · 10 · 29.68 · 1137.78 · 17 · 44.10 · 15
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read5.16 · 25 · 7.24 · 2518.68 · 60 · 28.66 · 607.54 · 25 · 14.15 · 2560.79 · 2 · 27.83 · 1027.25 · 23 · 16.11 · 1748.27 · 30 · 55.33 · 3036.13 · 8 · 45.54 · 744.45 · 10 · 24.72 · 1128.36 · 17 · 39.74 · 15
Jev 1.13Closed · EfficientLeague 1 group4.82 · 25 · 7.24 · 2520.21 · 60 · 23.69 · 606.34 · 25 · 10.79 · 2522.94 · 2 · 35.43 · 1033.90 · 23 · 13.43 · 1763.62 · 30 · 42.43 · 3048.48 · 8 · 37.32 · 736.46 · 10 · 17.33 · 1119.34 · 17 · 43.56 · 15
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group7.01 · 25 · 9.57 · 2516.69 · 60 · 35.63 · 609.65 · 25 · 25.92 · 2522.94 · 2 · 32.07 · 1044.92 · 23 · 14.08 · 1760.56 · 30 · 64.87 · 3036.76 · 8 · 45.54 · 763.71 · 10 · 30.99 · 1124.12 · 17 · 42.09 · 15
celeris-1-decisionClosed · EfficientLeague 1 group7.35 · 25 · 8.75 · 2533.69 · 60 · 24.63 · 607.12 · 25 · 19.44 · 2522.94 · 2 · 37.21 · 1035.99 · 23 · 30.60 · 1751.28 · 30 · 59.76 · 3031.16 · 8 · 40.57 · 723.29 · 10 · 14.27 · 1139.10 · 17 · 25.76 · 15
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group6.83 · 25 · 0.85 · 2519.69 · 60 · 37.37 · 606.19 · 25 · 8.59 · 2522.94 · 2 · 31.19 · 1056.92 · 23 · 16.55 · 1765.89 · 30 · 48.33 · 3037.56 · 8 · 35.51 · 725.72 · 10 · 25.86 · 1140.48 · 17 · 44.86 · 15
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group5.20 · 25 · 4.13 · 2528.33 · 60 · 27.23 · 605.97 · 25 · 13.21 · 2560.79 · 2 · 27.83 · 1036.62 · 23 · 18.67 · 1766.50 · 30 · 31.26 · 3049.87 · 8 · 40.66 · 746.30 · 10 · 17.92 · 1127.39 · 17 · 54.16 · 15
SubQ Snap PreviewClosed · EfficientLeague 1 group4.44 · 25 · 1.02 · 2530.16 · 60 · 37.55 · 605.27 · 25 · 2.75 · 2517.04 · 2 · 51.48 · 1042.33 · 23 · 34.81 · 1760.08 · 30 · 42.23 · 3048.45 · 8 · 48.65 · 732.21 · 10 · 16.02 · 1147.68 · 17 · 46.33 · 15
Mercury DecideClosed · EfficientLeague 1 group5.22 · 25 · 1.42 · 2526.72 · 60 · 30.21 · 607.74 · 25 · 25.88 · 2522.94 · 2 · 35.43 · 1058.55 · 23 · 10.87 · 1760.18 · 30 · 47.77 · 3044.31 · 8 · 45.54 · 745.57 · 10 · 21.24 · 1136.00 · 17 · 44.77 · 15
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group6.94 · 25 · 3.95 · 2524.97 · 60 · 32.92 · 6011.29 · 25 · 12.97 · 2554.89 · 2 · 27.91 · 1041.91 · 23 · 10.87 · 1773.23 · 30 · 76.68 · 3045.79 · 8 · 41.48 · 765.55 · 10 · 35.18 · 1136.43 · 17 · 56.47 · 15
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group4.39 · 25 · 0.63 · 2527.28 · 60 · 27.14 · 607.57 · 25 · 8.51 · 2517.04 · 2 · 43.32 · 1075.68 · 23 · 14.51 · 1766.47 · 30 · 26.41 · 3047.99 · 8 · 51.90 · 721.48 · 10 · 20.23 · 1137.18 · 17 · 43.06 · 15
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group7.52 · 25 · 9.30 · 2532.27 · 60 · 33.12 · 6010.73 · 25 · 24.51 · 2535.25 · 2 · 47.05 · 1085.22 · 23 · 51.71 · 1746.43 · 30 · 75.18 · 3044.13 · 8 · 80.19 · 756.50 · 10 · 32.79 · 1171.75 · 17 · 80.81 · 15
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group7.48 · 25 · 1.74 · 2545.60 · 60 · 29.70 · 6011.32 · 25 · 10.76 · 2581.86 · 2 · 52.70 · 1098.15 · 23 · 58.85 · 1763.23 · 30 · 81.96 · 3042.77 · 8 · 70.45 · 781.20 · 10 · 40.10 · 11102.23 · 17 · 78.53 · 15
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group0.00 · 25 · 0.00 · 2512.39 · 60 · 13.21 · 600.43 · 25 · 0.18 · 2511.09 · 2 · 17.22 · 1012.66 · 23 · 10.78 · 1713.41 · 30 · 7.15 · 3013.16 · 8 · 9.99 · 720.29 · 10 · 12.75 · 1111.68 · 17 · 7.25 · 15
GPT-6.1 SollowClosed · WorkhorseLeague 2 group1.59 · 25 · 0.00 · 2512.75 · 60 · 16.48 · 601.10 · 25 · 0.18 · 2522.94 · 2 · 17.22 · 1011.77 · 23 · 12.98 · 1713.70 · 30 · 7.37 · 3037.19 · 8 · 21.59 · 720.24 · 10 · 10.68 · 1110.05 · 17 · 6.89 · 15
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group0.00 · 25 · 0.07 · 2514.50 · 60 · 18.95 · 601.85 · 25 · 2.86 · 2511.09 · 2 · 39.47 · 1024.68 · 23 · 10.09 · 1713.68 · 30 · 7.60 · 3041.27 · 8 · 36.58 · 723.58 · 10 · 30.71 · 1118.25 · 17 · 20.13 · 15
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group2.15 · 25 · 0.07 · 2514.94 · 60 · 16.01 · 603.31 · 25 · 0.64 · 2522.94 · 2 · 27.83 · 1029.33 · 23 · 6.66 · 1714.24 · 30 · 13.35 · 3036.72 · 8 · 29.48 · 721.48 · 10 · 15.09 · 1129.13 · 17 · 14.61 · 15
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group2.02 · 25 · 0.03 · 2519.52 · 60 · 19.53 · 603.36 · 25 · 0.18 · 2522.94 · 2 · 28.66 · 1028.28 · 23 · 8.77 · 1713.15 · 30 · 11.62 · 3038.14 · 8 · 27.07 · 740.99 · 10 · 16.96 · 1132.83 · 17 · 22.56 · 15
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group4.43 · 25 · 0.94 · 2513.39 · 60 · 27.97 · 606.31 · 25 · 13.97 · 2522.94 · 2 · 27.83 · 1035.63 · 23 · 11.36 · 1747.28 · 30 · 62.50 · 3040.47 · 8 · 45.54 · 725.00 · 10 · 27.94 · 1150.20 · 17 · 37.11 · 15
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group5.56 · 25 · 9.54 · 2519.21 · 60 · 32.36 · 607.24 · 25 · 11.56 · 2522.94 · 2 · 23.84 · 1047.64 · 23 · 11.86 · 1761.66 · 30 · 49.21 · 3044.31 · 8 · 45.54 · 743.95 · 10 · 24.30 · 1121.69 · 17 · 47.71 · 15
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group5.92 · 25 · 9.74 · 2533.94 · 60 · 37.10 · 6011.24 · 25 · 19.24 · 2568.33 · 2 · 34.16 · 1058.45 · 23 · 14.94 · 1742.82 · 30 · 72.20 · 3051.18 · 8 · 53.66 · 767.40 · 10 · 21.02 · 1177.99 · 17 · 79.43 · 15
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group20.60 · 25 · 25.09 · 2540.54 · 60 · 56.04 · 6024.63 · 25 · 26.07 · 2522.94 · 2 · 45.59 · 1076.34 · 23 · 125.24 · 1788.29 · 30 · 73.95 · 3062.22 · 8 · 45.08 · 790.40 · 10 · 21.21 · 1143.26 · 17 · 53.51 · 15
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read0.00 · 25 · 0.00 · 2510.73 · 60 · 14.81 · 600.25 · 25 · 0.18 · 2517.04 · 2 · 23.32 · 1016.79 · 23 · 6.51 · 1714.04 · 30 · 5.40 · 3016.31 · 8 · 19.97 · 75.79 · 10 · 18.60 · 118.55 · 17 · 10.54 · 15
GPT-6 AstramediumClosed · FlagshipLeague 3 group0.03 · 25 · 0.07 · 258.84 · 60 · 12.14 · 600.83 · 25 · 0.18 · 2511.09 · 2 · 17.22 · 1010.20 · 23 · 10.78 · 1713.57 · 30 · 4.68 · 3021.16 · 8 · 9.99 · 723.69 · 10 · 8.27 · 1118.28 · 17 · 4.48 · 15
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read0.00 · 25 · 1.11 · 2512.73 · 60 · 22.83 · 602.30 · 25 · 0.93 · 2539.75 · 2 · 18.61 · 1023.28 · 23 · 13.20 · 1716.84 · 30 · 14.39 · 3040.59 · 8 · 45.54 · 720.82 · 10 · 19.01 · 1133.27 · 17 · 7.71 · 15
GPT-6.1 Terrano group (exploratory)1.76 · 25 · 0.00 · 2513.35 · 60 · 20.77 · 600.56 · 25 · 0.64 · 2539.75 · 2 · 31.19 · 1018.82 · 23 · 6.51 · 1714.00 · 30 · 19.69 · 3028.12 · 8 · 50.56 · 720.24 · 10 · 12.01 · 1128.52 · 17 · 7.94 · 15
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)0.23 · 25 · 0.34 · 2515.22 · 60 · 18.13 · 602.09 · 25 · 2.60 · 250.00 · 2 · 21.46 · 1022.27 · 23 · 8.73 · 1713.58 · 30 · 31.67 · 3032.68 · 8 · 35.49 · 715.73 · 10 · 28.54 · 1126.43 · 17 · 7.02 · 15
GNUbg weakyardstick5.15 · 25 · 1.53 · 2538.78 · 60 · 31.32 · 600.39 · 25 · 3.40 · 2517.04 · 2 · 22.11 · 1043.29 · 23 · 56.39 · 1722.63 · 30 · 26.00 · 3032.99 · 8 · 37.50 · 747.08 · 10 · 17.63 · 1137.05 · 17 · 27.14 · 15
Randomyardstick13.92 · 25 · 14.46 · 2595.63 · 60 · 84.31 · 6019.17 · 25 · 22.57 · 2557.49 · 2 · 104.58 · 10125.59 · 23 · 143.55 · 17113.57 · 30 · 121.18 · 3092.05 · 8 · 113.21 · 7135.92 · 10 · 81.52 · 11142.31 · 17 · 118.93 · 15

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Cost per decision

Swipe for more columns →

Cost per decision of the run (recorded cost and API-equivalent never added; 0 = nothing recorded of that kind)
Setup (group)Decisions, mainRecorded cost, USD per 1,000 decisions, mainAPI-equivalent USD per 1,000 decisions, mainPositions recorded, all (incl. timed out)Recorded cost, USD per 1,000, allAPI-equivalent USD per 1,000, all
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group4000.0000.0001,0000.0000.000
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read4000.0000.0001,0000.0000.000
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read4000.1050.0001,0000.1050.000
Jev 1.13Closed · EfficientLeague 1 group4000.1290.0001,0000.1290.000
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group4000.2670.0001,0000.2670.000
celeris-1-decisionClosed · EfficientLeague 1 group4000.1120.0001,0000.1120.000
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group4000.0560.0001,0000.0560.000
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group4000.0000.0001,0000.0000.000
SubQ Snap PreviewClosed · EfficientLeague 1 group4000.1400.0001,0000.1400.000
Mercury DecideClosed · EfficientLeague 1 group4000.0560.0001,0000.0560.000
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group4000.0000.0001,0000.0000.000
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group4000.0560.0001,0000.0560.000
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group4000.0280.0001,0000.0280.000
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group4000.0000.0001,0000.0000.000
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group4008.1480.0001,0008.0660.000
GPT-6.1 SollowClosed · WorkhorseLeague 2 group4006.0220.0001,0006.0160.000
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group4008.7650.0001,0008.7260.000
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group4002.8450.0001,0002.8850.000
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group4000.0000.0001,0000.0000.000
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group4000.2930.0001,0000.2670.000
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group4000.0000.0001,0000.0000.000
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group4000.0000.0001,0000.0000.000
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group4000.0000.0001,0000.0000.000
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read40072.9060.0001,00073.6700.000
GPT-6 AstramediumClosed · FlagshipLeague 3 group40029.6140.0001,00029.5150.000
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read40064.0870.0001,00063.4400.000
GPT-6.1 Terrano group (exploratory)4003.7230.0001,0003.7090.000
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)4005.9060.0001,0005.9070.000
GNUbg weakyardstick4000.0000.0001,0000.0000.000

Recorded cost (the field billed_usd_per_1000) is the cost stored with each answer: the provider's reported cost per response where the route reports one, otherwise the runner's list-price computation (Mercury Decide: list price; its provider-billed amount was not retained and is unknown). No invoice was checked. API-equivalent is a subscription setup priced at list price, not money paid. Setups on our GPUs have no list price.

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Secondary analyses

Swipe for more columns →

Secondary: kept advantage over random play, main 200 pairs
Setup (group)Kept advantage K95% intervalStandard advantage over Random (PR)95% intervalUnstable
GNUbg weakyardstick0.98[0.77, 1.25]59.76[47.61, 72.84]no
celeris-1-decisionClosed · EfficientLeague 1 group0.94[0.76, 1.15]57.74[47.55, 68.65]no
SubQ Snap PreviewClosed · EfficientLeague 1 group0.96[0.76, 1.19]55.54[45.29, 66.11]no
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group1.04[0.86, 1.27]56.12[46.07, 66.46]no
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group1.14[0.93, 1.39]52.77[42.90, 62.93]no
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group0.90[0.73, 1.09]64.71[52.87, 77.53]no
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group0.87[0.71, 1.05]63.54[53.09, 74.34]no
Jev 1.13Closed · EfficientLeague 1 group0.96[0.80, 1.16]60.92[50.62, 71.41]no
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group0.82[0.64, 1.03]59.18[49.11, 69.51]no
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group0.95[0.76, 1.17]57.03[46.86, 67.46]no
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group0.92[0.68, 1.22]49.25[38.34, 61.49]no
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group0.92[0.76, 1.10]59.32[49.48, 69.69]no
Mercury DecideClosed · EfficientLeague 1 group0.99[0.81, 1.21]54.71[45.13, 64.95]no
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group0.93[0.74, 1.16]53.12[43.32, 63.11]no
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group0.79[0.51, 1.19]36.70[25.62, 47.88]no
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group1.19[0.84, 1.83]35.37[22.97, 48.64]no
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group0.84[0.58, 1.16]47.68[36.50, 60.40]no
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group0.99[0.83, 1.18]71.53[59.26, 84.64]no
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group0.93[0.78, 1.10]73.49[61.59, 86.24]no
GPT-6.1 SollowClosed · WorkhorseLeague 2 group0.96[0.81, 1.12]76.30[64.19, 89.41]no
GPT-6 AstramediumClosed · FlagshipLeague 3 group0.97[0.83, 1.13]77.79[65.54, 90.83]no
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group0.96[0.82, 1.11]77.57[65.47, 90.77]no
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group1.01[0.84, 1.21]69.10[57.09, 82.14]no
GPT-6.1 Terrano group (exploratory)0.93[0.78, 1.10]73.93[61.79, 86.84]no
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read0.94[0.80, 1.09]78.32[66.07, 91.51]no
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read0.95[0.79, 1.12]72.25[60.31, 85.00]no
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read0.86[0.71, 1.05]63.20[52.20, 74.21]no
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read0.92[0.75, 1.13]64.52[52.71, 77.38]no
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)0.91[0.77, 1.07]73.66[61.40, 86.70]no

Swipe for more columns →

Secondary: kept advantage, all 500 pairs (extended setups)
Setup (group)Kept advantage K95% intervalStandard advantage over Random (PR)95% intervalUnstable
GNUbg weakyardstick1.01[0.88, 1.16]59.81[52.45, 67.38]no
celeris-1-decisionClosed · EfficientLeague 1 group0.87[0.76, 0.99]58.30[51.64, 65.13]no
SubQ Snap PreviewClosed · EfficientLeague 1 group0.90[0.78, 1.03]53.91[47.30, 60.50]no
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group0.93[0.83, 1.05]56.70[50.32, 63.36]no
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group0.98[0.84, 1.14]53.44[46.69, 60.36]no
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group0.91[0.79, 1.05]58.83[52.10, 65.71]no
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group0.84[0.75, 0.95]61.80[55.23, 68.46]no
Jev 1.13Closed · EfficientLeague 1 group0.89[0.80, 1.00]60.49[54.05, 67.08]no
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group0.83[0.72, 0.96]55.60[49.00, 62.17]no
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group0.92[0.80, 1.05]53.89[47.43, 60.46]no
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group0.82[0.68, 0.98]47.38[40.64, 54.20]no
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group0.85[0.76, 0.96]58.27[51.96, 64.55]no
Mercury DecideClosed · EfficientLeague 1 group0.88[0.77, 1.01]53.81[47.34, 60.28]no
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group0.88[0.77, 1.01]53.22[46.79, 59.83]no
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group0.79[0.61, 1.00]38.72[31.77, 45.62]no
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group0.98[0.81, 1.23]38.09[30.72, 45.47]no
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group0.74[0.59, 0.92]44.93[37.85, 52.08]no
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group0.96[0.86, 1.08]68.45[61.28, 75.91]no
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group0.96[0.86, 1.08]68.71[61.60, 76.16]no
GPT-6.1 SollowClosed · WorkhorseLeague 2 group0.97[0.87, 1.08]72.95[65.81, 80.40]no
GPT-6 AstramediumClosed · FlagshipLeague 3 group0.98[0.88, 1.08]74.29[67.10, 81.78]no
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group0.98[0.88, 1.08]74.01[66.80, 81.46]no
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group0.96[0.85, 1.08]67.19[60.13, 74.56]no
GPT-6.1 Terrano group (exploratory)0.91[0.81, 1.01]71.98[64.84, 79.45]no
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read0.94[0.85, 1.03]75.52[68.27, 83.14]no
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read0.93[0.83, 1.05]69.47[62.42, 76.93]no
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read0.83[0.74, 0.94]59.85[53.18, 66.58]no
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read0.89[0.78, 1.00]59.43[52.73, 66.34]no
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)0.91[0.81, 1.01]70.45[63.31, 77.95]no

Swipe for more columns →

Secondary: ranking, yardsticks left out
AnalysisModel setupsSpearman correlation95% interval
Standard against Nackgammon PR, main 200 pairs280.90[0.79, 0.92]

Swipe for more columns →

Secondary: the five largest rank changes (rank 1 = lowest PR; ties by name)
SetupRank, standardRank, NackgammonChange
Perplexity Decider 27B v1.22411-13
GPT-6 Luna17247
Cygnet2014-6
GLiDEv1.211154
Gemini 3.8 Flashlow95-4

Swipe for more columns →

Secondary: risk calls, main 200 pairs
Setup (group)Risk pays / Safety pays, standardRisk pays / Safety pays, NackgammonAbility d′, standardd′, NackgammonChange in d′ [95%]Caution c, standardc, NackgammonChange in c [95%]Thin (a class under 50 positions)
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group66 / 6666 / 660.820.37−0.45 [−1.15, +0.19]0.600.65+0.05 [−0.29, +0.38]no
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read66 / 6666 / 660.720.48−0.24 [−0.87, +0.38]0.590.71+0.12 [−0.19, +0.45]no
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read66 / 6666 / 660.910.52−0.39 [−1.15, +0.27]0.490.52+0.03 [−0.32, +0.37]no
Jev 1.13Closed · EfficientLeague 1 group66 / 6666 / 660.591.10+0.51 [−0.17, +1.25]0.600.66+0.06 [−0.28, +0.43]no
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group66 / 6666 / 660.950.14−0.81 [−1.57, −0.11]0.660.66+0.00 [−0.36, +0.38]no
celeris-1-decisionClosed · EfficientLeague 1 group66 / 6666 / 660.430.85+0.42 [−0.32, +1.11]0.860.65−0.21 [−0.56, +0.15]no
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group66 / 6666 / 660.440.45+0.01 [−0.67, +0.66]0.560.61+0.05 [−0.27, +0.38]no
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group66 / 6666 / 660.620.52−0.10 [−0.79, +0.60]0.700.69−0.01 [−0.34, +0.35]no
SubQ Snap PreviewClosed · EfficientLeague 1 group66 / 6666 / 660.560.29−0.27 [−0.95, +0.34]0.360.45+0.09 [−0.23, +0.42]no
Mercury DecideClosed · EfficientLeague 1 group66 / 6666 / 660.310.48+0.17 [−0.51, +0.86]0.790.83+0.04 [−0.29, +0.37]no
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group66 / 6666 / 660.400.39−0.01 [−0.65, +0.60]0.350.23−0.12 [−0.43, +0.18]no
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group66 / 6666 / 660.300.88+0.58 [−0.10, +1.27]0.530.63+0.10 [−0.25, +0.45]no
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group66 / 6666 / 660.300.53+0.23 [−0.42, +0.85]0.080.04−0.04 [−0.35, +0.25]no
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group66 / 6666 / 660.420.23−0.19 [−0.82, +0.43]0.060.23+0.17 [−0.14, +0.46]no
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group66 / 6666 / 661.182.05+0.87 [+0.16, +1.71]0.630.76+0.13 [−0.22, +0.54]no
GPT-6.1 SollowClosed · WorkhorseLeague 2 group66 / 6666 / 661.141.86+0.72 [+0.04, +1.56]0.650.85+0.20 [−0.12, +0.63]no
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group66 / 6666 / 660.991.44+0.45 [−0.32, +1.27]0.800.91+0.11 [−0.25, +0.51]no
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group66 / 6666 / 660.792.17+1.38 [+0.73, +1.92]0.901.35+0.45 [+0.12, +0.72]no
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group66 / 6666 / 660.661.56+0.90 [+0.04, +1.81]1.171.00−0.17 [−0.61, +0.30]no
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group66 / 6666 / 660.780.60−0.18 [−0.90, +0.55]0.500.65+0.15 [−0.20, +0.49]no
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group66 / 6666 / 660.310.37+0.06 [−0.64, +0.78]0.790.82+0.03 [−0.29, +0.36]no
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group66 / 6666 / 660.530.47−0.06 [−0.70, +0.52]0.170.23+0.06 [−0.24, +0.35]no
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group66 / 6666 / 660.230.45+0.22 [−0.39, +0.86]0.19−0.00−0.19 [−0.49, +0.11]no
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read66 / 6666 / 661.332.01+0.68 [+0.00, +1.47]0.550.78+0.23 [−0.11, +0.62]no
GPT-6 AstramediumClosed · FlagshipLeague 3 group66 / 6666 / 661.532.05+0.52 [−0.18, +1.32]0.730.76+0.03 [−0.32, +0.43]no
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read66 / 6666 / 660.631.47+0.84 [+0.16, +1.76]0.470.89+0.42 [+0.09, +0.87]no
GPT-6.1 Terrano group (exploratory)66 / 6666 / 660.911.44+0.53 [−0.29, +1.45]0.761.06+0.30 [−0.07, +0.75]no
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)66 / 6666 / 661.171.26+0.09 [−0.48, +0.71]0.360.67+0.31 [+0.02, +0.61]no
GNUbg weakyardstick66 / 6666 / 660.710.95+0.24 [−0.37, +0.82]−0.28−0.47−0.19 [−0.48, +0.09]no
Randomyardstick66 / 6666 / 660.270.37+0.10 [−0.11, +0.30]−0.77−0.82−0.05 [−0.15, +0.04]no

Swipe for more columns →

Secondary: risk calls, all 500 pairs, extended setups
Setup (group)Risk pays / Safety pays, standardRisk pays / Safety pays, NackgammonAbility d′, standardd′, NackgammonChange in d′ [95%]Caution c, standardc, NackgammonChange in c [95%]Thin (a class under 50 positions)
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group165 / 165165 / 1650.490.37−0.12 [−0.54, +0.30]0.610.61+0.00 [−0.21, +0.21]no
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read165 / 165165 / 1650.500.40−0.10 [−0.51, +0.33]0.680.68+0.00 [−0.19, +0.20]no
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read165 / 165165 / 1650.920.43−0.49 [−0.92, −0.09]0.560.52−0.04 [−0.25, +0.16]no
Jev 1.13Closed · EfficientLeague 1 group165 / 165165 / 1650.710.76+0.05 [−0.40, +0.48]0.750.57−0.18 [−0.39, +0.04]no
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group165 / 165165 / 1650.730.34−0.39 [−0.86, +0.02]0.790.58−0.21 [−0.43, +0.01]no
celeris-1-decisionClosed · EfficientLeague 1 group165 / 165165 / 1650.540.60+0.06 [−0.38, +0.46]0.800.58−0.22 [−0.44, −0.01]no
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group165 / 165165 / 1650.300.33+0.03 [−0.38, +0.42]0.560.63+0.07 [−0.13, +0.27]no
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group165 / 165165 / 1650.720.43−0.29 [−0.70, +0.13]0.690.58−0.11 [−0.33, +0.09]no
SubQ Snap PreviewClosed · EfficientLeague 1 group165 / 165165 / 1650.550.36−0.19 [−0.57, +0.19]0.420.42+0.00 [−0.20, +0.20]no
Mercury DecideClosed · EfficientLeague 1 group165 / 165165 / 1650.370.28−0.09 [−0.52, +0.33]0.840.83−0.01 [−0.23, +0.21]no
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group165 / 165165 / 1650.460.31−0.15 [−0.53, +0.22]0.370.19−0.18 [−0.38, +0.02]no
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group165 / 165165 / 1650.330.62+0.29 [−0.10, +0.67]0.530.59+0.06 [−0.13, +0.27]no
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group165 / 165165 / 1650.350.44+0.09 [−0.30, +0.48]0.05−0.05−0.10 [−0.29, +0.10]no
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group165 / 165165 / 1650.490.21−0.28 [−0.66, +0.11]0.120.11−0.01 [−0.19, +0.19]no
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group165 / 165165 / 1651.331.74+0.41 [−0.06, +0.89]0.660.71+0.05 [−0.18, +0.29]no
GPT-6.1 SollowClosed · WorkhorseLeague 2 group165 / 165165 / 1651.281.43+0.15 [−0.32, +0.61]0.680.72+0.04 [−0.18, +0.28]no
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group165 / 165165 / 1651.081.22+0.14 [−0.31, +0.58]0.860.79−0.07 [−0.29, +0.16]no
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group165 / 165165 / 1650.991.35+0.36 [−0.12, +0.89]0.991.02+0.03 [−0.21, +0.30]no
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group165 / 165165 / 1651.051.15+0.10 [−0.50, +0.65]1.240.95−0.29 [−0.57, −0.02]no
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group165 / 165165 / 1650.830.61−0.22 [−0.67, +0.20]0.680.62−0.06 [−0.27, +0.15]no
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group165 / 165165 / 1650.250.20−0.05 [−0.52, +0.37]0.900.87−0.03 [−0.24, +0.19]no
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group165 / 165165 / 1650.700.34−0.36 [−0.74, +0.00]0.200.19−0.01 [−0.19, +0.18]no
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group165 / 165165 / 1650.200.60+0.40 [+0.01, +0.80]0.18−0.05−0.23 [−0.44, −0.03]no
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read165 / 165165 / 1651.471.44−0.03 [−0.47, +0.39]0.620.56−0.06 [−0.27, +0.15]no
GPT-6 AstramediumClosed · FlagshipLeague 3 group165 / 165165 / 1651.471.71+0.24 [−0.21, +0.72]0.740.72−0.02 [−0.25, +0.22]no
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read165 / 165165 / 1650.881.05+0.17 [−0.26, +0.60]0.700.68−0.02 [−0.23, +0.20]no
GPT-6.1 Terrano group (exploratory)165 / 165165 / 1651.130.99−0.14 [−0.64, +0.36]0.910.86−0.05 [−0.31, +0.19]no
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)165 / 165165 / 1650.981.18+0.20 [−0.21, +0.60]0.480.57+0.09 [−0.10, +0.29]no
GNUbg weakyardstick165 / 165165 / 1651.041.16+0.12 [−0.27, +0.50]−0.31−0.42−0.11 [−0.30, +0.09]no

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Secondary, as fixed in §10.11. K = (Random's Nackgammon PR − setup's Nackgammon PR) ÷ (Random's standard PR − setup's standard PR): 1 = keeps its whole advantage over random play, 0 = falls to random. Unstable: standard advantage under 20 PR or its interval reaches 0. Risk: a class with fewer than 50 positions is thin and not read into.

Engine check

Swipe for more columns →

Engine check: 2-ply against rollout values on the subset
Setup (group)2-ply: standard PR2-ply: Nackgammon PR2-ply: dropRollout: standard PRRollout: Nackgammon PRRollout: drop
GNUbg weakyardstick25.2729.43+4.1628.1228.10−0.02
celeris-1-decisionClosed · EfficientLeague 1 group19.9453.19+33.2420.9051.27+30.37
SubQ Snap PreviewClosed · EfficientLeague 1 group17.1135.79+18.6718.4734.63+16.16
CygnetNVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 12BLeague 1 group17.7122.08+4.3719.0719.93+0.85
Perplexity Decider 27B v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group15.0920.47+5.3916.1719.36+3.19
GLiDEv1.2 · NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group13.4320.28+6.8614.3019.14+4.85
DeepSeek-V4.1-Flashoff · API + NVIDIA RTX PRO 6000 Blackwell · first releaseOpen · 552B MoE · 16B activeLeague 2 group10.1922.97+12.7811.0520.58+9.53
Jev 1.13Closed · EfficientLeague 1 group14.6625.27+10.6115.8223.79+7.97
GPT-6 LunaDecisions APIClosed · EfficientLeague 1 group11.2738.90+27.6312.1736.34+24.17
Perplexity Decider 27B v1.1NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 1 group17.2123.64+6.4318.3621.31+2.95
LFM2.5-24B-A2Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 24B MoE · 2.5B activeLeague 2 group24.0846.03+21.9525.8144.30+18.49
Qwen3.8-27Boff · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 27BLeague 2 group9.6323.71+14.0810.5222.07+11.55
Mercury DecideClosed · EfficientLeague 1 group15.2832.27+16.9916.1130.64+14.54
Winnow-12BNVIDIA RTX PRO 6000 BlackwellOpen · 12BLeague 1 group19.4040.94+21.5420.8239.56+18.74
LFM2.5-24B-A2Boff · NVIDIA RTX PRO 6000 BlackwellOpen · 24B MoE · 2.5B activeLeague 2 group36.1472.68+36.5437.0971.22+34.12
Open-Jev-27B-v1.1NVIDIA RTX PRO 6000 BlackwellOpen · 27BLeague 1 group51.0354.35+3.3152.1052.82+0.72
Perplexity Decider Flash v1.2NVIDIA RTX PRO 6000 BlackwellOpen · 4.0BLeague 1 group29.5163.81+34.3132.1560.75+28.60
Gemini 3.8 FlashlowClosed · WorkhorseLeague 2 group15.2110.19−5.0216.338.38−7.95
Claude Sonnet 5.5lowClosed · WorkhorseLeague 2 group12.9618.62+5.6614.1717.06+2.89
GPT-6.1 SollowClosed · WorkhorseLeague 2 group13.5610.52−3.0415.098.70−6.39
GPT-6 AstramediumClosed · FlagshipLeague 3 group8.467.75−0.719.606.25−3.35
GPT-6.1 SolhighClosed · WorkhorseLeague 2 group11.7310.46−1.2812.538.87−3.66
Gemma 4 31Bon · NVIDIA RTX PRO 6000 Blackwell + NVIDIA DGX SparkOpen · 31BLeague 2 group14.9318.07+3.1316.0516.73+0.69
GPT-6.1 Terrano group (exploratory)11.438.61−2.8112.646.74−5.90
Claude Fable 5.1highClosed · FlagshipLeague 3 group · added under §10.14(g), after the results were read12.198.18−4.0113.136.25−6.88
Grok 4.7highClosed · WorkhorseLeague 3 group · added under §10.14(g), after the results were read11.7716.37+4.6012.8214.18+1.36
d1-ultraClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read13.9225.38+11.4614.9421.96+7.03
GLiDEAPIClosed · EfficientLeague 1 group · added under §10.14(g), after the results were read13.4622.48+9.0214.2321.32+7.08
Gemini 4 ArgonlowClosed · Flagshipno group (exploratory)8.7311.33+2.609.049.02−0.02
Randomyardstick83.8493.55+9.7286.1092.16+6.06

Swipe for more columns →

Engine check: the primary comparisons with both engines, with the §10.14(g) additions (primary)
Comparison2-ply difference99% intervalRollout difference99% intervalDirection changed
League 1 group − League 2 group+5.57[−2.33, +14.99]+5.35[−2.63, +15.07]no
League 1 group − League 3 group+15.03[−1.57, +36.32]+14.90[−1.73, +35.94]no
League 2 group − League 3 group+9.46[−1.84, +23.64]+9.55[−1.79, +23.36]no
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff−14.59[−55.64, +18.77]−15.63[−56.64, +16.95]no
GPT-6.1 Solhigh − GPT-6.1 Sollow+1.76[−2.81, +6.99]+2.73[−1.97, +8.46]no

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Swipe for more columns →

Engine check: the primary comparisons with both engines, original members only
Comparison2-ply difference99% intervalRollout difference99% intervalDirection changed
League 1 group − League 2 group+6.36[−3.28, +18.64]+6.17[−3.57, +18.72]no
League 1 group − League 3 group+16.49[−1.68, +40.04]+16.11[−2.20, +39.49]no
League 2 group − League 3 group+10.13[−1.63, +24.58]+9.94[−1.65, +24.21]no
LFM2.5-24B-A2Bon − LFM2.5-24B-A2Boff−14.59[−55.64, +18.77]−15.63[−56.64, +16.95]no
GPT-6.1 Solhigh − GPT-6.1 Sollow+1.76[−2.81, +6.99]+2.73[−1.97, +8.46]no

Groups by original league (Method §10.14, amendment of 2026-10-27). Method §10 and its notes were committed before any model call, and the per-setup first looks were computed under the frozen extension rule. Separately, before the amendment, six full-report calls were issued on partial data under the earlier group definitions; that all six completed successfully has not been independently established. One returned numeric old-group primary results into the site agent's context before the amendment. Whether a person saw them, and any causation, is not established. Nothing was published before the amendment; the aggregate report and data were pushed to the public repository in commit 3e4f413 on 2026-10-28 (01:03 UTC), before release approval. See §10.14(e) and its dated correction (f) of 2026-10-28.

Original members only: the League 3 group has 1 setup (GPT-6 Astramedium); comparisons 2 and 3 rest on it.

Swipe for more columns →

Engine check: counts
SubsetPairsPlays rolled outUnrolled plays used (2-ply + shift)
Engine-check subset40442441

Settings: GNUbg 1.08.003 rollout: 1,296 trials per play, cubeless, variance reduction on, quasi-random dice, no truncation, no late evaluation, chequer play 1-ply, seed 1, 8 threads, nice 19; candidates: top 5 at 2-ply plus every play any setup chose. The additions' chosen plays that had no rollout yet were rolled out later with the same settings, alone rather than alongside the other candidates of their positions; the earlier rollouts are unchanged.

No primary comparison changes direction between 2-ply and rollout values, either way. GNUbg's network learned standard play, so its values for early Nackgammon positions are likely less accurate; the check bounds that, it does not remove it.

Scope. These results hold for these rosters only; each group is the setups in the run from that original league.

The League 3 group currently has 3 setups (GPT-6 Astramedium, Claude Fable 5.1high, Grok 4.7high); comparisons 2 and 3 rest on them.

Data: nack-paired.json · Method (Method spec 2026-10-24 §10 (groups: §10.14))

Finished. The knockout: Leagues.