Where models disagree
Where do models disagree?
Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.
Earlier version: the v1 set
Earlier versionSwipe for more columns →
| Position | Phase | Legal plays | Different plays chosen | Setups choosing the engine's best | Most common play | Engine's best play |
|---|---|---|---|---|---|---|
| v1-0292 | Opening | 19 | 10 | 6% | 13/9 9/835% of setups, loses 0.012 | 8/4* 24/23 |
| v1-0370 | Contact | 23 | 8 | 0% | 6/3 6/535% of setups, loses 0.048 | 6/3 3/2 |
| v1-0436 | Race | 13 | 8 | 6% | 11/9 9/829% of setups, loses 0.028 | 8/6 13/12 |
| v1-0796 | Contact | 18 | 8 | 12% | 6/1 6/418% of setups, loses 0.051 | 6/1 19/17 |
| v1-0708 | Contact | 22 | 8 | 24% | 6/5 5/141% of setups, loses 0.065 | 4/3 6/2 |
| v1-0801 | Contact | 17 | 8 | 24% | 24/18* 18/1629% of setups, loses 0.019 | 13/11 24/18* |
| v1-0079 | Contact | 13 | 7 | 0% | 6/1 5/329% of setups, loses 0.952 | 21/16* 11/9 |
| v1-0735 | Contact | 14 | 7 | 0% | 7/3 15/1335% of setups, loses 0.050 | 15/11 11/9 |
| v1-0210 | Contact | 11 | 7 | 6% | 3/1 1/off53% of setups, loses 0.025 | 6/4 1/off |
| v1-0602 | Contact | 17 | 7 | 6% | 8/6 24/2041% of setups, loses 0.006 | 3/1 24/20 |
| v1-0791 | Race | 16 | 7 | 6% | 8/7 7/453% of setups, loses 0.005 | 14/13 8/5 |
| v1-0671 | Contact | 19 | 7 | 12% | bar/24 6/5 6/5 2/147% of setups, loses 0.016 | bar/24 6/5 3/2 2/1 |
| v1-0260 | Opening | 18 | 7 | 18% | 13/11 11/529% of setups, loses 0.282 | 13/11 24/18* |
| v1-0281 | Race | 21 | 7 | 18% | 8/5 5/329% of setups, loses 0.139 | 19/16 19/17 |
| v1-0014 | Contact | 12 | 7 | 29% | 24/22 24/18*29% of setups, loses 0.000 | 24/22 24/18* |
| v1-0854 | Contact | 20 | 7 | 35% | 24/22* 24/2135% of setups, loses 0.000 | 24/22* 24/21 |
| v1-0489 | Opening | 16 | 7 | 47% | 24/20 20/15*47% of setups, loses 0.000 | 24/20 20/15* |
| v1-0667 | Contact | 12 | 7 | 47% | 13/10 10/4*47% of setups, loses 0.000 | 13/10 10/4* |
| v1-0495 | Opening | 17 | 7 | 59% | 24/21 13/1159% of setups, loses 0.000 | 24/21 13/11 |
| v1-0287 | Contact | 10 | 7 | 65% | 16/11 4/365% of setups, loses 0.000 | 16/11 4/3 |
| v1-0041 | Opening | 11 | 6 | 0% | 13/11 6/541% of setups, loses 0.034 | 24/22 6/5 |
| v1-0261 | Contact | 13 | 6 | 0% | 13/9 13/753% of setups, loses 0.152 | 24/18 18/14 |
| v1-0304 | Opening | 10 | 6 | 0% | 13/7 7/547% of setups, loses 0.017 | 24/22 22/16 |
| v1-0309 | Opening | 12 | 6 | 0% | 13/11 11/629% of setups, loses 0.075 | 24/22 6/1* |
| v1-0557 | Opening | 18 | 6 | 0% | 13/9 13/1047% of setups, loses 0.007 | 24/20 13/10 |
| v1-0624 | Contact | 21 | 6 | 0% | 7/2 6/529% of setups, loses 0.345 | 13/8 24/23 |
| v1-0742 | Race | 14 | 6 | 0% | 8/3 3/147% of setups, loses 0.330 | 13/8 8/6 |
| v1-0785 | Contact | 21 | 6 | 0% | 6/3 8/365% of setups, loses 0.173 | 24/21 21/16 |
| v1-0805 | Contact | 19 | 6 | 0% | 2/1* 4/153% of setups, loses 0.414 | 6/5 24/21 |
| v1-0827 | Contact | 13 | 6 | 0% | 24/18 24/18 14/8 8/241% of setups, loses 0.201 | 24/18 20/14 18/12 14/8 |
| v1-0829 | Contact | 12 | 6 | 0% | 6/3 6/447% of setups, loses 0.314 | 10/7 10/8 |
| v1-0031 | Holding game | 18 | 6 | 6% | 18/15 18/15 15/12 15/1265% of setups, loses 0.036 | 18/15 15/12 12/9 9/6 |
| v1-0074 | Contact | 25 | 6 | 6% | 6/1* 6/4*35% of setups, loses 0.123 | 21/16 16/14* |
| v1-0101 | Race | 8 | 6 | 6% | 19/13 14/1235% of setups, loses 0.001 | 19/13 13/11 |
| v1-0131 | Contact | 18 | 6 | 6% | 6/5 7/559% of setups, loses 0.030 | 13/12 24/22 |
| v1-0146 | Opening | 15 | 6 | 6% | 13/11 6/565% of setups, loses 0.011 | 13/11 24/23 |
| v1-0214 | Race | 23 | 6 | 6% | 20/16 3/153% of setups, loses 0.038 | 16/12 20/18 |
| v1-0283 | Contact | 9 | 6 | 6% | 12/7 3/253% of setups, loses 0.099 | 22/17 12/11 |
| v1-0313 | Contact | 10 | 6 | 6% | 13/11 5/271% of setups, loses 0.021 | 13/11 11/8 |
| v1-0460 | Contact | 13 | 6 | 6% | 7/2 7/535% of setups, loses 0.458 | 16/11 11/9 |
| v1-0530 | Race | 14 | 6 | 6% | 12/7 13/741% of setups, loses 0.017 | 13/8 12/6 |
| v1-0572 | Contact | 9 | 6 | 6% | bar/19 19/13 13/7 7/147% of setups, loses 0.125 | bar/19 19/13 15/9 13/7 |
| v1-0699 | Contact | 13 | 6 | 6% | 13/9 9/629% of setups, loses 0.100 | 8/4 8/5 |
| v1-0746 | Contact | 18 | 6 | 6% | 7/2 3/247% of setups, loses 0.145 | 14/9 7/6 |
| v1-0909 | Opening | 16 | 6 | 6% | 13/7 8/447% of setups, loses 0.048 | 13/7 24/20 |
| v1-0957 | Race | 14 | 6 | 6% | 19/14 19/14 19/14 13/853% of setups, loses 0.024 | 19/14 19/14 19/14 14/9 |
| v1-0993 | Opening | 18 | 6 | 6% | 13/9 13/1041% of setups, loses 0.063 | 13/9 24/21 |
| v1-0005 | Contact | 13 | 6 | 12% | 9/6 6/453% of setups, loses 0.083 | 9/6 13/11 |
| v1-0377 | Race | 6 | 6 | 12% | 10/8 8/241% of setups, loses 0.046 | 10/8 17/11 |
| v1-0428 | Contact | 15 | 6 | 12% | 6/5 18/1359% of setups, loses 0.018 | 18/17 17/12 |
| v1-0482 | Contact | 25 | 6 | 12% | 13/9 9/847% of setups, loses 0.096 | 13/9 6/5 |
| v1-0511 | Contact | 14 | 6 | 12% | 6/2 7/1*35% of setups, loses 0.305 | 24/20 20/14 |
| v1-0629 | Contact | 14 | 6 | 12% | 17/14 18/1453% of setups, loses 0.014 | 17/14 14/10 |
| v1-0700 | Race | 19 | 6 | 12% | 13/10 5/447% of setups, loses 0.018 | 10/7 7/6 |
| v1-0826 | Contact | 14 | 6 | 12% | 3/2 6/247% of setups, loses 0.069 | 18/14 14/13 |
| v1-0956 | Contact | 16 | 6 | 12% | 11/5 6/259% of setups, loses 0.021 | 11/5 8/4 |
| v1-0118 | Contact | 16 | 6 | 18% | 13/11 8/347% of setups, loses 0.032 | 13/11 11/6 |
| v1-0282 | Race | 21 | 6 | 18% | 9/8 8/553% of setups, loses 0.000 | 13/12 12/9 |
| v1-0293 | Contact | 15 | 6 | 18% | 24/19* 19/1329% of setups, loses 0.079 | 20/14 24/19* |
| v1-0533 | Contact | 12 | 6 | 18% | 10/7 11/559% of setups, loses 0.042 | 11/8 10/4 |
| v1-0639 | Race | 19 | 6 | 24% | 13/9 13/1129% of setups, loses 0.002 | 8/4 7/5 |
| v1-0645 | Contact | 17 | 6 | 24% | 13/9 16/1024% of setups, loses 0.000 | 13/9 16/10 |
| v1-0806 | Bear-off | 15 | 6 | 24% | 4/3 1/off 1/off 1/off29% of setups, loses 0.000 | 2/1 1/off 1/off 1/off |
| v1-0906 | Contact | 22 | 6 | 24% | 13/10 7/129% of setups, loses 0.237 | 24/21 24/18* |
| v1-0053 | Race | 18 | 6 | 29% | 12/10 12/10 10/8 10/847% of setups, loses 0.070 | 12/10 12/10 10/8 8/6 |
| v1-0470 | Opening | 14 | 6 | 29% | 13/7 8/5*29% of setups, loses 0.000 | 13/7 8/5* |
| v1-0536 | Race | 6 | 6 | 29% | 20/19 20/1429% of setups, loses 0.000 | 20/19 20/14 |
| v1-0824 | Contact | 24 | 6 | 29% | 13/12* 5/247% of setups, loses 0.058 | 13/12* 12/9 |
| v1-0619 | Opening | 17 | 6 | 35% | 13/11 13/1035% of setups, loses 0.000 | 13/11 13/10 |
| v1-0535 | Contact | 13 | 6 | 41% | 7/4 13/1141% of setups, loses 0.000 | 7/4 13/11 |
| v1-0666 | Race | 11 | 6 | 41% | 14/11 11/941% of setups, loses 0.000 | 14/11 11/9 |
| v1-0714 | Opening | 16 | 6 | 41% | 6/2* 2/1*41% of setups, loses 0.000 | 6/2* 2/1* |
| v1-0011 | Opening | 17 | 6 | 47% | 13/9 13/1047% of setups, loses 0.000 | 13/9 13/10 |
| v1-0515 | Contact | 22 | 6 | 47% | 13/10 10/647% of setups, loses 0.000 | 13/10 10/6 |
| v1-0622 | Opening | 11 | 6 | 47% | 24/18 13/1147% of setups, loses 0.000 | 24/18 13/11 |
| v1-0637 | Contact | 23 | 6 | 47% | 17/16 16/1247% of setups, loses 0.000 | 17/16 16/12 |
| v1-0774 | Contact | 19 | 6 | 47% | 4/2 5/247% of setups, loses 0.000 | 4/2 5/2 |
| v1-0771 | Race | 24 | 6 | 53% | 18/16 16/14 14/12 12/1053% of setups, loses 0.000 | 18/16 16/14 14/12 12/10 |
| v1-0553 | Race | 8 | 6 | 59% | 17/15 18/1259% of setups, loses 0.000 | 17/15 18/12 |
| v1-0396 | Contact | 20 | 6 | 65% | 20/18 8/265% of setups, loses 0.000 | 20/18 8/2 |
| v1-0718 | Contact | 11 | 6 | 65% | 10/7 7/4 7/4 4/165% of setups, loses 0.000 | 10/7 7/4 7/4 4/1 |
| v1-0734 | Contact | 23 | 6 | 65% | 8/5 7/4 7/4 7/465% of setups, loses 0.000 | 8/5 7/4 7/4 7/4 |
| v1-0818 | Contact | 14 | 6 | 65% | 24/23 24/19*65% of setups, loses 0.000 | 24/23 24/19* |
| v1-0934 | Contact | 6 | 6 | 65% | 18/17 17/1365% of setups, loses 0.000 | 18/17 17/13 |
| v1-0984 | Contact | 14 | 6 | 71% | 18/16* 18/1371% of setups, loses 0.000 | 18/16* 18/13 |
| v1-0018 | Contact | 13 | 5 | 0% | 3/2* 2/off53% of setups, loses 0.044 | 5/3 1/off |
| v1-0035 | Opening | 10 | 5 | 0% | 13/8 13/965% of setups, loses 0.098 | 24/20 20/15 |
| v1-0061 | Contact | 22 | 5 | 0% | 13/9 6/541% of setups, loses 0.135 | 24/20 6/5 |
| v1-0168 | Opening | 23 | 5 | 0% | 6/5 13/1176% of setups, loses 0.007 | 6/5 24/22 |
| v1-0183 | Opening | 18 | 5 | 0% | 22/16 16/1376% of setups, loses 0.021 | 22/16 24/21 |
| v1-0366 | Contact | 19 | 5 | 0% | 10/5 3/241% of setups, loses 0.196 | 21/16 16/15 |
| v1-0376 | Contact | 9 | 5 | 0% | 13/9 9/641% of setups, loses 0.136 | 13/9 24/21 |
| v1-0434 | Contact | 14 | 5 | 0% | 7/1 8/747% of setups, loses 0.396 | 24/23 23/17 |
| v1-0450 | Contact | 16 | 5 | 0% | 6/4 6/276% of setups, loses 0.129 | 6/4 24/20 |
| v1-0503 | Contact | 25 | 5 | 0% | 8/5 6/535% of setups, loses 0.152 | 24/21 6/5 |
| v1-0559 | Contact | 8 | 5 | 0% | 8/2 6/341% of setups, loses 0.046 | 14/8 8/5 |
| v1-0751 | Contact | 15 | 5 | 0% | 3/2 6/241% of setups, loses 0.109 | 2/1 24/20 |
| v1-0769 | Contact | 15 | 5 | 0% | 24/18 24/2147% of setups, loses 0.341 | 24/18 8/5* |
| v1-0807 | Contact | 14 | 5 | 0% | 13/9 9/353% of setups, loses 0.191 | 24/20 22/16* |
| v1-0845 | Holding game | 16 | 5 | 0% | 20/19* 19/1476% of setups, loses 0.131 | 20/19* 20/15 |
| v1-0870 | Contact | 12 | 5 | 0% | 13/12 6/341% of setups, loses 0.102 | 13/12 24/21 |
| v1-0879 | Contact | 13 | 5 | 0% | 8/2 6/365% of setups, loses 0.020 | 8/2 8/5 |
| v1-0889 | Contact | 21 | 5 | 0% | 20/17 17/14 14/11 11/871% of setups, loses 0.040 | 20/17 17/14 14/11 8/5 |
| v1-0890 | Opening | 15 | 5 | 0% | 13/7* 8/647% of setups, loses 0.122 | 13/7* 24/22 |
| v1-0901 | Contact | 20 | 5 | 0% | 5/3 7/353% of setups, loses 0.102 | 11/9 5/1* |
| v1-0925 | Contact | 20 | 5 | 0% | 6/3 6/553% of setups, loses 0.404 | 8/5 24/23 |
| v1-0948 | Contact | 24 | 5 | 0% | 13/11 13/1035% of setups, loses 0.035 | 24/22 13/10 |
| v1-0994 | Contact | 25 | 5 | 0% | 6/5 8/559% of setups, loses 0.437 | 6/5 24/21* |
| v1-0154 | Contact | 8 | 5 | 6% | 20/18* 18/1371% of setups, loses 0.032 | 24/19 20/18* |
| v1-0180 | Contact | 17 | 5 | 6% | 11/10 10/571% of setups, loses 0.025 | 22/17 17/16 |
| v1-0226 | Contact | 5 | 5 | 6% | 10/4 10/571% of setups, loses 0.009 | 10/4 13/8 |
| v1-0244 | Opening | 16 | 5 | 6% | 6/2 8/247% of setups, loses 0.111 | 24/20 20/14 |
| v1-0325 | Contact | 13 | 5 | 6% | 5/4 5/4 3/2 3/247% of setups, loses 0.057 | 5/4 5/4 4/3 4/3 |
| v1-0744 | Contact | 19 | 5 | 6% | 5/4 4/253% of setups, loses 0.021 | 3/2 15/13 |
| v1-0834 | Contact | 12 | 5 | 6% | 10/6 10/447% of setups, loses 0.124 | 20/16 20/14 |
| v1-0875 | Contact | 13 | 5 | 6% | 8/3 6/371% of setups, loses 0.219 | 24/19* 19/16 |
| v1-0878 | Contact | 14 | 5 | 6% | 7/1 6/176% of setups, loses 0.149 | 21/15* 21/16 |
| v1-0990 | Contact | 20 | 5 | 6% | 21/18 18/1771% of setups, loses 0.007 | 14/13 21/18 |
| v1-0992 | Contact | 15 | 5 | 6% | 6/4 9/471% of setups, loses 0.005 | 24/22* 8/3 |
| v1-0258 | Contact | 12 | 5 | 12% | 8/2 8/547% of setups, loses 0.305 | 24/21 21/15 |
| v1-0300 | Holding game | 17 | 5 | 12% | 9/8 13/953% of setups, loses 0.006 | 2/1 13/9 |
| v1-0512 | Contact | 20 | 5 | 12% | 13/11 6/553% of setups, loses 0.013 | 6/4* 4/3 |
| v1-0569 | Contact | 14 | 5 | 12% | 9/3 22/18*59% of setups, loses 0.067 | 24/18* 22/18 |
| v1-0621 | Contact | 7 | 5 | 12% | 13/11 6/365% of setups, loses 0.015 | 13/11 11/8 |
| v1-0652 | Opening | 10 | 5 | 12% | 24/22 13/859% of setups, loses 0.013 | 13/11 13/8 |
| v1-0966 | Contact | 13 | 5 | 12% | 11/8 8/465% of setups, loses 0.034 | 11/8 17/13 |
| v1-0999 | Contact | 11 | 5 | 12% | 5/2 5/365% of setups, loses 0.079 | 12/9 12/10 |
| v1-0025 | Opening | 10 | 5 | 18% | 13/7 7/2*41% of setups, loses 0.070 | 24/18 18/13 |
| v1-0098 | Contact | 18 | 5 | 18% | 3/1 3/235% of setups, loses 0.171 | 24/22 22/21 |
| v1-0412 | Contact | 15 | 5 | 18% | 16/11 13/8 11/6 8/335% of setups, loses 0.136 | 13/8 13/8 13/8 8/3 |
| v1-0664 | Race | 15 | 5 | 18% | 6/2 3/259% of setups, loses 0.055 | 8/4 3/2 |
| v1-0747 | Contact | 9 | 5 | 18% | 8/6 8/559% of setups, loses 0.234 | 5/3 5/2 |
| v1-0773 | Contact | 8 | 5 | 18% | 20/17 24/1965% of setups, loses 0.193 | 20/17 17/12 |
| v1-0298 | Opening | 15 | 5 | 24% | 6/5 13/1159% of setups, loses 0.003 | 24/23 13/11 |
| v1-0371 | Contact | 21 | 5 | 24% | 13/11 10/941% of setups, loses 0.117 | 6/4 10/9 |
| v1-0641 | Bear-off | 22 | 5 | 24% | 6/5 5/4 4/3 2/141% of setups, loses 0.010 | 2/1 2/1 1/off 1/off |
| v1-0686 | Contact | 9 | 5 | 24% | 21/16 5/353% of setups, loses 0.057 | 10/5 6/4 |
| v1-0814 | Contact | 20 | 5 | 24% | 7/6 7/559% of setups, loses 0.038 | 5/4 3/1 |
| v1-0872 | Contact | 24 | 5 | 24% | 6/2 3/235% of setups, loses 0.725 | 22/18* 13/12 |
| v1-0082 | Contact | 20 | 5 | 29% | 6/2* 7/247% of setups, loses 0.058 | 13/9 13/8 |
| v1-0097 | Contact | 15 | 5 | 29% | 8/5 8/653% of setups, loses 0.166 | 6/3 3/1 |
| v1-0321 | Race | 10 | 5 | 29% | 11/9 4/2 3/1 3/153% of setups, loses 0.122 | 11/9 9/7 7/5 2/off |
| v1-0390 | Contact | 6 | 5 | 29% | 24/18* 18/1647% of setups, loses 0.077 | 24/22 24/18* |
| v1-0454 | Opening | 16 | 5 | 29% | 11/10 13/829% of setups, loses 0.322 | 24/23 23/18* |
| v1-0518 | Contact | 17 | 5 | 29% | 18/13 13/9*29% of setups, loses 0.000 | 18/13 13/9* |
| v1-0655 | Contact | 18 | 5 | 29% | 4/3 4/3 3/2* 3/235% of setups, loses 0.174 | 1/off 1/off 1/off 1/off |
| v1-0704 | Contact | 17 | 5 | 29% | 10/8 8/553% of setups, loses 0.354 | 21/18 18/16* |
| v1-0707 | Contact | 5 | 5 | 29% | bar/20 6/535% of setups, loses 0.029 | bar/20 7/6 |
| v1-0736 | Contact | 12 | 5 | 29% | 6/2 6/453% of setups, loses 0.168 | 10/6 10/8 |
| v1-0896 | Contact | 24 | 5 | 29% | 8/6 8/747% of setups, loses 0.006 | 6/4 5/4 |
1,000 positions, 17 setups. All setups chose the same play in 76 positions. Earlier version: plays were listed in the runner's order, which tends to put the best play early. When Gemini 4 Argon answered the same 1,000 positions with the plays shuffled, its error rate changed by +1.9 (95% interval −0.2 to +4.2), so the effect is not proven.
How many different plays the setups chose
| Different plays chosen | Positions |
|---|---|
| 1 | 76 |
| 2 | 256 |
| 3 | 238 |
| 4 | 214 |
| 5 | 131 |
| 6 | 65 |
| 7 | 14 |
| 8 | 5 |
| 10 | 1 |