Hilde · Decision models · answer key v2

Grammar · topic 16

Questions combining several grammar topics

220 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model98.6%96.1% to 99.5%217 of 220
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model98.6%96.1% to 99.5%217 of 220
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model97.7%94.8% to 99.0%215 of 220
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model97.3%94.2% to 98.7%214 of 220
GPT-6 Astralow reasoning · 16 questions per request · Chat model95.9%92.4% to 97.8%211 of 220
GPT-6 Astralow reasoning · 1 question per request · Chat model95.9%92.4% to 97.8%211 of 220
GPT-6.1 Sollow reasoning · 1 question per request · Chat model95.0%91.3% to 97.2%209 of 220
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model94.5%90.7% to 96.9%208 of 220
Claude Fable 5.1high reasoning · 16 questions per request · Chat model94.1%90.2% to 96.5%207 of 220
Claude Opus 5.5high reasoning · 16 questions per request · Chat model94.1%90.2% to 96.5%207 of 220
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model91.8%87.4% to 94.8%202 of 220
Claude Opus 5.5low reasoning · 1 question per request · Chat model90.5%85.8% to 93.7%199 of 220
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model88.2%83.2% to 91.8%194 of 220
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model86.4%81.2% to 90.3%190 of 220
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model86.4%81.2% to 90.3%190 of 220
Grok 4.7high reasoning · 16 questions per request · Chat model85.0%79.7% to 89.1%187 of 220
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model83.6%78.2% to 87.9%184 of 220
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model81.8%76.2% to 86.4%180 of 220
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model77.3%71.3% to 82.3%170 of 220
Claude Opus 5.5low reasoning · 16 questions per request · Chat model74.1%67.9% to 79.4%163 of 220
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model67.3%60.8% to 73.1%148 of 220
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model62.7%56.2% to 68.8%138 of 220
GLM-5.3low reasoning · 16 questions per request · Chat model62.7%56.2% to 68.8%138 of 220
Liquid d1-ultra1 question per request · Decision model61.8%55.2% to 68.0%136 of 220
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model50.5%43.9% to 57.0%111 of 220
Gemma 4 31Breasoning off · 16 questions per request · Chat model49.5%43.0% to 56.1%109 of 220
Fastino GLiDE v1.2local · 1 question per request · Decision model47.3%40.8% to 53.9%104 of 220
Drex Max v11 question per request · Decision model46.4%39.9% to 53.0%102 of 220
Drex Max 1.11 question per request · Decision model46.4%39.9% to 53.0%102 of 220
Fastino GLiDE1 question per request · Decision model45.5%39.0% to 52.1%100 of 220
vLLM-SR d31 question per request · Decision model45.5%39.0% to 52.1%100 of 220
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model44.5%38.1% to 51.2%98 of 220
Perplexity Decider 27B v1.2local · 1 question per request · Decision model44.1%37.7% to 50.7%97 of 220
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model43.6%37.2% to 50.2%96 of 220
SubQ Snap Preview1 question per request · Decision model43.2%36.8% to 49.8%95 of 220
Perplexity Decider 27B v1.11 question per request · Decision model42.3%35.9% to 48.9%93 of 220
GPT-6 Lunalow reasoning · 16 questions per request · Chat model41.4%35.1% to 48.0%91 of 220
SubQ Snap 1.11 question per request · Decision model40.9%34.6% to 47.5%90 of 220
Liquid d1.11 question per request · Decision model40.9%34.6% to 47.5%90 of 220
Mammoth-11 question per request · Decision model40.0%33.8% to 46.6%88 of 220
Gemma 4 31Breasoning off · 1 question per request · Chat model40.0%33.8% to 46.6%88 of 220
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model37.3%31.2% to 43.8%82 of 220
Jev 1.14.01 question per request · Decision model36.8%30.7% to 43.4%81 of 220
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model36.4%30.3% to 42.9%80 of 220
Perplexity Decider 27B v11 question per request · Decision model36.4%30.3% to 42.9%80 of 220
Celeris-1 Decision1 question per request · Decision model35.5%29.4% to 42.0%78 of 220
OneJev 27B1 question per request · Decision model35.0%29.0% to 41.5%77 of 220
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model34.5%28.6% to 41.0%76 of 220
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model34.1%28.1% to 40.6%75 of 220
Clef1 question per request · Decision model34.1%28.1% to 40.6%75 of 220
Winnow-12B1 question per request · Decision model33.6%27.7% to 40.1%74 of 220
Drex 2.01 question per request · Decision model33.2%27.3% to 39.6%73 of 220
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model30.9%25.2% to 37.3%68 of 220
Jev 1.13.01 question per request · Decision model30.9%25.2% to 37.3%68 of 220
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model30.0%24.3% to 36.4%66 of 220
Elephant-11 question per request · Decision model29.5%23.9% to 35.9%65 of 220
Liquid d11 question per request · Decision model28.6%23.1% to 34.9%63 of 220
Cygnet1 question per request · Decision model28.6%23.1% to 34.9%63 of 220
Qwen3-14Breasoning off · 16 questions per request · Chat model27.7%22.2% to 34.0%61 of 220
Perplexity Decider Flash v1.2local · 1 question per request · Decision model27.3%21.8% to 33.5%60 of 220
Erk-14Breasoning off · 16 questions per request · Chat model27.3%21.8% to 33.5%60 of 220
Clef-flash1 question per request · Decision model25.9%20.6% to 32.1%57 of 220
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model25.5%20.2% to 31.6%56 of 220
Metask-Jev 4B1 question per request · Decision model25.0%19.7% to 31.1%55 of 220
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model23.6%18.5% to 29.7%52 of 220
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model23.6%18.5% to 29.7%52 of 220
Strands Decider 2B1 question per request · Decision model22.3%17.3% to 28.2%49 of 220
Laya1 question per request · Decision model21.4%16.5% to 27.2%47 of 220
Liquid d1-2B (open weights)1 question per request · Decision model20.9%16.1% to 26.8%46 of 220
Laya multilingual1 question per request · Decision model19.1%14.4% to 24.8%42 of 220
Open-Jev 27B v1.11 question per request · Decision model18.2%13.6% to 23.8%40 of 220
GLiNER2.5-multi-Decide1 question per request · Decision model17.7%13.2% to 23.3%39 of 220
meraGPT Decider 11 question per request · Decision model17.3%12.9% to 22.8%38 of 220
GLiNER2.5 Multi (base)1 question per request · Chat model14.5%10.5% to 19.8%32 of 220

Its questions

IDPageNumberAnswer keySetups rightDecision models right
22973011E34 of 7412 of 37
22983013C48 of 7419 of 37
22993014A32 of 7412 of 37
23003012C46 of 7419 of 37
23013015B31 of 746 of 37
23133046E28 of 745 of 37
23143049D26 of 743 of 37
23153047B24 of 743 of 37
231630410E63 of 7432 of 37
23173048B44 of 7416 of 37
231830411C59 of 7424 of 37
23313071A46 of 7419 of 37
23323074B23 of 741 of 37
23333072A26 of 745 of 37
23343075E42 of 7414 of 37
23353073B46 of 7420 of 37
23363076D31 of 746 of 37
23373087A26 of 743 of 37
233830810B34 of 748 of 37
23393088C41 of 7416 of 37
234030811D59 of 7426 of 37
23413089E25 of 745 of 37
234230812D45 of 7425 of 37
23433091C43 of 7414 of 37
23443094B27 of 742 of 37
23453092E35 of 748 of 37
23463095A42 of 7418 of 37
23473093D41 of 7417 of 37
23483096D28 of 745 of 37
23493107A25 of 749 of 37
235031010E7 of 742 of 37
235131011B53 of 7427 of 37
23523108C27 of 743 of 37
235331012A21 of 742 of 37
23543109D34 of 748 of 37
23643131E39 of 7415 of 37
23653134E25 of 743 of 37
23663135B58 of 7424 of 37
23673132D27 of 741 of 37
23683136C39 of 7412 of 37
23693133E31 of 7410 of 37
23813167D64 of 7429 of 37
238231610C39 of 7412 of 37
238331611A51 of 7421 of 37
23843168E32 of 748 of 37
238531612E45 of 7416 of 37
23863169E62 of 7428 of 37
23873171C37 of 7411 of 37
23883174D29 of 743 of 37
23893175E52 of 7422 of 37
23903172B19 of 740 of 37
23913176B34 of 745 of 37
23923173A51 of 7422 of 37
23983191E15 of 742 of 37
23993194D46 of 7421 of 37
24003192D12 of 741 of 37
24013195B33 of 748 of 37
24023193C32 of 746 of 37
24033196E14 of 744 of 37
24163227E42 of 7416 of 37
241732210E61 of 7432 of 37
241832211B58 of 7424 of 37
24193228A30 of 744 of 37
242032212C55 of 7423 of 37
24213229D43 of 7416 of 37
24463271D59 of 7428 of 37
24473274B45 of 7418 of 37
24483275A53 of 7422 of 37
24493272E25 of 744 of 37
24503276A55 of 7420 of 37
24513273C44 of 7416 of 37
24523287E28 of 743 of 37
245332810C22 of 746 of 37
24543288B56 of 7425 of 37
245532811A31 of 7415 of 37
245632812D29 of 747 of 37
24573289D37 of 7418 of 37
24583291A42 of 7414 of 37
24593295B39 of 7412 of 37
24603292B25 of 741 of 37
24613296E22 of 743 of 37
24623293D43 of 7414 of 37
24633297C35 of 7413 of 37
24643294E25 of 744 of 37
24713311E34 of 7410 of 37
24723314D27 of 742 of 37
24743312D39 of 749 of 37
24753316C23 of 742 of 37
24763313B57 of 7427 of 37
24833331D59 of 7427 of 37
24843334B47 of 7419 of 37
24853335E60 of 7426 of 37
24863332A34 of 7415 of 37
24873333C17 of 741 of 37
24883336E27 of 743 of 37
24893347A33 of 7413 of 37
249033410E16 of 7413 of 37
24913348D57 of 7427 of 37
249233411D40 of 7415 of 37
24933349C50 of 7418 of 37
249433412B35 of 748 of 37
38323026C38 of 748 of 37
38333029B54 of 7427 of 37
383430210E27 of 742 of 37
38353027D35 of 7414 of 37
383630211A35 of 749 of 37
38373028B27 of 743 of 37
38383031A31 of 749 of 37
38393033D17 of 742 of 37
38403034B49 of 7418 of 37
38413032C25 of 744 of 37
38423035C37 of 7415 of 37
38433051D30 of 745 of 37
38443054A41 of 7413 of 37
38453055D26 of 745 of 37
38463052A36 of 7412 of 37
38473056C36 of 749 of 37
38483053E40 of 7424 of 37
38493067E50 of 7424 of 37
385030610B45 of 7416 of 37
385130611C22 of 740 of 37
38523068B40 of 7411 of 37
385330612D53 of 7424 of 37
38543069A34 of 7410 of 37
38553111D41 of 7410 of 37
38563113E49 of 7421 of 37
38573112D36 of 7414 of 37
38583114C24 of 746 of 37
38593125D41 of 7413 of 37
38603128C27 of 748 of 37
38613126E33 of 749 of 37
38623129B36 of 747 of 37
38633127A45 of 7420 of 37
38643147D41 of 7419 of 37
38653149A42 of 7412 of 37
386631410E27 of 743 of 37
38673148D29 of 745 of 37
386831411B36 of 748 of 37
38693151A28 of 744 of 37
38703154E40 of 7415 of 37
38713155C43 of 7413 of 37
38723152D63 of 7428 of 37
38733156D53 of 7420 of 37
38743153A28 of 746 of 37
38753187C37 of 7410 of 37
38763189A43 of 7414 of 37
387731810C21 of 746 of 37
38783188C44 of 7419 of 37
387931811B26 of 744 of 37
38803207D58 of 7427 of 37
388132010C49 of 7422 of 37
38823208E38 of 7416 of 37
388332011B46 of 7420 of 37
388432012A27 of 744 of 37
38853209E29 of 744 of 37
38863211B45 of 7424 of 37
38873214D32 of 7410 of 37
38883212E28 of 745 of 37
38893215A48 of 7419 of 37
38903213C52 of 7425 of 37
38913216E63 of 7431 of 37
38923231E20 of 745 of 37
38933234A48 of 7424 of 37
38943235E46 of 7418 of 37
38953232E41 of 7416 of 37
38963236E27 of 747 of 37
38973233A38 of 7417 of 37
38983247E26 of 747 of 37
389932410B34 of 746 of 37
390032411D50 of 7420 of 37
39013248A21 of 743 of 37
390232412A44 of 7414 of 37
39033249C35 of 7412 of 37
39043251E28 of 745 of 37
39053254B28 of 748 of 37
39063252D37 of 7412 of 37
39073255D50 of 7420 of 37
39083256C35 of 7417 of 37
39093253E14 of 742 of 37
39103267E39 of 7410 of 37
391132610E62 of 7428 of 37
391232611B40 of 7418 of 37
39133268A30 of 747 of 37
391432612E13 of 743 of 37
39153269C29 of 747 of 37
39163308B39 of 7412 of 37
391733011E44 of 7416 of 37
39183309E26 of 741 of 37
391933012D35 of 746 of 37
392033013A31 of 749 of 37
392133010C32 of 744 of 37
39223327D52 of 7423 of 37
392333210C34 of 749 of 37
39243328B60 of 7431 of 37
392533211C25 of 744 of 37
39263329E26 of 743 of 37
392733212A34 of 745 of 37
39283351C58 of 7425 of 37
39293354D54 of 7426 of 37
39303355E40 of 7413 of 37
39313352D30 of 746 of 37
39323353E35 of 7411 of 37
39333356B40 of 7411 of 37
39343367E35 of 747 of 37
393533610A40 of 7413 of 37
393633611B47 of 7417 of 37
39373368C13 of 744 of 37
393833612E12 of 745 of 37
39393369D46 of 7420 of 37
39403371E33 of 7415 of 37
39413374C25 of 742 of 37
39423375D31 of 748 of 37
39433372A48 of 7425 of 37
39443373B22 of 743 of 37
39453376E48 of 7423 of 37
39463387A54 of 7428 of 37
39473389C46 of 7421 of 37
394833810B32 of 746 of 37
39493388C26 of 744 of 37
395033811D48 of 7419 of 37