Hilde · Decision models · answer key v2

Grammar · topic 7

Word types, such as nouns and verbs

140 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model99.3%96.1% to 99.9%139 of 140
GPT-6 Astralow reasoning · 16 questions per request · Chat model99.3%96.1% to 99.9%139 of 140
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model98.6%94.9% to 99.6%138 of 140
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model98.6%94.9% to 99.6%138 of 140
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model98.6%94.9% to 99.6%138 of 140
GPT-6 Astralow reasoning · 1 question per request · Chat model97.9%93.9% to 99.3%137 of 140
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model97.9%93.9% to 99.3%137 of 140
GPT-6.1 Sollow reasoning · 1 question per request · Chat model97.9%93.9% to 99.3%137 of 140
Claude Opus 5.5low reasoning · 16 questions per request · Chat model97.9%93.9% to 99.3%137 of 140
Claude Opus 5.5high reasoning · 16 questions per request · Chat model97.1%92.9% to 98.9%136 of 140
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model97.1%92.9% to 98.9%136 of 140
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model96.4%91.9% to 98.5%135 of 140
Claude Fable 5.1high reasoning · 16 questions per request · Chat model96.4%91.9% to 98.5%135 of 140
Claude Opus 5.5low reasoning · 1 question per request · Chat model95.7%91.0% to 98.0%134 of 140
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model95.0%90.0% to 97.6%133 of 140
Grok 4.7high reasoning · 16 questions per request · Chat model93.6%88.2% to 96.6%131 of 140
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model93.6%88.2% to 96.6%131 of 140
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model93.6%88.2% to 96.6%131 of 140
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model93.6%88.2% to 96.6%131 of 140
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model92.1%86.5% to 95.6%129 of 140
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model83.6%76.6% to 88.8%117 of 140
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model82.9%75.8% to 88.2%116 of 140
Liquid d1-ultra1 question per request · Decision model78.6%71.1% to 84.6%110 of 140
GLM-5.3low reasoning · 16 questions per request · Chat model77.9%70.3% to 83.9%109 of 140
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model77.9%70.3% to 83.9%109 of 140
Mammoth-11 question per request · Decision model69.3%61.2% to 76.3%97 of 140
Gemma 4 31Breasoning off · 16 questions per request · Chat model67.1%59.0% to 74.4%94 of 140
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model67.1%59.0% to 74.4%94 of 140
Drex Max v11 question per request · Decision model66.4%58.3% to 73.7%93 of 140
Perplexity Decider 27B v1.2local · 1 question per request · Decision model66.4%58.3% to 73.7%93 of 140
vLLM-SR d31 question per request · Decision model65.7%57.5% to 73.1%92 of 140
Drex Max 1.11 question per request · Decision model65.7%57.5% to 73.1%92 of 140
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model65.0%56.8% to 72.4%91 of 140
Gemma 4 31Breasoning off · 1 question per request · Chat model65.0%56.8% to 72.4%91 of 140
Fastino GLiDE v1.2local · 1 question per request · Decision model64.3%56.1% to 71.7%90 of 140
SubQ Snap 1.11 question per request · Decision model64.3%56.1% to 71.7%90 of 140
Perplexity Decider 27B v1.11 question per request · Decision model63.6%55.3% to 71.1%89 of 140
Fastino GLiDE1 question per request · Decision model62.1%53.9% to 69.8%87 of 140
SubQ Snap Preview1 question per request · Decision model61.4%53.2% to 69.1%86 of 140
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model60.0%51.7% to 67.7%84 of 140
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model56.4%48.2% to 64.4%79 of 140
Jev 1.14.01 question per request · Decision model55.7%47.4% to 63.7%78 of 140
GPT-6 Lunalow reasoning · 16 questions per request · Chat model55.0%46.7% to 63.0%77 of 140
Celeris-1 Decision1 question per request · Decision model55.0%46.7% to 63.0%77 of 140
Perplexity Decider 27B v11 question per request · Decision model54.3%46.0% to 62.3%76 of 140
Jev 1.13.01 question per request · Decision model54.3%46.0% to 62.3%76 of 140
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model53.6%45.3% to 61.6%75 of 140
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model52.1%43.9% to 60.2%73 of 140
OneJev 27B1 question per request · Decision model50.7%42.5% to 58.9%71 of 140
Clef1 question per request · Decision model50.7%42.5% to 58.9%71 of 140
Open-Jev 27B v1.11 question per request · Decision model50.7%42.5% to 58.9%71 of 140
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model50.0%41.8% to 58.2%70 of 140
Liquid d1.11 question per request · Decision model50.0%41.8% to 58.2%70 of 140
Cygnet1 question per request · Decision model50.0%41.8% to 58.2%70 of 140
Liquid d11 question per request · Decision model49.3%41.1% to 57.5%69 of 140
Drex 2.01 question per request · Decision model48.6%40.4% to 56.8%68 of 140
Elephant-11 question per request · Decision model46.4%38.4% to 54.7%65 of 140
Winnow-12B1 question per request · Decision model45.7%37.7% to 54.0%64 of 140
Perplexity Decider Flash v1.2local · 1 question per request · Decision model45.0%37.0% to 53.3%63 of 140
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model43.6%35.6% to 51.8%61 of 140
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model42.1%34.3% to 50.4%59 of 140
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model42.1%34.3% to 50.4%59 of 140
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model39.3%31.6% to 47.6%55 of 140
Clef-flash1 question per request · Decision model37.1%29.6% to 45.4%52 of 140
Metask-Jev 4B1 question per request · Decision model33.6%26.3% to 41.7%47 of 140
Qwen3-14Breasoning off · 16 questions per request · Chat model32.1%25.0% to 40.3%45 of 140
Erk-14Breasoning off · 16 questions per request · Chat model32.1%25.0% to 40.3%45 of 140
meraGPT Decider 11 question per request · Decision model30.0%23.0% to 38.0%42 of 140
Strands Decider 2B1 question per request · Decision model25.0%18.6% to 32.8%35 of 140
Liquid d1-2B (open weights)1 question per request · Decision model20.0%14.2% to 27.4%28 of 140
Laya multilingual1 question per request · Decision model18.6%13.0% to 25.8%26 of 140
GLiNER2.5-multi-Decide1 question per request · Decision model17.9%12.4% to 25.0%25 of 140
GLiNER2.5 Multi (base)1 question per request · Chat model16.4%11.2% to 23.4%23 of 140
Laya1 question per request · Decision model12.9%8.3% to 19.4%18 of 140

Its questions

IDPageNumberAnswer keySetups rightDecision models right
6891694B66 of 7430 of 37
6901691E64 of 7430 of 37
6911695D53 of 7427 of 37
6921692D36 of 746 of 37
6931696B30 of 745 of 37
6941693C41 of 7418 of 37
6951707E68 of 7433 of 37
69617010D46 of 7417 of 37
6971708B32 of 743 of 37
69817011E59 of 7427 of 37
6991709C48 of 7417 of 37
70017012A26 of 743 of 37
7081728A57 of 7425 of 37
70917211D29 of 744 of 37
71017212C58 of 7425 of 37
7111729E67 of 7430 of 37
71217210B39 of 7410 of 37
71317213A63 of 7428 of 37
32891651B63 of 7427 of 37
32901653C54 of 7422 of 37
32911652E61 of 7428 of 37
32921654A64 of 7430 of 37
32931665D41 of 7412 of 37
32941668D42 of 7417 of 37
32951666A68 of 7432 of 37
32961669E58 of 7428 of 37
32971667C68 of 7432 of 37
32981674E21 of 741 of 37
32991671D67 of 7433 of 37
33001672B55 of 7424 of 37
33011675C69 of 7434 of 37
33021673C66 of 7431 of 37
33031676B54 of 7422 of 37
33041687E41 of 7416 of 37
330516810D65 of 7432 of 37
330616811E34 of 748 of 37
33071688A26 of 746 of 37
330816812D29 of 743 of 37
33091689B33 of 748 of 37
33101711E34 of 748 of 37
33111715C29 of 743 of 37
33121712D69 of 7432 of 37
33131716A25 of 743 of 37
33141713B40 of 7414 of 37
33151717D53 of 7425 of 37
33161714A31 of 748 of 37
33171731C58 of 7427 of 37
33181734D44 of 7417 of 37
33191735C58 of 7428 of 37
33201732E54 of 7426 of 37
33211736A51 of 7420 of 37
33221733B25 of 742 of 37
33231747D22 of 744 of 37
33241749E54 of 7422 of 37
332517410A62 of 7427 of 37
33261748C68 of 7433 of 37
332717411B59 of 7425 of 37
33281751B36 of 748 of 37
33291754E52 of 7423 of 37
33301755A68 of 7432 of 37
33311752C53 of 7424 of 37
33321756D42 of 7413 of 37
33331757C43 of 7412 of 37
33341753D64 of 7431 of 37
33351768B67 of 7431 of 37
333617611E22 of 741 of 37
333717612D26 of 744 of 37
33381769A54 of 7424 of 37
333917610C31 of 746 of 37
334017613C46 of 7414 of 37
33411771D19 of 740 of 37
33421774E66 of 7432 of 37
33431772B35 of 7410 of 37
33441775A68 of 7432 of 37
33451773D52 of 7419 of 37
33461776C58 of 7426 of 37
33471787A66 of 7430 of 37
334817810E69 of 7434 of 37
334917811B57 of 7426 of 37
33501788E69 of 7434 of 37
33511789C21 of 741 of 37
335217812D62 of 7427 of 37
33531791D38 of 7410 of 37
33541795C67 of 7431 of 37
33551792C39 of 7413 of 37
33561796A59 of 7428 of 37
33571793E66 of 7432 of 37
33581797B56 of 7424 of 37
33591794E58 of 7426 of 37
33601808D20 of 743 of 37
336118011B19 of 741 of 37
336218012C38 of 7411 of 37
33631809E41 of 7412 of 37
336418013D39 of 7414 of 37
336518010A65 of 7433 of 37
33661811B57 of 7424 of 37
33671814B57 of 7424 of 37
33681812C56 of 7420 of 37
33691815E26 of 743 of 37
33701813D53 of 7421 of 37
33711826A40 of 7414 of 37
33721829B50 of 7423 of 37
337318210A61 of 7428 of 37
33741827D71 of 7435 of 37
33751828C24 of 745 of 37
337618211B27 of 744 of 37
33771831B64 of 7431 of 37
33781834D60 of 7429 of 37
33791835E31 of 742 of 37
33801832B26 of 742 of 37
33811836A16 of 747 of 37
33821833C46 of 7416 of 37
338318410B47 of 7418 of 37
33841847B24 of 745 of 37
338518411E5 of 743 of 37
33861848A35 of 747 of 37
33871849E28 of 744 of 37
338818412D49 of 7417 of 37
33891851B30 of 743 of 37
33901854D36 of 749 of 37
33911852E60 of 7426 of 37
33921855C63 of 7429 of 37
33931856A54 of 7423 of 37
33941853C33 of 746 of 37
33951867C35 of 7421 of 37
339618610E72 of 7436 of 37
33971868B66 of 7430 of 37
339818611A61 of 7427 of 37
33991869E21 of 744 of 37
340018612E47 of 7421 of 37
34011874A26 of 744 of 37
34021871D26 of 742 of 37
34031875B46 of 7418 of 37
34041872E64 of 7431 of 37
34051873C20 of 740 of 37
34061876D50 of 7422 of 37
34071887E46 of 7419 of 37
34081889C58 of 7428 of 37
340918810D62 of 7430 of 37
34101888B32 of 7411 of 37