Hilde · Decision models · answer key v2

Grammar · topic 14

How verbs show who acts and who receives the action

74 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model100.0%95.1% to 100.0%74 of 74
Claude Fable 5.1high reasoning · 16 questions per request · Chat model100.0%95.1% to 100.0%74 of 74
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model98.6%92.7% to 99.8%73 of 74
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model98.6%92.7% to 99.8%73 of 74
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model98.6%92.7% to 99.8%73 of 74
GPT-6 Astralow reasoning · 16 questions per request · Chat model98.6%92.7% to 99.8%73 of 74
GPT-6 Astralow reasoning · 1 question per request · Chat model98.6%92.7% to 99.8%73 of 74
GPT-6.1 Sollow reasoning · 1 question per request · Chat model98.6%92.7% to 99.8%73 of 74
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model98.6%92.7% to 99.8%73 of 74
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model98.6%92.7% to 99.8%73 of 74
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model98.6%92.7% to 99.8%73 of 74
Grok 4.7high reasoning · 16 questions per request · Chat model97.3%90.7% to 99.3%72 of 74
Claude Opus 5.5high reasoning · 16 questions per request · Chat model95.9%88.7% to 98.6%71 of 74
Claude Opus 5.5low reasoning · 16 questions per request · Chat model95.9%88.7% to 98.6%71 of 74
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model94.6%86.9% to 97.9%70 of 74
Claude Opus 5.5low reasoning · 1 question per request · Chat model94.6%86.9% to 97.9%70 of 74
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model94.6%86.9% to 97.9%70 of 74
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model91.9%83.4% to 96.2%68 of 74
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model90.5%81.7% to 95.3%67 of 74
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model89.2%80.1% to 94.4%66 of 74
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model87.8%78.5% to 93.5%65 of 74
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model83.8%73.8% to 90.5%62 of 74
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model82.4%72.2% to 89.4%61 of 74
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model79.7%69.2% to 87.3%59 of 74
Liquid d1-ultra1 question per request · Decision model77.0%66.3% to 85.1%57 of 74
Gemma 4 31Breasoning off · 16 questions per request · Chat model77.0%66.3% to 85.1%57 of 74
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model75.7%64.8% to 84.0%56 of 74
GPT-6 Lunalow reasoning · 16 questions per request · Chat model74.3%63.3% to 82.9%55 of 74
Perplexity Decider 27B v1.11 question per request · Decision model71.6%60.5% to 80.6%53 of 74
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model71.6%60.5% to 80.6%53 of 74
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model71.6%60.5% to 80.6%53 of 74
GLM-5.3low reasoning · 16 questions per request · Chat model70.3%59.1% to 79.5%52 of 74
Fastino GLiDE v1.2local · 1 question per request · Decision model70.3%59.1% to 79.5%52 of 74
Mammoth-11 question per request · Decision model70.3%59.1% to 79.5%52 of 74
Fastino GLiDE1 question per request · Decision model68.9%57.7% to 78.3%51 of 74
vLLM-SR d31 question per request · Decision model67.6%56.3% to 77.1%50 of 74
Drex Max v11 question per request · Decision model67.6%56.3% to 77.1%50 of 74
SubQ Snap Preview1 question per request · Decision model67.6%56.3% to 77.1%50 of 74
Perplexity Decider 27B v1.2local · 1 question per request · Decision model66.2%54.9% to 76.0%49 of 74
Perplexity Decider 27B v11 question per request · Decision model66.2%54.9% to 76.0%49 of 74
Gemma 4 31Breasoning off · 1 question per request · Chat model64.9%53.5% to 74.8%48 of 74
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model64.9%53.5% to 74.8%48 of 74
Drex Max 1.11 question per request · Decision model60.8%49.4% to 71.1%45 of 74
Drex 2.01 question per request · Decision model60.8%49.4% to 71.1%45 of 74
SubQ Snap 1.11 question per request · Decision model59.5%48.1% to 69.9%44 of 74
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model59.5%48.1% to 69.9%44 of 74
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model59.5%48.1% to 69.9%44 of 74
Jev 1.13.01 question per request · Decision model59.5%48.1% to 69.9%44 of 74
Liquid d1.11 question per request · Decision model58.1%46.7% to 68.7%43 of 74
OneJev 27B1 question per request · Decision model58.1%46.7% to 68.7%43 of 74
Jev 1.14.01 question per request · Decision model58.1%46.7% to 68.7%43 of 74
Elephant-11 question per request · Decision model58.1%46.7% to 68.7%43 of 74
Celeris-1 Decision1 question per request · Decision model55.4%44.1% to 66.2%41 of 74
Liquid d11 question per request · Decision model54.1%42.8% to 64.9%40 of 74
Open-Jev 27B v1.11 question per request · Decision model54.1%42.8% to 64.9%40 of 74
Clef1 question per request · Decision model51.4%40.2% to 62.4%38 of 74
Cygnet1 question per request · Decision model51.4%40.2% to 62.4%38 of 74
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model45.9%35.1% to 57.2%34 of 74
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model44.6%33.8% to 55.9%33 of 74
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model44.6%33.8% to 55.9%33 of 74
Winnow-12B1 question per request · Decision model43.2%32.6% to 54.6%32 of 74
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model40.5%30.1% to 51.9%30 of 74
Perplexity Decider Flash v1.2local · 1 question per request · Decision model37.8%27.6% to 49.2%28 of 74
Clef-flash1 question per request · Decision model35.1%25.2% to 46.5%26 of 74
Qwen3-14Breasoning off · 16 questions per request · Chat model33.8%24.0% to 45.1%25 of 74
Liquid d1-2B (open weights)1 question per request · Decision model33.8%24.0% to 45.1%25 of 74
Erk-14Breasoning off · 16 questions per request · Chat model28.4%19.4% to 39.5%21 of 74
Metask-Jev 4B1 question per request · Decision model25.7%17.1% to 36.7%19 of 74
GLiNER2.5-multi-Decide1 question per request · Decision model24.3%16.0% to 35.2%18 of 74
meraGPT Decider 11 question per request · Decision model21.6%13.8% to 32.3%16 of 74
Strands Decider 2B1 question per request · Decision model20.3%12.7% to 30.8%15 of 74
GLiNER2.5 Multi (base)1 question per request · Chat model20.3%12.7% to 30.8%15 of 74
Laya1 question per request · Decision model16.2%9.5% to 26.2%12 of 74
Laya multilingual1 question per request · Decision model14.9%8.5% to 24.7%11 of 74

Its questions

IDPageNumberAnswer keySetups rightDecision models right
13302751A30 of 744 of 37
13312753E65 of 7430 of 37
13322754D34 of 7410 of 37
13332752C50 of 7421 of 37
13342755D59 of 7427 of 37
13352766B65 of 7430 of 37
13362769E65 of 7431 of 37
13372767A43 of 7414 of 37
133827610D39 of 7410 of 37
133927611C54 of 7423 of 37
13402768B65 of 7429 of 37
13552791B49 of 7418 of 37
13562795D39 of 7412 of 37
13572792A66 of 7433 of 37
13582796E67 of 7431 of 37
13592793D67 of 7431 of 37
13602797A48 of 7419 of 37
13612794D28 of 745 of 37
13682811C54 of 7423 of 37
13692814C28 of 744 of 37
13702812D43 of 7413 of 37
13712815B60 of 7427 of 37
13722813E38 of 749 of 37
13732816A49 of 7421 of 37
13802831C60 of 7428 of 37
13812834B30 of 744 of 37
13822832B58 of 7426 of 37
13832835A65 of 7430 of 37
13842833D50 of 7419 of 37
13852836E57 of 7424 of 37
13862847B18 of 741 of 37
138728410D40 of 749 of 37
138828411A58 of 7423 of 37
13892848B15 of 740 of 37
13902849E60 of 7430 of 37
139128412B67 of 7431 of 37
13922851A40 of 7412 of 37
13932854D46 of 7413 of 37
13942852E31 of 745 of 37
13952855B55 of 7423 of 37
13962853D57 of 7427 of 37
13972856C61 of 7428 of 37
13982867B57 of 7423 of 37
139928610C42 of 7415 of 37
14002868B50 of 7421 of 37
140128611E32 of 747 of 37
14022869D68 of 7434 of 37
140328612B59 of 7424 of 37
37882771C63 of 7429 of 37
37892775A34 of 748 of 37
37902776C32 of 744 of 37
37912772C43 of 7415 of 37
37922777C57 of 7423 of 37
37932773B41 of 7411 of 37
37942778E61 of 7429 of 37
37952774C55 of 7423 of 37
37962789E57 of 7424 of 37
379727812D33 of 745 of 37
379827810E52 of 7432 of 37
379927813A69 of 7435 of 37
380027811C51 of 7421 of 37
380127814A23 of 749 of 37
38022808B40 of 7412 of 37
380328011C55 of 7423 of 37
380428012E51 of 7427 of 37
38052809D34 of 745 of 37
380628010D50 of 7420 of 37
380728013D54 of 7426 of 37
38082827B47 of 7418 of 37
380928210E65 of 7430 of 37
381028211B51 of 7425 of 37
38112828E22 of 742 of 37
381228212C58 of 7426 of 37
38132829E61 of 7426 of 37