Hilde · Decision models · answer key v2

Grammar · topic 12

How words are built

69 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model97.1%90.0% to 99.2%67 of 69
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model95.7%88.0% to 98.5%66 of 69
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model95.7%88.0% to 98.5%66 of 69
GPT-6 Astralow reasoning · 1 question per request · Chat model95.7%88.0% to 98.5%66 of 69
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model95.7%88.0% to 98.5%66 of 69
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model95.7%88.0% to 98.5%66 of 69
Claude Fable 5.1high reasoning · 16 questions per request · Chat model95.7%88.0% to 98.5%66 of 69
GPT-6 Astralow reasoning · 16 questions per request · Chat model94.2%86.0% to 97.7%65 of 69
GPT-6.1 Sollow reasoning · 1 question per request · Chat model94.2%86.0% to 97.7%65 of 69
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model94.2%86.0% to 97.7%65 of 69
Claude Opus 5.5high reasoning · 16 questions per request · Chat model94.2%86.0% to 97.7%65 of 69
Claude Opus 5.5low reasoning · 1 question per request · Chat model92.8%84.1% to 96.9%64 of 69
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model92.8%84.1% to 96.9%64 of 69
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model91.3%82.3% to 96.0%63 of 69
Grok 4.7high reasoning · 16 questions per request · Chat model88.4%78.8% to 94.0%61 of 69
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model88.4%78.8% to 94.0%61 of 69
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model88.4%78.8% to 94.0%61 of 69
Claude Opus 5.5low reasoning · 16 questions per request · Chat model84.1%73.7% to 90.9%58 of 69
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model84.1%73.7% to 90.9%58 of 69
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model78.3%67.2% to 86.4%54 of 69
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model76.8%65.6% to 85.2%53 of 69
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model73.9%62.5% to 82.8%51 of 69
GLM-5.3low reasoning · 16 questions per request · Chat model55.1%43.4% to 66.2%38 of 69
Gemma 4 31Breasoning off · 16 questions per request · Chat model55.1%43.4% to 66.2%38 of 69
GPT-6 Lunalow reasoning · 16 questions per request · Chat model52.2%40.6% to 63.5%36 of 69
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model50.7%39.2% to 62.2%35 of 69
Mammoth-11 question per request · Decision model49.3%37.8% to 60.8%34 of 69
Gemma 4 31Breasoning off · 1 question per request · Chat model47.8%36.5% to 59.4%33 of 69
Liquid d1-ultra1 question per request · Decision model46.4%35.1% to 58.0%32 of 69
Fastino GLiDE1 question per request · Decision model44.9%33.8% to 56.6%31 of 69
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model44.9%33.8% to 56.6%31 of 69
Drex Max 1.11 question per request · Decision model43.5%32.4% to 55.2%30 of 69
vLLM-SR d31 question per request · Decision model42.0%31.1% to 53.8%29 of 69
Fastino GLiDE v1.2local · 1 question per request · Decision model40.6%29.8% to 52.4%28 of 69
Drex Max v11 question per request · Decision model40.6%29.8% to 52.4%28 of 69
Perplexity Decider 27B v1.2local · 1 question per request · Decision model40.6%29.8% to 52.4%28 of 69
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model40.6%29.8% to 52.4%28 of 69
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model40.6%29.8% to 52.4%28 of 69
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model39.1%28.5% to 50.9%27 of 69
Perplexity Decider 27B v1.11 question per request · Decision model39.1%28.5% to 50.9%27 of 69
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model37.7%27.2% to 49.5%26 of 69
Celeris-1 Decision1 question per request · Decision model37.7%27.2% to 49.5%26 of 69
Winnow-12B1 question per request · Decision model37.7%27.2% to 49.5%26 of 69
SubQ Snap Preview1 question per request · Decision model36.2%25.9% to 48.0%25 of 69
Clef1 question per request · Decision model36.2%25.9% to 48.0%25 of 69
Drex 2.01 question per request · Decision model36.2%25.9% to 48.0%25 of 69
Cygnet1 question per request · Decision model36.2%25.9% to 48.0%25 of 69
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model36.2%25.9% to 48.0%25 of 69
Qwen3-14Breasoning off · 16 questions per request · Chat model36.2%25.9% to 48.0%25 of 69
SubQ Snap 1.11 question per request · Decision model34.8%24.6% to 46.6%24 of 69
Jev 1.14.01 question per request · Decision model34.8%24.6% to 46.6%24 of 69
Jev 1.13.01 question per request · Decision model34.8%24.6% to 46.6%24 of 69
Liquid d11 question per request · Decision model34.8%24.6% to 46.6%24 of 69
Liquid d1.11 question per request · Decision model33.3%23.4% to 45.1%23 of 69
OneJev 27B1 question per request · Decision model33.3%23.4% to 45.1%23 of 69
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model31.9%22.1% to 43.6%22 of 69
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model30.4%20.8% to 42.1%21 of 69
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model30.4%20.8% to 42.1%21 of 69
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model30.4%20.8% to 42.1%21 of 69
Metask-Jev 4B1 question per request · Decision model30.4%20.8% to 42.1%21 of 69
Perplexity Decider 27B v11 question per request · Decision model29.0%19.6% to 40.6%20 of 69
Elephant-11 question per request · Decision model29.0%19.6% to 40.6%20 of 69
Open-Jev 27B v1.11 question per request · Decision model27.5%18.4% to 39.0%19 of 69
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model27.5%18.4% to 39.0%19 of 69
Perplexity Decider Flash v1.2local · 1 question per request · Decision model26.1%17.2% to 37.5%18 of 69
Clef-flash1 question per request · Decision model24.6%16.0% to 36.0%17 of 69
Erk-14Breasoning off · 16 questions per request · Chat model23.2%14.8% to 34.4%16 of 69
Liquid d1-2B (open weights)1 question per request · Decision model21.7%13.6% to 32.8%15 of 69
meraGPT Decider 11 question per request · Decision model20.3%12.5% to 31.2%14 of 69
GLiNER2.5-multi-Decide1 question per request · Decision model20.3%12.5% to 31.2%14 of 69
Laya multilingual1 question per request · Decision model15.9%9.1% to 26.3%11 of 69
Strands Decider 2B1 question per request · Decision model14.5%8.1% to 24.7%10 of 69
Laya1 question per request · Decision model14.5%8.1% to 24.7%10 of 69
GLiNER2.5 Multi (base)1 question per request · Chat model13.0%7.0% to 23.0%9 of 69

Its questions

IDPageNumberAnswer keySetups rightDecision models right
114524410E41 of 7411 of 37
11462447D18 of 740 of 37
11472448E32 of 7411 of 37
114824411C56 of 7423 of 37
114924412C36 of 7418 of 37
11502449E54 of 7421 of 37
36342391E34 of 745 of 37
36352394D29 of 7419 of 37
36362392B47 of 7417 of 37
36372395B33 of 747 of 37
36382396A25 of 742 of 37
36392393D24 of 744 of 37
364124010C45 of 7415 of 37
36422408B46 of 7416 of 37
364324011A50 of 7421 of 37
36442409C52 of 7419 of 37
364524012D39 of 7411 of 37
36462411B45 of 7415 of 37
36472414B59 of 7427 of 37
36482412C56 of 7428 of 37
36492415B23 of 740 of 37
36502413A33 of 746 of 37
36512416B69 of 7433 of 37
36522427A38 of 747 of 37
365324210C36 of 748 of 37
36542428E38 of 7416 of 37
365524211D63 of 7429 of 37
36562429C25 of 744 of 37
365724212D23 of 746 of 37
36582431B29 of 742 of 37
36592434E59 of 7429 of 37
36602435B35 of 7410 of 37
36612432A24 of 742 of 37
36622436C25 of 743 of 37
36632433A25 of 742 of 37
36642451B28 of 742 of 37
36652454E39 of 7413 of 37
36662452E49 of 7420 of 37
36672455D46 of 7417 of 37
36682453C38 of 7415 of 37
36692456C67 of 7432 of 37
367024610D36 of 7411 of 37
36712467E39 of 7412 of 37
36722468B21 of 740 of 37
367324611D35 of 7411 of 37
36742469B45 of 7415 of 37
367524612D20 of 7411 of 37
36762471D29 of 745 of 37
36772474B39 of 7414 of 37
36782475D46 of 7420 of 37
36792472D32 of 743 of 37
36802473C48 of 7416 of 37
36812476B51 of 7425 of 37
36822487D55 of 7425 of 37
368324810C28 of 743 of 37
368424811E40 of 7411 of 37
36852488C19 of 743 of 37
368624812C38 of 7414 of 37
36872489D31 of 745 of 37
36882491C27 of 7414 of 37
36892494B34 of 748 of 37
36902492E53 of 7423 of 37
36912495D27 of 742 of 37
36922493D21 of 740 of 37
36932506C47 of 7417 of 37
36942508E18 of 742 of 37
36952509A18 of 741 of 37
36962507E28 of 745 of 37
369725010B62 of 7429 of 37