Hilde · Decision models · answer key v2

Grammar · topic 20

Problems in sentence wording

175 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model98.3%95.1% to 99.4%172 of 175
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model98.3%95.1% to 99.4%172 of 175
GPT-6 Astralow reasoning · 16 questions per request · Chat model98.3%95.1% to 99.4%172 of 175
GPT-6 Astralow reasoning · 1 question per request · Chat model98.3%95.1% to 99.4%172 of 175
GPT-6.1 Sollow reasoning · 1 question per request · Chat model98.3%95.1% to 99.4%172 of 175
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model97.7%94.3% to 99.1%171 of 175
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model96.6%92.7% to 98.4%169 of 175
Claude Fable 5.1high reasoning · 16 questions per request · Chat model96.6%92.7% to 98.4%169 of 175
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model96.0%92.0% to 98.0%168 of 175
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model96.0%92.0% to 98.0%168 of 175
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model95.4%91.2% to 97.7%167 of 175
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model94.9%90.5% to 97.3%166 of 175
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model94.9%90.5% to 97.3%166 of 175
Claude Opus 5.5high reasoning · 16 questions per request · Chat model93.7%89.1% to 96.5%164 of 175
Claude Opus 5.5low reasoning · 1 question per request · Chat model93.7%89.1% to 96.5%164 of 175
Grok 4.7high reasoning · 16 questions per request · Chat model92.6%87.7% to 95.6%162 of 175
Claude Opus 5.5low reasoning · 16 questions per request · Chat model91.4%86.3% to 94.7%160 of 175
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model89.7%84.3% to 93.4%157 of 175
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model88.0%82.4% to 92.0%154 of 175
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model85.7%79.8% to 90.1%150 of 175
Liquid d1-ultra1 question per request · Decision model81.7%75.3% to 86.7%143 of 175
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model80.6%74.1% to 85.8%141 of 175
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model79.4%72.8% to 84.8%139 of 175
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model79.4%72.8% to 84.8%139 of 175
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model77.1%70.4% to 82.7%135 of 175
GLM-5.3low reasoning · 16 questions per request · Chat model74.3%67.3% to 80.2%130 of 175
Fastino GLiDE v1.2local · 1 question per request · Decision model73.1%66.1% to 79.2%128 of 175
Fastino GLiDE1 question per request · Decision model73.1%66.1% to 79.2%128 of 175
GPT-6 Lunalow reasoning · 16 questions per request · Chat model72.6%65.5% to 78.6%127 of 175
Perplexity Decider 27B v1.2local · 1 question per request · Decision model72.6%65.5% to 78.6%127 of 175
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model72.6%65.5% to 78.6%127 of 175
SubQ Snap Preview1 question per request · Decision model72.6%65.5% to 78.6%127 of 175
Perplexity Decider 27B v1.11 question per request · Decision model72.0%64.9% to 78.1%126 of 175
vLLM-SR d31 question per request · Decision model71.4%64.3% to 77.6%125 of 175
Mammoth-11 question per request · Decision model71.4%64.3% to 77.6%125 of 175
Gemma 4 31Breasoning off · 1 question per request · Chat model71.4%64.3% to 77.6%125 of 175
Drex Max v11 question per request · Decision model70.9%63.7% to 77.1%124 of 175
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model70.3%63.1% to 76.6%123 of 175
Drex Max 1.11 question per request · Decision model70.3%63.1% to 76.6%123 of 175
Gemma 4 31Breasoning off · 16 questions per request · Chat model69.7%62.5% to 76.0%122 of 175
SubQ Snap 1.11 question per request · Decision model69.1%61.9% to 75.5%121 of 175
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model68.0%60.8% to 74.5%119 of 175
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model66.3%59.0% to 72.9%116 of 175
Celeris-1 Decision1 question per request · Decision model65.7%58.4% to 72.3%115 of 175
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model64.6%57.2% to 71.3%113 of 175
Liquid d1.11 question per request · Decision model64.0%56.7% to 70.7%112 of 175
Liquid d11 question per request · Decision model62.9%55.5% to 69.7%110 of 175
Perplexity Decider 27B v11 question per request · Decision model62.3%54.9% to 69.1%109 of 175
OneJev 27B1 question per request · Decision model62.3%54.9% to 69.1%109 of 175
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model62.3%54.9% to 69.1%109 of 175
Open-Jev 27B v1.11 question per request · Decision model62.3%54.9% to 69.1%109 of 175
Jev 1.14.01 question per request · Decision model61.1%53.8% to 68.1%107 of 175
Jev 1.13.01 question per request · Decision model60.6%53.2% to 67.5%106 of 175
Clef1 question per request · Decision model60.6%53.2% to 67.5%106 of 175
Winnow-12B1 question per request · Decision model60.0%52.6% to 67.0%105 of 175
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model60.0%52.6% to 67.0%105 of 175
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model59.4%52.0% to 66.4%104 of 175
Elephant-11 question per request · Decision model59.4%52.0% to 66.4%104 of 175
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model58.3%50.9% to 65.3%102 of 175
Cygnet1 question per request · Decision model57.7%50.3% to 64.8%101 of 175
Drex 2.01 question per request · Decision model56.6%49.2% to 63.7%99 of 175
Erk-14Breasoning off · 16 questions per request · Chat model55.4%48.0% to 62.6%97 of 175
Perplexity Decider Flash v1.2local · 1 question per request · Decision model52.6%45.2% to 59.8%92 of 175
Qwen3-14Breasoning off · 16 questions per request · Chat model52.0%44.6% to 59.3%91 of 175
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model47.4%40.2% to 54.8%83 of 175
Clef-flash1 question per request · Decision model46.3%39.1% to 53.7%81 of 175
Metask-Jev 4B1 question per request · Decision model40.6%33.6% to 48.0%71 of 175
meraGPT Decider 11 question per request · Decision model33.1%26.6% to 40.4%58 of 175
Liquid d1-2B (open weights)1 question per request · Decision model32.0%25.5% to 39.2%56 of 175
Strands Decider 2B1 question per request · Decision model29.1%22.9% to 36.3%51 of 175
GLiNER2.5 Multi (base)1 question per request · Chat model25.7%19.8% to 32.7%45 of 175
Laya multilingual1 question per request · Decision model21.7%16.2% to 28.4%38 of 175
GLiNER2.5-multi-Decide1 question per request · Decision model20.6%15.2% to 27.2%36 of 175
Laya1 question per request · Decision model19.4%14.2% to 25.9%34 of 175

Its questions

IDPageNumberAnswer keySetups rightDecision models right
28463951A56 of 7422 of 37
28473954D42 of 7415 of 37
28483955A67 of 7432 of 37
28493952D66 of 7429 of 37
28503956A35 of 7412 of 37
28513953A50 of 7420 of 37
28523967C38 of 7412 of 37
285339610C38 of 746 of 37
28543968D56 of 7423 of 37
285539611C21 of 7413 of 37
28563969A64 of 7429 of 37
285739612D35 of 7413 of 37
28583971A42 of 7416 of 37
28593974A68 of 7431 of 37
28603975E39 of 7413 of 37
28613972E54 of 7425 of 37
28623973A25 of 747 of 37
28633976A55 of 7423 of 37
28764007C50 of 7421 of 37
287740010E66 of 7436 of 37
287840011C41 of 7414 of 37
28794008A46 of 7417 of 37
28804009A38 of 7412 of 37
288140012D61 of 7426 of 37
28824011E58 of 7422 of 37
28834014B39 of 7417 of 37
28844015A22 of 743 of 37
28854012C66 of 7430 of 37
28864016D49 of 7419 of 37
28874013C17 of 742 of 37
28884027D52 of 7422 of 37
288940210D34 of 7412 of 37
28904028A67 of 7432 of 37
289140211E40 of 7412 of 37
28924029A17 of 741 of 37
289340212A39 of 7414 of 37
28944031B65 of 7430 of 37
28954034E58 of 7428 of 37
28964035E50 of 7423 of 37
28974032C39 of 7414 of 37
28984033D53 of 7421 of 37
28994036D46 of 7412 of 37
29004047C52 of 7420 of 37
290140410A56 of 7420 of 37
290240411B61 of 7427 of 37
29034048D62 of 7427 of 37
29044049B59 of 7426 of 37
290540412E60 of 7427 of 37
29124067E39 of 7416 of 37
291340610C29 of 746 of 37
29144068E64 of 7430 of 37
291540611D42 of 7410 of 37
29164069C51 of 7421 of 37
291740612E53 of 7426 of 37
29184071B55 of 7421 of 37
29194074B67 of 7431 of 37
29204072D56 of 7423 of 37
29214075C34 of 748 of 37
29224073C62 of 7430 of 37
29234076B41 of 7413 of 37
29244087A67 of 7432 of 37
292540810C34 of 748 of 37
292640811B20 of 744 of 37
29274088E63 of 7430 of 37
29284089B66 of 7430 of 37
292940812B15 of 742 of 37
29304091B57 of 7423 of 37
29314094D57 of 7424 of 37
29324092D27 of 745 of 37
29334095C24 of 743 of 37
29344096E42 of 7417 of 37
29354093A49 of 7417 of 37
29364107D31 of 747 of 37
293741010B63 of 7430 of 37
293841011C53 of 7420 of 37
29394108A69 of 7432 of 37
294041012E71 of 7434 of 37
29414109B59 of 7429 of 37
29504159D57 of 7424 of 37
295141511B68 of 7432 of 37
295241510A28 of 745 of 37
295341512E71 of 7435 of 37
297842239B67 of 7431 of 37
297942237D72 of 7435 of 37
298042240C65 of 7430 of 37
298142238A72 of 7435 of 37
29864245E71 of 7435 of 37
29874247A69 of 7433 of 37
29884246B69 of 7433 of 37
29894248E60 of 7432 of 37
299042511C68 of 7432 of 37
29914259A70 of 7433 of 37
299242510D36 of 7410 of 37
299342512C69 of 7433 of 37
299842717B67 of 7431 of 37
299942719E69 of 7434 of 37
300042718D53 of 7422 of 37
300142720C31 of 747 of 37
300242821D56 of 7426 of 37
300342823B71 of 7435 of 37
300442822D68 of 7432 of 37
300542824A34 of 747 of 37
300642925E69 of 7433 of 37
300742927C68 of 7431 of 37
300842928A34 of 747 of 37
300942926E59 of 7428 of 37
301043029C69 of 7433 of 37
301143031C69 of 7433 of 37
301243030C35 of 746 of 37
301343032E31 of 748 of 37
301843239E68 of 7434 of 37
301943237B53 of 7422 of 37
302043238A70 of 7433 of 37
302143240C68 of 7432 of 37
41703987D41 of 7416 of 37
417139810A69 of 7432 of 37
41723988A35 of 749 of 37
417339811E54 of 7424 of 37
41743989A68 of 7432 of 37
417539812A64 of 7429 of 37
41763991C53 of 7418 of 37
41773994D62 of 7428 of 37
41783995A56 of 7422 of 37
41793992C24 of 743 of 37
41803996B27 of 748 of 37
41813993B51 of 7423 of 37
41824051C40 of 747 of 37
41834054E28 of 747 of 37
41844052C44 of 7413 of 37
41854055C58 of 7424 of 37
41864053A51 of 7421 of 37
41874056C26 of 746 of 37
41884131B68 of 7432 of 37
41894133A34 of 746 of 37
41904134C45 of 7420 of 37
41914132D67 of 7430 of 37
41924145E69 of 7433 of 37
41934147E66 of 7430 of 37
41944146C70 of 7434 of 37
41954148D55 of 7420 of 37
419641613C59 of 7430 of 37
419741614D70 of 7434 of 37
419841615E51 of 7420 of 37
419941716B26 of 741 of 37
420041719E71 of 7435 of 37
420141717E55 of 7422 of 37
420241720B33 of 748 of 37
420341718B54 of 7421 of 37
420441821E69 of 7433 of 37
420541823D36 of 7411 of 37
420741824A27 of 746 of 37
420841925E70 of 7434 of 37
420941927A68 of 7432 of 37
421041926B48 of 7418 of 37
421141928D69 of 7433 of 37
421242029C41 of 7415 of 37
421342031C38 of 7412 of 37
421442030B38 of 7414 of 37
421542032E27 of 747 of 37
421642133C67 of 7431 of 37
421742135A11 of 742 of 37
421842134B39 of 746 of 37
421942136E71 of 7435 of 37
42204231B68 of 7432 of 37
42214233B55 of 7425 of 37
42224232D68 of 7433 of 37
42234234E64 of 7430 of 37
422442613A69 of 7432 of 37
422542615B31 of 747 of 37
422642614B6 of 742 of 37
422742616B4 of 742 of 37
422843133E69 of 7433 of 37
422943135B69 of 7433 of 37
423043134C52 of 7421 of 37
423143136D67 of 7431 of 37