Hilde · Decision models · answer key v2

Grammar · topic 10

Verb forms used like nouns or modifiers

72 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Grok 4.7high reasoning · 16 questions per request · Chat model100.0%94.9% to 100.0%72 of 72
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
GPT-6 Astralow reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model98.6%92.5% to 99.8%71 of 72
GPT-6.1 Sollow reasoning · 1 question per request · Chat model98.6%92.5% to 99.8%71 of 72
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
Claude Fable 5.1high reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
Claude Opus 5.5high reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model98.6%92.5% to 99.8%71 of 72
Claude Opus 5.5low reasoning · 1 question per request · Chat model98.6%92.5% to 99.8%71 of 72
GPT-6 Astralow reasoning · 1 question per request · Chat model97.2%90.4% to 99.2%70 of 72
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model97.2%90.4% to 99.2%70 of 72
Claude Opus 5.5low reasoning · 16 questions per request · Chat model97.2%90.4% to 99.2%70 of 72
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model95.8%88.5% to 98.6%69 of 72
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model94.4%86.6% to 97.8%68 of 72
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model94.4%86.6% to 97.8%68 of 72
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model93.1%84.8% to 97.0%67 of 72
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model93.1%84.8% to 97.0%67 of 72
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model90.3%81.3% to 95.2%65 of 72
GLM-5.3low reasoning · 16 questions per request · Chat model90.3%81.3% to 95.2%65 of 72
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model84.7%74.7% to 91.2%61 of 72
Gemma 4 31Breasoning off · 16 questions per request · Chat model79.2%68.4% to 86.9%57 of 72
Liquid d1-ultra1 question per request · Decision model76.4%65.4% to 84.7%55 of 72
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model70.8%59.5% to 80.1%51 of 72
vLLM-SR d31 question per request · Decision model70.8%59.5% to 80.1%51 of 72
Fastino GLiDE1 question per request · Decision model69.4%58.0% to 78.9%50 of 72
Drex Max v11 question per request · Decision model69.4%58.0% to 78.9%50 of 72
GPT-6 Lunalow reasoning · 16 questions per request · Chat model68.1%56.6% to 77.7%49 of 72
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model68.1%56.6% to 77.7%49 of 72
Drex Max 1.11 question per request · Decision model68.1%56.6% to 77.7%49 of 72
SubQ Snap Preview1 question per request · Decision model68.1%56.6% to 77.7%49 of 72
Fastino GLiDE v1.2local · 1 question per request · Decision model66.7%55.2% to 76.5%48 of 72
Perplexity Decider 27B v1.2local · 1 question per request · Decision model65.3%53.8% to 75.2%47 of 72
Perplexity Decider 27B v1.11 question per request · Decision model65.3%53.8% to 75.2%47 of 72
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model63.9%52.4% to 74.0%46 of 72
SubQ Snap 1.11 question per request · Decision model63.9%52.4% to 74.0%46 of 72
Liquid d1.11 question per request · Decision model63.9%52.4% to 74.0%46 of 72
Gemma 4 31Breasoning off · 1 question per request · Chat model62.5%51.0% to 72.8%45 of 72
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model62.5%51.0% to 72.8%45 of 72
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model61.1%49.6% to 71.5%44 of 72
Mammoth-11 question per request · Decision model58.3%46.8% to 69.0%42 of 72
Jev 1.13.01 question per request · Decision model56.9%45.4% to 67.7%41 of 72
Liquid d11 question per request · Decision model55.6%44.1% to 66.5%40 of 72
Jev 1.14.01 question per request · Decision model52.8%41.4% to 63.9%38 of 72
Perplexity Decider Flash v1.2local · 1 question per request · Decision model52.8%41.4% to 63.9%38 of 72
OneJev 27B1 question per request · Decision model51.4%40.1% to 62.6%37 of 72
Cygnet1 question per request · Decision model51.4%40.1% to 62.6%37 of 72
Celeris-1 Decision1 question per request · Decision model50.0%38.7% to 61.3%36 of 72
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model48.6%37.4% to 59.9%35 of 72
Drex 2.01 question per request · Decision model48.6%37.4% to 59.9%35 of 72
Perplexity Decider 27B v11 question per request · Decision model45.8%34.8% to 57.3%33 of 72
Open-Jev 27B v1.11 question per request · Decision model45.8%34.8% to 57.3%33 of 72
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model43.1%32.3% to 54.6%31 of 72
Winnow-12B1 question per request · Decision model43.1%32.3% to 54.6%31 of 72
Metask-Jev 4B1 question per request · Decision model43.1%32.3% to 54.6%31 of 72
Clef1 question per request · Decision model41.7%31.0% to 53.2%30 of 72
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model38.9%28.5% to 50.4%28 of 72
Elephant-11 question per request · Decision model38.9%28.5% to 50.4%28 of 72
Erk-14Breasoning off · 16 questions per request · Chat model38.9%28.5% to 50.4%28 of 72
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model37.5%27.2% to 49.0%27 of 72
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model37.5%27.2% to 49.0%27 of 72
Clef-flash1 question per request · Decision model34.7%24.8% to 46.2%25 of 72
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model33.3%23.5% to 44.8%24 of 72
Qwen3-14Breasoning off · 16 questions per request · Chat model33.3%23.5% to 44.8%24 of 72
Strands Decider 2B1 question per request · Decision model23.6%15.3% to 34.6%17 of 72
Liquid d1-2B (open weights)1 question per request · Decision model20.8%13.1% to 31.6%15 of 72
meraGPT Decider 11 question per request · Decision model19.4%12.0% to 30.0%14 of 72
Laya multilingual1 question per request · Decision model16.7%9.8% to 26.9%12 of 72
GLiNER2.5 Multi (base)1 question per request · Chat model15.3%8.8% to 25.3%11 of 72
Laya1 question per request · Decision model13.9%7.7% to 23.7%10 of 72
GLiNER2.5-multi-Decide1 question per request · Decision model12.5%6.7% to 22.1%9 of 72

Its questions

IDPageNumberAnswer keySetups rightDecision models right
10242251A52 of 7423 of 37
10252254E48 of 7419 of 37
10262255E30 of 746 of 37
10272252D32 of 746 of 37
10282253D31 of 746 of 37
10292256C58 of 7423 of 37
34892151D49 of 7422 of 37
34902153E59 of 7427 of 37
34912152B57 of 7426 of 37
34922154B45 of 7415 of 37
34932165B27 of 742 of 37
34942168C41 of 7411 of 37
34952169D63 of 7429 of 37
34962166C41 of 7412 of 37
349721610C45 of 7418 of 37
34982167D42 of 7415 of 37
34992171B64 of 7429 of 37
35002174C67 of 7431 of 37
35012172E36 of 747 of 37
35022175D57 of 7425 of 37
35032173B58 of 7424 of 37
35042176B28 of 744 of 37
35052187C55 of 7422 of 37
350621810C67 of 7432 of 37
35072188E21 of 740 of 37
350821811E63 of 7432 of 37
35092189C64 of 7429 of 37
351021812C35 of 747 of 37
35112191C45 of 7414 of 37
35122194B67 of 7431 of 37
35132195E36 of 7413 of 37
35142192B24 of 741 of 37
35152196D64 of 7429 of 37
35162193C48 of 7418 of 37
35172197C60 of 7427 of 37
35182208A42 of 7413 of 37
351922011A51 of 7419 of 37
352022012B33 of 744 of 37
35212209C59 of 7427 of 37
352222013E49 of 7419 of 37
352322010E44 of 7422 of 37
352422014B51 of 7419 of 37
35252211C45 of 7414 of 37
35262214D67 of 7431 of 37
35272215C44 of 7416 of 37
35282212C36 of 745 of 37
35292216C66 of 7431 of 37
35302213B15 of 744 of 37
35312227E68 of 7433 of 37
353222210E55 of 7428 of 37
35332228D46 of 7413 of 37
353422211A65 of 7430 of 37
35352229A53 of 7424 of 37
353622212E61 of 7429 of 37
35372231E67 of 7432 of 37
35382234D30 of 746 of 37
35392235A63 of 7428 of 37
35402232B67 of 7432 of 37
35412233E39 of 7410 of 37
35422236C31 of 746 of 37
35432247E44 of 7416 of 37
354422410B26 of 741 of 37
35452248B23 of 743 of 37
354622411A38 of 7410 of 37
354722412C55 of 7430 of 37
35482249B63 of 7428 of 37
35492267E52 of 7423 of 37
355022610B32 of 748 of 37
355122611B43 of 7411 of 37
35522268D59 of 7427 of 37
35532269A59 of 7425 of 37
355422612D35 of 747 of 37