Hilde · Decision models · answer key v2

Grammar · topic 17

Sound patterns and changes

77 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model100.0%95.2% to 100.0%77 of 77
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model98.7%93.0% to 99.8%76 of 77
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model98.7%93.0% to 99.8%76 of 77
GPT-6 Astralow reasoning · 1 question per request · Chat model97.4%91.0% to 99.3%75 of 77
GPT-6 Astralow reasoning · 16 questions per request · Chat model96.1%89.2% to 98.7%74 of 77
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model96.1%89.2% to 98.7%74 of 77
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model94.8%87.4% to 98.0%73 of 77
Claude Fable 5.1high reasoning · 16 questions per request · Chat model94.8%87.4% to 98.0%73 of 77
GPT-6.1 Sollow reasoning · 1 question per request · Chat model93.5%85.7% to 97.2%72 of 77
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model93.5%85.7% to 97.2%72 of 77
Claude Opus 5.5high reasoning · 16 questions per request · Chat model90.9%82.4% to 95.5%70 of 77
Grok 4.7high reasoning · 16 questions per request · Chat model89.6%80.8% to 94.6%69 of 77
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model87.0%77.7% to 92.8%67 of 77
Claude Opus 5.5low reasoning · 1 question per request · Chat model85.7%76.2% to 91.8%66 of 77
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model85.7%76.2% to 91.8%66 of 77
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model85.7%76.2% to 91.8%66 of 77
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model79.2%68.9% to 86.8%61 of 77
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model74.0%63.3% to 82.5%57 of 77
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model71.4%60.5% to 80.3%55 of 77
Claude Opus 5.5low reasoning · 16 questions per request · Chat model70.1%59.2% to 79.2%54 of 77
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model51.9%41.0% to 62.7%40 of 77
Gemma 4 31Breasoning off · 16 questions per request · Chat model49.4%38.5% to 60.3%38 of 77
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model42.9%32.4% to 54.0%33 of 77
Gemma 4 31Breasoning off · 1 question per request · Chat model41.6%31.2% to 52.7%32 of 77
Celeris-1 Decision1 question per request · Decision model41.6%31.2% to 52.7%32 of 77
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model40.3%30.0% to 51.4%31 of 77
Liquid d1-ultra1 question per request · Decision model39.0%28.8% to 50.1%30 of 77
Mammoth-11 question per request · Decision model39.0%28.8% to 50.1%30 of 77
GPT-6 Lunalow reasoning · 16 questions per request · Chat model37.7%27.7% to 48.8%29 of 77
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model37.7%27.7% to 48.8%29 of 77
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model36.4%26.5% to 47.5%28 of 77
Fastino GLiDE v1.2local · 1 question per request · Decision model35.1%25.3% to 46.2%27 of 77
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model35.1%25.3% to 46.2%27 of 77
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model35.1%25.3% to 46.2%27 of 77
GLM-5.3low reasoning · 16 questions per request · Chat model33.8%24.2% to 44.9%26 of 77
Drex Max 1.11 question per request · Decision model33.8%24.2% to 44.9%26 of 77
Jev 1.14.01 question per request · Decision model33.8%24.2% to 44.9%26 of 77
Winnow-12B1 question per request · Decision model33.8%24.2% to 44.9%26 of 77
Metask-Jev 4B1 question per request · Decision model33.8%24.2% to 44.9%26 of 77
Fastino GLiDE1 question per request · Decision model32.5%23.1% to 43.5%25 of 77
Drex Max v11 question per request · Decision model32.5%23.1% to 43.5%25 of 77
Liquid d1.11 question per request · Decision model32.5%23.1% to 43.5%25 of 77
OneJev 27B1 question per request · Decision model32.5%23.1% to 43.5%25 of 77
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model32.5%23.1% to 43.5%25 of 77
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model31.2%21.9% to 42.2%24 of 77
vLLM-SR d31 question per request · Decision model31.2%21.9% to 42.2%24 of 77
Perplexity Decider 27B v11 question per request · Decision model31.2%21.9% to 42.2%24 of 77
Clef1 question per request · Decision model31.2%21.9% to 42.2%24 of 77
Cygnet1 question per request · Decision model31.2%21.9% to 42.2%24 of 77
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model31.2%21.9% to 42.2%24 of 77
Qwen3-14Breasoning off · 16 questions per request · Chat model31.2%21.9% to 42.2%24 of 77
Liquid d11 question per request · Decision model28.6%19.7% to 39.5%22 of 77
Clef-flash1 question per request · Decision model28.6%19.7% to 39.5%22 of 77
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model27.3%18.6% to 38.1%21 of 77
Perplexity Decider 27B v1.2local · 1 question per request · Decision model27.3%18.6% to 38.1%21 of 77
Jev 1.13.01 question per request · Decision model27.3%18.6% to 38.1%21 of 77
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model26.0%17.5% to 36.7%20 of 77
SubQ Snap Preview1 question per request · Decision model26.0%17.5% to 36.7%20 of 77
Drex 2.01 question per request · Decision model26.0%17.5% to 36.7%20 of 77
Erk-14Breasoning off · 16 questions per request · Chat model26.0%17.5% to 36.7%20 of 77
Perplexity Decider 27B v1.11 question per request · Decision model24.7%16.4% to 35.4%19 of 77
Liquid d1-2B (open weights)1 question per request · Decision model24.7%16.4% to 35.4%19 of 77
Open-Jev 27B v1.11 question per request · Decision model23.4%15.3% to 34.0%18 of 77
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model23.4%15.3% to 34.0%18 of 77
meraGPT Decider 11 question per request · Decision model22.1%14.3% to 32.5%17 of 77
Laya1 question per request · Decision model22.1%14.3% to 32.5%17 of 77
Elephant-11 question per request · Decision model20.8%13.2% to 31.1%16 of 77
SubQ Snap 1.11 question per request · Decision model19.5%12.2% to 29.7%15 of 77
Strands Decider 2B1 question per request · Decision model19.5%12.2% to 29.7%15 of 77
GLiNER2.5-multi-Decide1 question per request · Decision model19.5%12.2% to 29.7%15 of 77
Laya multilingual1 question per request · Decision model16.9%10.1% to 26.8%13 of 77
GLiNER2.5 Multi (base)1 question per request · Chat model16.9%10.1% to 26.8%13 of 77
Perplexity Decider Flash v1.2local · 1 question per request · Decision model15.6%9.1% to 25.3%12 of 77
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model0.0%0.0% to 4.8%0 of 77

Its questions

IDPageNumberAnswer keySetups rightDecision models right
25303411E17 of 741 of 37
25313414B27 of 747 of 37
25323412C24 of 742 of 37
25333415C41 of 7416 of 37
25343416A30 of 745 of 37
25353413D44 of 7418 of 37
39513391A29 of 747 of 37
39523394B28 of 747 of 37
39533395E41 of 7417 of 37
39543392C28 of 746 of 37
39553396C22 of 742 of 37
39563393D58 of 7429 of 37
39573407C40 of 7414 of 37
395834010A26 of 745 of 37
39593408E24 of 744 of 37
396034011B23 of 742 of 37
39613409D25 of 743 of 37
396234012E41 of 7413 of 37
39633427B49 of 7423 of 37
396434210D50 of 7420 of 37
396534211B31 of 745 of 37
39663428C57 of 7427 of 37
396734212E20 of 742 of 37
396834213C49 of 7422 of 37
39693429E37 of 7411 of 37
39703431E36 of 749 of 37
39713434E64 of 7430 of 37
39723435B47 of 7418 of 37
39733432D13 of 742 of 37
39743436B45 of 7421 of 37
39753433C34 of 7411 of 37
39763437A21 of 746 of 37
39773448E40 of 7416 of 37
397834411A33 of 7410 of 37
397934412D34 of 7410 of 37
39803449C34 of 748 of 37
398134413E19 of 745 of 37
398234410B24 of 740 of 37
398334414C15 of 747 of 37
39843451E45 of 7417 of 37
39853455E21 of 748 of 37
39863456D46 of 7423 of 37
39873452D40 of 7416 of 37
39883457D25 of 743 of 37
39893453A20 of 744 of 37
39903458A19 of 743 of 37
39913454C54 of 7424 of 37
39923469E22 of 743 of 37
399334612C32 of 748 of 37
399434613C44 of 7420 of 37
399534610B19 of 743 of 37
399634614D32 of 747 of 37
399734611A38 of 7411 of 37
39983471C37 of 7411 of 37
39993474A21 of 745 of 37
40003475D35 of 7415 of 37
40013472B18 of 740 of 37
40023476D44 of 7417 of 37
40033473E48 of 7417 of 37
40043487A43 of 7416 of 37
400534810E48 of 7424 of 37
40063488B46 of 7416 of 37
400734811C20 of 740 of 37
400834812C27 of 746 of 37
40093489D36 of 746 of 37
40103491A24 of 743 of 37
40113494E51 of 7420 of 37
40123492D35 of 7410 of 37
40133495B38 of 7410 of 37
40143493C45 of 7419 of 37
40153496D29 of 745 of 37
40163507D27 of 743 of 37
401735010B39 of 747 of 37
401835011A40 of 749 of 37
40193508E19 of 744 of 37
40203509C32 of 748 of 37
402135012C44 of 7418 of 37