Hilde · Decision models · answer key v2

Grammar · topic 8

Groups of words built around a noun

65 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model100.0%94.4% to 100.0%65 of 65
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model98.5%91.8% to 99.7%64 of 65
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model98.5%91.8% to 99.7%64 of 65
GPT-6 Astralow reasoning · 16 questions per request · Chat model98.5%91.8% to 99.7%64 of 65
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model98.5%91.8% to 99.7%64 of 65
GPT-6.1 Sollow reasoning · 1 question per request · Chat model98.5%91.8% to 99.7%64 of 65
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model98.5%91.8% to 99.7%64 of 65
Claude Opus 5.5high reasoning · 16 questions per request · Chat model98.5%91.8% to 99.7%64 of 65
Claude Opus 5.5low reasoning · 16 questions per request · Chat model98.5%91.8% to 99.7%64 of 65
GPT-6 Astralow reasoning · 1 question per request · Chat model96.9%89.5% to 99.2%63 of 65
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model96.9%89.5% to 99.2%63 of 65
Claude Fable 5.1high reasoning · 16 questions per request · Chat model96.9%89.5% to 99.2%63 of 65
Claude Opus 5.5low reasoning · 1 question per request · Chat model96.9%89.5% to 99.2%63 of 65
Grok 4.7high reasoning · 16 questions per request · Chat model96.9%89.5% to 99.2%63 of 65
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model96.9%89.5% to 99.2%63 of 65
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model95.4%87.3% to 98.4%62 of 65
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model95.4%87.3% to 98.4%62 of 65
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model93.8%85.2% to 97.6%61 of 65
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model90.8%81.3% to 95.7%59 of 65
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model89.2%79.4% to 94.7%58 of 65
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model84.6%73.9% to 91.4%55 of 65
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model84.6%73.9% to 91.4%55 of 65
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model80.0%68.7% to 87.9%52 of 65
Liquid d1-ultra1 question per request · Decision model76.9%65.4% to 85.5%50 of 65
GLM-5.3low reasoning · 16 questions per request · Chat model64.6%52.5% to 75.1%42 of 65
Gemma 4 31Breasoning off · 16 questions per request · Chat model64.6%52.5% to 75.1%42 of 65
Fastino GLiDE1 question per request · Decision model61.5%49.4% to 72.4%40 of 65
Fastino GLiDE v1.2local · 1 question per request · Decision model60.0%47.9% to 71.0%39 of 65
vLLM-SR d31 question per request · Decision model60.0%47.9% to 71.0%39 of 65
Drex Max v11 question per request · Decision model60.0%47.9% to 71.0%39 of 65
Mammoth-11 question per request · Decision model58.5%46.3% to 69.6%38 of 65
Perplexity Decider 27B v1.2local · 1 question per request · Decision model58.5%46.3% to 69.6%38 of 65
Perplexity Decider 27B v1.11 question per request · Decision model58.5%46.3% to 69.6%38 of 65
Drex Max 1.11 question per request · Decision model56.9%44.8% to 68.2%37 of 65
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model55.4%43.3% to 66.8%36 of 65
SubQ Snap 1.11 question per request · Decision model55.4%43.3% to 66.8%36 of 65
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model53.8%41.9% to 65.4%35 of 65
SubQ Snap Preview1 question per request · Decision model53.8%41.9% to 65.4%35 of 65
Gemma 4 31Breasoning off · 1 question per request · Chat model52.3%40.4% to 64.0%34 of 65
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model52.3%40.4% to 64.0%34 of 65
Celeris-1 Decision1 question per request · Decision model52.3%40.4% to 64.0%34 of 65
Open-Jev 27B v1.11 question per request · Decision model52.3%40.4% to 64.0%34 of 65
Perplexity Decider 27B v11 question per request · Decision model50.8%38.9% to 62.5%33 of 65
Liquid d11 question per request · Decision model50.8%38.9% to 62.5%33 of 65
Jev 1.13.01 question per request · Decision model49.2%37.5% to 61.1%32 of 65
GPT-6 Lunalow reasoning · 16 questions per request · Chat model47.7%36.0% to 59.6%31 of 65
Liquid d1.11 question per request · Decision model47.7%36.0% to 59.6%31 of 65
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model47.7%36.0% to 59.6%31 of 65
OneJev 27B1 question per request · Decision model47.7%36.0% to 59.6%31 of 65
Jev 1.14.01 question per request · Decision model47.7%36.0% to 59.6%31 of 65
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model46.2%34.6% to 58.1%30 of 65
Winnow-12B1 question per request · Decision model43.1%31.8% to 55.2%28 of 65
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model41.5%30.4% to 53.7%27 of 65
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model41.5%30.4% to 53.7%27 of 65
Clef1 question per request · Decision model41.5%30.4% to 53.7%27 of 65
Cygnet1 question per request · Decision model41.5%30.4% to 53.7%27 of 65
Drex 2.01 question per request · Decision model40.0%29.0% to 52.1%26 of 65
Elephant-11 question per request · Decision model38.5%27.6% to 50.6%25 of 65
Perplexity Decider Flash v1.2local · 1 question per request · Decision model38.5%27.6% to 50.6%25 of 65
Clef-flash1 question per request · Decision model38.5%27.6% to 50.6%25 of 65
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model33.8%23.5% to 46.0%22 of 65
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model33.8%23.5% to 46.0%22 of 65
Metask-Jev 4B1 question per request · Decision model33.8%23.5% to 46.0%22 of 65
Erk-14Breasoning off · 16 questions per request · Chat model30.8%20.9% to 42.8%20 of 65
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model29.2%19.6% to 41.2%19 of 65
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model29.2%19.6% to 41.2%19 of 65
Qwen3-14Breasoning off · 16 questions per request · Chat model29.2%19.6% to 41.2%19 of 65
meraGPT Decider 11 question per request · Decision model26.2%17.0% to 38.0%17 of 65
Liquid d1-2B (open weights)1 question per request · Decision model23.1%14.5% to 34.6%15 of 65
Strands Decider 2B1 question per request · Decision model23.1%14.5% to 34.6%15 of 65
Laya1 question per request · Decision model15.4%8.6% to 26.1%10 of 65
GLiNER2.5 Multi (base)1 question per request · Chat model13.8%7.5% to 24.3%9 of 65
GLiNER2.5-multi-Decide1 question per request · Decision model12.3%6.4% to 22.5%8 of 65
Laya multilingual1 question per request · Decision model12.3%6.4% to 22.5%8 of 65

Its questions

IDPageNumberAnswer keySetups rightDecision models right
8181911D65 of 7432 of 37
8191914E57 of 7429 of 37
8201915B33 of 747 of 37
8211912C65 of 7432 of 37
8221913A35 of 747 of 37
8231916D66 of 7431 of 37
8241927C58 of 7426 of 37
82519210B27 of 743 of 37
8261928B37 of 748 of 37
82719211E27 of 745 of 37
8281929A52 of 7425 of 37
8461967A30 of 745 of 37
84719610E41 of 7416 of 37
84819611B59 of 7428 of 37
8491968C41 of 7414 of 37
85019612A44 of 7414 of 37
8511969D51 of 7423 of 37
8521971C27 of 744 of 37
8531974D38 of 7411 of 37
8541972B68 of 7432 of 37
8551975E38 of 7413 of 37
8561973A63 of 7429 of 37
8682007B31 of 745 of 37
8692009A26 of 744 of 37
87020010E63 of 7431 of 37
8712008D64 of 7430 of 37
87220011C48 of 7431 of 37
34111893D40 of 7413 of 37
34121891A46 of 7417 of 37
34131892D63 of 7430 of 37
34141894D48 of 7418 of 37
34151905E41 of 7416 of 37
34161908C43 of 7413 of 37
34171909C62 of 7431 of 37
34181906D33 of 746 of 37
341919010A40 of 7414 of 37
34201907B55 of 7425 of 37
34211931D20 of 741 of 37
34221934B27 of 743 of 37
34231935C30 of 744 of 37
34241932E50 of 7422 of 37
34251936E52 of 7422 of 37
34261933A44 of 7419 of 37
34271947A35 of 7410 of 37
342819410D49 of 7421 of 37
34291948C59 of 7425 of 37
343019411B34 of 749 of 37
34311949D46 of 7421 of 37
34321951D43 of 7415 of 37
34331954E26 of 744 of 37
34341952E63 of 7429 of 37
34351955E37 of 748 of 37
34361953B57 of 7428 of 37
34371956C44 of 7413 of 37
34381986D40 of 7412 of 37
34391988E55 of 7424 of 37
34401989A40 of 7415 of 37
34411987B28 of 743 of 37
344219810C24 of 740 of 37
34431991D36 of 748 of 37
34441994D48 of 7417 of 37
34451995C39 of 7413 of 37
34461992D31 of 7410 of 37
34471996A38 of 7413 of 37
34481993B64 of 7430 of 37