Hilde · Decision models · answer key v2

Reading · topic 1

Word and phrase meanings

131 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model97.7%93.5% to 99.2%128 of 131
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model97.7%93.5% to 99.2%128 of 131
GPT-6 Astralow reasoning · 16 questions per request · Chat model97.7%93.5% to 99.2%128 of 131
GPT-6 Astralow reasoning · 1 question per request · Chat model97.7%93.5% to 99.2%128 of 131
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model97.7%93.5% to 99.2%128 of 131
Claude Opus 5.5low reasoning · 1 question per request · Chat model97.7%93.5% to 99.2%128 of 131
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model96.9%92.4% to 98.8%127 of 131
GPT-6.1 Sollow reasoning · 1 question per request · Chat model96.9%92.4% to 98.8%127 of 131
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model96.9%92.4% to 98.8%127 of 131
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model96.9%92.4% to 98.8%127 of 131
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model96.2%91.4% to 98.4%126 of 131
Claude Opus 5.5high reasoning · 16 questions per request · Chat model96.2%91.4% to 98.4%126 of 131
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model96.2%91.4% to 98.4%126 of 131
Claude Fable 5.1high reasoning · 16 questions per request · Chat model95.4%90.4% to 97.9%125 of 131
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model95.4%90.4% to 97.9%125 of 131
Grok 4.7high reasoning · 16 questions per request · Chat model95.4%90.4% to 97.9%125 of 131
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model95.4%90.4% to 97.9%125 of 131
Claude Opus 5.5low reasoning · 16 questions per request · Chat model95.4%90.4% to 97.9%125 of 131
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model94.7%89.4% to 97.4%124 of 131
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model91.6%85.6% to 95.2%120 of 131
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model90.8%84.7% to 94.7%119 of 131
Fastino GLiDE v1.2local · 1 question per request · Decision model89.3%82.9% to 93.5%117 of 131
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model88.5%82.0% to 92.9%116 of 131
Liquid d1-ultra1 question per request · Decision model88.5%82.0% to 92.9%116 of 131
GPT-6 Lunalow reasoning · 16 questions per request · Chat model87.0%80.2% to 91.7%114 of 131
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model87.0%80.2% to 91.7%114 of 131
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model86.3%79.3% to 91.1%113 of 131
Gemma 4 31Breasoning off · 16 questions per request · Chat model86.3%79.3% to 91.1%113 of 131
Gemma 4 31Breasoning off · 1 question per request · Chat model86.3%79.3% to 91.1%113 of 131
vLLM-SR d31 question per request · Decision model85.5%78.5% to 90.5%112 of 131
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model85.5%78.5% to 90.5%112 of 131
GLM-5.3low reasoning · 16 questions per request · Chat model84.7%77.6% to 89.9%111 of 131
Drex Max 1.11 question per request · Decision model84.7%77.6% to 89.9%111 of 131
Mammoth-11 question per request · Decision model84.7%77.6% to 89.9%111 of 131
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model84.7%77.6% to 89.9%111 of 131
Drex Max v11 question per request · Decision model84.0%76.7% to 89.3%110 of 131
Perplexity Decider 27B v1.11 question per request · Decision model84.0%76.7% to 89.3%110 of 131
Perplexity Decider 27B v1.2local · 1 question per request · Decision model83.2%75.9% to 88.6%109 of 131
Fastino GLiDE1 question per request · Decision model82.4%75.0% to 88.0%108 of 131
SubQ Snap 1.11 question per request · Decision model82.4%75.0% to 88.0%108 of 131
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model81.7%74.2% to 87.4%107 of 131
SubQ Snap Preview1 question per request · Decision model80.9%73.3% to 86.7%106 of 131
OneJev 27B1 question per request · Decision model80.9%73.3% to 86.7%106 of 131
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model80.9%73.3% to 86.7%106 of 131
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model80.2%72.5% to 86.1%105 of 131
Jev 1.13.01 question per request · Decision model80.2%72.5% to 86.1%105 of 131
Celeris-1 Decision1 question per request · Decision model79.4%71.7% to 85.4%104 of 131
Jev 1.14.01 question per request · Decision model79.4%71.7% to 85.4%104 of 131
Perplexity Decider 27B v11 question per request · Decision model78.6%70.8% to 84.8%103 of 131
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model78.6%70.8% to 84.8%103 of 131
Liquid d11 question per request · Decision model78.6%70.8% to 84.8%103 of 131
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model77.9%70.0% to 84.1%102 of 131
Liquid d1.11 question per request · Decision model77.1%69.2% to 83.5%101 of 131
Clef1 question per request · Decision model77.1%69.2% to 83.5%101 of 131
Open-Jev 27B v1.11 question per request · Decision model77.1%69.2% to 83.5%101 of 131
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model76.3%68.4% to 82.8%100 of 131
Cygnet1 question per request · Decision model76.3%68.4% to 82.8%100 of 131
Winnow-12B1 question per request · Decision model75.6%67.6% to 82.1%99 of 131
Drex 2.01 question per request · Decision model73.3%65.1% to 80.1%96 of 131
Elephant-11 question per request · Decision model73.3%65.1% to 80.1%96 of 131
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model69.5%61.1% to 76.7%91 of 131
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model65.6%57.2% to 73.2%86 of 131
Erk-14Breasoning off · 16 questions per request · Chat model64.1%55.6% to 71.8%84 of 131
Perplexity Decider Flash v1.2local · 1 question per request · Decision model61.8%53.3% to 69.7%81 of 131
Qwen3-14Breasoning off · 16 questions per request · Chat model61.1%52.5% to 69.0%80 of 131
Clef-flash1 question per request · Decision model58.8%50.2% to 66.8%77 of 131
Metask-Jev 4B1 question per request · Decision model50.4%41.9% to 58.8%66 of 131
Liquid d1-2B (open weights)1 question per request · Decision model38.2%30.3% to 46.7%50 of 131
Strands Decider 2B1 question per request · Decision model32.1%24.7% to 40.5%42 of 131
meraGPT Decider 11 question per request · Decision model26.7%19.9% to 34.9%35 of 131
Laya1 question per request · Decision model25.2%18.5% to 33.3%33 of 131
GLiNER2.5-multi-Decide1 question per request · Decision model22.1%15.9% to 30.0%29 of 131
Laya multilingual1 question per request · Decision model19.1%13.3% to 26.7%25 of 131
GLiNER2.5 Multi (base)1 question per request · Chat model18.3%12.6% to 25.8%24 of 131

Its questions

IDPageNumberAnswer keySetups rightDecision models right
44171C69 of 7433 of 37
45174A56 of 7423 of 37
46175C44 of 7415 of 37
47172D52 of 7417 of 37
48176E31 of 748 of 37
49173B69 of 7433 of 37
302291B66 of 7431 of 37
302393D70 of 7433 of 37
302592C46 of 7412 of 37
3026105C49 of 7418 of 37
3027108A62 of 7428 of 37
3028106B47 of 7417 of 37
3029109D61 of 7426 of 37
3030107E69 of 7432 of 37
3031111C61 of 7429 of 37
3032114B54 of 7423 of 37
3033112D70 of 7434 of 37
3034115A67 of 7432 of 37
3035113E24 of 7413 of 37
3036129E64 of 7428 of 37
3037126E35 of 7411 of 37
30381210B40 of 7410 of 37
3039127B41 of 7412 of 37
3040128C64 of 7428 of 37
30411211D65 of 7430 of 37
3042131B62 of 7429 of 37
3043134A62 of 7427 of 37
3044132D64 of 7428 of 37
3045135C71 of 7435 of 37
3046136B69 of 7433 of 37
3047133E29 of 747 of 37
3048147E67 of 7431 of 37
3049149C69 of 7433 of 37
30501410A25 of 742 of 37
3051148D43 of 7416 of 37
30521411B59 of 7426 of 37
3053151B61 of 7428 of 37
3054154C54 of 7422 of 37
3055152D29 of 741 of 37
3056155D25 of 743 of 37
3057153A66 of 7430 of 37
3058156E61 of 7429 of 37
3059167C66 of 7430 of 37
30601610D20 of 743 of 37
3061168E68 of 7433 of 37
30621611D60 of 7428 of 37
3063169B60 of 7427 of 37
30641612C40 of 7413 of 37
3065187E36 of 7413 of 37
30661810E41 of 7419 of 37
30671811D70 of 7434 of 37
3068188C66 of 7430 of 37
3069189B70 of 7434 of 37
30701812D54 of 7421 of 37
3071191E62 of 7430 of 37
3072193C38 of 7410 of 37
3073194D63 of 7430 of 37
3074192B66 of 7431 of 37
3075195B63 of 7430 of 37
3076206B65 of 7429 of 37
3077209B51 of 7417 of 37
3078207C43 of 7417 of 37
30792010D68 of 7433 of 37
3080208A61 of 7425 of 37
30812011E63 of 7428 of 37
3082211B1 of 740 of 37
3083214A72 of 7435 of 37
3084215E71 of 7434 of 37
3085212D61 of 7427 of 37
3086213C67 of 7431 of 37
3088227E71 of 7435 of 37
30892210A65 of 7428 of 37
3090228B47 of 7426 of 37
30912211E66 of 7431 of 37
3092229D65 of 7428 of 37
30932212C69 of 7433 of 37
3094231D19 of 746 of 37
3095234B69 of 7434 of 37
3096232A70 of 7434 of 37
3097235D63 of 7427 of 37
3098233E39 of 7411 of 37
3099236C70 of 7433 of 37
3100247D57 of 7426 of 37
31012410A56 of 7424 of 37
3102248E69 of 7433 of 37
31032411B65 of 7429 of 37
3104249C63 of 7427 of 37
31052412D72 of 7436 of 37
3106251B68 of 7432 of 37
3107254A71 of 7434 of 37
3108255E68 of 7434 of 37
3109252D66 of 7430 of 37
3110256B67 of 7431 of 37
3111253C66 of 7430 of 37
3112267E70 of 7434 of 37
31132610E70 of 7434 of 37
3114268D66 of 7431 of 37
31152611A70 of 7434 of 37
31162612B69 of 7433 of 37
3117269C66 of 7432 of 37
3118271D67 of 7431 of 37
3119274A63 of 7430 of 37
3120272B70 of 7434 of 37
3121275E71 of 7435 of 37
3122273E70 of 7435 of 37
3123286C5 of 745 of 37
3124288B71 of 7435 of 37
3125289E64 of 7431 of 37
3126287D69 of 7432 of 37
3127291D56 of 7427 of 37
3128293B41 of 7414 of 37
3129294A48 of 7421 of 37
3130292C51 of 7422 of 37
3131295E70 of 7434 of 37
3132306D68 of 7432 of 37
3133309D55 of 7425 of 37
3134307E66 of 7431 of 37
31353010C65 of 7428 of 37
31363011B65 of 7430 of 37
3137308A62 of 7428 of 37
3138311E68 of 7432 of 37
3139313B67 of 7431 of 37
3140314A69 of 7432 of 37
3141312D54 of 7421 of 37
3142315D64 of 7429 of 37
3143326C65 of 7430 of 37
3144329A49 of 7423 of 37
3145327E32 of 744 of 37
31463210D61 of 7431 of 37
3147328B68 of 7432 of 37
31483211A58 of 7427 of 37