Hilde · Decision models · answer key v2

Reading · topic 6

Details that support a paragraph’s main point

97 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model99.0%94.4% to 99.8%96 of 97
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model99.0%94.4% to 99.8%96 of 97
GPT-6.1 Sollow reasoning · 1 question per request · Chat model99.0%94.4% to 99.8%96 of 97
Claude Opus 5.5low reasoning · 1 question per request · Chat model99.0%94.4% to 99.8%96 of 97
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model99.0%94.4% to 99.8%96 of 97
Mammoth-11 question per request · Decision model99.0%94.4% to 99.8%96 of 97
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model97.9%92.8% to 99.4%95 of 97
GPT-6 Astralow reasoning · 16 questions per request · Chat model97.9%92.8% to 99.4%95 of 97
GPT-6 Astralow reasoning · 1 question per request · Chat model97.9%92.8% to 99.4%95 of 97
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model97.9%92.8% to 99.4%95 of 97
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model97.9%92.8% to 99.4%95 of 97
Claude Fable 5.1high reasoning · 16 questions per request · Chat model97.9%92.8% to 99.4%95 of 97
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model97.9%92.8% to 99.4%95 of 97
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model97.9%92.8% to 99.4%95 of 97
Claude Opus 5.5high reasoning · 16 questions per request · Chat model96.9%91.3% to 98.9%94 of 97
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model96.9%91.3% to 98.9%94 of 97
Grok 4.7high reasoning · 16 questions per request · Chat model96.9%91.3% to 98.9%94 of 97
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model96.9%91.3% to 98.9%94 of 97
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model95.9%89.9% to 98.4%93 of 97
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model95.9%89.9% to 98.4%93 of 97
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model95.9%89.9% to 98.4%93 of 97
Gemma 4 31Breasoning off · 16 questions per request · Chat model95.9%89.9% to 98.4%93 of 97
Fastino GLiDE v1.2local · 1 question per request · Decision model95.9%89.9% to 98.4%93 of 97
Drex Max v11 question per request · Decision model95.9%89.9% to 98.4%93 of 97
SubQ Snap Preview1 question per request · Decision model95.9%89.9% to 98.4%93 of 97
Perplexity Decider 27B v11 question per request · Decision model95.9%89.9% to 98.4%93 of 97
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model94.8%88.5% to 97.8%92 of 97
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model94.8%88.5% to 97.8%92 of 97
Claude Opus 5.5low reasoning · 16 questions per request · Chat model93.8%87.2% to 97.1%91 of 97
GPT-6 Lunalow reasoning · 16 questions per request · Chat model93.8%87.2% to 97.1%91 of 97
vLLM-SR d31 question per request · Decision model93.8%87.2% to 97.1%91 of 97
Perplexity Decider 27B v1.2local · 1 question per request · Decision model93.8%87.2% to 97.1%91 of 97
Perplexity Decider 27B v1.11 question per request · Decision model93.8%87.2% to 97.1%91 of 97
Gemma 4 31Breasoning off · 1 question per request · Chat model93.8%87.2% to 97.1%91 of 97
SubQ Snap 1.11 question per request · Decision model93.8%87.2% to 97.1%91 of 97
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model93.8%87.2% to 97.1%91 of 97
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model92.8%85.8% to 96.5%90 of 97
Liquid d1-ultra1 question per request · Decision model92.8%85.8% to 96.5%90 of 97
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model92.8%85.8% to 96.5%90 of 97
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model92.8%85.8% to 96.5%90 of 97
OneJev 27B1 question per request · Decision model92.8%85.8% to 96.5%90 of 97
Open-Jev 27B v1.11 question per request · Decision model92.8%85.8% to 96.5%90 of 97
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model92.8%85.8% to 96.5%90 of 97
Fastino GLiDE1 question per request · Decision model91.8%84.6% to 95.8%89 of 97
Drex Max 1.11 question per request · Decision model91.8%84.6% to 95.8%89 of 97
Jev 1.14.01 question per request · Decision model91.8%84.6% to 95.8%89 of 97
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model91.8%84.6% to 95.8%89 of 97
GLM-5.3low reasoning · 16 questions per request · Chat model90.7%83.3% to 95.0%88 of 97
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model90.7%83.3% to 95.0%88 of 97
Celeris-1 Decision1 question per request · Decision model90.7%83.3% to 95.0%88 of 97
Clef1 question per request · Decision model90.7%83.3% to 95.0%88 of 97
Liquid d1.11 question per request · Decision model89.7%82.1% to 94.3%87 of 97
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model89.7%82.1% to 94.3%87 of 97
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model89.7%82.1% to 94.3%87 of 97
Jev 1.13.01 question per request · Decision model88.7%80.8% to 93.5%86 of 97
Liquid d11 question per request · Decision model87.6%79.6% to 92.8%85 of 97
Cygnet1 question per request · Decision model87.6%79.6% to 92.8%85 of 97
Drex 2.01 question per request · Decision model86.6%78.4% to 92.0%84 of 97
Winnow-12B1 question per request · Decision model86.6%78.4% to 92.0%84 of 97
Elephant-11 question per request · Decision model85.6%77.2% to 91.2%83 of 97
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model85.6%77.2% to 91.2%83 of 97
Perplexity Decider Flash v1.2local · 1 question per request · Decision model80.4%71.4% to 87.1%78 of 97
Clef-flash1 question per request · Decision model79.4%70.3% to 86.2%77 of 97
Erk-14Breasoning off · 16 questions per request · Chat model79.4%70.3% to 86.2%77 of 97
Qwen3-14Breasoning off · 16 questions per request · Chat model78.4%69.2% to 85.4%76 of 97
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model75.3%65.8% to 82.8%73 of 97
Metask-Jev 4B1 question per request · Decision model68.0%58.2% to 76.5%66 of 97
Liquid d1-2B (open weights)1 question per request · Decision model41.2%32.0% to 51.2%40 of 97
meraGPT Decider 11 question per request · Decision model37.1%28.2% to 47.0%36 of 97
GLiNER2.5-multi-Decide1 question per request · Decision model24.7%17.2% to 34.2%24 of 97
Laya multilingual1 question per request · Decision model16.5%10.4% to 25.1%16 of 97
Laya1 question per request · Decision model15.5%9.6% to 24.0%15 of 97
GLiNER2.5 Multi (base)1 question per request · Chat model14.4%8.8% to 22.8%14 of 97
Strands Decider 2B1 question per request · Decision model12.4%7.2% to 20.4%12 of 97

Its questions

IDPageNumberAnswer keySetups rightDecision models right
5691391E68 of 7432 of 37
5701392B59 of 7429 of 37
5711403C69 of 7433 of 37
5721405C62 of 7426 of 37
5731406E63 of 7429 of 37
5741404D68 of 7432 of 37
5751411E66 of 7432 of 37
5761413D66 of 7431 of 37
5771414E68 of 7431 of 37
5781412B66 of 7431 of 37
5791425E69 of 7433 of 37
5801427A60 of 7423 of 37
5811426E69 of 7433 of 37
5821428E68 of 7432 of 37
5961465D67 of 7431 of 37
5971467E33 of 7421 of 37
5981466A66 of 7430 of 37
5991468A69 of 7432 of 37
6001471A69 of 7433 of 37
6011473E69 of 7433 of 37
6021472D69 of 7433 of 37
6031474C66 of 7430 of 37
6041485D61 of 7426 of 37
6051487A60 of 7429 of 37
6061486E69 of 7433 of 37
6071488D45 of 7416 of 37
6201525B66 of 7430 of 37
6211527D69 of 7433 of 37
6221526C69 of 7433 of 37
6231528B61 of 7427 of 37
6281547C61 of 7426 of 37
6291545E68 of 7432 of 37
6301548E68 of 7433 of 37
6311546E71 of 7434 of 37
6321551B65 of 7431 of 37
6331553A69 of 7432 of 37
6341552E69 of 7433 of 37
6351554B64 of 7431 of 37
6401571E45 of 7424 of 37
6411572B67 of 7431 of 37
6421583C63 of 7430 of 37
6431585A65 of 7429 of 37
6441584D68 of 7432 of 37
6451586D69 of 7433 of 37
6461591D69 of 7434 of 37
6471593B69 of 7433 of 37
6481592D48 of 7418 of 37
6491594A68 of 7432 of 37
6501605C65 of 7428 of 37
6511607E65 of 7429 of 37
6531608E68 of 7432 of 37
6541611C41 of 7414 of 37
6551612E62 of 7428 of 37
6561613C32 of 748 of 37
6611632C70 of 7434 of 37
6621631E69 of 7433 of 37
6631633D69 of 7433 of 37
32491433D66 of 7432 of 37
32501431D67 of 7431 of 37
32511434B66 of 7430 of 37
32521432B68 of 7432 of 37
32531445D67 of 7431 of 37
32541447B71 of 7435 of 37
32551446D70 of 7434 of 37
32561448C19 of 7410 of 37
32571451D67 of 7431 of 37
32581453E69 of 7433 of 37
32591452E69 of 7433 of 37
32601454E70 of 7434 of 37
32611491C70 of 7434 of 37
32621493A71 of 7434 of 37
32631492C68 of 7432 of 37
32641494E68 of 7431 of 37
32651505E54 of 7422 of 37
32661507B54 of 7425 of 37
32671506E67 of 7431 of 37
32681508A69 of 7432 of 37
32691511C67 of 7431 of 37
32701513B70 of 7434 of 37
32711512B35 of 749 of 37
32721514B67 of 7431 of 37
32731531E65 of 7429 of 37
32741533A57 of 7424 of 37
32751532A60 of 7426 of 37
32761534E56 of 7422 of 37
32771565B69 of 7434 of 37
32781567E67 of 7431 of 37
32791568A69 of 7433 of 37
32801566B64 of 7428 of 37
32811624E71 of 7435 of 37
32821626E65 of 7431 of 37
32831625E60 of 7427 of 37
32841627B64 of 7429 of 37
32851644A71 of 7434 of 37
32861646B67 of 7431 of 37
32871645A28 of 746 of 37
32881647E63 of 7429 of 37