Hilde · Decision models · answer key v2

Reading · topic 5

How a paragraph fits together

106 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model96.2%90.7% to 98.5%102 of 106
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model95.3%89.4% to 98.0%101 of 106
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model95.3%89.4% to 98.0%101 of 106
GPT-6 Astralow reasoning · 1 question per request · Chat model95.3%89.4% to 98.0%101 of 106
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model95.3%89.4% to 98.0%101 of 106
Claude Opus 5.5low reasoning · 16 questions per request · Chat model95.3%89.4% to 98.0%101 of 106
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model95.3%89.4% to 98.0%101 of 106
GPT-6 Astralow reasoning · 16 questions per request · Chat model94.3%88.2% to 97.4%100 of 106
Claude Opus 5.5low reasoning · 1 question per request · Chat model94.3%88.2% to 97.4%100 of 106
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model93.4%87.0% to 96.8%99 of 106
GPT-6.1 Sollow reasoning · 1 question per request · Chat model93.4%87.0% to 96.8%99 of 106
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model93.4%87.0% to 96.8%99 of 106
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model92.5%85.8% to 96.1%98 of 106
Grok 4.7high reasoning · 16 questions per request · Chat model92.5%85.8% to 96.1%98 of 106
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model92.5%85.8% to 96.1%98 of 106
Claude Fable 5.1high reasoning · 16 questions per request · Chat model91.5%84.6% to 95.5%97 of 106
Claude Opus 5.5high reasoning · 16 questions per request · Chat model90.6%83.5% to 94.8%96 of 106
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model90.6%83.5% to 94.8%96 of 106
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model90.6%83.5% to 94.8%96 of 106
Liquid d1-ultra1 question per request · Decision model90.6%83.5% to 94.8%96 of 106
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model88.7%81.2% to 93.4%94 of 106
Drex Max v11 question per request · Decision model87.7%80.1% to 92.7%93 of 106
Mammoth-11 question per request · Decision model86.8%79.0% to 92.0%92 of 106
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model85.8%78.0% to 91.2%91 of 106
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model85.8%78.0% to 91.2%91 of 106
Perplexity Decider 27B v1.2local · 1 question per request · Decision model84.9%76.9% to 90.5%90 of 106
Gemma 4 31Breasoning off · 16 questions per request · Chat model84.0%75.8% to 89.7%89 of 106
vLLM-SR d31 question per request · Decision model84.0%75.8% to 89.7%89 of 106
Fastino GLiDE1 question per request · Decision model83.0%74.7% to 89.0%88 of 106
Perplexity Decider 27B v1.11 question per request · Decision model83.0%74.7% to 89.0%88 of 106
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model82.1%73.7% to 88.2%87 of 106
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model82.1%73.7% to 88.2%87 of 106
SubQ Snap Preview1 question per request · Decision model82.1%73.7% to 88.2%87 of 106
GLM-5.3low reasoning · 16 questions per request · Chat model81.1%72.6% to 87.4%86 of 106
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model81.1%72.6% to 87.4%86 of 106
Drex Max 1.11 question per request · Decision model81.1%72.6% to 87.4%86 of 106
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model80.2%71.6% to 86.7%85 of 106
SubQ Snap 1.11 question per request · Decision model80.2%71.6% to 86.7%85 of 106
Gemma 4 31Breasoning off · 1 question per request · Chat model79.2%70.6% to 85.9%84 of 106
Fastino GLiDE v1.2local · 1 question per request · Decision model78.3%69.5% to 85.1%83 of 106
Celeris-1 Decision1 question per request · Decision model78.3%69.5% to 85.1%83 of 106
Perplexity Decider 27B v11 question per request · Decision model78.3%69.5% to 85.1%83 of 106
OneJev 27B1 question per request · Decision model78.3%69.5% to 85.1%83 of 106
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model78.3%69.5% to 85.1%83 of 106
Jev 1.14.01 question per request · Decision model77.4%68.5% to 84.3%82 of 106
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model77.4%68.5% to 84.3%82 of 106
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model76.4%67.5% to 83.5%81 of 106
Liquid d1.11 question per request · Decision model75.5%66.5% to 82.7%80 of 106
Clef1 question per request · Decision model75.5%66.5% to 82.7%80 of 106
Open-Jev 27B v1.11 question per request · Decision model75.5%66.5% to 82.7%80 of 106
GPT-6 Lunalow reasoning · 16 questions per request · Chat model74.5%65.5% to 81.9%79 of 106
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model74.5%65.5% to 81.9%79 of 106
Drex 2.01 question per request · Decision model74.5%65.5% to 81.9%79 of 106
Winnow-12B1 question per request · Decision model74.5%65.5% to 81.9%79 of 106
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model74.5%65.5% to 81.9%79 of 106
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model74.5%65.5% to 81.9%79 of 106
Jev 1.13.01 question per request · Decision model73.6%64.5% to 81.0%78 of 106
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model71.7%62.5% to 79.4%76 of 106
Liquid d11 question per request · Decision model71.7%62.5% to 79.4%76 of 106
Cygnet1 question per request · Decision model71.7%62.5% to 79.4%76 of 106
Elephant-11 question per request · Decision model70.8%61.5% to 78.6%75 of 106
Qwen3-14Breasoning off · 16 questions per request · Chat model67.9%58.5% to 76.0%72 of 106
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model63.2%53.7% to 71.8%67 of 106
Clef-flash1 question per request · Decision model61.3%51.8% to 70.0%65 of 106
Erk-14Breasoning off · 16 questions per request · Chat model60.4%50.9% to 69.2%64 of 106
Perplexity Decider Flash v1.2local · 1 question per request · Decision model57.5%48.0% to 66.5%61 of 106
Metask-Jev 4B1 question per request · Decision model49.1%39.7% to 58.4%52 of 106
Strands Decider 2B1 question per request · Decision model33.0%24.8% to 42.4%35 of 106
meraGPT Decider 11 question per request · Decision model31.1%23.1% to 40.5%33 of 106
Liquid d1-2B (open weights)1 question per request · Decision model28.3%20.6% to 37.5%30 of 106
Laya1 question per request · Decision model26.4%19.0% to 35.5%28 of 106
GLiNER2.5 Multi (base)1 question per request · Chat model20.8%14.1% to 29.4%22 of 106
Laya multilingual1 question per request · Decision model18.9%12.6% to 27.4%20 of 106
GLiNER2.5-multi-Decide1 question per request · Decision model14.2%8.8% to 22.0%15 of 106

Its questions

IDPageNumberAnswer keySetups rightDecision models right
4601091C43 of 7412 of 37
4611093A40 of 7415 of 37
4621094D64 of 7429 of 37
4631092B55 of 7424 of 37
4641105B65 of 7430 of 37
4651107E68 of 7432 of 37
4661106A25 of 7412 of 37
4671111D64 of 7430 of 37
4681113A65 of 7430 of 37
4691112E43 of 7415 of 37
4701124C41 of 7419 of 37
4711126B62 of 7429 of 37
4721125C23 of 744 of 37
4731127D60 of 7425 of 37
4781145C67 of 7431 of 37
4791147E70 of 7433 of 37
4801148A66 of 7431 of 37
4811146D10 of 743 of 37
4821151A32 of 746 of 37
4831153B67 of 7431 of 37
4841152C70 of 7434 of 37
4851164E66 of 7430 of 37
4861166C53 of 7427 of 37
4871167D67 of 7432 of 37
4881165C46 of 7420 of 37
4921184A40 of 7425 of 37
4931186C65 of 7429 of 37
4941185E67 of 7431 of 37
4951187E34 of 7412 of 37
4961191A65 of 7429 of 37
4971193B71 of 7435 of 37
4981192C69 of 7434 of 37
4991194D68 of 7432 of 37
5001205A69 of 7432 of 37
5011207E68 of 7432 of 37
5021206D29 of 7416 of 37
5031208B44 of 7416 of 37
5181251C64 of 7428 of 37
5191253C68 of 7432 of 37
5201252B64 of 7428 of 37
5211264C56 of 7425 of 37
5221266D70 of 7433 of 37
5231267E67 of 7432 of 37
5241265A70 of 7434 of 37
5251271A69 of 7432 of 37
5261273D60 of 7428 of 37
5271272C63 of 7429 of 37
5281274D53 of 7424 of 37
5291285D65 of 7431 of 37
5301287E66 of 7431 of 37
5311286B70 of 7434 of 37
5321293E64 of 7433 of 37
5331291B33 of 748 of 37
5341292D71 of 7434 of 37
5351304A66 of 7430 of 37
5361306C68 of 7432 of 37
5371305B51 of 7420 of 37
5381307D55 of 7421 of 37
5431325C66 of 7430 of 37
5441327A69 of 7432 of 37
5451326D57 of 7422 of 37
5461328E50 of 7419 of 37
5471331D38 of 7413 of 37
5481333A69 of 7432 of 37
5491332E62 of 7428 of 37
5501346D50 of 7416 of 37
5511344D67 of 7431 of 37
5521347B70 of 7434 of 37
5531345A71 of 7434 of 37
5621371C69 of 7433 of 37
5631373D54 of 7422 of 37
5641372E64 of 7431 of 37
5651374B70 of 7434 of 37
32141133B64 of 7431 of 37
32151131A71 of 7434 of 37
32161132E67 of 7431 of 37
32171134C63 of 7428 of 37
32181171E22 of 748 of 37
32191173D19 of 746 of 37
32201172B71 of 7435 of 37
32211211C44 of 7415 of 37
32221213D54 of 7423 of 37
32241224E72 of 7436 of 37
32251226A72 of 7435 of 37
32261227C62 of 7427 of 37
32271225D22 of 741 of 37
32281233B57 of 7424 of 37
32291231B65 of 7430 of 37
32301232E33 of 7412 of 37
32311244B59 of 7428 of 37
32321246A68 of 7431 of 37
32331245D44 of 7416 of 37
32341247D22 of 742 of 37
32351311D39 of 7417 of 37
32361313D66 of 7430 of 37
32371312E57 of 7423 of 37
32381314C58 of 7424 of 37
32391351B55 of 7424 of 37
32401353D60 of 7428 of 37
32411354C22 of 7413 of 37
32431365A13 of 749 of 37
32441366A40 of 7413 of 37
32451367A68 of 7432 of 37
32461387D52 of 7422 of 37
32471385C68 of 7432 of 37
32481386A68 of 7431 of 37