Hilde · Decision models · answer key v2

Reading · topic 4

Topic and main point of a paragraph

98 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model99.0%94.4% to 99.8%97 of 98
Claude Opus 5.5high reasoning · 16 questions per request · Chat model99.0%94.4% to 99.8%97 of 98
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model98.0%92.9% to 99.4%96 of 98
GPT-6 Astralow reasoning · 1 question per request · Chat model98.0%92.9% to 99.4%96 of 98
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model98.0%92.9% to 99.4%96 of 98
Claude Fable 5.1high reasoning · 16 questions per request · Chat model98.0%92.9% to 99.4%96 of 98
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model98.0%92.9% to 99.4%96 of 98
Claude Opus 5.5low reasoning · 16 questions per request · Chat model98.0%92.9% to 99.4%96 of 98
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model98.0%92.9% to 99.4%96 of 98
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model98.0%92.9% to 99.4%96 of 98
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model96.9%91.4% to 99.0%95 of 98
GPT-6 Astralow reasoning · 16 questions per request · Chat model96.9%91.4% to 99.0%95 of 98
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model96.9%91.4% to 99.0%95 of 98
Claude Opus 5.5low reasoning · 1 question per request · Chat model96.9%91.4% to 99.0%95 of 98
Gemma 4 31Breasoning off · 16 questions per request · Chat model96.9%91.4% to 99.0%95 of 98
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
GPT-6.1 Sollow reasoning · 1 question per request · Chat model95.9%90.0% to 98.4%94 of 98
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
Grok 4.7high reasoning · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model95.9%90.0% to 98.4%94 of 98
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
Liquid d1-ultra1 question per request · Decision model95.9%90.0% to 98.4%94 of 98
GLM-5.3low reasoning · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
vLLM-SR d31 question per request · Decision model95.9%90.0% to 98.4%94 of 98
Mammoth-11 question per request · Decision model95.9%90.0% to 98.4%94 of 98
Gemma 4 31Breasoning off · 1 question per request · Chat model95.9%90.0% to 98.4%94 of 98
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model95.9%90.0% to 98.4%94 of 98
Cygnet1 question per request · Decision model95.9%90.0% to 98.4%94 of 98
GPT-6 Lunalow reasoning · 16 questions per request · Chat model94.9%88.6% to 97.8%93 of 98
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model94.9%88.6% to 97.8%93 of 98
Perplexity Decider 27B v1.11 question per request · Decision model94.9%88.6% to 97.8%93 of 98
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model94.9%88.6% to 97.8%93 of 98
Liquid d1.11 question per request · Decision model94.9%88.6% to 97.8%93 of 98
Jev 1.14.01 question per request · Decision model94.9%88.6% to 97.8%93 of 98
Jev 1.13.01 question per request · Decision model94.9%88.6% to 97.8%93 of 98
Clef1 question per request · Decision model94.9%88.6% to 97.8%93 of 98
Liquid d11 question per request · Decision model94.9%88.6% to 97.8%93 of 98
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model93.9%87.3% to 97.2%92 of 98
Drex Max v11 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Drex Max 1.11 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Perplexity Decider 27B v1.2local · 1 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Perplexity Decider 27B v11 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Drex 2.01 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Open-Jev 27B v1.11 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Elephant-11 question per request · Decision model93.9%87.3% to 97.2%92 of 98
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model92.9%86.0% to 96.5%91 of 98
Fastino GLiDE v1.2local · 1 question per request · Decision model92.9%86.0% to 96.5%91 of 98
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model92.9%86.0% to 96.5%91 of 98
OneJev 27B1 question per request · Decision model92.9%86.0% to 96.5%91 of 98
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model92.9%86.0% to 96.5%91 of 98
Winnow-12B1 question per request · Decision model92.9%86.0% to 96.5%91 of 98
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model92.9%86.0% to 96.5%91 of 98
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model92.9%86.0% to 96.5%91 of 98
SubQ Snap Preview1 question per request · Decision model91.8%84.7% to 95.8%90 of 98
SubQ Snap 1.11 question per request · Decision model91.8%84.7% to 95.8%90 of 98
Fastino GLiDE1 question per request · Decision model90.8%83.5% to 95.1%89 of 98
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model90.8%83.5% to 95.1%89 of 98
Erk-14Breasoning off · 16 questions per request · Chat model90.8%83.5% to 95.1%89 of 98
Celeris-1 Decision1 question per request · Decision model88.8%81.0% to 93.6%87 of 98
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model87.8%79.8% to 92.9%86 of 98
Clef-flash1 question per request · Decision model87.8%79.8% to 92.9%86 of 98
Perplexity Decider Flash v1.2local · 1 question per request · Decision model86.7%78.6% to 92.1%85 of 98
Qwen3-14Breasoning off · 16 questions per request · Chat model85.7%77.4% to 91.3%84 of 98
Metask-Jev 4B1 question per request · Decision model85.7%77.4% to 91.3%84 of 98
Strands Decider 2B1 question per request · Decision model60.2%50.3% to 69.3%59 of 98
Liquid d1-2B (open weights)1 question per request · Decision model55.1%45.2% to 64.6%54 of 98
meraGPT Decider 11 question per request · Decision model54.1%44.2% to 63.6%53 of 98
GLiNER2.5-multi-Decide1 question per request · Decision model22.4%15.3% to 31.7%22 of 98
Laya multilingual1 question per request · Decision model21.4%14.5% to 30.5%21 of 98
GLiNER2.5 Multi (base)1 question per request · Chat model21.4%14.5% to 30.5%21 of 98
Laya1 question per request · Decision model18.4%11.9% to 27.2%18 of 98

Its questions

IDPageNumberAnswer keySetups rightDecision models right
360831A71 of 7435 of 37
361832C68 of 7432 of 37
362843C68 of 7432 of 37
363845B48 of 7425 of 37
364844E34 of 7413 of 37
365846A64 of 7429 of 37
366851E69 of 7433 of 37
367853A20 of 748 of 37
368852B68 of 7432 of 37
369854E69 of 7434 of 37
370865C68 of 7432 of 37
371867B69 of 7433 of 37
372866E71 of 7435 of 37
373868D67 of 7432 of 37
374871B70 of 7434 of 37
375873E67 of 7431 of 37
376874B67 of 7431 of 37
377872B70 of 7434 of 37
378885C68 of 7432 of 37
380886B27 of 749 of 37
381888A73 of 7436 of 37
382891D67 of 7431 of 37
383893E61 of 7427 of 37
384894A71 of 7434 of 37
385892E69 of 7433 of 37
386905A72 of 7435 of 37
387907A70 of 7433 of 37
388906E69 of 7434 of 37
389908D69 of 7433 of 37
398931A71 of 7435 of 37
399933B67 of 7431 of 37
400932B58 of 7422 of 37
401934C72 of 7435 of 37
402945A60 of 7426 of 37
403947E70 of 7434 of 37
404946C72 of 7435 of 37
405948C71 of 7434 of 37
414971E70 of 7434 of 37
415973D70 of 7434 of 37
416972B47 of 7418 of 37
417974E66 of 7432 of 37
418985D66 of 7431 of 37
419987E72 of 7436 of 37
420988D47 of 7415 of 37
421986B68 of 7432 of 37
422991C71 of 7434 of 37
423993B65 of 7433 of 37
424992C70 of 7434 of 37
425994D70 of 7434 of 37
4261005D71 of 7434 of 37
4271007D70 of 7434 of 37
4281006E71 of 7435 of 37
4291008C68 of 7432 of 37
4321025D68 of 7433 of 37
4331023D68 of 7432 of 37
4341024C71 of 7434 of 37
4351026B68 of 7432 of 37
4361031E69 of 7433 of 37
4371033C67 of 7432 of 37
4381032B66 of 7432 of 37
4391034B68 of 7432 of 37
4441051C69 of 7432 of 37
4451053A69 of 7432 of 37
4461054B67 of 7433 of 37
4471052D70 of 7434 of 37
4521071E69 of 7434 of 37
4531073A52 of 7419 of 37
4541072E69 of 7433 of 37
4551074E63 of 7429 of 37
4561085E70 of 7434 of 37
4571087C8 of 742 of 37
4581086A69 of 7432 of 37
3188913D70 of 7434 of 37
3189911B55 of 7422 of 37
3190914B70 of 7434 of 37
3191912B69 of 7433 of 37
3192925E73 of 7437 of 37
3193927D70 of 7434 of 37
3194928A64 of 7431 of 37
3195926A73 of 7436 of 37
3196951B67 of 7432 of 37
3197953C55 of 7424 of 37
3198954B70 of 7434 of 37
3199952A72 of 7435 of 37
3200965E70 of 7434 of 37
3201967A71 of 7434 of 37
3202966B71 of 7435 of 37
3203968A66 of 7430 of 37
32041011E71 of 7435 of 37
32051012E71 of 7435 of 37
32061045E69 of 7433 of 37
32071047B68 of 7432 of 37
32081048B70 of 7434 of 37
32091046D68 of 7432 of 37
32101065D68 of 7432 of 37
32111067C71 of 7435 of 37
32121066A68 of 7432 of 37
32131068E70 of 7434 of 37