Hilde · Decision models · answer key v2

Grammar · topic 19

Punctuation

117 scored questions. Accuracy by topic compares this topic with the others.

Every setup's result

Accuracy on this topic's questions, with its 95% interval. Few questions make wide intervals.

SetupAccuracyRight
Gemini 3.8 Flashhigh reasoning · 16 questions per request · Chat model98.3%94.0% to 99.5%115 of 117
GPT-6 Astrahigh reasoning · 16 questions per request · Chat model97.4%92.7% to 99.1%114 of 117
GPT-6 Astralow reasoning · 16 questions per request · Chat model96.6%91.5% to 98.7%113 of 117
GPT-6.1 Solhigh reasoning · 16 questions per request · Chat model95.7%90.4% to 98.2%112 of 117
Gemini 3.8 Flashlow reasoning · 1 question per request · Chat model95.7%90.4% to 98.2%112 of 117
Gemini 4 Argonlow reasoning · 16 questions per request · Chat model95.7%90.4% to 98.2%112 of 117
GPT-6.1 Sollow reasoning · 1 question per request · Chat model94.0%88.2% to 97.1%110 of 117
GPT-6.1 Sollow reasoning · 16 questions per request · Chat model94.0%88.2% to 97.1%110 of 117
GPT-6 Astralow reasoning · 1 question per request · Chat model93.2%87.1% to 96.5%109 of 117
Claude Fable 5.1high reasoning · 16 questions per request · Chat model92.3%86.0% to 95.9%108 of 117
Claude Opus 5.5high reasoning · 16 questions per request · Chat model88.9%81.9% to 93.4%104 of 117
Gemini 3.8 Flashlow reasoning · 16 questions per request · Chat model88.9%81.9% to 93.4%104 of 117
GPT-6.1 Terrahigh reasoning · 16 questions per request · Chat model85.5%78.0% to 90.7%100 of 117
Grok 4.7high reasoning · 16 questions per request · Chat model85.5%78.0% to 90.7%100 of 117
Claude Opus 5.5low reasoning · 16 questions per request · Chat model84.6%77.0% to 90.0%99 of 117
DeepSeek-V4.1-Flash, first releasereasoning on · 16 questions per request · Chat model83.8%76.0% to 89.4%98 of 117
GPT-6.1 Terralow reasoning · 16 questions per request · Chat model83.8%76.0% to 89.4%98 of 117
Claude Opus 5.5low reasoning · 1 question per request · Chat model82.1%74.1% to 88.0%96 of 117
Claude Sonnet 5.5high reasoning · 16 questions per request · Chat model80.3%72.2% to 86.5%94 of 117
Claude Sonnet 5.5high reasoning · 1 question per request · Chat model76.1%67.6% to 82.9%89 of 117
Liquid d1-ultra1 question per request · Decision model75.2%66.7% to 82.2%88 of 117
Kimi K3.1 Prolow reasoning · 16 questions per request · Chat model71.8%63.0% to 79.2%84 of 117
GLM-5.3-Flashreasoning on · 16 questions per request · Chat model70.1%61.3% to 77.6%82 of 117
Gemma 4 31Breasoning off · 16 questions per request · Chat model70.1%61.3% to 77.6%82 of 117
Qwen3.8-27Blow reasoning · 16 questions per request · Chat model65.8%56.8% to 73.8%77 of 117
Kimi K3.1 Proreasoning none · 16 questions per request · Chat model65.0%56.0% to 73.0%76 of 117
Fastino GLiDE v1.2local · 1 question per request · Decision model65.0%56.0% to 73.0%76 of 117
GPT-6 Lunalow reasoning · 16 questions per request · Chat model65.0%56.0% to 73.0%76 of 117
Mammoth-11 question per request · Decision model64.1%55.1% to 72.2%75 of 117
Perplexity Decider 27B v1.2local · 1 question per request · Decision model63.2%54.2% to 71.4%74 of 117
Gemma 4 31Breasoning off · 1 question per request · Chat model63.2%54.2% to 71.4%74 of 117
GLM-5.3low reasoning · 16 questions per request · Chat model62.4%53.4% to 70.6%73 of 117
Perplexity Decider 27B v1.11 question per request · Decision model62.4%53.4% to 70.6%73 of 117
vLLM-SR d31 question per request · Decision model61.5%52.5% to 69.9%72 of 117
vLLM-SR d3-ultratransformers, rev 52ba9e7 · 1 question per request · Decision model61.5%52.5% to 69.9%72 of 117
Drex Max v11 question per request · Decision model61.5%52.5% to 69.9%72 of 117
Drex Max 1.11 question per request · Decision model61.5%52.5% to 69.9%72 of 117
SubQ Snap Preview1 question per request · Decision model60.7%51.6% to 69.1%71 of 117
SubQ Snap 1.11 question per request · Decision model59.8%50.8% to 68.3%70 of 117
Liquid d1.11 question per request · Decision model59.8%50.8% to 68.3%70 of 117
DeepSeek-V4.1-Flash, first releasereasoning off · 16 questions per request · Chat model55.6%46.5% to 64.2%65 of 117
Fastino GLiDE1 question per request · Decision model54.7%45.7% to 63.4%64 of 117
GPT-6 LunaDecisions API, beta · 1 question per request · Decision model53.0%44.0% to 61.8%62 of 117
Claude Sonnet 5.5low reasoning · 16 questions per request · Chat model53.0%44.0% to 61.8%62 of 117
OneJev 27B1 question per request · Decision model52.1%43.2% to 61.0%61 of 117
Celeris-1 Decision1 question per request · Decision model51.3%42.3% to 60.2%60 of 117
Jev 1.14.01 question per request · Decision model51.3%42.3% to 60.2%60 of 117
Perplexity Decider 27B v11 question per request · Decision model50.4%41.5% to 59.3%59 of 117
Inception Mercury Decidevia OpenRouter · 1 question per request · Decision model50.4%41.5% to 59.3%59 of 117
Qwen3.8-27Bvote share · reasoning off · 1 question per request · Chat model50.4%41.5% to 59.3%59 of 117
Qwen3.8-27Bstated confidence · reasoning off · 1 question per request · Chat model50.4%41.5% to 59.3%59 of 117
GLM-5.3-Flashreasoning off · 16 questions per request · Chat model49.6%40.7% to 58.5%58 of 117
Drex 2.01 question per request · Decision model49.6%40.7% to 58.5%58 of 117
Liquid d11 question per request · Decision model48.7%39.8% to 57.7%57 of 117
Winnow-12B1 question per request · Decision model48.7%39.8% to 57.7%57 of 117
Jev 1.13.01 question per request · Decision model47.0%38.2% to 56.0%55 of 117
Clef1 question per request · Decision model47.0%38.2% to 56.0%55 of 117
Open-Jev 27B v1.11 question per request · Decision model47.0%38.2% to 56.0%55 of 117
Elephant-11 question per request · Decision model47.0%38.2% to 56.0%55 of 117
Cygnet1 question per request · Decision model46.2%37.4% to 55.2%54 of 117
Qwen3-14Breasoning off · 16 questions per request · Chat model45.3%36.6% to 54.3%53 of 117
Qwen3.8-27Byes/no scoring · 1 question per request · Chat model40.2%31.7% to 49.2%47 of 117
Perplexity Decider Flash v1.2local · 1 question per request · Decision model39.3%30.9% to 48.4%46 of 117
Clef-flash1 question per request · Decision model38.5%30.1% to 47.5%45 of 117
Microsoft-Decision-1via OpenRouter · 1 question per request · Decision model35.9%27.8% to 44.9%42 of 117
Erk-14Breasoning off · 16 questions per request · Chat model34.2%26.2% to 43.2%40 of 117
Metask-Jev 4B1 question per request · Decision model27.4%20.1% to 36.1%32 of 117
meraGPT Decider 11 question per request · Decision model25.6%18.6% to 34.2%30 of 117
Strands Decider 2B1 question per request · Decision model24.8%17.8% to 33.3%29 of 117
GLiNER2.5-multi-Decide1 question per request · Decision model23.1%16.4% to 31.5%27 of 117
Liquid d1-2B (open weights)1 question per request · Decision model22.2%15.6% to 30.6%26 of 117
Laya multilingual1 question per request · Decision model17.1%11.3% to 24.9%20 of 117
Laya1 question per request · Decision model14.5%9.3% to 22.0%17 of 117
GLiNER2.5 Multi (base)1 question per request · Chat model14.5%9.3% to 22.0%17 of 117

Its questions

IDPageNumberAnswer keySetups rightDecision models right
27293731D21 of 744 of 37
27303734E50 of 7423 of 37
27313732A16 of 741 of 37
27323735C69 of 7433 of 37
27333733D48 of 7420 of 37
27343736D65 of 7430 of 37
27353747B36 of 748 of 37
273637410A42 of 7414 of 37
27373748E49 of 7421 of 37
273837411A67 of 7433 of 37
27393749B40 of 7410 of 37
27403754B11 of 746 of 37
27413751A25 of 742 of 37
27423752D28 of 749 of 37
27433755C24 of 746 of 37
27443753E50 of 7425 of 37
27453756A68 of 7431 of 37
27513771D5 of 743 of 37
27523774C63 of 7429 of 37
27533775A26 of 745 of 37
27543772D49 of 7416 of 37
27553773A37 of 7411 of 37
27563776C61 of 7425 of 37
27573787E65 of 7431 of 37
275837810E46 of 7419 of 37
27593788C23 of 743 of 37
276037811C32 of 7411 of 37
27613789B47 of 7417 of 37
276237812D18 of 746 of 37
27633791B35 of 749 of 37
27643794A49 of 7417 of 37
27653795E53 of 7425 of 37
27663792E8 of 743 of 37
27673796D24 of 742 of 37
27683793E51 of 7420 of 37
27693807C66 of 7431 of 37
277038010B50 of 7421 of 37
27713808B61 of 7427 of 37
27723809E41 of 7414 of 37
277338011A61 of 7430 of 37
27803827B56 of 7426 of 37
27813829C30 of 7416 of 37
278238210D58 of 7427 of 37
27833828C54 of 7420 of 37
278438211C32 of 745 of 37
27853831D40 of 7411 of 37
27863833D36 of 7411 of 37
27873834E64 of 7430 of 37
27883832E52 of 7424 of 37
27893835E13 of 746 of 37
27903846D20 of 740 of 37
27913848E47 of 7418 of 37
27923847B45 of 7413 of 37
27933849C54 of 7424 of 37
27993866D66 of 7431 of 37
28003868D43 of 7416 of 37
28013867C67 of 7431 of 37
28023869C59 of 7425 of 37
28033871B58 of 7426 of 37
28043874E41 of 7414 of 37
28053872B41 of 7416 of 37
28063875E43 of 7411 of 37
28073873C52 of 7419 of 37
28083886B27 of 7410 of 37
28093888D69 of 7433 of 37
28103889B48 of 7417 of 37
28113887B38 of 7411 of 37
281238810E37 of 7412 of 37
28133891E47 of 7417 of 37
28143894A46 of 7417 of 37
28153892A57 of 7423 of 37
28163895C51 of 7416 of 37
28173896A59 of 7429 of 37
28183893D43 of 7417 of 37
28193907B55 of 7422 of 37
282039010E63 of 7426 of 37
28213908C54 of 7426 of 37
282239011C51 of 7422 of 37
282339012E31 of 744 of 37
28243909D31 of 747 of 37
28253911D21 of 741 of 37
28263914A37 of 7419 of 37
28273912C31 of 7412 of 37
28283915C68 of 7432 of 37
28293913B57 of 7424 of 37
28353931D55 of 7422 of 37
28363933E17 of 743 of 37
28373934E3 of 742 of 37
28383932B12 of 740 of 37
28393935B59 of 7425 of 37
28403946B45 of 7416 of 37
28413949B67 of 7432 of 37
28423947E62 of 7427 of 37
284339410B42 of 7414 of 37
28443948A47 of 7425 of 37
284539411D41 of 7414 of 37
41493767A40 of 7417 of 37
41503769B44 of 7411 of 37
415137610E65 of 7430 of 37
41523768A50 of 7419 of 37
415337611C41 of 7410 of 37
41543811C68 of 7432 of 37
41553814A60 of 7427 of 37
41563815D58 of 7430 of 37
41573812D58 of 7428 of 37
41583816D10 of 740 of 37
41593813A44 of 7420 of 37
41603851D54 of 7426 of 37
41613853C65 of 7429 of 37
41623854C23 of 743 of 37
41633852A52 of 7424 of 37
41643855B61 of 7428 of 37
41653926A48 of 7417 of 37
41663929E52 of 7421 of 37
41673927E68 of 7432 of 37
416839210D44 of 7415 of 37
41693928E24 of 743 of 37