Skip to content
Hilde · Decision models · answer key v2
Every question
Where models disagree
Topics
Compare
Method
Data
Dark
background
Light
background
Compare two setups
First
Gemini 3.8 Flash · high reasoning · 16 questions per request
GPT-6 Astra · high reasoning · 16 questions per request
GPT-6.1 Sol · high reasoning · 16 questions per request
GPT-6 Astra · low reasoning · 16 questions per request
GPT-6 Astra · low reasoning · 1 question per request
Gemini 3.8 Flash · low reasoning · 1 question per request
GPT-6.1 Sol · low reasoning · 1 question per request
GPT-6.1 Sol · low reasoning · 16 questions per request
Gemini 4 Argon · low reasoning · 16 questions per request
Claude Fable 5.1 · high reasoning · 16 questions per request
Claude Opus 5.5 · high reasoning · 16 questions per request
GPT-6.1 Terra · high reasoning · 16 questions per request
Claude Opus 5.5 · low reasoning · 1 question per request
Gemini 3.8 Flash · low reasoning · 16 questions per request
Grok 4.7 · high reasoning · 16 questions per request
DeepSeek-V4.1-Flash, first release · reasoning on · 16 questions per request
GPT-6.1 Terra · low reasoning · 16 questions per request
Claude Opus 5.5 · low reasoning · 16 questions per request
Claude Sonnet 5.5 · high reasoning · 1 question per request
Claude Sonnet 5.5 · high reasoning · 16 questions per request
Kimi K3.1 Pro · low reasoning · 16 questions per request
GLM-5.3-Flash · reasoning on · 16 questions per request
Liquid d1-ultra · 1 question per request
GLM-5.3 · low reasoning · 16 questions per request
Qwen3.8-27B · low reasoning · 16 questions per request
Gemma 4 31B · reasoning off · 16 questions per request
Kimi K3.1 Pro · reasoning none · 16 questions per request
Fastino GLiDE v1.2 · local · 1 question per request
GPT-6 Luna · low reasoning · 16 questions per request
Fastino GLiDE · 1 question per request
vLLM-SR d3 · 1 question per request
vLLM-SR d3-ultra · transformers, rev 52ba9e7 · 1 question per request
Drex Max v1 · 1 question per request
Drex Max 1.1 · 1 question per request
Mammoth-1 · 1 question per request
Perplexity Decider 27B v1.2 · local · 1 question per request
Perplexity Decider 27B v1.1 · 1 question per request
Gemma 4 31B · reasoning off · 1 question per request
DeepSeek-V4.1-Flash, first release · reasoning off · 16 questions per request
SubQ Snap Preview · 1 question per request
SubQ Snap 1.1 · 1 question per request
GPT-6 Luna · Decisions API, beta · 1 question per request
Liquid d1.1 · 1 question per request
Claude Sonnet 5.5 · low reasoning · 16 questions per request
Celeris-1 Decision · 1 question per request
Perplexity Decider 27B v1 · 1 question per request
OneJev 27B · 1 question per request
Jev 1.14.0 · 1 question per request
Inception Mercury Decide · via OpenRouter · 1 question per request
GLM-5.3-Flash · reasoning off · 16 questions per request
Jev 1.13.0 · 1 question per request
Clef · 1 question per request
Liquid d1 · 1 question per request
Drex 2.0 · 1 question per request
Cygnet · 1 question per request
Open-Jev 27B v1.1 · 1 question per request
Winnow-12B · 1 question per request
Qwen3.8-27B · vote share · reasoning off · 1 question per request
Qwen3.8-27B · stated confidence · reasoning off · 1 question per request
Elephant-1 · 1 question per request
Qwen3.8-27B · yes/no scoring · 1 question per request
Perplexity Decider Flash v1.2 · local · 1 question per request
Microsoft-Decision-1 · via OpenRouter · 1 question per request
Clef-flash · 1 question per request
Qwen3-14B · reasoning off · 16 questions per request
Erk-14B · reasoning off · 16 questions per request
Metask-Jev 4B · 1 question per request
Liquid d1-2B (open weights) · 1 question per request
Strands Decider 2B · 1 question per request
meraGPT Decider 1 · 1 question per request
GLiNER2.5-multi-Decide · 1 question per request
Laya · 1 question per request
Laya multilingual · 1 question per request
GLiNER2.5 Multi (base) · 1 question per request
Second
Gemini 3.8 Flash · high reasoning · 16 questions per request
GPT-6 Astra · high reasoning · 16 questions per request
GPT-6.1 Sol · high reasoning · 16 questions per request
GPT-6 Astra · low reasoning · 16 questions per request
GPT-6 Astra · low reasoning · 1 question per request
Gemini 3.8 Flash · low reasoning · 1 question per request
GPT-6.1 Sol · low reasoning · 1 question per request
GPT-6.1 Sol · low reasoning · 16 questions per request
Gemini 4 Argon · low reasoning · 16 questions per request
Claude Fable 5.1 · high reasoning · 16 questions per request
Claude Opus 5.5 · high reasoning · 16 questions per request
GPT-6.1 Terra · high reasoning · 16 questions per request
Claude Opus 5.5 · low reasoning · 1 question per request
Gemini 3.8 Flash · low reasoning · 16 questions per request
Grok 4.7 · high reasoning · 16 questions per request
DeepSeek-V4.1-Flash, first release · reasoning on · 16 questions per request
GPT-6.1 Terra · low reasoning · 16 questions per request
Claude Opus 5.5 · low reasoning · 16 questions per request
Claude Sonnet 5.5 · high reasoning · 1 question per request
Claude Sonnet 5.5 · high reasoning · 16 questions per request
Kimi K3.1 Pro · low reasoning · 16 questions per request
GLM-5.3-Flash · reasoning on · 16 questions per request
Liquid d1-ultra · 1 question per request
GLM-5.3 · low reasoning · 16 questions per request
Qwen3.8-27B · low reasoning · 16 questions per request
Gemma 4 31B · reasoning off · 16 questions per request
Kimi K3.1 Pro · reasoning none · 16 questions per request
Fastino GLiDE v1.2 · local · 1 question per request
GPT-6 Luna · low reasoning · 16 questions per request
Fastino GLiDE · 1 question per request
vLLM-SR d3 · 1 question per request
vLLM-SR d3-ultra · transformers, rev 52ba9e7 · 1 question per request
Drex Max v1 · 1 question per request
Drex Max 1.1 · 1 question per request
Mammoth-1 · 1 question per request
Perplexity Decider 27B v1.2 · local · 1 question per request
Perplexity Decider 27B v1.1 · 1 question per request
Gemma 4 31B · reasoning off · 1 question per request
DeepSeek-V4.1-Flash, first release · reasoning off · 16 questions per request
SubQ Snap Preview · 1 question per request
SubQ Snap 1.1 · 1 question per request
GPT-6 Luna · Decisions API, beta · 1 question per request
Liquid d1.1 · 1 question per request
Claude Sonnet 5.5 · low reasoning · 16 questions per request
Celeris-1 Decision · 1 question per request
Perplexity Decider 27B v1 · 1 question per request
OneJev 27B · 1 question per request
Jev 1.14.0 · 1 question per request
Inception Mercury Decide · via OpenRouter · 1 question per request
GLM-5.3-Flash · reasoning off · 16 questions per request
Jev 1.13.0 · 1 question per request
Clef · 1 question per request
Liquid d1 · 1 question per request
Drex 2.0 · 1 question per request
Cygnet · 1 question per request
Open-Jev 27B v1.1 · 1 question per request
Winnow-12B · 1 question per request
Qwen3.8-27B · vote share · reasoning off · 1 question per request
Qwen3.8-27B · stated confidence · reasoning off · 1 question per request
Elephant-1 · 1 question per request
Qwen3.8-27B · yes/no scoring · 1 question per request
Perplexity Decider Flash v1.2 · local · 1 question per request
Microsoft-Decision-1 · via OpenRouter · 1 question per request
Clef-flash · 1 question per request
Qwen3-14B · reasoning off · 16 questions per request
Erk-14B · reasoning off · 16 questions per request
Metask-Jev 4B · 1 question per request
Liquid d1-2B (open weights) · 1 question per request
Strands Decider 2B · 1 question per request
meraGPT Decider 1 · 1 question per request
GLiNER2.5-multi-Decide · 1 question per request
Laya · 1 question per request
Laya multilingual · 1 question per request
GLiNER2.5 Multi (base) · 1 question per request
This page needs JavaScript. The
leaderboard
lists every setup.