Vision

Logic & Math questions, as text and as a page image

Logic & Math bench · its 358 figure questions (306 Logic + 52 Math) · original page images · five options · Info

The picture alone (B) · 306 Logic questions

Share answered right from a picture of the question alone (crops of the original page) · higher is better · the line: chance (20%) · a shared letter: not told apart

  1. Gemini Decision 1 20261111

    Gemini Decision 1 (20261111): 33.7%, 103 right of 306; Wilson 95% interval 28.6% to 39.1%

    33.7% a
  2. KnowLine-31B-Gen1.5 Run 2Run 2 of 12 Nov; first run 99/306 kept on Experiments

    KnowLine-31B-Gen1.5 (Run 2): 32.0%, 98 right of 306; Wilson 95% interval 27.0% to 37.4%; Run 2 of 12 Nov; first run 99/306 kept on Experiments

    32.0% a
  3. GLiDE 27B

    GLiDE 27B: 30.7%, 94 right of 306; Wilson 95% interval 25.8% to 36.1%

    30.7% a
  4. Drex Max v1.1

    Drex Max v1.1: 29.7%, 91 right of 306; Wilson 95% interval 24.9% to 35.1%

    29.7% a
  5. d3-ultra

    d3-ultra: 29.1%, 89 right of 306; Wilson 95% interval 24.3% to 34.4%

    29.1% a
  6. SubQ Snap 1.1

    SubQ Snap 1.1: 28.4%, 87 right of 306; Wilson 95% interval 23.7% to 33.7%

    28.4% a
  7. d1-ultra

    d1-ultra: 27.5%, 84 right of 306; Wilson 95% interval 22.8% to 32.7%

    27.5% a

Not in the chart, no complete run here: 1 paused, 10 queued, 1 blocked, 1 partial · Table

Google PelaAI Fastino Nace.AI vLLM Semantic Router Subquadratic Liquid AI Chance, 20% (five options)

Every setup of the track

20 setups of the models chosen; scored only when a cell is complete (306 Logic, 52 Math).

ModelstateAnswers storedLogic · Math
d1-ultraPublishedA text only: 306/306 · 52/52B picture alone: 306/306 · 52/52C text + picture: 306/306 · 52/52D ocr route: 0/306 · 0/52
d3-ultraPublishedA text only: 306/306 · 52/52B picture alone: 306/306 · 52/52C text + picture: 306/306 · 52/52D ocr route: 0/306 · 0/52
Drex Max v1.1PublishedA text only: 306/306 · 52/52B picture alone: 306/306 · 52/52C text + picture: 306/306 · 52/52D ocr route: 0/306 · 0/52
Gemini Decision 1 20261111PublishedA text only: 306/306 · 52/52B picture alone: 306/306 · 52/52C text + picture: 306/306 · 52/52D ocr route: 0/306 · 0/52
GLiDE 27BPublishedA text only: 306/306 · 52/52B picture alone: 306/306 · 52/52C text + picture: 306/306 · 52/52D ocr route: 0/306 · 0/52
KnowLine-31B-Gen1.5PublishedA text only: 306/306 · 52/52B picture alone: 306/306 · 52/52C text + picture: 306/306 · 52/52D ocr route: 0/306 · 0/52
SubQ Snap 1.1PublishedA text only: 306/306 · 52/52B picture alone: 306/306 · 52/52C text + picture: 306/306 · 52/52D ocr route: 0/306 · 0/52
Jev 1.14Partly runLogic B and C not measured (304/306 accepted): 2 crops refused by the provider, 36×280 px, below TypeSafe's 280 px minimum side; A and Math run in full (Zeki #5277)A text only: 306/306 · 52/52B picture alone: 304/306 · 52/52C text + picture: 304/306 · 52/52D ocr route: 0/306 · 0/52
OneJev-27BPausedstopped during its first full run when spark2 went to other models (Ümit #4187); needs a cardA text only: 136/306 · 0/52B picture alone: 0/306 · 0/52C text + picture: 0/306 · 0/52D ocr route: 0/306 · 0/52
d1-2BQueuednew d1 sizes; needs a card (bariton: no d1 environment on spark2)none stored yet
d1-3BQueuednew d1 sizes; needs a card (bariton: no d1 environment on spark2)none stored yet
d1-Omni-2BQueuednew d1 sizes; needs a card (bariton: no d1 environment on spark2)none stored yet
d3Queuedserved setup prepared on spark2; needs a cardnone stored yet
Drex MaxQueuedserved setup prepared on spark2; needs a cardnone stored yet
KnowLine 4B Gen5.5Queuedlocal setup; needs a card (spark2)none stored yet
KnowLine-0.8B-Gen1Queuedlocal setup; needs a card (spark2)none stored yet
OneJev1.6-0.8BQueuedlocal setup; needs a card (spark2)none stored yet
StartLux-Decision-v1.1-0.8B-InstantQueuedlocal setup; needs a card (spark2)none stored yet
StartLux-Decision-v1.1-2B-InstantQueuedlocal setup; needs a card (spark2)none stored yet
GPT-6 LunaBlockedAPI setup; its OpenAI account has no credit (HTTP 429 credit_balance_exhausted, 12 Nov 2026)none stored yet
Info: conditions, averages, classes

The questions are asked in Turkish, as original page images, unlike the Text tab, which uses English translations. A, text only, is the control, not a score; B and C are for setups that take images.

Against the control, on all 358 questions (Logic and Math pooled):

Each line is the mean, over the models with both cells complete, of a model's accuracy in that condition minus its accuracy in A; one point is one percentage point.

Class: Size by total parameters, as disclosed by the maker (for MoE models the label also shows active parameters): Tiny under 6B · Small 6B–15B · Medium over 15B up to 40B · Large over 40B. Models of undisclosed size, by standard list price per 1M input tokens: Efficient up to $0.50 · Workhorse up to $3 · Flagship over $3. Undisclosed: neither a disclosed size nor a list price.

9,662 graded answers in 27 runs, read at build time.

Where the self-hosted models ran

API setups ran on their providers' services.

Method: questions, crops, conditions, OCR, runs and cost

Questions

Crops

Conditions

The method follows the Turkish decision-model benchmark's page-image track. One question per request; the request is built by the benchmark runner's own code, with the images in each provider's format.

OCR (condition D)

No OCR model has read the crops yet, so condition D has no results. Planned, as in the Turkish decision-model benchmark's page-image track: Chandra OCR 2, dots.mocr, Surya OCR 2 and Baidu Unlimited-OCR-2, each its own D route, with each OCR model's character error rate measured against the clean text where it exists (normalised Levenshtein, micro-averaged over crops, the median also given).

Measures

Runs

Decision setups ran each condition once on both benchmarks: a 20-question smoke run per benchmark, the runner's probability check, then the full run (306 Logic, 52 Math). Chat setups (setup lm-chat-vision-1, condition B only) ran once per benchmark with no smoke gate and no probabilities: one question per request, the letter of the reply's last “CEVAP: X” line scored, and a reply cut off or without that line counted wrong; their measure is accuracy only. Only complete cells are scored; every other setup of the track is listed with its state and the answers stored so far.

ModelCellStartedUTCRunnercommitAnswersLogic / MathInput tokensas reportedRequest timesummedCostwhole run
d1-ultraA2026-11-12 11:44:51 UTCdf9ae27306 / 52109,23591 s$0.005 (provider price × reported usage)
d1-ultraB2026-11-12 11:28:02 UTCdf9ae27306 / 52287,577333 s$0.012 (provider price × reported usage)
d1-ultraC2026-11-12 11:59:56 UTCdf9ae27306 / 52381,060274 s$0.016 (provider price × reported usage)
d3-ultraA2026-11-10 15:01:24 UTC9ba1b40306 / 52157,406241 s$0.015estimate · DGX Spark
d3-ultraB2026-11-10 15:06:00 UTC9ba1b40306 / 52214,661315 s$0.020estimate · DGX Spark
d3-ultraC2026-11-10 15:11:59 UTC9ba1b40306 / 52308,813392 s$0.024estimate · DGX Spark
Drex Max v1.1A2026-11-12 11:47:02 UTCdf9ae27306 / 52130,310183 s$0.008 (provider price × reported usage)
Drex Max v1.1B2026-11-12 11:28:11 UTCdf9ae27306 / 52210,723406 s$0.013 (provider price × reported usage)
Drex Max v1.1C2026-11-12 12:20:38 UTCdf9ae27306 / 52305,949433 s$0.018 (provider price × reported usage)
Gemini Decision 1 20261111A2026-11-12 11:35:54 UTCdf9ae27306 / 52124,83495 s$0.012 (provider price × reported usage)
Gemini Decision 1 20261111B2026-11-12 11:28:02 UTCdf9ae27306 / 52765,797261 s$0.077 (provider price × reported usage)
Gemini Decision 1 20261111C2026-11-12 11:49:37 UTCdf9ae27306 / 52853,041256 s$0.085 (provider price × reported usage)
Gemma 4 12B Unified chatB2026-11-12 15:39:02 UTCdf9ae27306 / 52200,6371,137 s—
Gemma 4 E4B chatB2026-11-12 16:02:00 UTCdf9ae27306 / 52199,205524 s—
GLiDE 27BA2026-11-10 13:07:22 UTC9ba1b40306 / 52120,966195 s$0.012estimate · DGX Spark
GLiDE 27BB2026-11-10 13:11:13 UTC9ba1b40306 / 52198,193297 s$0.018estimate · DGX Spark
GLiDE 27BC2026-11-10 13:16:53 UTC9ba1b40306 / 52296,963422 s$0.026estimate · DGX Spark
Jev 1.14A2026-11-12 11:40:58 UTCdf9ae27306 / 52230,08060 s$0.010 (provider price × reported usage)
Jev 1.14B2026-11-12 11:30:58 UTCdf9ae27304 / 52600,252103 s$0.025 (provider price × reported usage)
Jev 1.14C2026-11-12 11:51:30 UTCdf9ae27304 / 52708,01799 s$0.030 (provider price × reported usage)
KnowLine-31B-Gen1.5A2026-11-11 00:45:13 UTC5fb5178306 / 52249,920348 s$0.022estimate · DGX Spark
KnowLine-31B-Gen1.5B2026-11-11 00:51:52 UTC5fb5178306 / 52212,863402 s$0.025estimate · DGX Spark
KnowLine-31B-Gen1.5C2026-11-11 00:59:38 UTC5fb5178306 / 52302,560540 s$0.033estimate · DGX Spark
Qwen3.5-4B on · chatB2026-11-13 07:31:38 UTCdf9ae27306 / 52202,9644,593 s—
SubQ Snap 1.1A2026-11-12 11:42:58 UTCdf9ae27306 / 52130,310139 s$0.005 (provider price × reported usage)
SubQ Snap 1.1B2026-11-12 11:28:07 UTCdf9ae27306 / 52210,723320 s$0.008 (provider price × reported usage)
SubQ Snap 1.1C2026-11-12 11:53:53 UTCdf9ae27306 / 52305,949322 s$0.012 (provider price × reported usage)

Cost

API setups: the provider's price × the usage it reported. Models served on our NVIDIA DGX Spark (GB10) show “DGX Spark”; hover, focus or tap it for the estimate: the run's GPU time (first to last answer, pauses over 10 minutes left out) × $0.213/h, as on the Text tab (see its Method). Drex Max keeps the maker's token list price, as in its text runs: that price does not depend on the hardware.