Experiments
Repeat checks
Turkish Text2,198 questions; the reading and grammar parts are subsets of the same run
| Rank | Model | Run 2% · right / questions | First run% · right / questions | Questions | Status |
|---|---|---|---|---|---|
| 1 | KnowLine-31B-Gen1.5Done KnowLine-31B-Gen1.5 · PelaAI First run: run 957 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Run 2: run 998 · 11 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Questions: 2,198 One DGX Spark (GB10), BF16, behind the KnowLine server (the card's serve_knowline.sh, PelaAI/KnowLine-31B-Gen1.5 at 8e56aa1), as run 957: repeat 2 on 11 November 2026 (logic's server on spark2, shared by the three benches). | 81.8%1,798/2,198 | 81.8%1,798/2,198 | 2,198 | Done |
| 2 | REAX 1.3Done REAX 1.3 · REAX First run: run 827 · 6 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.41 at GPU-time estimate. Run 2: run 990 · 11 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.49 at GPU-time estimate. Questions: 2,198 One RTX PRO 6000 Blackwell (96 GB), FP8 (block-128), behind the REAX Engine server, as run 827: repeat 2 on 11 November 2026 (logic's offline server, the same checkpoint and engine). | 77.9%1,713/2,198 | 77.9%1,713/2,198 | 2,198 | Done |
| 3 | d1-ultraRetained d1-ultra · Liquid AI First run: run 199 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price. Run 2: run 200 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price. Run 3: run 201 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price. Run 4: run 202 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price. Run 5: run 203 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price. Questions: 2,198 Liquid AI's decisions API; its repeats 2 to 5 of 16 October 2026, also on the Stability tab. | 76.7%1,686/2,198 | 77.1%1,694/2,198 | 2,198 | Retained |
| 4 | GLiDE v1.2Retained GLiDE v1.2 · Fastino First run: run 248 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model. Run 2: run 250 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model. Run 3: run 251 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model. Run 4: run 252 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model. Run 5: run 253 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model. Questions: 2,198 fastino/GLiDE at a7e4589 on one DGX Spark (GB10), BF16, behind the server its checkpoint ships; its repeats 2 to 5 of 22 October 2026, also on the Stability tab. | 68.5%1,505/2,198 | 68.5%1,505/2,198 | 2,198 | Retained |
| 5 | Drex Max v1.1Retained Drex Max v1.1 · Nace.AI First run: run 326 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price. Run 2: run 329 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price. Run 3: run 330 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price. Run 4: run 331 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price. Run 5: run 332 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price. Questions: 2,198 Nace's Drex Max v1.1 API (drex-max-v1.1); its repeats 2 to 5 of 31 October 2026, also on the Stability tab. | 67.4%1,481/2,198 | 67.4%1,481/2,198 | 2,198 | Retained |
| 6 | GLiDE 2BDone GLiDE 2B · Fastino First run: run 915 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Run 2: run 1098 · 13 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Questions: 2,198 fastino/GLiDE-2B at 00f122c on one DGX Spark (GB10), BF16, behind the GLiDE server its repository ships (one request at a time), as run 915: repeat 2 on 13 November 2026 (logic's server on spark2, reloaded as on 12 November; Ümit #6410, #6416), every one of its 2,198 answers as in run 915. | 35.3%776/2,198 | 35.3%776/2,198 | 2,198 | Done |
English reading547 kept questions
| Rank | Model | Run 2% · right / questions | First run% · right / questions | Questions | Status |
|---|---|---|---|---|---|
| 1 | KnowLine-31B-Gen1.5Done KnowLine-31B-Gen1.5 · PelaAI First run: run 953 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Run 2: run 996 · 11 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Questions: 547 One DGX Spark (GB10), BF16, behind the KnowLine server (the card's serve_knowline.sh, PelaAI/KnowLine-31B-Gen1.5 at 8e56aa1), as run 953: repeat 2 on 11 November 2026 (logic's server on spark2, shared by the three benches). | 93.6%512/547 | 93.6%512/547 | 547 | Done |
| 2 | REAX 1.3Done REAX 1.3 · REAX First run: run 829 · 6 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.092 at GPU-time estimate. Run 2: run 983 · 11 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.12 at GPU-time estimate. Questions: 547 One RTX PRO 6000 Blackwell (96 GB), FP8 (block-128), behind the REAX Engine server, as run 829: repeat 2 on 11 November 2026 (logic's offline server, the same checkpoint and engine). | 92.5%506/547 | 92.5%506/547 | 547 | Done |
| 3 | GLiDE 27BDone GLiDE 27B · Fastino First run: run 901 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Run 2: run 994 · 11 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Questions: 547 One DGX Spark (GB10), BF16, behind the server its repository ships (fastino/GLiDE-27B at 4d32743, one request at a time), as run 901: repeat 2 on 11 November 2026 (logic's server on spark2, shared by the three benches). | 90.9%497/547 | 90.9%497/547 | 547 | Done |
| 4 | GLiDE v1.2Done GLiDE v1.2 · Fastino First run: run 521 · 2 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.008 at the maker's list price for the same model. Run 2: run 992 · 11 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.008 at the maker's list price for the same model. Questions: 547 One RTX PRO 6000 Blackwell (96 GB), BF16, behind the server its checkpoint ships (fastino/GLiDE at a7e4589), as run 521 (its Text tab run 248 used a DGX Spark): repeat 2 on 11 November 2026 (logic's server on bariton GPU 0, shared by the three benches). | 90.3%494/547 | 90.3%494/547 | 547 | Done |
| 5 | Drex Max v1.1Done Drex Max v1.1 · Nace.AI First run: run 399 · 2 November 2026; Nace.AI System One API; cost: $0.013 at API list price. Run 2: run 999 · 11 November 2026; Nace.AI System One API; cost: $0.013 at API list price. Questions: 547 Nace's Drex Max v1.1 API, as run 399: repeat 2 on 11 November 2026, after logic's Logic & Math repeat on the same API (its rate limit). | 88.5%484/547 | 88.5%484/547 | 547 | Done |
| 6 | LFM3-1.7B reasoning on · chatDone LFM3-1.7B reasoning on · chat · Liquid AI First run: run 1061 · 12 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.21 at the maker's list price for the same model. Run 2: run 1091 · 13 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.23 at the maker's list price for the same model. Questions: 547 One RTX PRO 6000 Blackwell (96 GB), BF16, vLLM nightly 0.34.0rc1.dev302 (commit be47189f), thinking on, with run 1061's request settings: repeat 2 on 13 November 2026 (logic's server on bariton GPU 1; Ümit #6410, #6416). Its English “I don't know” run was repeated the same way: run 1092, 374 right, 20 “I don't know”, 153 wrong (its first run 1062: 380, 26, 141). | 71.7%392/547 | 71.1%389/547 | 547 | Done |
| 7 | GLiDE 2BDone GLiDE 2B · Fastino First run: run 917 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Run 2: run 1100 · 13 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware). Questions: 547 fastino/GLiDE-2B at 00f122c on one DGX Spark (GB10), BF16, behind the GLiDE server its repository ships (one request at a time), as run 917: repeat 2 on 13 November 2026 (logic's server on spark2; Ümit #6410, #6416), every one of its 547 answers as in run 917. Its English “I don't know” run was repeated the same way: run 1102, 368 right, 47 “I don't know”, 132 wrong, every answer as in its first run 919. | 70.0%383/547 | 70.0%383/547 | 547 | Done |
| 8 | d1-2BDone d1-2B · Liquid AI First run: run 708 · 5 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.004 at GPU-time estimate. Run 2: run 1131 · 14 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.007 at GPU-time estimate. Questions: 547 LiquidAI/d1-2B at 10e7671 on one RTX PRO 6000 Blackwell (96 GB), BF16, behind d1_server.py and its model card's own call, as run 708: repeat 2 on 14 November 2026 (logic's servers.sh on bariton GPU 2, jevgammon's booking; Ümit #6410, #6416), every one of its 547 answers as in run 708. Its English “I don't know” run was repeated the same way: run 1132, 299 right, 151 “I don't know”, 97 wrong, every answer as in its first run 710. | 65.6%359/547 | 65.6%359/547 | 547 | Done |
Vision, picture only763 questions, the page's picture alone (condition B)
| Rank | Model | Run 2% · right / questions | First run% · right / questions | Questions | Status |
|---|---|---|---|---|---|
| 1 | GLiDE 27BDone GLiDE 27B · Fastino First run: run 68 · 10 November 2026; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware). Run 2: pass 2, run 6 · 11 November 2026 · 0 of 763 picks differ; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware). Questions: 763 One DGX Spark (GB10), BF16, as run 68: pass 2 (run 6 of vision/repeats/repeat-2.sqlite), 11 November 2026, on logic's server on spark2 (fastino/GLiDE-27B at 4d32743, one request at a time). | 49.9%381/763 | 49.9%381/763 | 763 | Done |
| 2 | KnowLine-31B-Gen1.5Done KnowLine-31B-Gen1.5 · PelaAI First run: run 86 · 10 November 2026; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware). Run 2: pass 2, run 8 · 11 November 2026 · 0 of 763 picks differ; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware). Questions: 763 One DGX Spark (GB10), BF16, behind the KnowLine server (the card's serve_knowline.sh, PelaAI/KnowLine-31B-Gen1.5 at 8e56aa1), as run 86: pass 2 (run 8 of vision/repeats/repeat-2.sqlite), 11 November 2026, on logic's server on spark2. | 49.8%380/763 | 49.8%380/763 | 763 | Done |
| 3 | d1-ultraDone d1-ultra · Liquid AI First run: runs 2, 5 · 27 October 2026; Liquid AI decisions API; cost: $0.026 at API list price. Run 2: pass 2, run 3 · 11 November 2026 · 181 of 763 picks differ; Liquid AI decisions API; cost: API list price. Questions: 763 Liquid AI's decisions API with the first pass's settings (3 requests in flight, 30 s timeout): pass 2 (run 3 of vision/repeats/repeat-2.sqlite), $0.0256 (610,157 input tokens at $0.042 per 1M, output free). | 49.7%379/763 | 50.1%382/763 | 763 | Done |
| 4 | d3Done d3 · vLLM Semantic Router First run: run 41 · 27 October 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; vLLM 0.32.1rc1.dev613+g8977b13a1 with --max-num-seqs 64 --gpu-memory-utilization 0.78 (another process held 18 GB of the GPU); cost: not counted (our hardware). Run 2: pass 2, run 4 · 11 November 2026 · 2 of 763 picks differ; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; vLLM 0.32.1rc1.dev613+g8977b13a1 with --max-num-seqs 64 --gpu-memory-utilization 0.78; cost: not counted (our hardware). Questions: 763 One RTX PRO 6000 Blackwell (96 GB), BF16, vLLM 0.32.1rc1.dev613+g8977b13a1 with the first pass's flags (the same server arguments as run 41's log), as run 41: pass 2 (run 4 of vision/repeats/repeat-2.sqlite), 11 November 2026, on bariton GPU 0 under jevgammon's reservation (#4873). | 48.2%368/763 | 48.5%370/763 | 763 | Done |
| 5 | Drex MaxDone Drex Max · Nace.AI First run: run 56 · 31 October 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; drex_max_server.py with Nace's drex_max at e206b27 (transformers 5.17.0, torch 2.15.0, flash-linear-attention 0.6.0; causal_conv1d not installed, so its reference PyTorch path); cost: not counted (our hardware). Run 2: pass 2, run 5 · 11 November 2026 · 0 of 763 picks differ; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; drex_max_server.py with Nace's drex_max at e206b27 (transformers 5.17.0, torch 2.15.0, flash-linear-attention 0.6.0; causal_conv1d not installed, so its reference PyTorch path); cost: not counted (our hardware). Questions: 763 One RTX PRO 6000 Blackwell (96 GB), BF16, drex_max_server.py with Nace's drex_max at e206b27 and nace-ai/drex-max-v1 at cac65e1, as run 56: pass 2 (run 5 of vision/repeats/repeat-2.sqlite), 11 November 2026, on bariton GPU 0 under jevgammon's reservation (#4873). | 46.8%357/763 | 46.8%357/763 | 763 | Done |
| 6 | Drex Max v1.1Done Drex Max v1.1 · Nace.AI First run: run 58 · 31 October 2026; Nace.AI System One API; cost: $0.026 at API list price. Run 2: pass 2, run 7 · 11 November 2026 · 0 of 763 picks differ; Nace.AI System One API; cost: API list price. Questions: 763 Nace's Drex Max v1.1 API with the first pass's settings (4 requests in flight, 30 s timeout): pass 2 (run 7 of vision/repeats/repeat-2.sqlite), 11 November 2026. | 45.1%344/763 | 45.1%344/763 | 763 | Done |
The published scores stay each setup's first run; a repeat changes no score or rank. Data: results.json (repeat_checks).
Tests beside the ranking
Hallucination (“I don't know”)
The Turkish questions with a sixth option, “I don't know”: how often a model says it does not know instead of answering wrong.
Running: 1 chat setup still running in Turkish · 2 not run
Web grounding
One model with web search off and on: Perplexity Decider 27B v1.2 · API against Perplexity Decider 27B v1.2 · API · with web.
Done: 18 runs
PWS-Decisions
Parallel Web Systems' benchmark of dated web facts, asked of the decision models and chat models; not part of the Turkish ranking.
Done: 66 setups · Waiting: 8 setups need an API key or API credit
Artificial Analysis comparison
Our numbers for the decision models of Artificial Analysis's Decision Index v1.0.
Done: 13 of its 14 models in our benchmark