Experiments

Repeat checks

Turkish Text2,198 questions; the reading and grammar parts are subsets of the same run

RankModelRun 2% · right / questionsFirst run% · right / questionsQuestionsStatus
1KnowLine-31B-Gen1.5Done

KnowLine-31B-Gen1.5 · PelaAI

First run: run 957 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Run 2: run 998 · 11 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Questions: 2,198

One DGX Spark (GB10), BF16, behind the KnowLine server (the card's serve_knowline.sh, PelaAI/KnowLine-31B-Gen1.5 at 8e56aa1), as run 957: repeat 2 on 11 November 2026 (logic's server on spark2, shared by the three benches).

Open its model page →

81.8%1,798/2,19881.8%1,798/2,1982,198Done
2REAX 1.3Done

REAX 1.3 · REAX

First run: run 827 · 6 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.41 at GPU-time estimate.

Run 2: run 990 · 11 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.49 at GPU-time estimate.

Questions: 2,198

One RTX PRO 6000 Blackwell (96 GB), FP8 (block-128), behind the REAX Engine server, as run 827: repeat 2 on 11 November 2026 (logic's offline server, the same checkpoint and engine).

Open its model page →

77.9%1,713/2,19877.9%1,713/2,1982,198Done
3d1-ultraRetained

d1-ultra · Liquid AI

First run: run 199 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price.

Run 2: run 200 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price.

Run 3: run 201 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price.

Run 4: run 202 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price.

Run 5: run 203 · 16 October 2026; Liquid AI decisions API; cost: $0.032 at API list price.

Questions: 2,198

Liquid AI's decisions API; its repeats 2 to 5 of 16 October 2026, also on the Stability tab.

Open its model page →

76.7%1,686/2,19877.1%1,694/2,1982,198Retained
4GLiDE v1.2Retained

GLiDE v1.2 · Fastino

First run: run 248 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model.

Run 2: run 250 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model.

Run 3: run 251 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model.

Run 4: run 252 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model.

Run 5: run 253 · 22 October 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: $0.032 at the maker's list price for the same model.

Questions: 2,198

fastino/GLiDE at a7e4589 on one DGX Spark (GB10), BF16, behind the server its checkpoint ships; its repeats 2 to 5 of 22 October 2026, also on the Stability tab.

Open its model page →

68.5%1,505/2,19868.5%1,505/2,1982,198Retained
5Drex Max v1.1Retained

Drex Max v1.1 · Nace.AI

First run: run 326 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price.

Run 2: run 329 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price.

Run 3: run 330 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price.

Run 4: run 331 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price.

Run 5: run 332 · 31 October 2026; Nace.AI System One API; cost: $0.051 at API list price.

Questions: 2,198

Nace's Drex Max v1.1 API (drex-max-v1.1); its repeats 2 to 5 of 31 October 2026, also on the Stability tab.

Open its model page →

67.4%1,481/2,19867.4%1,481/2,1982,198Retained
6GLiDE 2BDone

GLiDE 2B · Fastino

First run: run 915 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Run 2: run 1098 · 13 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Questions: 2,198

fastino/GLiDE-2B at 00f122c on one DGX Spark (GB10), BF16, behind the GLiDE server its repository ships (one request at a time), as run 915: repeat 2 on 13 November 2026 (logic's server on spark2, reloaded as on 12 November; Ümit #6410, #6416), every one of its 2,198 answers as in run 915.

Open its model page →

35.3%776/2,19835.3%776/2,1982,198Done

English reading547 kept questions

RankModelRun 2% · right / questionsFirst run% · right / questionsQuestionsStatus
1KnowLine-31B-Gen1.5Done

KnowLine-31B-Gen1.5 · PelaAI

First run: run 953 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Run 2: run 996 · 11 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Questions: 547

One DGX Spark (GB10), BF16, behind the KnowLine server (the card's serve_knowline.sh, PelaAI/KnowLine-31B-Gen1.5 at 8e56aa1), as run 953: repeat 2 on 11 November 2026 (logic's server on spark2, shared by the three benches).

Open its model page →

93.6%512/54793.6%512/547547Done
2REAX 1.3Done

REAX 1.3 · REAX

First run: run 829 · 6 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.092 at GPU-time estimate.

Run 2: run 983 · 11 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); FP8 (block-128); cost: $0.12 at GPU-time estimate.

Questions: 547

One RTX PRO 6000 Blackwell (96 GB), FP8 (block-128), behind the REAX Engine server, as run 829: repeat 2 on 11 November 2026 (logic's offline server, the same checkpoint and engine).

Open its model page →

92.5%506/54792.5%506/547547Done
3GLiDE 27BDone

GLiDE 27B · Fastino

First run: run 901 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Run 2: run 994 · 11 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Questions: 547

One DGX Spark (GB10), BF16, behind the server its repository ships (fastino/GLiDE-27B at 4d32743, one request at a time), as run 901: repeat 2 on 11 November 2026 (logic's server on spark2, shared by the three benches).

Open its model page →

90.9%497/54790.9%497/547547Done
4GLiDE v1.2Done

GLiDE v1.2 · Fastino

First run: run 521 · 2 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.008 at the maker's list price for the same model.

Run 2: run 992 · 11 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.008 at the maker's list price for the same model.

Questions: 547

One RTX PRO 6000 Blackwell (96 GB), BF16, behind the server its checkpoint ships (fastino/GLiDE at a7e4589), as run 521 (its Text tab run 248 used a DGX Spark): repeat 2 on 11 November 2026 (logic's server on bariton GPU 0, shared by the three benches).

Open its model page →

90.3%494/54790.3%494/547547Done
5Drex Max v1.1Done

Drex Max v1.1 · Nace.AI

First run: run 399 · 2 November 2026; Nace.AI System One API; cost: $0.013 at API list price.

Run 2: run 999 · 11 November 2026; Nace.AI System One API; cost: $0.013 at API list price.

Questions: 547

Nace's Drex Max v1.1 API, as run 399: repeat 2 on 11 November 2026, after logic's Logic & Math repeat on the same API (its rate limit).

Open its model page →

88.5%484/54788.5%484/547547Done
6LFM3-1.7B reasoning on · chatDone

LFM3-1.7B reasoning on · chat · Liquid AI

First run: run 1061 · 12 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.21 at the maker's list price for the same model.

Run 2: run 1091 · 13 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.23 at the maker's list price for the same model.

Questions: 547

One RTX PRO 6000 Blackwell (96 GB), BF16, vLLM nightly 0.34.0rc1.dev302 (commit be47189f), thinking on, with run 1061's request settings: repeat 2 on 13 November 2026 (logic's server on bariton GPU 1; Ümit #6410, #6416). Its English “I don't know” run was repeated the same way: run 1092, 374 right, 20 “I don't know”, 153 wrong (its first run 1062: 380, 26, 141).

Open its model page →

71.7%392/54771.1%389/547547Done
7GLiDE 2BDone

GLiDE 2B · Fastino

First run: run 917 · 10 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Run 2: run 1100 · 13 November 2026; 1 × DGX Spark, GB10 (128 GB unified memory); BF16; cost: not counted (our hardware).

Questions: 547

fastino/GLiDE-2B at 00f122c on one DGX Spark (GB10), BF16, behind the GLiDE server its repository ships (one request at a time), as run 917: repeat 2 on 13 November 2026 (logic's server on spark2; Ümit #6410, #6416), every one of its 547 answers as in run 917. Its English “I don't know” run was repeated the same way: run 1102, 368 right, 47 “I don't know”, 132 wrong, every answer as in its first run 919.

Open its model page →

70.0%383/54770.0%383/547547Done
8d1-2BDone

d1-2B · Liquid AI

First run: run 708 · 5 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.004 at GPU-time estimate.

Run 2: run 1131 · 14 November 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; cost: $0.007 at GPU-time estimate.

Questions: 547

LiquidAI/d1-2B at 10e7671 on one RTX PRO 6000 Blackwell (96 GB), BF16, behind d1_server.py and its model card's own call, as run 708: repeat 2 on 14 November 2026 (logic's servers.sh on bariton GPU 2, jevgammon's booking; Ümit #6410, #6416), every one of its 547 answers as in run 708. Its English “I don't know” run was repeated the same way: run 1132, 299 right, 151 “I don't know”, 97 wrong, every answer as in its first run 710.

Open its model page →

65.6%359/54765.6%359/547547Done

Vision, picture only763 questions, the page's picture alone (condition B)

RankModelRun 2% · right / questionsFirst run% · right / questionsQuestionsStatus
1GLiDE 27BDone

GLiDE 27B · Fastino

First run: run 68 · 10 November 2026; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware).

Run 2: pass 2, run 6 · 11 November 2026 · 0 of 763 picks differ; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware).

Questions: 763

One DGX Spark (GB10), BF16, as run 68: pass 2 (run 6 of vision/repeats/repeat-2.sqlite), 11 November 2026, on logic's server on spark2 (fastino/GLiDE-27B at 4d32743, one request at a time).

Open its model page →

49.9%381/76349.9%381/763763Done
2KnowLine-31B-Gen1.5Done

KnowLine-31B-Gen1.5 · PelaAI

First run: run 86 · 10 November 2026; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware).

Run 2: pass 2, run 8 · 11 November 2026 · 0 of 763 picks differ; 1 × DGX Spark (GB10); BF16; cost: not counted (our hardware).

Questions: 763

One DGX Spark (GB10), BF16, behind the KnowLine server (the card's serve_knowline.sh, PelaAI/KnowLine-31B-Gen1.5 at 8e56aa1), as run 86: pass 2 (run 8 of vision/repeats/repeat-2.sqlite), 11 November 2026, on logic's server on spark2.

Open its model page →

49.8%380/76349.8%380/763763Done
3d1-ultraDone

d1-ultra · Liquid AI

First run: runs 2, 5 · 27 October 2026; Liquid AI decisions API; cost: $0.026 at API list price.

Run 2: pass 2, run 3 · 11 November 2026 · 181 of 763 picks differ; Liquid AI decisions API; cost: API list price.

Questions: 763

Liquid AI's decisions API with the first pass's settings (3 requests in flight, 30 s timeout): pass 2 (run 3 of vision/repeats/repeat-2.sqlite), $0.0256 (610,157 input tokens at $0.042 per 1M, output free).

Open its model page →

49.7%379/76350.1%382/763763Done
4d3Done

d3 · vLLM Semantic Router

First run: run 41 · 27 October 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; vLLM 0.32.1rc1.dev613+g8977b13a1 with --max-num-seqs 64 --gpu-memory-utilization 0.78 (another process held 18 GB of the GPU); cost: not counted (our hardware).

Run 2: pass 2, run 4 · 11 November 2026 · 2 of 763 picks differ; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; vLLM 0.32.1rc1.dev613+g8977b13a1 with --max-num-seqs 64 --gpu-memory-utilization 0.78; cost: not counted (our hardware).

Questions: 763

One RTX PRO 6000 Blackwell (96 GB), BF16, vLLM 0.32.1rc1.dev613+g8977b13a1 with the first pass's flags (the same server arguments as run 41's log), as run 41: pass 2 (run 4 of vision/repeats/repeat-2.sqlite), 11 November 2026, on bariton GPU 0 under jevgammon's reservation (#4873).

Open its model page →

48.2%368/76348.5%370/763763Done
5Drex MaxDone

Drex Max · Nace.AI

First run: run 56 · 31 October 2026; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; drex_max_server.py with Nace's drex_max at e206b27 (transformers 5.17.0, torch 2.15.0, flash-linear-attention 0.6.0; causal_conv1d not installed, so its reference PyTorch path); cost: not counted (our hardware).

Run 2: pass 2, run 5 · 11 November 2026 · 0 of 763 picks differ; 1 × RTX PRO 6000 Blackwell (96 GB); BF16; drex_max_server.py with Nace's drex_max at e206b27 (transformers 5.17.0, torch 2.15.0, flash-linear-attention 0.6.0; causal_conv1d not installed, so its reference PyTorch path); cost: not counted (our hardware).

Questions: 763

One RTX PRO 6000 Blackwell (96 GB), BF16, drex_max_server.py with Nace's drex_max at e206b27 and nace-ai/drex-max-v1 at cac65e1, as run 56: pass 2 (run 5 of vision/repeats/repeat-2.sqlite), 11 November 2026, on bariton GPU 0 under jevgammon's reservation (#4873).

Open its model page →

46.8%357/76346.8%357/763763Done
6Drex Max v1.1Done

Drex Max v1.1 · Nace.AI

First run: run 58 · 31 October 2026; Nace.AI System One API; cost: $0.026 at API list price.

Run 2: pass 2, run 7 · 11 November 2026 · 0 of 763 picks differ; Nace.AI System One API; cost: API list price.

Questions: 763

Nace's Drex Max v1.1 API with the first pass's settings (4 requests in flight, 30 s timeout): pass 2 (run 7 of vision/repeats/repeat-2.sqlite), 11 November 2026.

Open its model page →

45.1%344/76345.1%344/763763Done

The published scores stay each setup's first run; a repeat changes no score or rank. Data: results.json (repeat_checks).

Tests beside the ranking