Pre-registration · addendum · 2026-11-09

Decision models (added 9 Nov)

← Results: Did they improve?

Results will be published whichever way they go.

The decision models are pre-registered in an addendum to the plan, committed before their first call. Its rule, cohort, writer, money and open decisions follow as written (sections 1, 2, 3, 4, 7, 9).

Addendum: Did they improve - decision models - addendum 2026-11-09.md, sha256 b370f4dc9660a618aece4586b8c8f5259688372fab580305b4454aaa2d71a967

1. Question

For each decision model of the cohort (§3), at its leaderboard setting: does it take risks better on the pilot's 200-position test after 10 games against GNUbg weak (GNU Backgammon 1.08 Beginner), with a block of text about its own earlier games in its prompt, than with no block?

There are two arms, each judged on its own:

Each arm has its own games: same dice schedules, never the other arm's games, decisions or trajectories. Notes and examples reset at the start of every series. Games count nowhere (no league, no leaderboard).

Verdicts are reported per arm, under the same rule. Neither arm is picked as "the" result.

2. The rule (verbatim from the live plan §2; decided by Ümit, #3060 and #3065)

Improved = both of these hold:

  1. risks found with lessons (the mean of the 5 tests after game 10) minus risks found with no memory (the mean of the 3 empty runs) is more than max(range of the model's own 3 empty runs, 5 points);
  2. right safety with lessons minus right safety with no memory is ≥ −5 points (right safety drops by no more than 5 points).

For Part 3, "with lessons" means "with the arm's block" (notes in Arm N, worst mistakes in Arm W). "With no memory" means that arm's own empty runs (§6).

3. Cohort (Ümit #3744, #3746 and #3758)

Each row is one model at its leaderboard setup, frozen from v3.db players.settings: setup name, route, model id sent, request parameters and option cap. The only changes are name, parallel (never above the league's), the block file, this experiment's budget ledger (never the league's), the route check and the context gate.

The allowed response ids are the ids observed in the league's own replies. There is no successor, alias or substitute (D7). A reply naming another id stops the arm: "not tested: model id not served as on the leaderboard". There is no hosted/local swap.

Setup (v3.db)Chosen byRouteResponse idDelivery evidenceStatus before preflight
REAX 1.3Ümit #3744local reax13/serve.py (reax-engine @ ff8f595)reax13pending Lrneeds D5 (read-twice → read-once switch); local render check; a GPU
GLiDEv1.2Ümit #3744local glide/server.pyglide-v1.2pending Lrlocal render check; a GPU
d1-ultraÜmit #3744api.liquid.ai/decisions/v1/systemone, d1-ultrad1-ultradelivery unverified (our evidence gap)selected and shown; not tested: native context limit unverified (Zeki #3761; §6.1, §6.2)
Drex Max v1.1Ümit #3744Nace console.nace.ai/v1/systemone, drex-max-v1.1, 25 optionsdrex-max-v1.1HcP1, then runs; Nace subcap (§7)
d3-ultraÜmit #3744———not started: added 9 Nov, no league record, no backgammon route yet
Mammoth-1Ümit #3744local baselabs_server.py (baselabs-serve:81ce62f), 120 optionsbaselabs/mammoth-1pending Lrneeds D5 (both arms past the trained 16,384 tokens); local render check; a GPU
Elephant-1Ümit #3746local, as Mammoth-1baselabs/elephant-1pending Lrneeds D5 for Arm N (past the trained 16,384 tokens); local render check; a GPU
Perplexity Decider 27B v1.2Ümit #3744local pplx_decider_server.py (decider-v1.2.sh 27b), 25 optionspplx-decider-v1.2pending Lrlocal render check; a GPU
Jev 1.14Ümit #3744api.typesafe.ai/v1/systemone, jev-1.14.0jev-1.14.0Hcnot tested: conservative context guard (§6.2: the guard refuses 1.3% of its league requests even without a block)
SubQ Snap 1.1Ümit #3758api.subq.ai/v1/systemone, subq-snap-1.1subq-snap-1.1HcP1, then runs

Every decision model of the leaderboard outside this list is out of scope for Part 3. Its row stays as the live plan states it.

Page notes:

4. The writer (Arm N only)

Qwen3.8 Flashoff via OpenRouter, pinned once for the whole cohort, no swap mid-cohort. It is the live plan's Qwen3.8 Flashoff lesson call, unchanged (improve.py player_config of that setup):

Prompt: the pilot's lessons_system.txt + lessons_user.txt, byte-identical (sha256 d755001513b60546cbcc4159db0965dbeda09a65d65377d874b22e247f74550e).

Cut rule (the pilot's): lines starting - are lessons; at most 5 lessons and 150 words are kept; cuts are counted and shown. The writer's raw reply, the kept notes and the exact block delivered (by sha256) are stored privately. After the cut rule nothing in a block is ever shortened.

What the writer reads: the live plan's lesson record (improve.py record_line, game_header, eligible) of that model's game in Arm N, scored by GNUbg 1.08 2-ply. Left out:

The block (Arm N), at the start of state (OpenAI: input), then a blank line, then the league state unchanged:

=== NOTES FROM YOUR EARLIER GAMES IN THIS SERIES ===
Notes a helper wrote from your earlier games in this series, oldest first:

After game 1:
<kept notes>

…
=== END OF NOTES ===

Writer calls are booked in the same ledgers as the players.

7. Money

9. Decisions

Results will be published whichever way they go.