decision model · v1 set (earlier version)

Open-Jev 27B

Earlier version

Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
Missing
Provider or endpoint
our GPUs
Date run
Missing
Price
no price per token; own hardware; price date not recorded
Model type
decision model
Weights
open
Runs on
our GPUs
Reasoning setting
none (decision model)
Confidence source
decision-model probability
Rules text
short (the v1 rules text; the same stored request text for every model)
Option groups
not recorded; no decision in this data was split
Sampling settings
not sent (decision request)
Notes
Open-Jev-27B-v1.1: a LoRA adapter and decision head on a 27B base model; 4,096-token limit per scored sequence; its server takes no model field

Missing B7 fields: option_groups, date_run

Results in v1 set (earlier version)

Earlier version: plays were listed in the runner's order, which tends to put the best play early. When Gemini 4 Argon answered the same 1,000 positions with the plays shuffled, its error rate changed by +1.9 (95% interval −0.2 to +4.2), so the effect is not proven.

Error rate (PR)
59.954.1 to 66.1
Decisions
1,000 (1,000 scored, 0 invalid)
Engine's best play
39.9%
Blunder rate
36.4%
P95 equity loss
0.593
Mean loss, worst 5%
0.788
Median / P90 s per decision
1.38 / 1.91
$ per 1,000 decisions
Not measured
Risk judgement, margin 0.001
53.1 (95% interval 49.6 to 56.5); risks found 40.3% (143 / 355), right safety 65.9% (243 / 369)
Risk judgement, margin 0.02
51.9 (95% interval 48.3 to 55.3); risks found 38.7% (123 / 318), right safety 65.0% (264 / 406)
No trade-off
276 (left out of every risk metric)

JSON

Risk call card

53.1risk judgement, margin 0.001

right risks143
wrong risks86
needless risks126
missed risks126
right safety243

Hit risk only: being hit. Gammon risk and priming are not in it.

By game phase

PhaseError rate (PR)Decisions
OpeningToo few decisions143
Contact82.0641
Holding gameToo few decisions20
RaceToo few decisions136
Bear-offToo few decisions60