decision model · v1 set (earlier version)

Jev 1.13

Earlier version

Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
jev-1.13.0
Provider or endpoint
TypeSafe System One API
Date run
Missing
Price
$0.042 / $0.0 per million tokens in / out; API price; price date 2026-10-08
Model type
decision model
Weights
Missing
Runs on
API
Reasoning setting
none (decision model)
Confidence source
decision-model probability
Rules text
short (the v1 rules text; the same stored request text for every model)
Option groups
not recorded; no decision in this data was split
Sampling settings
not sent (decision request)
Notes
Missing

Missing B7 fields: weights, option_groups, date_run

Results in v1 set (earlier version)

Earlier version: plays were listed in the runner's order, which tends to put the best play early. When Gemini 4 Argon answered the same 1,000 positions with the plays shuffled, its error rate changed by +1.9 (95% interval −0.2 to +4.2), so the effect is not proven.

Error rate (PR)
33.629.6 to 37.9
Decisions
1,000 (1,000 scored, 0 invalid)
Engine's best play
47.8%
Blunder rate
24.0%
P95 equity loss
0.308
Mean loss, worst 5%
0.531
Median / P90 s per decision
0.25 / 0.29
$ per 1,000 decisions
Not measured
Risk judgement, margin 0.001
59.1 (95% interval 56.3 to 62.0); risks found 28.7% (102 / 355), right safety 89.4% (330 / 369)
Risk judgement, margin 0.02
58.6 (95% interval 55.7 to 61.5); risks found 28.6% (91 / 318), right safety 88.7% (360 / 406)
No trade-off
276 (left out of every risk metric)

JSON

Risk call card

59.1risk judgement, margin 0.001

right risks102
wrong risks28
needless risks39
missed risks225
right safety330

Hit risk only: being hit. Gammon risk and priming are not in it.

By game phase

PhaseError rate (PR)Decisions
OpeningToo few decisions143
Contact39.2641
Holding gameToo few decisions20
RaceToo few decisions136
Bear-offToo few decisions60