chat model, answer-letter probabilities · New models: first look (50 smoke positions)

Beam 1.0low

Smoke test

Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.

How we called it

Model ID
reflection/beam-1.0-501b-a23b
Provider or endpoint
OpenRouter, pinned to Nebius (fp8)
Date run
2026-10-21
Price
$0.25 / $1.0 per million tokens in / out; API price; price date 2026-10-21
Model type
chat model, answer-letter probabilities
Weights
open
Runs on
API
Reasoning setting
reasoning effort low (reasoning is mandatory)
Confidence source
letter probability
Rules text
long
Option groups
one request (no split)
Sampling settings
temperature 1.0
Notes
Missing

Smoke test

Positions answered
50 of 50 (0 failed, 0 timed out)
Invalid answers
0
Error rate (PR)
15.1 (95% interval 7.3 to 24.2)
Median / P90 request s
7.7 / 16.2
$ per 1,000 decisions
1.99