chat model, answer-letter probabilities · New models: first look (50 smoke positions)
Beam 1.0low
Smoke test
Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- reflection/beam-1.0-501b-a23b
- Provider or endpoint
- OpenRouter, pinned to Nebius (fp8)
- Date run
- 2026-10-21
- Price
- $0.25 / $1.0 per million tokens in / out; API price; price date 2026-10-21
- Model type
- chat model, answer-letter probabilities
- Weights
- open
- Runs on
- API
- Reasoning setting
- reasoning effort low (reasoning is mandatory)
- Confidence source
- letter probability
- Rules text
- long
- Option groups
- one request (no split)
- Sampling settings
- temperature 1.0
- Notes
- Missing
Smoke test
- Positions answered
- 50 of 50 (0 failed, 0 timed out)
- Invalid answers
- 0
- Error rate (PR)
- 15.1 (95% interval 7.3 to 24.2)
- Median / P90 request s
- 7.7 / 16.2
- $ per 1,000 decisions
- 1.99