chat model, answer-letter probabilities · New models: first look (50 smoke positions)
Beam 1.0high
Smoke test
Results f24d1f41, updated 16 November 2026, 22:20 UTC; scored by GNU Backgammon 1.08, 2-ply.
How we called it
- Model ID
- reflection/beam-1.0-501b-a23b
- Provider or endpoint
- OpenRouter, pinned to Nebius (fp8)
- Date run
- 2026-10-21
- Price
- $0.25 / $1.0 per million tokens in / out; API price; price date 2026-10-21
- Model type
- chat model, answer-letter probabilities
- Weights
- open
- Runs on
- API
- Reasoning setting
- reasoning effort high (reasoning is mandatory)
- Confidence source
- letter probability
- Rules text
- long
- Option groups
- one request (no split)
- Sampling settings
- temperature 1.0
- Notes
- Missing
Smoke test
- Positions answered
- 50 of 50 (0 failed, 0 timed out)
- Invalid answers
- 0
- Error rate (PR)
- 15.0 (95% interval 5.9 to 26.7)
- Median / P90 request s
- 34.5 / 72.9
- $ per 1,000 decisions
- 8.67