Machine Learning Research Scientist, Evaluations
Scale AI · San Francisco, CA · Seattle, WA
- Level
- Not stated
- Type
- full-time
- Mode
- On-site / hybrid
- Posted
- 2026-08-26
- Team
- Research
Role description
Scale works with the industry's leading AI labs to provide high quality data and accelerate progress in GenAI research. We are looking for Research Scientists and Research Engineers with expertise in LLM post-training (SFT, RLHF, reward modeling) and evaluation. This role is on the evaluation pod within the GenAI Research Organization and will focus on building benchmarks and diagnosing model failure modes in both text and multimodal modalities. In this role, you will develop rigorous evaluations and diagnostic methods that reveal where frontier models fail and why. You will collaborate with researchers and engineers to define best practices in evaluation-driven AI development.…
Technologies named in the posting
How Rolefit scores this role
- Stack overlap (50%). Every technology named in the posting is compared with your selected stack, weighted by how rare it is across the whole index — a niche match counts for more than Docker.
- Seniority fit (16%). The level in the posting is compared with the level assessed from your years and depth.
- Title alignment (16%). Whether the role is actually pitched at your field rather than adjacent to it.
- Freshness and filters (18%). Newer postings, your work type and your target region all move the score.
The score you see on a shortlist is specific to you — the same posting scores differently for a different stack. Run the assessment once and this role either makes your top 30 or it does not.
Build my shortlistSimilar roles
- 7d agoApply ↗
- Frontier Agent Engineering Manager, Enterprise
Scale AI · London, UK
Team leadershipPrompt engineeringETL / ELTRAG10d agoApply ↗ - 24d agoApply ↗
- Engineering Manager, Customer Platform (GenAI)
Scale AI · San Francisco, CA · New York, NY
LLMsTeam leadershipSystem design1mo agoApply ↗ - 1mo agoApply ↗
- 1mo agoApply ↗
Your turn
See the 30 roles that actually fit your stack.
Three steps, no account, no CV upload. Every match opens with its score breakdown and a cover letter written for it.