About the Job
We launched in January 2023 with the first human-like AI voice model. Today, we serve millions of users and thousands of businesses - from fast-growing startups to large enterprises like Deutsche Telekom and Meta. Our investors are some of the world's most prominent, including Andreessen Horowitz, ICONIQ Growth and ...
We have expanded from voice into three main platforms:
ElevenAgents enables businesses to deliver seamless and intelligent customer experiences, with the integrations, testing, monitoring, and reliability necessary to deploy voice and chat agents at scale.
Key Responsibilities
We're looking for a Data Scientist to take ownership of the datasets and evaluation workflows that underpin our AI Safety work. You'll turn real-world product data into reliable, well-structured datasets for training and evaluating models, and help build the processes and infrastructure to continuously assess how those models perform in production.
This is a hands-on role at the intersection of data science, ML, AI safety, and policy. You'll work closely with safety researchers, engineers, and policy specialists to translate complex safety requirements into practical data and evaluation systems - starting hands-on with collection, analysis, and labelling, then building the methodologies, contributor networks, and quality standards that let this work scale.
Own safety datasets end-to-end: collection, cleaning, labelling, quality control, versioning, and readiness for training and evaluation.
Translate safety policy into clear, consistent labelling and evaluation criteria, working closely with policy specialists.
Design and manage labelling processes, including sourcing, onboarding, and overseeing external contributors to a high quality bar.
Build evaluation workflows for models in production, using real-world data to track performance and surface issues.
Develop lightweight Python/SQL pipelines and tooling to make data work faster and reproducible, partnering with ML engineers on what "training-ready" looks like.
Required Skills & Abilities
Experience as a Data Scientist, or similar, working with real-world datasets and ML models.
Strong grasp of what makes a good dataset: collection, cleaning, sampling, labelling, quality control, evaluation.
Comfortable with Python and SQL for analysis and practical data workflows.
Strong critical thinking and judgement - comfortable with ambiguity and nuance, and able to turn complex guidelines into consistent, scalable decisions.
Autonomous and a clear communicator, able to work across disciplines (policy, engineering, research) in a fast-moving, still-forming environment.
Experience in AI safety, trust & safety, content moderation, or fraud/abuse detection.
Experience building datasets or evaluation frameworks, or with adversarial/safety-focused ML.
Experience managing human-in-the-loop labelling processes, dataset versioning, or ML pipeline tooling.
Experience operationalising guidelines with policy or legal teams.
