Member of Technical Staff, Head of Quality
Summary
Plato, an applied research lab building simulated environments and data infrastructure for training specialized AI agents, is hiring a hands-on Head of Quality in San Francisco. They review datasets and environment outputs, build automated QA/evaluation systems that catch reward hacking and broken verifiers, and hire and lead the QA team.
Member of Technical Staff, Head of Quality
Company: Confidential — venture-backed technology company
Location: San Francisco, CA (SoMa office, 5 days per week)
Compensation: $180,000 - $280,000 + competitive equity
Employment Type: Full-time
Visa Sponsorship: Visa transfers (OPT, H-1B transfer)
About the Company
The client is an applied research lab building the infrastructure to train specialized AI agents. It turns real-world data streams into high-fidelity simulated environments that generate the training signal needed for capable models, supporting frontier labs, hyperscalers and enterprises. As compute and algorithms commoditize, reinforcement learning data remains the bottleneck, and the client automatically scales it.
The client has raised a $19M seed.
The Role
The client is hiring a Head of Quality (1-5 years) to own the quality bar for every environment and dataset it delivers, and to build the systems and team that enforce it at scale. This is hands-on: you will be deep in the data reviewing samples, building automated QA, and making sure what ships produces real learning signal. You will also hire and lead the QA team as the client scales.
What You Will Do
- Review human data and environment outputs hands-on to keep quality extremely high.
- Build automated evaluation, QA agents and grading systems that catch reward hacking, broken verifiers and distribution mismatches before delivery.
- Define quality metrics and acceptance criteria for environments, tasks, verifiers, rewards and datasets.
- Own delivery acceptance for engagements with frontier labs, hyperscalers and enterprises.
- Hire, onboard and manage QA engineers and reviewers.
What You Bring
- 1-5 years of experience in ML, RL, data quality or evaluation
- Understanding of what a dataset actually teaches a model, beyond checklists
- Experience building automated QA or evaluation systems
- Ability to work in the San Francisco office 5 days a week
Tech Stack
Machine learning, reinforcement learning, LLMs, data quality, QA automation, evaluation frameworks
Before you apply: get a free recruiter-readiness score for your resume at CareerStanding — about a minute, no signup, and your resume stays in your browser.
