Member of Technical Staff, Head of Quality
Summary
Hands-on quality lead at an applied AI research company building training environments and datasets: the person reviews human data and environment outputs, builds automated evals, graders and QA agents, defines quality metrics, owns delivery acceptance, and manages a growing quality team. Fully onsite in San Francisco.
About the company
Our client is an applied AI research company developing training environments and datasets. Its technical teams build the systems that help AI models learn from complex tasks and data.
The role
Raydar is recruiting for this opportunity through the Paraform network. The position is with our client. Own quality across environments and datasets as a Member of Technical Staff, Head of Quality. This is a hands-on evaluation and data-review role with responsibility for automation and a growing quality team.
What you'll do
- Review human data and environment outputs in detail.
- Build automated evaluations, graders, and QA agents.
- Detect reward hacking, broken verifiers, and distribution mismatches.
- Define measurable quality metrics and acceptance criteria.
- Own delivery acceptance for customer projects.
- Hire, onboard, and manage quality engineers and reviewers.
Requirements
What we're looking for
- 1 to 5 years of relevant LLM evaluation, QA engineering, or ML data-quality experience.
- Experience building automated quality systems for ML data, agents, or large-scale pipelines.
- Technical understanding of what makes training data and environments useful for learning.
- Hands-on work with LLM evaluations, annotation, or human-in-the-loop review.
- Ability to build graders, judge models, telemetry, and dashboards.
- Strong attention to detail and willingness to spend substantial time reviewing data manually.
Bonus points
- RL environment, verifier, or reward-design experience.
- Founder or zero-to-one product-building experience.
- A STEM degree.
Benefits
Compensation and benefits
- Base salary: USD 180,000 to 280,000 per year.
- Equity: Competitive equity.
Location and work model
- San Francisco, California, United States.
- Fully onsite role; exact weekly office-day count is not specified.
- Visa transfers are supported.
Skills
As published by workable · 3 questions
Basics
First name, Last name, Email, Headline, Phone, Address, Photo, Education, Experience, Summary, Resume, Cover letter
Short answers (1)
- Public LinkedIn profile URL (https://www.linkedin.com/in/...)
Pick from a list (2)
- Will you require work authorization of any kind?
- Are you able to work onsite in San Francisco?