Member of Technical Staff - Data Quality Engineer Pre-training
Posted 3
views
Summary
Reflection is hiring a Data Quality Engineer for LLM pre-training to define measurable data-quality standards, build automated QA pipelines and human-in-the-loop checks, and monitor quality trends using Python and ML/LLM workflows. On-site roles in San Francisco, New York, or London.
You will define and operationalize measurable data-quality standards for LLM pre-training. You will analyze data, build and validate automated QA methods alongside human-in-the-loop processes, create reusable quality pipelines, provide vendor feedback, and monitor quality trends to improve acceptance criteria.
Responsibilities
- Own upstream data quality for LLM pre-training
- Translate requirements into measurable quality signals
- Provide actionable feedback to external data vendors
- Design, validate, and scale automated QA methods
- Build reusable QA pipelines for model training
- Monitor and report data quality over time
- Improve quality standards, processes, and acceptance criteria
Requirements
- Experience building data pipelines, QA systems, or evaluation workflows for pre-training data
- Understanding of data quality impacts on pre-training
- Experience designing automated quality checks
- Proficiency in Python
- Experience building ML or LLM workflows
- Experience with large datasets and automated evaluation or quality-checking systems
- Familiarity with LLM training and evaluation
- Communication skills
Benefits
- Stock options
- Medical, dental, vision, and life insurance
- Annual wellness allowance
- Daily office lunch and dinner
- 22 weeks of paid parental leave
- Unlimited paid time off in the U.S.
- 30 vacation days in the U.K.
- Visa sponsorship support
- Regular off-sites, happy hours, and team celebrations
