Hybrid AI Data Quality Analyst: Human Data for AI
Posted Updated
Description
Prolific is seeking a Human Data Quality Analyst, AI Business, to ensure high-quality human data powering AI research. You will perform analytical QA on datasets, identify failure modes, and translate findings into practical guidance for cross-functional teams.
You’ll work across data, engineering and operations to build quality controls, validate data pipelines, and help shape new data programs from design to launch. This role offers hybrid working and exposure to cutting-edge AI data workflows.
Run day-to-day quality measurement across live human-data programmes, combining statistical analysis with regular hands-on review to determine whether the data is fit for its intended use. Find the signal behind quality issues by investigating changes, quantifying their impact and distinguishing errors from wider problems in guidance, task design, training or tooling. Help build the data-quality pipeline alongside our quality engineers, turning repeated analysis into reliable validations and automated systems, including rule-based, LLM, and ML assisted approaches. Turn complex analysis into a clear story through reporting and visualisations that help stakeholders understand what is happening, why it matters, and what action the evidence supports. Support new programmes from design through to launch by helping develop rubrics, guidelines, gold sets, calibration exercises, and production gates to measure quality from the start. Use what we learn from live programmes to improve annotation design, guidance, training and quality processes, then measure whether those changes work. 1–2 years of practical data analysis in a QA or data-focused role. Proficient in SQL and Python for querying and analysis. Strong understanding of statistics (sampling, distributions). Ability to translate findings into actionable recommendations for non-technical stakeholders. Competitive salary Hybrid working Benefits packageSkills
Sign in to read and join the discussion.