Search Quality Analyst (Member of Technical Staff)
Posted Updated 18
views
Description
- Perplexity is looking for an experienced analyst to help us build and improve our core search technologies
- You’ll work at the intersection of data analysis and engineering — designing metrics, building data pipelines, and improving the quality of our search and answer systems
- You’ll find and diagnose quality issues in our search pipeline
- Design metrics from scratch to track and measure search quality
- Build datasets for model training, including LLM-as-a-judge labeling pipelines
- Improve search snippet quality and page selection algorithms for indexing
- Design and analyze A/B experiments to validate improvements
- 4+ years of experience as a data analyst or in a related role
- Proficiency with SQL and Python
- Building labeling pipelines using LLM-as-a-judge
- Strong coding skills — expected to write production-grade code at a mid-level backend engineer level
- Designing metrics from scratch (not just analyzing existing A/B experiments)
- Designing evals with known ground truth (e.g. SimpleQA, BrowseComp) or driving meaningful improvements on such evals
- Training ML models that shipped to production with measurable metric improvements
- Demonstrated hands-on experience with at least one of the following:
- Knowledge of statistics and A/B experiment design
- Experience with Apache Spark or Databricks
- Experience working on search-related products
