a16z AI Evaluation Engineer — Scale QA & Metrics
Summary
QA/AI Evaluation Engineer running large-scale evaluations of AI-assisted solutions, measuring factual grounding and accuracy, building metrics frameworks, and automating evaluation pipelines in CI/CD.
Appnovation Technologies in Toronto is seeking a QA / AI Evaluation Engineer to join a forward-leaning team that delivers high-quality AI-assisted solutions.
You will run evaluations at scale across large question sets, measure factual grounding and accuracy lift, and build the metrics framework that shows incremental improvement. You will design load tests and automate evaluation pipelines within CI/CD, collaborating with engineering and data science to reproduce and verify fixes.