Senior AI Evaluation & Data Infra Engineer - Auckland
Posted Updated
Description
Procore seeks a Senior Software Engineer for its Data Science team in Auckland, aiming to improve AI accuracy across the platform. You will own data, tooling, and evaluation infrastructure, focusing on measuring model quality and production readiness.
You will collaborate with engineering, product, and AI teams, using Python, SQL, and evaluation tools to raise AI product quality. The role requires in-office presence at least two days weekly in Auckland.
Build and maintain dataset and evaluation infrastructure with ground-truth controls. Diagnose measurement-path issues between offline evals and production accuracy. Own prompt optimization across the lifecycle, from candidate selection to production rollout and post-launch validation. Automate eval pipelines and tooling to run at scale across multiple AI products. Create reusable tooling for error analysis, audits, and experimentation. Iterate on labeling frameworks and knowledge capture systems. Analyze ontology/schema changes and their propagation across models and infrastructure. Prioritize work by likely impact. Strong Python for data manipulation, analysis, and tooling (pandas, NumPy). Solid SQL and database experience with large datasets. Data science background with focus on LLM accuracy, including visual reasoning. Equity compensation Bonus incentive compensationSkills
Sign in to read and join the discussion.