Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Procore Technologies

Senior LLM Evaluation & Data Quality Engineer

Posted Updated

Description

Procore Technologies in Auckland is seeking a Senior Software Engineer for its Data Science team to own the accuracy of LLM-powered features that convert drone and robotics data into actionable insights.

You will collaborate with engineering, product, and AI teams to build data, tooling, and evaluation infrastructure, using Python, SQL, and LLM evaluation to improve product quality and reliability. This role reports to the Director of Engineering and is based in our Auckland office.

Build and maintain dataset and evaluation infrastructure with quality controls. Diagnose gaps between offline evals and production accuracy. Own prompt optimization across lifecycle from candidate selection to production. Automate eval pipelines and scalable tooling for multiple AI products. Create internal tooling for error analysis and experiments. Iterate on labeling frameworks and schema propagation analyses. Prioritize work by likely impact and align with teams. Strong Python and SQL experience with data manipulation and tooling. Background in data science with focus on LLM accuracy and evaluation. Experience building evaluation datasets and error analysis tools. Familiarity with ground-truth labeling and data quality checks. Knowledge of prompt engineering and ontology/schema design. Equity compensation Bonus incentive

Skills

What Senior AI Engineering jobs ask for — and how much of it you have →
Apply

See also

AI Engineering jobs by country — openings, pay and top skills →