Contract. Senior Software Engineer, Data Science (AI Accuracy) (NZ)
Summary
Senior software engineer builds and scales AI evaluation pipelines, prompt optimization, and dataset infrastructure for a drone/AI field-team platform.
Contract. Senior Software Engineer, Data Science (AI Accuracy) (NZ)
DroneDeploy powers field teams with robotics and AI.
Key Responsibilities
- Build and maintain robust dataset and evaluation infrastructure, including ground-truth quality controls
- Diagnose and fix measurement-path issues between offline evals and production accuracy
- Iterate on and scale existing labeling frameworks and knowledge capture systems
- Own prompt optimization across its full lifecycle, from candidate selection through production rollout and post-launch validation
- Automate eval pipelines and build the tooling to run this at scale across multiple AI products
- Create reusable internal tooling for error analysis, audits, and experimentation, and prioritize work using likely impact
Requirements
- Strong professional experience with Python for data manipulation, analysis, and tooling (e.g., pandas, NumPy)
- Solid SQL and database experience: querying, data modeling, and working with large datasets
- A data science background with a knowledge of LLM accuracy and in particular visual reasoning of these models
- Building and maintaining evaluation datasets, running and debugging evals with tools such as Braintrust or Langfuse, and conducting error analysis
- Experience designing ground-truth and labeling systems, and measuring label agreement and data quality
- Familiarity with prompt engineering and the full prompt lifecycle, from candidate selection through deployment and post-launch validation
- Ability to reason about ontology and schema design and its downstream impact on models, evals, and tooling
- Domain knowledge in construction or industrial inspection is a plus
- A collaborative mindset and a preference for iterative, team-oriented development
- Comfortable using AI-assisted tools for coding, data exploration, and debugging, while applying strong engineering judgment to guard against hallucinations and maintain high code quality.
- Drone Certification: Not required for this role.