Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Unavailable

Principal ML Engineer: Agent Quality & AI Systems

Posted Updated

Description

Atlassian is seeking a Principal Machine Learning Engineer to set the technical direction for AI agent quality in Studio. This hands-on leadership role defines how we measure agent quality and leads efforts to improve it, across production systems.

The team operates in Sydney and Melbourne with remote options, collaborating with product, data science and central AI teams to advance applied ML on real agentic systems in production.

Own the agent quality architecture: define measurement, sampling, and roll-up of results. Design for cost and accuracy: determine deterministic checks vs. heuristics vs. model-based judgments. Architect evaluation and experimentation: benchmarks, datasets, evaluation environments, observability. Make quality part of shipping: regression gates, pre-release checks, drift detection. Lead the hardest quality problems hands-on (e.g., agent routing, tool-use reliability). Set multi-quarter roadmap with product and data science, align with central AI teams. Grow ML craft: mentor engineers through design reviews and hands-on work. 6+ years in ML/AI engineering with deep expertise in LLMs, NLP, information retrieval or recommendation. Proven track record of building and shipping LLM-powered or agentic systems used by real customers. Strong evaluation and experimental skills with benchmarks and leakage detection. Strong software and data engineering skills with data pipelines and production observability. Strong product judgment to choose between deterministic, classical ML and LLM-based approaches. History of influencing technical direction across multiple teams and mentoring engineers. Regular use of AI tools and translating research into production results. Sound judgment on trade-offs between quality, latency, cost, reliability, privacy.

Skills

What Principal ML / AI jobs ask for — and how much of it you have →
Apply

See also

ML / AI jobs by country — openings, pay and top skills →