Senior Data Scientist — AI Agent Improvement & Quality
Posted Updated
Who recruits here Pro
Up to three recruiters at Jobgether with their work email, so you can write to a person.
Part of Pro. How it works
Description
Jobgether is seeking a Staff Data Scientist to lead evaluation and optimization of AI-powered agents in a fast-paced, international environment. You will design evaluation strategies, analyze production traces, and translate insights into measurable improvements across product, engineering, and customer success teams.
You will mentor colleagues, contribute to the evolution of agent capabilities, and develop production-quality Python and TypeScript tools, shaping the quality and reliability of
Own and develop the evaluation strategy for AI agents. Analyze agent traces and conversations at scale to identify failure patterns. Drive improvement cycle with actionable fixes and before/after comparisons. Improve agent skills, orchestration, and supporting infrastructure. Collaborate with Delivery and Customer Success to enhance workflows. Design and validate LLM-as-a-judge evaluation systems. Develop production-quality Python and TypeScript code for tools. Mentor data scientists and engineers in measurement and evaluation. 8+ years delivering data science solutions with measurable impact. Hands-on evaluation of LLM-based systems and AI agents. Proficient in Python and SQL; TypeScript willingness. Strong software engineering practices and CI/CD experience. Experience with production GenAI systems and tools. Ability to translate traces into actionable insights. Unlimited PTO Fully remote Flexible hours around Central Time Professional development opportunities International teamSkills
Sign in to read and join the discussion.