Senior Product Manager, Multilingual AI and Evals
Summary
Owns AI-driven quality and evaluation infrastructure for multilingual AI agents, building datasets, metrics, and feedback loops to improve translation and localization across global products.
You will own quality assurance and evaluation infrastructure for multilingual AI agents. You will improve automated translation quality gates, build AI-driven localization testing, create datasets and annotation tools, establish evaluation metrics and dashboards, and turn human feedback into agent improvements across global products.
Responsibilities
- Enhance the evaluation agent that determines whether translations are ready for publication
- Balance automation coverage and risk across content tiers
- Diagnose systematic quality-gate failures and drive fixes into the agent
- Build an AI-driven tool that detects localization defects
- Ship an MVP that scans mobile and web applications for localization issues
- Integrate localization tests into internal product testing infrastructure
- Build golden datasets, annotation tools, automated evaluations, metrics dashboards, and feedback loops
- Integrate standardized real-time annotation workflows for linguists
Requirements
- Bachelor's degree or higher
- 3+ years in product management, with hands-on AI agent experience potentially substituting for part of the requirement
- Experience building content platforms, testing platforms, or SaaS products
- Hands-on experience with AI agents, agent harnesses, and evaluations
- Ability to design evaluation harnesses and golden datasets
- Experience shipping products across multiple markets or languages
- Cross-functional leadership across engineering, AI teams, linguists, design, and external partners
Benefits
- Education subsidy
- Team building programs
- Company events
- Wellness allowance
- Meal allowance
- Comprehensive healthcare schemes for employees and dependants