Senior AI Product Manager
Posted Updated
Description
We're looking for a deeply technical AI Product Manager, someone who can own the product and understand how the underlying AI system behaves.
Responsibilities:
- Own AI/agent products end-to-end: user problem, product strategy, requirements, and launch iteration.
- Define how AI agents should reason, retrieve, orchestrate tools, and complete research workflows.
- Build and own AI evaluation frameworks: eval datasets, LLM-as-judge, human evals, regression testing, hallucination/factuality, and failure analysis.
- Define metrics around accuracy, task completion, latency, reliability, and user trust.
- Work deeply with engineering/AI teams on LLMs, RAG, agentic workflows, prompting, orchestration, and system architecture.
- Translate AI failure modes and user feedback into concrete product and engineering improvements.
Requirements:
- Experience: 5-11 years.
- Strong product management fundamentals with experience shipping products 01 and iterating them at scale.
- Hands-on understanding of AI/LLM systems; this cannot be a surface-level AI PM.
- Strong experience with LLM/agent evaluations, experimentation, and AI quality measurement.
- Technical background: ideally prior experience in software engineering, ML/AI, NLP, applied science, or another engineering-heavy role.
- Comfortable discussing APIs, system architecture, RAG, agents, prompts, model behavior, and trade-offs directly with engineers.
- Experience with LangGraph, LangChain, or similar agent frameworks is a strong advantage.
- The bar: We are looking for a product manager + AI systems thinker + evals owner in one person, not a traditional PM who has only worked around AI.
