AI QA Engineer (GenAI & ML Systems)
Job Description\: AI QA Engineer (GenAI & ML Systems)
We hiring an AI QA Engineer to own the quality, reliability and safety of our GenAI and ML-driven engineering applications. You will define how we test non-deterministic systems, from prompts and RAG pipelines to agentic workflows, and set the release bar for AI features used across Powertrain Engineering.
Key Responsibilities \:
• Design evaluation frameworks for LLM, RAG and agentic systems\: accuracy, hallucination, consistency, latency and cost
• Build golden datasets, regression suites and automated eval pipelines integrated into CI/CD
• Run red-teaming and adversarial tests\: prompt injection, jailbreaks, data leakage, bias and toxicity
• Validate retrieval quality, tool-calling and multi-step agent behavior
• Define quality gates and release criteria for AI features
• Monitor production drift and failure modes; lead root-cause analysis with AI engineers
• Report quality metrics and risks to engineering leadership