Automation and AI product testing
Posted Updated
Description
Job Summary
We are looking for an experienced AI Automation Tester with hands-on experience in testing AI-powered products and applications. The ideal candidate should have strong expertise in Playwright automation, along with practical experience in LLM evaluations (Evals), Golden Datasets (Goldens), and AI Observability.
Key Responsibilities
- Design, develop and execute automated test scripts using Playwright.
- Test AI-powered applications, LLM-based products and Generative AI solutions.
- Work on Evals to assess AI response accuracy, relevance, quality and consistency.
- Create, maintain and validate Golden Datasets (Goldens) for benchmarking AI outputs.
- Implement and work with AI Observability to monitor model performance, response quality, latency and failures.
- Perform functional, regression, API and end-to-end testing.
- Identify, report and track defects to ensure product quality and reliability.
Skills
Sign in to read and join the discussion.