SDET - III (RAGas Automation Testing)
Posted Updated
Summary
Senior SDET role (Level III) focused on test automation for AI products: designing frameworks and test strategies for LLM-powered features, evaluating RAG pipelines with RAGas, and validating integrations like OpenAI and Claude. Core stack includes Python, JavaScript, Playwright/Cypress/Selenium, and API testing tools, plus mentoring junior SDETs.
About the job
Requirements
- 5+ years of overall experience in QA/SDET roles
- Proven experience working with AI tools (e.g., OpenAI, Claude, or similar LLM platforms)
- Strong understanding of testing AI/ML products and their unique challenges
- Hands-on knowledge of RAG pipelines and their testing methodologies
- Proficiency in automation frameworks (e.g., Playwright, Cypress, Selenium, or similar)
- Experience with API testing (Postman, REST, GraphQL, gRPC)
- Strong problem-solving skills with the ability to work independently
- Mentorship and leadership skills in guiding peers or junior engineers
Responsibilities
- Design, develop, and maintain automation frameworks and test strategies for AI products
- Test and validate AI-powered features (including OpenAI, Claude, and other AI integrations)
- Evaluate and improve RAG (Retrieval-Augmented Generation) flows, ensuring accuracy and performance
- Leverage AI automation tools to optimize test coverage and efficiency
- Collaborate closely with developers, product managers, and AI researchers to ensure high-quality releases
- Own tasks end-to-end, working independently to deliver on commitments.
- Mentor and guide junior SDETs, fostering a culture of quality and continuous learning
Nice to have
- Experience with CI/CD pipelines and cloud platforms (AWS/GCP/Azure)
- Familiarity with MLOps pipelines and monitoring AI systems
- Exposure to performance and scalability testing of AI workloads
Skills: llm testing,ragas,ai test engineer,retrieval testing,prompt engineering,rag evaluation,python,javascript
