Senior Quality Assurance Automation Engineer - Python/Playwright Testing

Hiring Update : Senior Automation QA Engineer

Experience : 6+ Years

Location : Bangalore, India

Mode of Work : Hybrid

Summary :

We are looking for a Senior Automation QA Engineer to define and execute test strategies for AI agent features in the Agent Development Squad. This role goes beyond standard API and UI automation. You must understand the unique challenges of testing non-deterministic LLM outputs, multi-step agent graphs, and audit-critical financial workflows. Pytest and Playwright are the primary tooling standards. LLM and agent testing awareness is required.

What You'll Do :

- Design and implement test automation strategies for backend APIs, agent orchestration logic, and Angular-based agent UIs.

- Build and maintain Pytest-based test suites for Python backend services and agent workflow validation.

- Implement Playwright automation for Angular-based UI, covering interaction flows, approval queues, and streaming response handling.

- Validate multi-step agent workflows by asserting on intermediate states, tool call outputs, and handoff results.

- Use Langfuse traces to validate agent reasoning chains, confidence scores, and execution paths.

- Build and maintain LLM evaluation harnesses covering prompt regression testing, RAG output validation, and model version change detection.

- Integrate test suites into Azure DevOps CI/CD pipelines as automated quality gates.

- Verify audit log integrity, including immutable log structure, correlation ID chains, and SOX-compliant event emission.

Tech Stack :

- Python, Pytest, Playwright, Angular, Azure DevOps, Langfuse, REST API.