Test Automation Engineer (pharma; 100% remote)
Description
We are looking for a Senior Automation QA Engineer who will focus on end-to-end quality of the multi-tenant, serverless AWS platform where teams build, deploy and run AI agents. Each team gets an isolated workspace with browser-based dev environments (VS Code, Langflow, n8n), automated agent deployment, an agent marketplace and real-time logs. The ideal candidate will have a strong background in software quality assurance and test automation, with a proven track record of defining test strategies and building and owning API and UI test automation for web platforms, as well as a practical understanding of how AI agents behave. You will play an important role in ensuring the performance, reliability, and accuracy of our intelligent systems.
Frontend: React / TypeScript single-page application
Backend / API: Go Lambda functions behind AWS API Gateway (OpenAPI-documented REST), WebSocket log streaming, Python CLI
Agents: AWS Bedrock AgentCore; templates for multiple frameworks (e.g., LangGraph, AWS Strands); MCP and A2A
100% remote work.
Requirements
- Test Strategy & Repository Ownership: 5+ years in software quality engineering. You have defined a test strategy for a web platform or SaaS product. You have also set up and owned a test automation repository (not only contributed tests to one), including its structure, conventions, CI integration, code reviews and long-term maintenance, in Python or TypeScript.
- API Test Automation: Strong hands-on experience automating REST API tests (e.g., pytest + httpx, Playwright API testing), including: schema validation against OpenAPI (contract testing is a plus) asynchronous flows (polling, state machines, queues, WebSockets) auth and access control (OAuth2/OIDC, JWT, role- and tenant-based access, negative cases)
- UI Automation: Proven experience with Playwright on React/SPA applications. You have kept a large suite stable and kept flaky tests measured and under control.
- CI/CD & Cloud: Experience building quality gates in GitLab CI (or a similar tool) with Docker. Experience testing cloud-hosted applications and debugging through logs, metrics and traces.
- AI Awareness & Daily Use: You understand how LLM-based agents behave (tools, memory, non-determinism). You use AI assistants (e.g., Claude Code, GitHub Copilot) every day in your own work: writing tests, analysing failures and documenting. You review AI-generated code critically.
- Communication: Strong collaboration and reporting skills; ability to work in cross-functional and international teams. Strong analytical and problem-solving skills with attention to detail.
- Nice to have: AWS experience (Lambda, API Gateway, DynamoDB, S3, CloudWatch, Bedrock AgentCore). Experience with Strands or other agent frameworks (e.g., LangGraph, CrewAI). Familiarity with LLM evaluation tools (e.g., DeepEval, Promptfoo, Ragas). Experience with MCP / A2A servers and protocols. Knowledge of vector-based retrieval systems. Experience in monitoring & observability. Experience with full-stack application testing. Security testing: working knowledge of the OWASP Top 10 (web and API) and hands-on penetration testing experience (e.g., Burp Suite, OWASP ZAP) Knowledge of GxP or regulated environments is a plus. Strong analytical mindset and attention to detail.
Skills
- Agentic AI
- AI
- API
- API Gateway
- API Testing
- Automation
- AWS
- AWS Bedrock
- Burp Suite
- CI/CD
- Claude Code
- CLI
- Cloud
- CloudWatch
- CrewAI
- Docker
- DynamoDB
- GitHub
- Github Copilot
- GitLab
- JWT
- Lambda
- LangGraph
- LLM
- MCP
- n8n
- OAuth
- Observability
- OpenAPI
- OpenID
- OWASP
- Penetration Testing
- Playwright
- pytest
- Python
- React
- REST
- S3
- SaaS
- Serverless
- Test Automation
- TypeScript
- Websockets
Sign in to read and join the discussion.