Test Automation Engineer
Summary
Senior test automation engineer owning the test strategy and automation framework for an AI agent development platform: building automated REST API and Playwright UI tests, testing async workflows and LLM/agent functionality, and wiring tests into GitLab CI/CD with AWS-based environments. Core stack: Python or TypeScript, Playwright, Docker, AWS.
ABOUT SEARGIN
Shape. Build. Run.
Seargin is an international technology transformation and delivery partner helping global enterprises turn complex technology challenges into measurable business outcomes.
With 1,000+ technology professionals and a presence across Europe, the United States and Asia, we work with leading international organisations, including Fortune 500 companies, on business-critical transformation and technology initiatives.
Our expertise spans SAP and Enterprise Transformation, Data & AI, M&A and Post-Merger Integration, Application & Managed Services, and complex technology delivery.
What makes Seargin different is our ability to combine strategic expertise with hands-on execution. We don’t just advise on transformation - we help design, build, implement and operate the solutions that make it successful.
At Seargin, you’ll have the opportunity to:
* Contribute to meaningful projects for globally recognised organisations.
* Work with experienced technology professionals in international environments.
* Develop your expertise across complex enterprise technologies and business challenges.
* Take ownership, share ideas and make a tangible impact.
* Grow your career in a collaborative, ambitious and internationally connected organisation.
Join Seargin and help shape the technology behind global business transformation.
About the role
We are looking for a Senior Test Automation Engineer who will take ownership of the test automation strategy and framework for an AI Agent Development Platform.
The successful candidate will be responsible for designing, developing and maintaining automated tests across frontend, backend and API layers, ensuring the reliability, security and scalability of the platform.
The role involves close collaboration with software engineers, platform engineers and AI specialists, with a strong focus on API automation, asynchronous workflows, cloud infrastructure and testing AI-powered functionality.
Requirements
- 5+ years of experience in QA / Software Testing.
- 3+ years of hands-on experience in Test Automation.
- Experience defining test strategies and owning test automation frameworks or repositories.
- Strong programming skills in Python or TypeScript.
- Practical experience with REST API test automation.
- Knowledge of OpenAPI, JSON Schema validation and contract testing.
- Experience testing asynchronous workflows, including polling, message queues and WebSockets.
- Understanding of authentication and authorization mechanisms, including OAuth2, OIDC, JWT and RBAC.
- Hands-on experience with Playwright, preferably for testing React-based single-page applications.
- Experience identifying, investigating and reducing flaky tests.
- Experience integrating automated tests into CI/CD pipelines, preferably GitLab CI.
- Knowledge of Docker and containerized environments.
- Experience with cloud-based applications and infrastructure, preferably AWS.
- Familiarity with application logs, metrics and distributed tracing.
- Understanding of LLM-based applications, AI agents and their testing challenges.
- Experience using AI coding assistants to support software development and test automation.
- Strong analytical, troubleshooting and communication skills.
Responsibilities
- Define and maintain the overall test strategy and appropriate test pyramid for the platform.
- Build, maintain and take ownership of the test automation repository and framework.
- Develop and maintain automated REST API tests, including functional, negative, integration and schema validation scenarios.
- Test asynchronous workflows involving polling, message queues and WebSockets.
- Develop and maintain end-to-end UI automation tests using Playwright.
- Implement automated smoke and integration tests for AI agent workflows, including deployment, authentication, MCP tool calls, logs, traces and memory-related functionality.
- Integrate automated tests into CI/CD pipelines, including merge-request quality gates, scheduled regression runs and post-deployment smoke tests.
- Support the provisioning and maintenance of AWS-based test environments.
- Investigate test failures, perform root cause analysis and improve automation reliability.
- Monitor application behaviour using logs, metrics and distributed tracing.
- Collaborate with developers, platform engineers and AI specialists to identify quality risks early in the development lifecycle.
- Review technical designs and provide recommendations to improve testability, reliability and overall product quality.
- Report test results, communicate risks and continuously improve testing processes.
Nice to Have
- Experience with AWS services such as Lambda, API Gateway, S3 and CloudWatch.
- Familiarity with AI agent frameworks such as LangGraph, CrewAI or AWS Strands.
- Experience with LLM evaluation tools such as DeepEval, Promptfoo or Ragas.
- Knowledge of MCP and A2A protocols.
- Experience with RAG pipelines, vector databases and retrieval-based applications.
- Understanding of OWASP security principles and application security testing.
- Experience working in regulated environments or with GxP requirements.
Contact: elina.reviakina@seargin.com
Discover the Power of IT Excellence. Apply!
To learn more about Seargin, please visit our web page: www.seargin.com
Skills
Sign in to read and join the discussion.