BigData Test Automation Specialist
Posted 12
views
We are looking for a BigData Test Automation Specialist to join our team in Zurich. This role focuses on designing and implementing a robust, multi-layered testing strategy for a modern big data platform within the banking sector. You will work with cutting-edge technologies including Trino, Kafka, Iceberg and Dataiku to ensure data quality, platform resilience and overall system integrity.
Responsibilities
- Design and implement comprehensive test automation strategies for a big data platform
- Develop and maintain scalable test frameworks
- Implement declarative data quality checks for schemas
- Create and manage automated tests for complex SQL transformations
- Test and validate Hive metastore schemas and data integrity
- Develop tests for streaming data pipelines using Kafka and Testcontainers
- Automate Dataiku DSS scenarios and integrate them into CI/CD pipelines
- Build and optimize CI/CD infrastructure for efficient big data testing
- Perform resilience and chaos testing to ensure platform stability
- Use data versioning patterns like Write-Audit-Publish for isolated and safe testing
Requirements
- Strong experience with Python and the Pytest framework for test automation
- Hands-on experience with big data technologies like Trino, Hive, Kafka or S3
- Proficiency with data quality and validation tools
- Experience with containerization and mocking tools like Docker, Testcontainers
- Proven ability to test streaming applications and event-driven architectures
- Knowledge of scripting for cloud object storage interaction
- Experience with MLOps platforms, preferably Dataiku
- Proven experience building and managing CI/CD pipelines (e.g. GitHub Actions, GitLab CI)
- Familiarity with SQL transformation tools like dbt or SQLMesh
- Understanding of data versioning concepts
- Fluent in English
Nice to have
- Experience in the banking or financial services sector
Benefits
- 5 weeks of vacation
- EPAM Employee Stock Purchase Plan (ESPP)
- Enhanced parental leave
- Extended pension plan
- Daily sickness allowance insurance
- Employee assistance program
- Global business travel medical and accident insurance
- Learning and development opportunities including in-house training and coaching, professional certifications, and courses