Senior Data Quality Engineer

We are seeking a skilled Senior Data Quality Engineer to join our team. The successful candidate will play a crucial role in ensuring the integrity, reliability, and quality of our data engineering pipelines and platforms.

Responsibilities

• Design and develop a robust automated testing framework and strategy for data engineering pipelines, with a focus on integration with Databricks, PySpark, Scala, Spark SQL, and other relevant technologies
• Implement automated tests to ensure data integrity, quality, and reliability across all data pipelines and platforms
• Integrate automated testing within Azure DevOps CI/CD pipelines to ensure seamless testing, build, and deployment processes
• Conduct and execute performance tests to assess the efficiency and scalability of data processing systems
• Work closely with data engineers and other stakeholders to understand requirements and ensure testing strategies are aligned with business objectives
• Establish monitoring for automated tests, providing regular reports on test coverage, defects, and data quality issues

Requirements

• Minimum of 3 years of experience in Data Quality Engineering or a related field
• Solid understanding of automated testing frameworks and data engineering principles
• Proficiency with Databricks, PySpark, Scala, Spark SQL, Terraform, and Azure Event Hubs. Solid understanding of automated testing frameworks and data engineering principles
• Experience in integrating testing frameworks with Azure DevOps and managing source control with Git
• Knowledge of pipeline configuration and deployments utilizing Terraform
• Familiarity with Jira
• Strong analytical abilities with a proven track record in troubleshooting and resolving complex data and testing challenges
• Fluency in English with a B2+ level

Nice to have

• Experience with Power BI, Azure Data Lake, Spark Streaming, and other data processing and visualization tools
• Prior experience in a data-intensive industry or with large-scale data environments
• Proficient in implementing solutions through Infrastructure as Code (IaC)

Benefits

With us you can:

• Work on a flexible schedule remotely or from any of our comfortable offices or coworking spaces in Ukraine
• Receive the necessary equipment to perform your work tasks
• Change projects and technology stacks within EPAM
• Gain experience in various business domains (Insurance, E-commerce, Healthcare, Finance, Travelling, Media, Artificial Intelligence, and more)
• Relocation opportunities may be available for eligible candidates, depending on the role and openings at other EPAM locations
• Participate in volunteer, charity programs and communities (both technical and interest-based)

We focus on your professional growth:

• You can plan your individual career path together with your manager
• Receive regular feedback from colleagues
• Improve your English for free with certified teachers (Speaking Clubs, client interview preparation courses, etc.)
• Get the opportunity to undergo free training and certification in AWS, GCP, or Azure Clouds
• Use the internal E-learn training program (18,200+ specialized training and mentoring programs)
• Access corporate accounts on LinkedIn Learning, Get Abstract and other partner resources
• Study at EPAM Solution Architecture School with the instructors who are practicing architects
• Develop as a leader, join Delivery Management, Resource Management, Leadership Essentials school and more
• Participate in internal communities (500+ meetups, technical discussions, brainstorming sessions, online events and conferences annually)

What we offer:

• Vacation and sick leave (including a sick leave without a medical certificate)
• A wide range of Voluntary Medical Insurance programs providing both medical treatment and various preventive options (including sports activities)
• Medical insurance for family members at corporate rates
• Company support during significant life events (childbirth or adoption, marriage, etc.)
• Support for psychological comfort: discounts on services from mental health specialists or coaches, thematic training
• E-kids program - a free programming language training program for EPAMers' children