Senior/Lead Data Quality Engineer

Our team is looking to hire a talented Senior/Lead Data Quality Engineer. In this role, you will be instrumental in safeguarding the integrity, reliability, and quality of our data engineering platforms and pipelines.

Responsibilities

• Build and architect a strong automated testing strategy and framework for data engineering pipelines, emphasizing integration with Databricks, PySpark, Scala, Spark SQL, and other pertinent technologies
• Deploy automated tests that guarantee data quality, integrity, and reliability across every data pipeline and platform
• Embed automated testing into Azure DevOps CI/CD pipelines to enable smooth testing, build, and deployment workflows
• Carry out performance testing to evaluate the scalability and efficiency of data processing systems
• Collaborate closely with data engineers and additional stakeholders to gather requirements and align testing strategies with business goals
• Set up monitoring for automated tests and deliver routine reports covering test coverage, defects, and data quality concerns

Requirements

• At least 3 years of experience working in Data Quality Engineering or an adjacent field
• Strong grasp of data engineering principles and automated testing frameworks
• Hands-on expertise with Databricks, PySpark, Scala, Spark SQL, Terraform, and Azure Event Hubs
• Background in connecting testing frameworks to Azure DevOps and handling source control through Git
• Understanding of pipeline configuration and deployment using Terraform
• Working knowledge of Jira
• Excellent analytical skills with a demonstrated history of diagnosing and resolving complex testing and data issues
• English proficiency at a B2+ level

Nice to have

• Exposure to Power BI, Azure Data Lake, Spark Streaming, and additional data visualization and processing tools
• Previous work in a data-heavy industry or within large-scale data environments
• Skill in delivering solutions via Infrastructure as Code (IaC)

Benefits

With us you can:

• Work on a flexible schedule remotely or from any of our comfortable offices or coworking spaces in Ukraine
• Receive the necessary equipment to perform your work tasks
• Change projects and technology stacks within EPAM
• Gain experience in various business domains (Insurance, E-commerce, Healthcare, Finance, Travelling, Media, Artificial Intelligence, and more)
• Relocation opportunities may be available for eligible candidates, depending on the role and openings at other EPAM locations
• Participate in volunteer, charity programs and communities (both technical and interest-based)

We focus on your professional growth:

• You can plan your individual career path together with your manager
• Receive regular feedback from colleagues
• Improve your English for free with certified teachers (Speaking Clubs, client interview preparation courses, etc.)
• Get the opportunity to undergo free training and certification in AWS, GCP, or Azure Clouds
• Use the internal E-learn training program (18,200+ specialized training and mentoring programs)
• Access corporate accounts on LinkedIn Learning, Get Abstract and other partner resources
• Study at EPAM Solution Architecture School with the instructors who are practicing architects
• Develop as a leader, join Delivery Management, Resource Management, Leadership Essentials school and more
• Participate in internal communities (500+ meetups, technical discussions, brainstorming sessions, online events and conferences annually)

What we offer:

• Vacation and sick leave (including a sick leave without a medical certificate)
• A wide range of Voluntary Medical Insurance programs providing both medical treatment and various preventive options (including sports activities)
• Medical insurance for family members at corporate rates
• Company support during significant life events (childbirth or adoption, marriage, etc.)
• Support for psychological comfort: discounts on services from mental health specialists or coaches, thematic training
• E-kids program - a free programming language training program for EPAMers' children