Lead Data Software Engineer

Our team is expanding, and we're in search of a Lead Data Software Engineer to join us. This position is well-suited for an individual who enjoys engineering resilient data systems that inform key business decisions. You'll work alongside a group committed to producing clean, well-validated code and maintaining disciplined engineering practices. For someone drawn to solving data problems at scale, this role presents considerable opportunity for growth and lasting impact

Responsibilities

• Build and structure scalable data pipelines that ensure reliable data flow across a variety of systems
• Develop and keep up ingestion pipelines that source data from multiple, varied origin systems
• Write clean, high-quality, well-tested code rooted in sound software engineering practices
• Work together with cross-departmental teams to understand data needs and convert them into viable technical solutions
• Observe data pipeline performance closely, addressing potential issues early to maintain consistent system reliability
• Contribute to code review processes to help sustain quality expectations and share insights across the team
• Investigate and correct data quality concerns to maintain dataset accuracy and consistency
• Play a part in continuously refining data architecture methods and engineering standards
• Compile documentation covering technical processes and solutions to facilitate knowledge sharing and support new hire onboarding
• Work inside an agile structure, engaging in sprint planning and other routine team gatherings

Requirements

• At least 5 years of relevant practical experience in the field
• A minimum of one year spent in roles leading and managing development teams
• Demonstrated skill with PySpark and Spark SQL for developing and managing large-scale data workflows
• Hands-on background constructing and supervising Lakeflow pipelines and Jobs for data orchestration purposes
• Proficient with Auto Loader and JDBC/CDC ingestion strategies, including experience with sources like DB2 and SQL Server
• Strong familiarity with Delta Lake and Unity Catalog for managing data storage and governance requirements
• Track record maintaining data quality and carrying out source reconciliation to ensure accurate, trustworthy data
• Background developing unit tests using pytest to validate code reliability and correctness
• Working knowledge of Git and Asset Bundles to support version control and deployment management
• Upper-intermediate English language skills or higher (B2)

Nice to have

• Experience participating in DataStage migration efforts
• Background extracting data from Dynamics 365/Dataverse systems
• Databricks certification held

Benefits

• International projects with top brands
• Work with global teams of highly skilled, diverse peers
• Healthcare benefits
• Employee financial programs
• Paid time off and sick leave
• Upskilling, reskilling and certification courses
• Unlimited access to the LinkedIn Learning library and 22,000+ courses
• Global career opportunities
• Volunteer and community involvement opportunities
• EPAM Employee Groups
• Award-winning culture recognized by Glassdoor, Newsweek and LinkedIn