Big Data Engineering Lead
Summary
The Big Data Engineering Lead will oversee the development and maintenance of enterprise data pipelines, data models, and services while leading a high-performing engineering team. The role requires expertise in Python, PySpark, SQL, and modern data architecture patterns to ensure scalable and reliable data solutions.
Responsibilities
- Lead the technical leadership, engineering delivery, and quality of the Data Foundation platform
- Build and maintain enterprise data pipelines, data products, canonical data models, and data services
- Partner with Service Owner, Architecture Team, and Platform Engineering Lead to deliver scalable, trusted, and maintainable data solutions
- Accountable for engineering execution, solution quality, and operational reliability
- Develop a high-performing data engineering team
Requirements
- 8+ years of experience in data engineering, software engineering, or enterprise data platform development
- 3+ years of technical leadership experience leading engineering teams or large-scale data initiatives
- Advanced SQL and data modeling expertise
- Strong Python and PySpark development experience
- Deep understanding of ETL and ELT architecture patterns
- Experience implementing Master Data Management solutions
- Experience building RESTful APIs and data services
- Familiarity with GraphQL and modern integration approaches
- Understanding of microservices-based architectures
- Experience implementing event-driven and real-time data processing solutions
- Strong understanding of data quality frameworks, metadata management, and data lineage concepts