Lead Data Engineer
Summary
A hands-on Lead Data Engineer role designing, building, and operating enterprise-scale data platforms and AI/ML pipelines, spanning legacy to AWS cloud-native environments. Day to day: build scalable pipelines and data models, operationalize ML/AI into production, and set standards for governance, testing, and CI/CD while guiding other engineers.
As Lead Data Engineer, you will drive the design, build, and operation of production-grade data platforms and AI-enabled solutions in an enterprise context. You’ll act as a senior technical anchor, delivering end-to-end data and AI capabilities while guiding others through examples and best practices. The role spans legacy to cloud-native environments, focusing on reliable systems, governance, and scalable data pipelines. You will partner with cross-functional teams to operationalize AI, improve data sharing, and enable secure, scalable analytics and ML workloads. This is a hands-on, impact-focused leadership role that shapes how data and AI are used safely and effectively at scale.
Responsibilities- Design, build, and enhance enterprise data platforms, pipelines, and services
- Lead complex data engineering work end-to-end from problem definition to deployment and support
- Develop cloud-native data platforms supporting analytics, ML, and AI workloads
- Create data pipelines for traditional analytics and AI/ML use cases
- Collaborate with stakeholders to translate problems into scalable data and AI solutions
- Design data models, integrations, and storage for maintainability and reuse
- Establish AI-led development frameworks including CI/CD, testing, governance, and deployment standards
- Operationalize ML/AI solutions into production
- Improve data sharing and onboarding of new data sources across teams
- Diagnose and resolve complex data/platform issues in enterprise environments
- Contribute to an engineering culture focused on quality and continuous improvement
- Significant hands-on experience as a Senior or Lead Data Engineer on complex, enterprise-scale systems
- Strong Python and Spark skills with production data pipelines
- Deep understanding of the full data engineering lifecycle (ingestion, transformation, storage, serving, reuse)
- Experience designing integrations across diverse data sources and legacy environments
- Experience building and supporting cloud-based data platforms, preferably AWS
- Proven ability to design scalable data models and architectures
- Solid understanding of data governance, security, compliance, and operational controls
- Experience with modern engineering practices: source control, automated testing, CI/CD, infrastructure as code, deployment automation
- Ability to communicate with both technical and non-technical stakeholders
- Experience supporting ML/AI workloads through production-grade data engineering solutions
- Excellent communication with technical and non-technical stakeholders
- Delivery-focused mindset and collaboration
- Problem-solving mindset under complex enterprise conditions
- Python
- Spark
- AWS data services (S3, Glue, EMR, Redshift, Athena, Lambda, SageMaker, Bedrock)