Data Engineer: PySpark on AWS for Scalable Analytics
Summary
Data Engineer building scalable PySpark ETL pipelines on AWS (EMR, Lambda, EC2) in a hybrid cloud/legacy environment to support reporting and analytics for DWP Digital's MI team.
DWP Digital's Management Information (MI) team is seeking a Data Engineer to shape data at scale using AWS technologies, PySpark, and ETL pipelines for reporting, analytics, and strategic decision-making.
Join a hybrid environment combining AWS with legacy platforms to build scalable data products used by analysts and data scientists. You will orchestrate PySpark ETL pipelines with EMR, Lambda, and EC2, while collaborating with stakeholders and supporting CI/CD automation.