Data Engineer — PySpark & AWS Data Pipelines
Summary
Builds PySpark ETL pipelines on AWS to deliver scalable data products for government reporting, ensuring quality and security in a hybrid cloud setup.
DWP Digital is seeking a Data Engineer to join the Management Information (MI) team. You will build scalable data products, develop PySpark-based ETL pipelines with EMR, Lambda, and EC2, and collaborate across stakeholders to translate reporting needs into robust solutions.
You will contribute to Agile delivery, CI/CD automation, and maintain documentation while ensuring data quality, performance, and security in a hybrid cloud environment.