SC-Cleared Cloud Data Engineer: Spark, PySpark & Airflow
Summary
Builds and maintains cloud-based data pipelines using Spark/PySpark and Airflow to process large datasets, ensuring data reliability and governance in an AWS-native environment with IaC.
Scrumconnect is seeking a hands-on Data Engineer to design, build and maintain data pipelines in a modern AWS-native stack. You will use Apache Spark, PySpark and Airflow to process large data volumes and ensure data assets are trusted and governed. Travel to the London office three days weekly; remote days from anywhere in the UK. Active SC clearance is required.
The role emphasises collaboration across teams to translate requirements into scalable solutions with IaC using Terraform.