Software Engineer
Summary
Senior Python Data Engineer designing, building and optimising large-scale cloud-native data pipelines and platforms on AWS (Spark, Airflow, Terraform, SQL) for a major UK public sector programme. Long-term contract, working remote/hybrid within an agile cloud data team.
Senior Python Data Engineer | AWS | Spark | Airflow | Public Sector
Location: UK Remote / Hybrid
Rate: Competitive Daily Rate
Duration: Long-Term Contract
Sector: Public Sector
We are supporting a major public sector programme and are looking for an experienced Senior Data Engineer to join a high-performing cloud data team delivering business-critical data solutions.
About the Role
This is an opportunity to work on large-scale data platforms, helping design, build and optimise modern cloud-native data services that support analytics, reporting and operational decision-making.
Responsibilities
- Design, develop and maintain scalable data pipelines and ETL/ELT solutions
- Build and optimise cloud-native data platforms within AWS
- Develop Spark and PySpark workloads for large-scale data processing
- Create and maintain Apache Airflow workflows and orchestration pipelines
- Implement Infrastructure as Code using Terraform
- Support batch and real-time data ingestion frameworks
- Work closely with data architects, analysts, data scientists and engineering teams
- Drive data quality, automation, monitoring and performance optimisation
- Contribute to CI/CD processes and engineering best practices
Qualifications
Experience Required
- Previous experience delivering within public sector or highly regulated environments
- Experience working on large-scale cloud data transformation programmes
- Strong stakeholder engagement and problem-solving skills
- Ability to work independently and contribute within agile delivery teams
Required Skills
- Python
- AWS
- Terraform
- SQL
- ETL / ELT Development
- Data Integration
- Data Warehousing
Preferred Skills
- AWS Glue
- EMR
- Lambda
- Databricks
- Kafka
- Docker
- Jupyter Notebooks
- Snowflake
- Data Governance & Data Quality Frameworks
Pay range and compensation package
- Rate: Competitive Daily Rate
- Duration: Long-Term Contract
Equal Opportunity Statement
We welcome applications from all suitably qualified candidates and are committed to creating an inclusive environment for all employees, including those working within public sector or highly regulated environments.