Senior/Lead Data Engineer
Summary
Hands-on Senior/Lead Data Engineer building and modernising large-scale data pipelines using Spark, PySpark, and Python on AWS in a hybrid London role.
Salary: £? - ? per year
Requirements:
- Strong Spark and PySpark engineering experience
- Experience developing and modernising large-scale data pipelines
- Strong Python skills and software engineering fundamentals
- Experience with performance, scalability, and workload optimisation
- Cloud data engineering experience, ideally on AWS
- Experience with CI/CD and modern engineering practices
- Experience working in complex, enterprise-scale data environments
- Senior or Lead-level data engineering experience
- Ability to use AI tools to accelerate development while maintaining strong engineering judgement
Responsibilities:
- Build, modernise, and optimise data workloads in a large-scale, complex environment
- Work hands-on across engineering rather than overseeing from a distance
- Develop and modernise large-scale data pipelines
- Apply performance and scalability improvements to data workloads
- Use AI to support the development lifecycle while reviewing and correcting AI-generated code where needed
- Deliver robust engineering solutions that scale properly
Technologies:
- AI
- AWS
- CI/CD
- Cloud
- Support
- Python
- PySpark
- Spark
More:
We are a major UK organisation making significant investment across our Data and AI estate. This is a Senior or Lead Data Engineer contract role based in London on a hybrid arrangement with three days onsite. The initial contract duration is six months, and the role is inside IR35. We are looking for a hands-on engineer who enjoys tackling genuinely large-scale data challenges and working in a modern, complex enterprise environment.
last updated 33 week of 2026