Senior/Lead Data Engineer | London | Contract | Inside IR35
Summary
Builds and optimizes large-scale data pipelines and AI-driven workflows in a complex enterprise environment using Spark/PySpark, Python, and AWS, with hands-on engineering focus.
Senior / Lead Data Engineer - Contract - Inside IR35
London - Hybrid, 3 days onsite
Initial 6 months
Inside IR35
I'm looking for a seriously hands‑on Senior/Lead Data Engineer to join a major UK organisation undertaking significant investment across its Data & AI estate.
The key word here is hands‑on.
We're not looking for someone who has progressed into overseeing engineering from a distance. You'll be actively building, modernising and optimising data workloads within a large‑scale, complex environment.
What we're looking for:
- Strong Spark / PySpark engineering
- Large‑scale data pipeline development & modernisation
- Python and strong software engineering fundamentals
- Performance, scalability & workload optimisation
- Cloud data engineering - ideally AWS
- CI/CD and modern engineering practices
- Experience working across complex, enterprise‑scale data environments
There's also a particularly interesting AI engineering angle.
We want someone who knows how to use AI to accelerate the development lifecycle - but has enough engineering depth to look at AI-generated code and say “that won't scale”, explain why, and build it properly.
This would suit an experienced Senior or Lead Data Engineer who still loves engineering and wants to tackle genuinely large-scale data challenges.
#DataEngineering #DataEngineer #PySpark #Spark #AWS #Python #AI #ContractJobs