Lead Data Engineer
Summary
A technical leadership role managing a team of 4 data engineers while specializing in Databricks on AWS — designing scalable data pipelines, overseeing batch and real-time integrations, and providing hands-on guidance. Core stack: Databricks, Python, SQL, Spark, Airflow, Terraform.
Lead Data Engineer | London | 2 days a month in office
I’m working with a leading UK business that is investing heavily in its data and technology capabilities, and they’re looking for a Lead Data Engineer to join the team.
This is a technical leadership role where you’ll specialise in Databricks and manage a team of 4 Data Engineers, while providing technical direction across the wider data engineering function.
The role
- Manage, mentor and develop a team of 4 Data Engineers
- Provide technical leadership across data engineering projects
- Design and deliver scalable data products and pipelines
- Work closely with Data Scientists, Analysts, Architects and Product teams
- Drive engineering standards around coding, testing and data quality
- Oversee batch and real-time data integrations
- Help shape the evolution of a modern cloud data platform
- Improve engineering processes and ways of working across the wider data community
- Provide hands-on technical guidance where required
- Support wider data, ML and AI initiatives
What they’re looking for
- Strong commercial Data Engineering experience
- Previous experience leading, mentoring or managing Data Engineers
- Strong Python and SQL skills
- Experience with Databricks on AWS
- Strong Spark and Airflow experience
- Experience building scalable data pipelines and integrations
- Good understanding of CI/CD and Git
- Experience with Terraform or similar IaC tools
- Strong understanding of data modelling and data quality
- A genuine interest in developing people as well as technology
If you’re a Data Engineer who has moved into technical leadership or people management and want to lead a team while staying close to the technology, get in touch.