Senior Data Engineer in London
Summary
Senior Data Engineer building and optimising scalable data pipelines and a single source of truth on Databricks (PySpark, Spark SQL, Delta Lake) for a UK software company. Mostly remote with occasional London office visits, plus mentoring a junior engineer and shaping data architecture.
UK HQ based & fast‑growing software business (400+ staff, globally), serving hundreds of enterprise customers worldwide, are looking for a Senior Data Engineer to help shape the future of its data platform. You'll take ownership of building scalable data solutions in Databricks primarily, establishing a single source of truth, and enabling trusted, self‑service access to data across the business.
Location: UK, remote with very occasional visits to the office in London. Salary: Up to £85,000 basic, plus 10% bonus and an extensive benefits package.
What you'll be doing
Design, build and maintain scalable data pipelines and data products using Databricks, PySpark, Spark SQL and Delta Lake.
Develop and optimise ETL/ELT processes to ingest, transform and curate data from internal and external systems.
Shape enterprise data architecture, balancing scalability, performance, security and maintainability.
Partner with stakeholders across Finance, Product, Sales, Marketing and Engineering to turn business challenges into technical solutions.
Embed data quality, governance and access controls throughout the data lifecycle.
Mentor a junior engineer and help mature a growing data function.
What you need to bring
“Senior” level experience in data engineering, with strong hands‑onDatabricks skills (PySpark, Spark SQL, Delta Lake, etc)
Advanced SQL and Python coding skills.
A track record of building and optimising enterprise‑scale data pipelines and ETL/ELT solutions.
Solid grounding in data architecture, modelling and modern data platform design.
Cloud experience, preferably AWS (but Azure is totally fine), and familiarity with CI/CD and source control best practice.
Desirable: More Data Science leaning background or want to explore in this role:ML / AI & data / predictive modelling strategies.
Why you'll love this job
A highly visible, hands‑on role with real influence over data architecture and platform strategy.
Join a small, high‑impact team at the heart of the company's data strategy, with genuine scope to shape how it evolves. We are near the beginning of our Data Ops’ journey – so an excellent opportunity to shape the future!
Extensive benefits package including health insurance, income protection, life assurance, subsidised gym membership and 25 days' holiday plus your birthday off.