Senior Data Engineer
Summary
Senior Data Engineer builds and optimizes Databricks-based ETL pipelines and data products for a large-scale transformation program, collaborating with analysts and architects.
Responsibilities
- Support the delivery of a large‑scale data transformation programme
- Build, optimise and maintain data pipelines within a modern Databricks environment
- Develop, orchestrate, and maintain ETL pipelines and related objects within Databricks utilising medallion architecture structures
- Create high‑quality, performant, and secure Data Products for analytics use cases
- Collaborate with Data Analysts, Architects and business stakeholders
- Act as an escalation point in supporting data integration and transformation activities
- Drive improvements in data quality, performance and reliability
- Contribute to engineering standards and best practices
- Support the adoption of AI‑assisted development approaches where appropriate
- Actively participate in design and architecture reviews, contributing pragmatically to ensure robust data solutions are proposed
Requirements
- Strong Databricks Data Engineering experience
- Databricks certification is highly advantageous
- Experience developing ETL pipelines and orchestration frameworks
- Strong Python development skills
- Experience working with cloud‑based data platforms
- AWS experience is an advantage
- Strong SQL and data modelling knowledge
- Excellent problem‑solving and analytical abilities
- Proven Data Engineering experience within enterprise environments
- Experience delivering data solutions using Databricks
- Experience working with large‑scale data transformation initiatives
- Experience with AI‑assisted development within Databricks
- Insurance or Financial Services experience would be highly beneficial