Senior Data Engineer
Summary
Senior Data Engineer at Howden Group Services owning metadata-driven data pipelines and orchestration, building with Databricks and Azure data services (ADF, ADLS, Azure Functions), promoting CI/CD, optimizing ETL, and mentoring junior engineers in a hybrid London role.
In this Senior Data Engineer role, you own and advance high-impact data initiatives within Howden Group Services, focusing on metadata-driven data solutions and orchestration. You work with Databricks and Azure data services to design scalable pipelines, and you mentor junior engineers while collaborating with stakeholders. You will balance multiple deliverables and push for robust, compliant data solutions that enable business insights and operational efficiency. This is a chance to shape a modern Azure-based data ecosystem in a globally connected, diverse team.
Pay / Benefits- reasonable adjustments
- flexible hours
- hybrid working
- diverse culture
- career development
- global opportunities
- Own metadata ingestion frameworks and orchestration processes
- Design and implement scalable, reusable metadata-driven services for data pipelines
- Build and maintain data solutions using Databricks and Azure services (Autoloader, Spark Declarative Pipelines, ADF, Azure Functions, ADLS)
- Promote CI/CD and DevOps practices for data engineering
- Collaborate with stakeholders to translate data needs into engineering solutions
- Optimize pipelines for performance, reliability and scalability
- Support data governance and security best practices
- Design, build and maintain ETL processes and analytical data products with minimal oversight
- Provide technical guidance and mentoring to junior engineers
- Act as escalation point for complex data issues with strong problem solving and stakeholder management skills
- 4-6 years of data engineering experience with Databricks (SQL & PySpark)
- Hands-on Azure data services experience (ADF, ADLS, Azure Functions)
- Proven experience designing metadata-driven data pipelines
- Strong understanding of orchestration and data workflow automation (Airflow, DBT, Spark Declarative Pipelines)
- Knowledge of CI/CD practices for data engineering
- Experience in technical design discussions and architectural decisions
- Good data governance, security, and compliance practices
- Experience with insurance data (preferred)
- Collaborative mindset
- Stakeholder management
- Problem solving and communication
- Databricks SQL & PySpark
- Azure Data Factory (ADF)
- Azure Data Lake Storage (ADLS)