Senior Data Engineer – Azure Databricks - Banking

The Opportunity

We are looking for an experienced Senior Data Engineer to join a high-performing engineering team responsible for delivering a modern, enterprise-scale Azure data platform. This is an exciting opportunity to work on large-scale data engineering initiatives, building robust, cloud-native solutions that power analytics, AI, machine learning, and business-critical applications.

Working within a collaborative Agile environment, you will design, develop and optimise high-performance data pipelines while helping shape the future of a modern Azure Lakehouse platform.

This role would suit someone who enjoys solving complex engineering challenges, building scalable solutions and working with the latest Microsoft Azure and Databricks technologies.

Key Responsibilities

• Design, develop and maintain scalable data engineering solutions using Microsoft Azure and Azure Databricks.
• Build robust ETL/ELT pipelines using Python, PySpark, SQL and Spark technologies.
• Develop batch and streaming data processing applications using Delta Lake and Lakehouse architecture.
• Design and optimise high-performance Spark workloads for scalability, reliability and cost efficiency.
• Develop reusable frameworks, libraries and engineering components to accelerate data delivery.
• Build and maintain CI/CD pipelines using Azure DevOps.
• Implement automated testing, monitoring, logging and data quality controls.
• Work closely with Architecture, Cloud, DevOps and AI teams to deliver enterprise-grade data solutions.
• Mentor junior engineers and contribute to engineering standards, best practices and code quality.

Required Experience

We're looking for someone with strong hands-on experience across most of the following:

• 5+ years of Data Engineering experience
• Python
• PySpark
• SQL
• Lakehouse Architecture
• Performance tuning and Spark optimisation
• Data modelling and transformation
• Version control using Git

Experience with any of the following would be highly beneficial:

• Scala
• Terraform or Infrastructure as Code
• Databricks Unity Catalog
• Machine Learning or AI data platforms
• Data governance, metadata and lineage
• MLOps
• Microsoft Fabric
• Enterprise-scale cloud migrations

Preferred Certifications

• Databricks Certified Data Engineer Associate or Professional

What We’re Looking For

You’ll be someone who enjoys writing clean, maintainable code, solving challenging technical problems and working collaboratively with engineering teams. You’ll have a passion for modern cloud technologies, continuous improvement and delivering high-quality, scalable data solutions that make a real business impact.

Excellent communication skills and the ability to work across technical and business teams are essential.