Databricks Data Engineer
Salary: £9,600 - 14,400 per year
Requirements:- 8–10 years of hands-on data engineering experience.
- Strong commercial experience with Azure Databricks and Azure Data Factory.
- Advanced Spark development using Python or Scala, alongside strong SQL skills.
- Experience with Azure Data Lake Storage Gen2 and Azure SQL or PostgreSQL.
- Strong understanding of Azure analytics services and Azure identity, including SPN, SAMI and UAMI.
- Experience designing data lake, data warehouse, data mesh and service-oriented integration solutions.
- Previous ETL development experience using Informatica, SSIS, Talend or similar.
- Experience delivering solutions within an Agile SDLC environment.
- Strong Git, GitLab or GitHub experience, including branching, pull requests and CI/CD.
- Hands-on scripting experience with Linux or PowerShell.
- Exposure to Docker and automated testing practices.
- Experience with pipeline orchestration tools such as Apache Airflow, Autosys or Control-M, with Airflow preferred.
- Excellent written and verbal English communication skills.
- Kafka or other streaming technologies.
- Terraform, ARM templates, YAML or Puppet.
- GitLab CI/CD pipeline creation.
- Azure Functions and Logic Apps.
- Test-driven development.
- Previous experience delivering data solutions within an HR or employee lifecycle environment
- Design and develop scalable data pipelines and analytics solutions using Azure Databricks and Azure Data Factory.
- Build and improve data products supporting HR and employee lifecycle processes.
- Translate business and technical requirements into production-ready solutions.
- Develop data lake, data warehouse and data mesh solutions using Azure services.
- Work with large, complex and multi-format datasets.
- Apply automated testing, CI/CD and DevOps practices throughout the development lifecycle.
- Build observability into solutions to monitor production health and support incident resolution.
- Ensure solutions meet security, reliability, compliance and performance standards.
- Collaborate with engineering, product, HR and other cross-functional teams.
- Contribute to technical decisions with long-term scalability and sustainability in mind.
- Airflow
- ARM
- Azure
- CI/CD
- Data Warehouse
- Databricks
- DevOps
- Docker
- ETL
- Git
- GitHub
- GitLab
- Informatica
- Support
- Kafka
- Linux
- PostgreSQL
- PowerShell
- Puppet
- Python
- SQL
- Scala
- Security
- Spark
- SSIS
- Talend
- Terraform
- Cloud
More:
We are hiring an Azure Databricks Data Engineer in London on a hybrid working pattern, with 3–4 days per week onsite. This is an inside IR35 contract role within an HR technology environment, where we focus on building reliable, scalable and production-quality data solutions across the employee lifecycle. Our team works closely with engineering, product, HR and other cross-functional partners, with a strong emphasis on automation, testing, security, observability and long-term sustainability.
last updated 36 week of 2026