DevSecOps / Data Engineer
Summary
Contract role building secure, scalable cloud data solutions on Azure as part of a major geospatial data delivery programme. Day to day: developing Databricks/Spark and Scala data processing modules, optimizing Azure data pipelines, and implementing infrastructure as code with Bicep, in a hybrid setup from Southampton.
In this role you will develop secure, scalable cloud data solutions within a modern Azure environment. You’ll work on large-scale data processing platforms using Databricks, Spark and Azure services as part of a major data delivery programme. You will design and implement IaC with Bicep and optimize data pipelines, contributing to critical geospatial data services. This contract sits in a hybrid setup with onboarding at least one day per week on site.
Responsibilities- Develop and maintain software components and data processing modules
- Build scalable, secure solutions using Azure and Databricks
- Develop Spark and Scala-based data processing capabilities
- Design and implement Infrastructure as Code using Bicep
- Build, maintain and optimise Azure data pipelines
- Strong Azure DevOps experience
- Hands-on with Azure Cloud Services (Entra ID, Azure Batch, ADLS, ADF)
- Databricks development experience (Unity Catalog)
- Strong Apache Spark and Scala expertise
- Infrastructure as Code using Bicep
- Python and PowerShell scripting
- Azure DevOps
- Entra ID
- Azure Batch
- Azure Data Lake Storage (ADLS)
- Azure Data Factory (ADF)
- Databricks (Unity Catalog)