Data platform engineer
Posted 1
view
Summary
The Data Platform Engineer designs and operates an automated Azure and Databricks environment using Terraform and CI/CD pipelines. The role involves developing platform tooling in Python or .NET, managing Unity Catalog, and ensuring platform reliability, security, and cost governance.
Role : Data Platform Engineer
Location : Sandton, Johannesburg( 3 Days in a week from office)
Duration : 12 Months renewal
PurposeSupport the design, build, and operation of the Data Platform, blending infrastructure engineering with data engineering practices to keep the Databricks & Azure environment reliable, automated, and self-service for platform users.
Key Responsibilities Design and maintain CI/CD pipelines (Dev Ops/Git Hub Actions) for infrastructure and application deployments Write and maintain Terraform Ia C for Azure services and Azure Databricks (Unity Catalog, workspaces, networking, compute policies) Develop and support application code in Python and/or. NET for platform tooling and automation Build and maintain shared libraries/modules used by data engineers and analysts across the platform Design and implement pipelines/processes that support platform management (cost tracking, governance, provisioning, monitoring) Collaborate with data engineers, analysts, and security/infra teams to translate requirements into platform capabilities Troubleshoot infrastructure, pipeline, and integration issues (networking, auth, connectivity) Contribute to platform documentation and best-practice standards Required Skills & Experience Solid experience with Microsoft Azure infrastructure (Entra ID, Key Vault, networking, storage) Hands-on Azure Databricks experience (Unity Catalog, cluster/job management) Terraform (Ia C) experience, ideally in a multi-environment setup Proficiency in Python; working knowledge of SQL;. NET is a plus Experience building/maintaining CI/CD pipelines (Azure Dev Ops, Git Hub Actions, or similar) Understanding of data platform governance and cost