Azure Databricks Platform Engineer - Automation & Security
Posted Updated
Description
Don Nelson Technology Limited, in partnership with NP Solution Limited, seeks an experienced Azure Databricks platform engineer to design secure, scalable environments for Hong Kong insurance data workloads. You will build pipelines, automate provisioning with Terraform, and collaborate across architects, engineers, and security teams to ensure seamless integration with legacy systems and cloud sources.
The role focuses on performance tuning, cost control, and regulatory compliance in a
Design and manage Azure Databricks environments for enterprise data workloads. Collaborate with architects, engineers, and security teams to define standards. Automate provisioning with Terraform and CI/CD using GitHub Actions. Monitor performance, scale resources, and optimize pipelines and analytics. Ensure RBAC, encryption, managed identities, and regulatory compliance. Integrate Informatica for metadata and data lineage and support audits. Maintain thorough infrastructure documentation and incident response processes. Bachelor's degree in Computer Science, IT, or a related discipline. 5+ years in Azure cloud platform design and administration with Databricks. Strong experience provisioning and managing Databricks clusters and workspaces. Experience integrating Azure Data Factory and Data Lake Storage Gen2 with Databricks. Proficiency with Terraform for infrastructure automation. Familiarity with Informatica tools for data governance and cataloguing. Deep understanding of security protocols, monitoring, and cost control.Skills
Sign in to read and join the discussion.