Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Don Nelson Recruitment Limited

Data Engineer - Databricks

Posted Updated

Description

Don Nelson Technology Limited seeks a skilled Data Engineer to design and maintain scalable Azure Databricks pipelines for insurance data. You will collaborate with architects, modellers, analysts and stakeholders to deliver robust data assets for analytics, operations, and regulatory reporting.

You will develop batch and streaming solutions using Azure Data Factory, Data Lake Gen2, Event Hubs and Kafka, with strong governance, security, and CI/CD practices.

Design and maintain scalable ETL/ELT pipelines in Azure Databricks. Collaborate with architects, modellers, analysts and stakeholders. Develop and optimize batch and streaming data solutions using ADF, Databricks, Kafka. Implement data validation and quality controls for critical use cases. Enforce governance, metadata management and cataloguing with Informatica. Apply domain expertise in Hong Kong Life and General Insurance. Automate deployment workflows with GitHub Actions. Troubleshoot pipelines and maintain detailed technical documentation. Bachelor's degree in CS/IT or related field. 3+ years with Azure cloud platforms and Databricks for insurance workloads. Experience provisioning and managing Databricks clusters/workspaces. Experience integrating Data Factory and Data Lake Gen2 with Databricks. Hands-on with Informatica for governance and metadata management. Strong Azure security, monitoring and cost optimization in regulated environments. CI/CD automation using GitHub Actions for deployment. Proven ability to troubleshoot data workloads in insurance.

Skills

Apply

See also

Data Engineering jobs by country — openings, pay and top skills →