Lead Data Engineer: Databricks & GenAI Platform
Summary
Lead Data Engineer in Sydney designing and delivering scalable Databricks data platforms that support GenAI initiatives — guiding architecture, building end-to-end pipelines, and setting data quality, governance and security standards. Core stack is Spark, PySpark, Python, SQL, Delta Lake and Lakehouse across batch and streaming workloads.
M&T Resources is seeking a Lead Data Engineer with deep Databricks expertise to design and deliver scalable data platforms supporting GenAI initiatives. You will guide architecture, build end-to-end pipelines and establish standards for data quality, governance, and security.
You will lead with hands-on capability in Spark, PySpark, Python and SQL, and collaborate to advance Delta Lake and Lakehouse architectures across batch and streaming workloads.