Senior Data Engineer: Real-Time Pipelines on Databricks
Summary
Senior Data Engineer at Showtime Consulting in Sydney, embedded in a large-scale enterprise data environment. Day to day you design, build and optimise high-throughput and real-time ETL/ELT pipelines using Azure Databricks, Spark (Scala), SQL, Python, and Delta Lake Medallion Architecture.
Showtime Consulting is seeking an experienced Data Engineer to join a large-scale enterprise data environment in Sydney. You will design, build and optimise high-throughput data pipelines using Azure Databricks, Spark (Scala), SQL and Python.
Responsibilities include delivering scalable ETL/ELT pipelines, supporting real-time integrations, and implementing Medallion Architecture with Delta Lake, while collaborating with data, engineering and business stakeholders across cloud platforms such as