Lead Data Engineer
Posted Updated
Summary
Lead Data Engineer in Auckland providing technical leadership over enterprise-scale data platforms: designing scalable Databricks-based pipelines (Spark, Delta Lake, PySpark) with Medallion architecture on Azure, mentoring engineers and driving data quality, governance and performance.
Salary: $130 - $140 per hour
The opportunityAn exciting opportunity for an experienced Lead Data Engineer to take a leading role in designing, building and optimising enterprise-scale data platforms.
You will provide technical leadership across data engineering initiatives, driving best practice in data architecture, engineering standards and the delivery of scalable, reliable data solutions. Working with modern cloud technologies, you will help shape data platform capabilities and enable the organisation to make better use of its data.
Key responsibilities
- Lead the design, development and delivery of scalable data engineering solutions using Databricks.
- Develop and optimise data pipelines, ETL/ELT processes and data transformation frameworks.
- Design and implement modern data architectures, including Medallion (Bronze, Silver and Gold) architectures.
- Provide technical guidance and leadership to data engineers, promoting engineering best practice.
- Work closely with architects, analysts and business stakeholders to translate requirements into technical solutions.
- Drive performance optimisation, data quality, governance and platform reliability.
- Contribute to cloud data platform strategy and continuous improvement.
- Proven experience as a Lead Data Engineer or Senior Data Engineer in complex enterprise environments.
- Strong experience with Databricks, including Spark, Delta Lake and PySpark.
- Advanced SQL and Python programming skills.
- Experience designing and building large-scale data pipelines and ETL/ELT solutions.
- Solid understanding of modern data architecture and Medallion data patterns.
- Experience working with Azure cloud data services.
- Knowledge of CI/CD, DevOps practices and data engineering automation.
- Excellent communication skills and the ability to provide technical leadership and direction.
- Lead important data engineering initiatives.
- Work with modern cloud and data technologies.
- Opportunity to influence data architecture and engineering standards.
- Collaborative environment with interesting technical challenges.
If you are an experienced Data Engineer with strong Databricks expertise and enjoy leading technical delivery, we'd love to hear from you.
Apply now or contact Cornelia Jacobs at cornelia@younity.co.nz for a confidential discussion.
#SCR-cornelia-jacobs