Databricks Data Engineer - Flexible AI Lakehouse Architect
Posted Updated
Description
Deloitte NZ is seeking a Databricks Data Engineer to design, build and maintain scalable data and AI solutions using Databricks in a hybrid Auckland Christchurch environment. You’ll work across ingestion, transformation, modelling, and production support with AI-assisted tooling and modern DevOps practices.
You will collaborate with architects, data scientists, and business stakeholders to translate requirements into practical data solutions, delivering scalable, secure, and automated data
Design, develop, and maintain scalable data and AI solutions on Databricks. Build robust data ingestion, ELT/ETL, and data processing pipelines using Databricks. Develop high-quality and reusable transformations through Lakeflow Declarative Pipelines. Design and implement Lakehouse architectures. Build dimensional, curated, and feature-ready data models to support reporting, analytics, ML and GenAI use cases. Develop and maintain integrations between Databricks and cloud platforms, APIs, databases, files, streaming platforms, and enterprise apps. Implement automated CI/CD pipelines and modern DevOps practices for Databricks deployments. Develop and orchestrate workloads using Lakeflow Jobs, Azure Data Factory, Airflow, or similar technologies. Implement security, governance, data quality, lineage, and monitoring controls. Optimise Databricks workloads for performance, scalability, reliability, and cost efficiency. Troubleshoot complex platform and pipeline issues across environments. Collaborate with architects, analysts, data scientists, ML engineers, and stakeholders. Contribute to engineering standards, reusable frameworks, technical docs, and improvement initiatives. Leverage Genie Code, GitHub Copilot, and AI tools across design, coding, testing, review, and documentation. Strong hands-on experience with Databricks. Experience designing and building modern Lakehouse architectures. Strong understanding of data engineering, data modelling, data warehousing, and distributed processing concepts. Experience building batch and/or streaming data pipelines. Experience with orchestration technologies such as Lakeflow Jobs, Azure Data Factory, Airflow, or similar tools. Understanding of cloud data services across Azure and/or AWS. Experience with Git, CI/CD, and modern software engineering practices. Experience using Python in data engineering solutions are advantageous. Familiarity with infrastructure-as-code technologies such as Terraform or similar. Strong analytical, troubleshooting, and problem-solving skills. Excellent communication and stakeholder engagement skills. Flexible work arrangements Subsidised health insurance Subsidised professional qualifications Income protection insurance 22 days annual leave Well-being focusSkills
Sign in to read and join the discussion.