Lead Data Engineer | Federal Government
Summary
Lead Data Engineer delivering enterprise analytics in a secure Australian Federal Government environment in Canberra: designing automated ingestion and ETL/ELT pipelines on Azure and Databricks, shaping data into Parquet/Delta formats, optimizing Spark and Azure SQL performance, and building Power BI dashboards. Initial 12-month contract with a 12-month extension option; NV1 clearance mandatory.
Lead Data Engineer | Federal Government
About the Opportunity
Join a dynamic, multi-disciplinary section driving critical enterprise analytics and modern data platforms within a secure Federal Government environment.
Deliver high-impact operational analytics by designing and maintaining cutting-edge ingestion patterns, transformation solutions, and data visualisations using Azure and Databricks technologies.
What's on Offer
Contract/Engagement: Initial 12-month contract + 1x 12-month extension option.
Location & Flexibility: Canberra, ACT (Only)
Role & Responsibilities
Design, build, and maintain automated ingestion and transformation patterns using modern cloud technologies (Azure & Databricks).
Transform complex datasets into Parquet and Delta formats tailored for advanced analytics and AI use cases.
Develop ETL/ELT data pipelines and create Power BI visualisations to support operational analytics and decision-making processes.
Optimize query performance across distributed computing engines (Spark), Azure SQL, and cloud storage solutions.
Collaborate within an Agile delivery team while providing quality reviews and constructive feedback to uplift delivery standards.
Skills & Experience
To be successful in this position, you will ideally bring:
Technical Toolkit: Strong expertise in Azure hosted services (Data Factory, Databricks, SQL Server, Data Lake Storage, Azure ML), DevOps, Visual Studio, Power BI, and data formats like Parquet and Delta.
Experience: 5+ years of experience as a Data Engineer, with proven expertise developing complex ETL/ELT processes in cloud environments and distributed computing engines (e.g., Spark).
Core Capabilities: High productivity, ownership, and exceptional written and oral communication skills to engage and negotiate effectively with multi-disciplinary team members.
Eligibility & Clearance Requirements
Security Clearance: Due to the nature of this environment, candidates must hold a valid Negative Vetting Level 1 (NV1) security clearance.