Data Engineer
Summary
Build and maintain enterprise-scale data pipelines using Microsoft Fabric, Azure, and PySpark to power analytics and Power BI reporting for a Queensland Government agency.
A long term contract opportunity is available for an experienced Data Engineer to join a major Queensland Government organisation delivering enterprise-scale data and analytics capabilities.
This is a hands-on role where you'll design, build and optimise modern data pipelines and products within an enterprise data platform, working with the latest Microsoft data technologies.
Key Responsibilities
- Design, build and maintain enterprise data pipelines using Microsoft Fabric and Azure technologies.
- Develop ingestion and transformation processes across Bronze, Silver and Gold data layers.
- Create curated datasets and semantic models to support analytics and Power BI reporting.
- Implement data quality checks, validation processes and automated testing.
- Integrate data governance practices, including metadata management, lineage and data classification.
- Apply security and privacy-by-design principles across data engineering activities.
- Develop and maintain CI/CD pipelines using Azure DevOps and Git-based workflows.
- Monitor, troubleshoot and optimise data pipelines for performance, reliability and cost efficiency.
- Produce technical documentation, runbooks and data dictionaries.
- Collaborate with product teams, analysts, architects and business stakeholders to deliver high-value data solutions.
Ideal skills and experience:
- Demonstrated experience delivering enterprise data engineering solutions within complex environments.
- Strong experience with Microsoft Fabric, including Pipelines, Dataflows Gen2, Lakehouse, OneLake, SQL Endpoints and Notebooks.
- Advanced SQL and PySpark skills.
- Experience with Azure data services such as Azure Data Factory, Synapse Analytics or Databricks.
- Knowledge of modern data architectures, including Medallion (Bronze/Silver/Gold) and dimensional modelling.
- Experience supporting Power BI through high-quality curated datasets and semantic models.
- Exposure to Microsoft Purview and enterprise data governance practices.
- Experience implementing CI/CD using Azure DevOps and Git.
- Strong understanding of data security, privacy, governance and data quality principles.
- Excellent communication skills and the ability to work collaboratively within multidisciplinary Agile teams.
Technology
This role will be offered as an initial 12 month contract + opportunity to extend.