Data Engineer MS Fabrics

Role: MS Fabric Data Engineer

Location: Sydney

Job Type: Permanent

Experience: 5+ yrs

Job Description:

Roles and Responsibilities:

• Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
• Develop and manage Lakehouse and One Lake architectures for structured and unstructured data.
• Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.
• Implement ETL/ELT processes using Spark (Pyspark) and SQL.
• Design and optimize data models to support analytics and reporting workloads.
• Develop and maintain Fabric Data Warehouse solutions using T-SQL.
• Ensure data quality, reliability, and consistency through validation and monitoring.
• Optimize pipeline performance, storage, and capacity utilization in Fabric.
• Implement security, access control (RBAC), and data sensitivity labels.
• Enable and support Power BI semantic models and downstream reporting use cases.
• Monitor, troubleshoot, and resolve data pipeline and performance issues.
• Implement data governance, lineage, and cataloguing using Microsoft Purview.
• Support CI/CD and DevOps practices for analytics and data engineering workloads.
• Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.

Skill Experience requirement:

• Design enterprise data architecture leveraging Fabric components (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, OneLake).
• Build and optimise Fabric Lakehouse tables, Warehouses, Pipelines, and Dataflows Gen2.
• Define data modelling standards, governance frameworks, and security patterns.
• Develop reusable ETL/ELT frameworks using PySpark, SQL, and Fabric notebooks.
• Manage CI/CD workflows via Azure DevOps and Git.
• Architect scalable, cost-efficient data solutions integrating Fabric with Azure/enterprise systems.
• Apply best practices for performance tuning, cost optimisation, and data quality.
• Provide technical leadership and guidance to engineering teams.
• Expertise in Delta Lake, Fabric Data Engineering, ADLS, PySpark/SQL.
• Strong understanding of data modelling and integration with Power BI datasets.
• Strong understanding of metadata management, Purview, access control, and FinOps.
• Certifications: DP-203, Fabric Analytics Engineer (preferred).

• Design, build, and maintain end-to-end data pipelines using Microsoft Fabric Data Engineering and Data Factory.
• Develop and manage Lakehouse and OneLake architectures for structured and unstructured data.
• Ingest data from multiple sources (databases, APIs, files, streaming sources) into Fabric.
• Implement ETL/ELT processes using Spark (PySpark) and SQL.
• Design and optimize data models to support analytics and reporting workloads.
• Develop and maintain Fabric Data Warehouse solutions using T-SQL.
• Ensure data quality, reliability, and consistency through validation and monitoring.
• Optimize pipeline performance, storage, and capacity utilization in Fabric.
• Implement security, access control (RBAC), and data sensitivity labels.
• Enable and support Power BI semantic models and downstream reporting use cases.
• Monitor, troubleshoot, and resolve data pipeline and performance issues.
• Implement data governance, lineage, and cataloging using Microsoft Purview.
• Support CI/CD and DevOps practices for analytics and data engineering workloads.
• Collaborate with architects, analysts, and stakeholders to translate business requirements into scalable data solutions.

Skill Experience requirement :

• Design enterprise data architecture leveraging Fabric components (Lakehouse, Warehouse, Dataflows Gen2, Pipelines, OneLake).
• Build and optimise Fabric Lakehouse tables, Warehouses, Pipelines, and Dataflows Gen2.
• Define data modelling standards, governance frameworks, and security patterns.
• Develop reusable ETL/ELT frameworks using PySpark, SQL, and Fabric notebooks.
• Manage CI/CD workflows via Azure DevOps and Git.
• Architect scalable, cost-efficient data solutions integrating Fabric with Azure/enterprise systems.
• Apply best practices for performance tuning, cost optimisation, and data quality.
• Provide technical leadership and guidance to engineering teams.
• Expertise in Delta Lake, Fabric Data Engineering, ADLS, PySpark/SQL.
• Strong understanding of data modelling and integration with Power BI datasets.
• Strong understanding of metadata management, Purview, access control, and FinOps.
• Certifications: DP-203, Fabric Analytics Engineer (preferred).