Data Engineer: Microsoft Fabric & PySpark Specialist
Posted Updated
Description
The HR Ally is seeking an experienced Data Engineer for a contract role in Melbourne. The position focuses on Microsoft Fabric, PySpark and Data Lake/Lakehouse architectures to design, develop and implement scalable data pipelines.
Responsibilities include building ETL/ELT workflows, transforming and validating data, and ensuring quality at scale. Proficiency with Azure Data Factory, Azure Databricks and Spark is preferred, along with Agile work experience.
Design, develop and maintain data pipelines using Microsoft Fabric. Build and optimise ETL/ELT workflows and data integration solutions. Develop PySpark scripts for large-scale data processing and transformation. Work with Data Lake, Lakehouse architectures and related components. Perform data transformation, cleansing, validation and optimisation. Troubleshoot performance issues and ensure data quality and reliability. Leverage AI-assisted coding tools to improve development efficiency. Hands-on experience with Microsoft Fabric and PySpark in enterprise data pipelines. Strong understanding of Data Lake and Lakehouse architectures. Experience building and optimising ETL/ELT workflows. Ability to design, develop and troubleshoot data engineering solutions independently.Skills
Sign in to read and join the discussion.