Data Engineer (Azure Fabric)
Summary
Designs and implements Azure Fabric-based ETL pipelines, ingesting and transforming data into medallion layers while building monitoring and reconciliation jobs.
1d ago , from Illawarra Mercury Job Board
About The Role
Ability to proactively collect client requirements and design solutionMinimum of 10 years overall experience with at least 5 years’ experience as Data Engineer
Location
Location: Sydney or Melbourne with Full working rights
Responsibilities
- Hands-on role as Data Engineer working on data ingestion and ETL on Azure Fabric.
- Connecting with various source systems and ingesting dataETL using Dataflow Gen2 and Insertion of raw data into Bronze layer
- Cleansing, de-normalisation and movement of data into Silver layer.
- Build scheduled ingestion jobs
- Build reconciliation jobs
- Build monitoring and logging of data flows
Required Skills
- Proven experience with Azure Fabric – should have executed projects on Fabric or Databricks
- Proven experience with Dataflow Gen2 or ADF
- Proven experience with Dataverse
- Familiarity with OData APIs
- Familiarity with DAB (Data API Builder)
- Hands on experience with Medallion Architecture across all layers