Data Engineer
Summary
The Data Engineer will design and maintain scalable data pipelines and warehousing solutions using the Microsoft Azure stack, including ADF, Databricks, and Synapse. The role involves developing Python-based solutions and APIs while collaborating in an Agile DevOps environment.
We are looking for a hands-on Data Engineer for Sydney to join a team focused on strengthening..
Must-Have Skills
- Strong proficiency in SQL and Python (or any programming language).
- Experience in Azure Data Factory (ADF) pipelines, Databricks, and Azure Synapse.
- Expertise in Data Modeling and Data Warehousing (DWH) concepts.
- Knowledge on MS Fabric
Job Responsibilities
- Develop and maintain APIs, Azure Functions, and Python-based solutions.
- Work with JSON inputs and process them using Python notebooks.
- Design and implement data pipelines using Azure Data Factory and Synapse on Microsoft Azure.
- Build and optimize SQL queries, Synapse packages, and related components.
- Manage and configure Azure cloud environment and its components.
- Apply data warehousing solutions and patterns for scalable architecture.
- Collaborate in Agile DevOps environments and implement CI/CD pipelines using GitHub and Azure DevOps.
- Engage directly with business and technical stakeholders to gather requirements and deliver solutions.
Experience
- 8+ years of experience in data engineering and related technologies.
- Proven ability to work independently and in cross-functional teams.
Preferred Skills
- Development of Azure Function Apps and APIs.
- Familiarity with proc frameworks.
All candidates should have full working rights in Australia. Only shortlisted candidates will be contacted for this role.