Azure Data Engineer (4(
Summary
This role involves designing, building, and maintaining scalable Azure-based data pipelines and ETL solutions using Python, T-SQL, Azure Databricks, and Azure Data Factory. The engineer will also focus on data warehouse modeling, performance optimization, and implementing CI/CD pipelines within an on-site contract environment.
Job Title
Description
We are seeking 4 experienced Azure Data Engineers to join our client in Sandton, Johannesburg. This is a 12-month on-site contract (work from office) with a subcontractor budget of R800/hr, negotiable for outstanding candidates. Permanent opportunities with Zensar are available for the right people. Candidates must have a minimum of 7 years total experience and 3 to 4 years of expert-level experience as an Azure Data Engineer. Highlighted skills are mandatory.
Responsibilities
- Design, build and maintain scalable Azure-based data pipelines and ETL solutions.
- Perform data analysis and exploration using T-SQL to support requirements, profiling and troubleshooting.
- Develop ETL and data transformation code in Python, primarily using Azure Databricks.
- Implement and maintain automated tests for data pipelines to ensure data quality and reliability.
- Design and optimize data warehouse models using dimensional modelling and Data Vault 2.0.
- Ingest, store and process data using Azure Data Factory, Azure Databricks, Azure Synapse Analytics and Azure Data Lake Storage Gen2.
- Optimize performance and ensure robustness of ETL processes and data solutions.
- Use source control and Azure DevOps to manage code, CI/CD pipelines and deployments.
- Apply data governance tools and practices (for example, Azure Purview) and ensure compliance with data policies.
- Collaborate with stakeholders, document technical designs, and provide guidance to development and data teams.
Qualifications
- Minimum 7 years total IT experience with 3 to 4 years in an Azure Data Engineer role at expert level.
- Demonstrated experience with Azure Data Factory, Azure Databricks, Azure Synapse Analytics and Azure Data Lake Storage Gen2.
- Excellent data analysis and exploration skills using T-SQL and advanced SQL programming.
- Proficiency in Python for ETL development, data wrangling and working within Databricks.
- Experience writing automated tests for data pipelines and validating data quality.
- Strong understanding of data warehouse modelling methodologies including Kimball/dimensional modelling and Data Vault 2.0.
- Experience building robust, performant ETL processes and applying performance optimization techniques.
- Experience with source control (Git), Azure DevOps and CI/CD deployment pipelines.
- Awareness and experience of data governance tools such as Azure Purview.
- Excellent analytical and problem-solving skills, strong communication, stakeholder engagement and documentation skills.
Skills
- Microsoft Fabric
- Git
- Python
- T-SQL
- SQL stored procedures and functions
- Automated testing for data pipelines
- Data warehouse modeling
- Data Vault 2.0
- ETL performance optimization
- CI/CD and deployment pipelines
- Data governance
- Data analysis and exploration
- Analytical problem solving
- Organizational skills
- Ownership and self-motivation
- Teamwork
- Resilience under pressure