Azure Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, load data from various sources into Azure Databricks.
• Collaborate with cross-functional teams to gather requirements for data processing needs and design solutions that meet business objectives.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Azure Synapse.
• Ensure high availability of the system by implementing monitoring tools such as Prometheus/Grafana.

Job Requirements :

• 5-10 years of experience in designing and developing big data solutions on Microsoft Azure platform.
• Strong expertise in Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Storage (ADLS), Azure Synapse Analytics.
• Proficiency in Python programming language with hands-on experience working with PySpark.