Azure Data Engineer (ID: 3870)

Responsibilities

• Design, develop, and maintain scalable data pipelines using Azure Data Factory (ADF) and Azure Databricks.
  
• Build and optimize large-scale data processing solutions using PySpark and Databricks.
• Develop robust ETL/ELT workflows using Python for data extraction, transformation, and loading.
• Implement and maintain CI/CD pipelines in Azure DevOps to automate deployment and testing of data solutions.
• Collaborate with business stakeholders, data architects, and engineering teams to translate data requirements into scalable solutions.
• Monitor, troubleshoot, and optimize data pipelines to ensure high performance, reliability, and scalability.
• Ensure data quality, security, governance, and compliance with industry best practices.
• Document technical solutions and contribute to continuous improvement of cloud data engineering practices.

What You Bring to the Table

• 5+ years of experience in Data Engineering with a strong focus on Microsoft Azure.
  
• Hands‑on experience with Azure Data Factory (ADF) for orchestrating data movement and transformation workflows.
• Proficiency in Python and PySpark for distributed data processing and ETL development.
• Strong knowledge of SQL for querying, transforming, and managing large datasets.
• Experience working with cloud‑based data platforms and modern data engineering best practices.
• Good understanding of data security, performance tuning, and optimization techniques.
• Strong analytical, problem‑solving, and debugging skills.
• Excellent communication, collaboration, and technical documentation skills.

You Should Possess the Ability to

• Design and develop scalable, high‑performance cloud data solutions.
  
• Build and optimize complex ETL/ELT pipelines for large datasets.
• Process and analyze big data efficiently using PySpark and Databricks.
• Automate deployments through Azure DevOps CI/CD pipelines.
• Troubleshoot and resolve data pipeline failures and performance bottlenecks.
• Collaborate effectively with cross‑functional technical and business teams.
• Work independently while managing multiple priorities in an Agile environment.
• Follow cloud security, governance, and data quality best practices.
• Continuously improve data engineering processes through automation and optimization.

What We Bring to the Table

• Opportunity to work on enterprise‑scale Azure cloud data engineering projects.
  
• Exposure to modern cloud‑native technologies including Azure Data Factory, Azure Databricks, PySpark, and Azure DevOps.
• Collaborative environment that encourages innovation and knowledge sharing.
  
• Opportunities to work with large‑scale data platforms and cloud transformation initiatives.
• Continuous learning and professional development in Azure cloud technologies.

Let’s Connect

Want to discuss this opportunity in more detail? Feel free to reach out.