Senior Azure Data Engineer (ID: 3907)
Summary
Design and maintain Azure cloud data pipelines using ADF, Databricks, and PySpark to process large-scale enterprise data for analytics and BI.
As a Senior Azure Data Engineer, you will:
- Design, develop, and maintain scalable, high-performance data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build and optimize large-scale data processing solutions using PySpark and Databricks to support enterprise analytics and business intelligence initiatives.
- Develop robust Python-based ETL/ELT solutions for extracting, transforming, and loading data from diverse data sources.
- Collaborate with architects, data analysts, business stakeholders, and engineering teams to understand data requirements and deliver scalable cloud-based data solutions.
- Implement and maintain CI/CD pipelines using Azure DevOps to automate testing, deployment, and release management of data engineering solutions.
- Monitor, troubleshoot, and optimize Azure data pipelines to ensure reliability, scalability, and operational excellence.
- Ensure data quality, governance, security, and compliance across the enterprise data platform.
- Optimize SQL queries and data processing workflows to improve system performance and efficiency.
- Participate in Agile ceremonies and contribute to continuous improvement of engineering processes and cloud data architecture.
- Stay current with emerging Azure technologies, cloud data engineering best practices, and modern data platform innovations.
What You Bring to the Table:
- 6–8 years of experience in Data Engineering, Cloud Data Platforms, or Big Data Engineering.
- Strong hands‑on expertise in Azure Data Factory (ADF) for data orchestration and workflow automation.
- Extensive experience with Azure Databricks for distributed data processing and analytics.
- Advanced programming skills in Python for ETL development, automation, and data engineering.
- Strong experience using PySpark for processing large-scale datasets and building scalable data pipelines.
- Proficiency in SQL for querying, optimizing, and managing large enterprise datasets.
- Strong understanding of ETL/ELT methodologies, data integration, and cloud-native data architectures.
- Experience working within Agile development environments.
- Excellent analytical, troubleshooting, problem‑solving, and communication skills.
You Should Possess the Ability to:
- Design scalable, reliable, and secure Azure-based data engineering solutions.
- Develop high-performance ETL pipelines using Python, Azure Data Factory, and Databricks.
- Build and optimize distributed data processing solutions using PySpark.
- Automate deployment processes through Azure DevOps CI/CD pipelines.
- Troubleshoot complex data processing and pipeline performance issues.
- Ensure enterprise data quality, governance, and security standards are maintained.
- Collaborate effectively with cross-functional teams to deliver business-driven data solutions.
- Continuously improve cloud data platforms by adopting modern engineering best practices.
What We Bring to the Table:
- Opportunity to work on enterprise-scale Azure cloud data engineering and analytics initiatives.
- Exposure to modern cloud-native technologies, big data platforms, and large-scale distributed data processing.
- Collaborative environment with experienced cloud architects, data engineers, and analytics professionals.
- Challenging projects involving data modernization, automation, and cloud transformation.
- Opportunities for continuous learning, Azure certifications, and technical career growth.
- A culture that encourages innovation, collaboration, and engin eering excellence.
Let’s Connect
Want to discuss this opportunity in more detail? Feel free to reach out.