Senior Azure Data Engineer (ID: 4146)
Posted
Summary
Senior data engineer building and optimizing enterprise-scale Azure data pipelines: Azure Data Factory and Databricks with PySpark, Python ETL, SQL for large datasets, and CI/CD via Azure DevOps. Requires 8-10 years in IT with strong data engineering experience.
As a Senior Azure Data Engineer, you will:
- Design, develop, and maintain complex data pipelines using Azure Data Factory (ADF) and Azure Databricks.
- Build, implement, and optimize large-scale data processing solutions using PySpark and Databricks.
- Develop robust ETL processes using Python for data extraction, transformation, and loading.
- Design and implement CI/CD pipelines using Azure DevOps to automate testing, integration, and deployment.
- Work with SQL to query, process, transform, and manage large datasets.
- Monitor, troubleshoot, and optimize data pipelines to ensure performance, scalability, and reliability.
- Collaborate with cross-functional teams to understand business and data requirements and deliver effective data solutions.
- Ensure data security, quality, and compliance with established standards and best practices.
- Stay up to date with emerging technologies and best practices in Azure cloud data engineering.
What You Bring to the Table:
- 8–10 years of overall experience in IT, with strong experience in data engineering and development.
- Hands-on experience with Azure Data Factory for data integration, orchestration, and pipeline automation.
- Strong expertise in Azure Databricks for scalable data processing and engineering solutions.
- Proficiency in Python and PySpark for big data processing and ETL development.
- Experience implementing CI/CD pipelines using Azure DevOps.
- Good knowledge of SQL and experience working with large datasets.
- Experience developing, maintaining, and optimizing complex data pipelines.
- Understanding of data security, data quality, and cloud data engineering best practices.
You should possess the ability to:
- Design and develop scalable, reliable, and high-performance data engineering solutions.
- Troubleshoot complex data pipeline issues and improve processing efficiency.
- Work independently as well as collaboratively within cross-functional teams.
- Analyze technical and business requirements and translate them into effective data solutions.
- Apply problem-solving and analytical skills to overcome technical challenges.
- Communicate technical concepts clearly and maintain proper technical documentation.
- Adapt to evolving Azure technologies and modern data engineering practices.
What We Bring to the Table:
- An opportunity to work on enterprise-scale Azure data engineering solutions.
- Exposure to modern cloud data technologies, including Azure Data Factory, Azure Databricks, and PySpark.
- An opportunity to contribute to large-scale data processing and automation initiatives.
Let’s Connect
Want to discuss this opportunity in more detail? Feel free to reach out.
Recruiter: Keerthika
Phone: +31 20 369 0609 ; Extn :148
Email: keerthika.p@stafide.nl
LinkedIn:
Skills
Sign in to read and join the discussion.