Azure Data Engineer
Summary
Contract, hybrid role in Amsterdam designing and maintaining large-scale data pipelines with Azure Data Factory and Azure Databricks. The engineer writes PySpark/Python ETL, implements CI/CD in Azure DevOps, and monitors and optimizes pipelines for performance, security, and reliability.
JOB DETAILS:
Role: Azure Data Engineer
Location: Amsterdam, Netherlands (Hybrid)
Type of Hiring: Contract
Job Description:
Key Responsibilities:
• Design, develop, and maintain complex data pipelines using Azure Data Factory and Azure Databricks.
• Build and optimize large-scale data processing solutions using PySpark and Databricks.
• Collaborate with cross-functional teams to understand data requirements and deliver efficient data solutions.
• Implement CI/CD pipelines in Azure DevOps for automated testing, integration, and deployment of data solutions.
• Develop, test, and maintain robust data solutions in Python for data extraction, transformation, and loading (ETL) processes.
• Monitor, troubleshoot, and optimize data pipelines for performance and reliability.
• Ensure data security and compliance with company standards and best practices.
• Stay updated with the latest trends and best practices in cloud data engineering and Azure services.
Required Skills and Qualifications:
• Experience with Azure Data Factory for orchestrating and automating data movement and transformation.
• Proficiency in Azure Databricks for building and managing scalable data processing frameworks.
• Strong programming skills in Python and experience working with PySpark for big data processing.
• Knowledge of Azure DevOps and experience with CI/CD pipeline implementation.
• Familiarity with SQL for querying and managing large datasets.
• Strong problem-solving skills and the ability to work independently or as part of a team.
• Excellent communication and documentation skills
Skills
Sign in to read and join the discussion.