Azure Data Engineer
Summary
A lead data engineer designs, builds, and maintains end-to-end data pipelines and warehouse solutions on Microsoft Azure, using Azure Data Factory, Synapse Analytics, Databricks/Spark, and ADLS Gen2 to ingest, transform, and validate data for reporting and business decision-making.
We are seeking a highly skilled and passionate Lead Data Engineer to join our growing Data & Analytics team. In this role, you will be responsible for designing, developing, and managing end-to-end data solutions on the Microsoft Azure platform. You will play a critical role in enabling reliable data ingestion, transformation, storage, and analytics capabilities that support business operations, reporting, and strategic decision‑making.
The ideal candidate will have strong experience in Azure Data Engineering technologies, data warehousing, ETL/ELT development, and cloud-native data architectures, with the ability to translate business requirements into scalable and efficient data solutions.
Key Responsibilities
- Design and develop end-to-end data pipelines using Azure Data Factory (ADF).
- Ingest data from REST APIs, SharePoint, Excel, and ADLS Gen2.
- Build scalable ETL/ELT solutions using ADF Mapping Data Flows and metadata-driven frameworks.
- Develop and optimize data models, data warehouses, and analytics solutions using Azure Synapse Analytics.
- Leverage Azure Databricks and Apache Spark for large-scale data processing.
- Implement data quality, validation, and monitoring processes to ensure reliable datasets.
- Collaborate with business and technical teams to deliver robust data solutions.
- Monitor, troubleshoot, and enhance production data pipelines.
Required Skills
- 8+ Years of Strong hands‑on experience with Azure Data Factory (ADF), including pipeline orchestration and Mapping Data Flows.
- Experience with Azure Synapse Analytics, ADLS Gen2, and Azure Databricks.
- Expertise in REST API integrations, authentication, pagination, and error handling.
- Strong knowledge of ETL/ELT, data pipeline design, incremental loading, and data validation.
- Experience in Data Warehousing, Dimensional Modeling, Star Schema, and Snowflake Schema.
- Good understanding of metadata-driven architecture and scalable cloud data platforms.
Preferred Skills
- Experience with Azure DevOps (CI/CD pipelines).
- Hands‑on exposure to Azure Functions and Azure Logic Apps.
- Knowledge of Microsoft Fabric and modern data architecture patterns.
- Azure certifications such as DP-203 are a plus.