Lead Data Engineer - Azure

Job Description :

Role & Responsibilities :

- Lead the design, development, and implementation of scalable Azure data engineering solutions.

- Design and develop high-volume ETL/ELT pipelines using Azure Data Factory, Azure Synapse, Databricks, PySpark and SQL.

- Build and optimize data lakes, data warehouses, Azure SQL and Synapse Dedicated SQL Pools.

- Design robust data models, datasets and analytical solutions to support business and reporting requirements.

- Develop data ingestion and migration solutions from legacy/on-premise systems to Azure.

- Handle large-scale and petabyte-level data processing with a focus on performance, reliability and scalability.

- Monitor, troubleshoot and optimize data pipelines, queries and data-processing workloads.

- Implement data quality, security, access control, governance and compliance standards.

- Collaborate with Product, Marketing, BI, Data Science and ML teams to understand requirements and deliver data solutions.

- Support Power BI, Tableau and real-time/offline analytics initiatives.

- Provide technical leadership, conduct code/design reviews, and establish engineering best practices.

- Mentor and guide junior and senior data engineers, supporting their technical growth.

- Participate in recruitment and help build and grow the Data Analytics/Data Engineering team.

- Drive continuous improvement, automation and operational excellence across the data platform.

Preferred Candidate Profile :

- 7+ years of experience in Data Engineering with strong Azure cloud expertise.

- Strong hands-on experience with Azure Data Factory, Azure Synapse, Azure Databricks, Azure SQL and Azure Data Lake.

- Excellent knowledge of PySpark, SparkSQL and SQL.

- Strong experience in ETL/ELT, data pipelines, data migration, data modelling, data warehousing and data lakes.

- Experience with Synapse Dedicated SQL Pool / Azure SQL Data Warehouse and large-scale data processing.

- Experience handling high-volume event data and petabyte-scale datasets is highly desirable.

- Good knowledge of SSIS, Azure Analysis Services, Power BI and Tableau.

- Strong understanding of data security, RBAC, access management, governance and data quality.

- Experience with performance tuning, monitoring and troubleshooting production data pipelines.

- Proven ability to lead, mentor and guide data engineering teams.

- Strong communication and stakeholder-management skills with experience working across Product, BI, Analytics, Data Science and ML teams.

- Experience working in Agile/Scrum environments.

- Strong analytical, problem-solving and solution-design capabilities.

- Exposure to modern data platform/Data Fabric architecture is an added advantage.