Lead Azure Data Engineer (Databricks)
Summary
Lead Azure Data Engineer architecting Azure cloud data platforms and Medallion lakehouses (Bronze/Silver/Gold), building batch and streaming ETL pipelines with Azure Databricks, Data Factory, and PySpark, and leading engineering teams through delivery, code reviews, and CI/CD for enterprise client analytics projects.
We are seeking a highly skilled Lead Azure Data Engineer for our client,a leading global consulting firm. The ideal candidate will have a proven track record in architecting modern cloud data landscapes, hands-on experience building Medallion lakehouses, and a strong background in leading engineering teams to drive digital transformation initiatives.
- Job Title: Lead Azure Data Engineer (Databricks)
- Location: India (Noida / Mumbai / Bengaluru /Gurgaon/ Kolkata/Bhubaneswar/Chennai/Coimbatore)
- Level: Lead / Senior Consultant / Manager
Application Steps
Do not apply without hands-on experience in Azure Databricks and PySpark as your profile will be rejected.To help us assess technical skills quickly and prioritize your profile over others, please complete a 5-minute free interview here: https://www.whitetick.com
Responsibilities
- Architect Data Landscapes: Architect Azure cloud data landscapes and modern Medallion lakehouses (Bronze, Silver, Gold layers) for enterprise analytics.
- Pipeline Development: Build batch and real-time/streaming ETL pipelines using Azure Data Factory (ADF), Azure Databricks, and PySpark.
- Performance Optimization: Optimize complex Spark SQL jobs, Python data transformations, and cluster performance for maximum efficiency.
- Technical Leadership: Lead engineering teams through project delivery, conduct rigorous code reviews, and drive CI/CD deployment pipelines.
- Stakeholder Collaboration: Collaborate with clients and cross-functional teams to translate business requirements into scalable technical solutions using Azure data services.
Qualifications & Technical Stack
- Experience: 6 to 12 years of overall experience with a strong focus on Azure data engineering and analytics (including prior technical leadership/mentoring experience).
- Technical Stack:
- Core: Azure Databricks, Apache Spark, PySpark, Azure Data Factory (ADF), and Azure Data Lake Storage Gen2 (ADLS Gen2).
- Languages & Querying: Python, SQL, Spark SQL, and Azure Synapse Analytics.
- Preferred Certifications: Microsoft Certified: Azure Data Engineer Associate (DP-203) or Databricks Certified Data Engineer Professional.
- Soft Skills: Excellent problem-solving, communication, and client-facing consulting skills.
Unfortunately, due to high volume of applicants, not everybody will receive a response.
About Us
curatAId is a hiring platform for the tech industry in India, providing pre-vetted candidates to recruiters. Our expert interviewers screen candidates based on their primary and secondary technical skills, as well as communication and behavioral traits, to provide recruiters with high-quality, qualified candidates. As a tech hiring platform, our goal is to simplify and streamline the recruitment process for both recruiters and job seekers.