Data Engineering Lead
Posted Updated
Summary
Verdantas is hiring a Data Engineering Lead in Pune (in-office) to design scalable cloud data platforms on Azure and Microsoft Fabric, build batch/real-time ETL/ELT pipelines with Spark, Airflow, dbt, and Azure Data Factory, and mentor a team of data engineers supporting AI/ML, analytics, and web applications.
Join Verdantas – A Top #ENR 81 Firm
Verdantas is seeking a Data Engineering Lead to design and deliver scalable, secure, and modern data solutions that support AI/ML, reporting, analytics, and web applications. The ideal candidate will have strong expertise in Microsoft Azure, Microsoft Fabric, data platform architecture, and team leadership.
Key Responsibilities
Architecture & Strategy
- Design and implement batch and real-time data architectures using modern cloud platforms.
- Define data engineering standards, governance, and best practices.
- Evaluate and recommend technologies for data ingestion, processing, storage, and reporting.
Development & Operations
- Lead development of ETL/ELT pipelines using Spark, Airflow, dbt, Azure Data Factory, and Microsoft Fabric.
- Build and manage data lakes, lakehouses, data warehouses, and data marts.
- Ensure data quality, lineage, observability, and governance.
- Develop scalable data solutions supporting enterprise reporting, dashboards, and web applications.
Leadership & Collaboration
- Mentor data engineers and drive technical excellence.
- Collaborate with business stakeholders, analysts, data scientists, and application teams.
- Manage project delivery, timelines, and resource allocation.
Cloud & DevOps
- Build and manage cloud-based data platforms on Azure and Microsoft Fabric.
- Implement CI/CD pipelines and Infrastructure as Code (IaC).
- Optimize performance, scalability, and cost efficiency.
Skills & Qualifications
- Bachelor's or Master's degree in Computer Science or related field.
- 8+ years of Data Engineering experience with 2+ years in a leadership role.
- Strong SQL and Python skills.
- Experience with Spark, Flink, ETL/ELT frameworks, and modern data architectures.
- Hands-on experience with Microsoft Azure services such as Azure Data Factory, Synapse, Data Lake, and Databricks.
- Experience with Microsoft Fabric (OneLake, Lakehouse, Warehouse, Data Factory, Power BI integration).
- Experience building data solutions for reporting, analytics, dashboards, and web applications.
- Strong understanding of data modeling, governance, metadata management, and cloud-native architectures.
- Excellent communication and stakeholder management skills.
Preferred Certifications
- Microsoft Certified: Azure Data Engineer Associate
- Microsoft Fabric Analytics Engineer Associate
- Google Professional Data Engineer
- AWS Certified Data Analytics – Specialty
Location and Work Set-up
Pune, Maharashtra, India
Work Mode: In Office
Skills
- AI
- Airflow
- Analytics
- AWS
- Azure
- Azure Data Factory
- CI/CD
- Cloud
- Cloud Native
- Data Analytics
- Data Engineering
- Data Ingestion
- Data Lake
- Data Modeling
- Data Quality
- Databricks
- dbt
- DevOps
- ELT
- ETL
- Flink
- Infrastructure as Code
- Lakehouse
- Machine Learning
- Metadata Management
- Microsoft Fabric
- Observability
- Power BI
- Python
- Spark
- SQL
- Stakeholder Management