Azure Data Engineer

Job Description: Azure Data Engineer / Data Engineering Team Lead

Location: Bangalore
Experience: 5+ Years
Qualification: 15 Years of Full-Time Education
Position: Data Engineer / Team Lead

Role Overview

We are seeking skilled Data Engineering professionals with hands-on experience in Azure Databricks, PySpark, Python, and cloud-based data solutions. The ideal candidate will be responsible for building and optimizing data pipelines, implementing ETL/ELT processes, and developing scalable data architectures.

Candidates with 5+ years of relevant experience and strong technical leadership skills are also encouraged to apply for the Team Lead position.

Key Responsibilities

• Develop, maintain, and optimize scalable data pipelines and ETL/ELT workflows.
• Implement data engineering solutions using Azure Databricks, Azure Data Factory, and other Azure data services.
• Build data processing applications using Python, PySpark, and Apache Spark.
• Manage data ingestion, transformation, integration, and storage across cloud platforms.
• Work with Azure Data Lake Storage Gen2 and Azure Blob Storage to manage structured and unstructured data.
• Design robust data architectures that support business requirements and large-scale data processing.
• Implement data validation, data quality checks, and monitoring mechanisms.
• Troubleshoot pipeline failures and improve data processing performance and reliability.
• Implement CI/CD practices for automated deployment and maintenance of data solutions.
• Collaborate with cross-functional teams to translate business requirements into effective data solutions.
• For Team Lead candidates, provide technical guidance, mentor team members, coordinate delivery activities, and contribute to architectural decisions.

Required Technical Skills

• Strong experience in Data Engineering and Data Architecture.
• Hands-on expertise in Azure Databricks and Microsoft Azure Data Services.
• Proficiency in Python, PySpark, and Apache Spark.
• Experience developing ETL/ELT processes and data pipelines.
• Knowledge of Azure Data Factory and Azure Data Lake Storage Gen2.
• Understanding of cloud data platforms, data modeling, and data quality practices.
• Experience in performance tuning, pipeline monitoring, troubleshooting, and CI/CD.

Good-to-Have Skills

• Azure Synapse Analytics
• Microsoft Fabric
• Azure Blob Storage
• DevOps and automated deployment
• Cloud data architecture and optimization
• Technical leadership and team management

Candidate Requirements

• Minimum 5 years of relevant Data Engineering experience.
• Strong hands-on experience with Azure Databricks and Azure data technologies.
• Excellent analytical, problem-solving, and communication skills.
• Ability to work independently and collaborate with cross-functional teams.
• Candidates with relevant technical leadership experience may apply for the Team Lead position.
• 15 years of full-time education.
• Willingness to work from the Bangalore office.