Data Engineer – Azure Databricks SME
NewBe an early applicantSummary
Senior (7–8+ yrs) data engineer role in Bangalore acting as an Azure Databricks subject-matter expert: building ETL/ELT pipelines with PySpark, Python, SQL and Scala on Delta Lake/Medallion architecture, using ADLS Gen2, Azure Data Factory, Databricks Workflows/DLT/Structured Streaming, Unity Catalog, and Azure DevOps CI/CD.
Job Role: Data Engineer – Azure Databricks SME
Experience: 7–8+ Years
Work Location: Bangalore
Job Type: Full-time
Required Skills:
- Hands on experience in Data Engineering.
- Strong hands-on experience with Azure Databricks.
- Good knowledge of PySpark, Python, SQL, and Scala.
- Experience in building ETL/ELT data pipelines.
- Strong understanding of Delta Lake and Medallion Architecture.
- Experience with Azure Data Lake Storage (ADLS Gen2).
- Knowledge of Azure Data Factory (ADF) for data pipeline orchestration.
- Experience with Databricks Jobs, Workflows, DLT, and Structured Streaming.
- Good understanding of Unity Catalog, data governance, and security.
- Experience with Azure DevOps and CI/CD pipelines.
- Knowledge of Bicep / Infrastructure as Code is preferred.
- Experience with Kafka or Azure Event Hubs is a plus.
- Strong experience in performance tuning and cost optimization in Databricks.
- Good understanding of data quality, validation, and data modeling.
- Experience working with large-scale data / big data environments.
- Good communication and stakeholder management skills.
- Ability to mentor junior data engineers.
- Azure certification such as AZ-900 or higher is preferred.
- Exposure to Power BI, Azure ML, or Microsoft Purview is an advantage.