Data Engineer (Azure Data Platforms)
Posted
Summary
Data Engineer in Bengaluru (onsite) who architects and maintains Azure-based data platforms: building end-to-end pipelines with Azure Data Factory, Databricks, and Synapse, streaming with Kafka and Airflow, storing data in Cosmos DB and Blob Storage, and governing it via Azure Purview. Targets ~3-6 years of experience at ₹10 LPA.
Data Engineer (Azure Data Platforms)
Summary
We are seeking a skilled Data Engineer to design, build, and maintain robust data solutions within the IT Services sector. In this pivotal role, you will leverage advanced Azure technologies to architect scalable data pipelines and ensure the integrity and accessibility of critical business information. Your expertise will drive the optimization of data workflows, enabling seamless integration across diverse platforms and supporting strategic decision-making through reliable data infrastructure.
Responsibilities
- Architect and develop end-to-end data solutions utilizing the Azure ecosystem, including Azure Data Factory, Azure Databricks, and Azure Synapse Analytics.
- Implement real-time data streaming and processing capabilities using Apache Airflow, Kafka, and Azure Functions.
- Manage and optimize data storage strategies across Cosmos DB and Blob Storage to ensure high availability and performance.
- Establish comprehensive data governance and lineage tracking frameworks using Azure Purview to maintain compliance and data quality.
- Collaborate with cross-functional teams to translate business requirements into technical data specifications and scalable architectures.
- Monitor data pipeline performance, troubleshoot issues, and implement continuous improvements to enhance system efficiency.
Requirements
Requirements:
- Possess 3-6 years of hands-on professional experience in data engineering within the IT Services industry.
- Demonstrate proven proficiency with Azure data services, specifically Azure Data Factory, Azure Databricks, Azure Synapse Analytics, and Azure Functions.
- Exhibit strong expertise in managing NoSQL databases (Cosmos DB) and object storage solutions (Blob Storage).
- Have practical experience implementing streaming data architectures using Apache Airflow and Kafka.
- Understand and apply data governance principles using tools like Azure Purview.
- Hold a competitive salary expectation aligned with the market rate of ₹10 LPA for the specified experience level.