Senior Azure Data Engineer - ETL/Data Warehousing
Posted Updated
Description
Senior Azure Data Engineer - ETL/Data Warehousing
Company: HR Works Consultancy
Job Location: Hyderabad, Telangana, India
Employment Type: Full-time
Job Description :
Design and implement scalable data pipelines using Azure Data Factory and Azure Databricks
Develop and optimize ETL/ELT processes for large-scale distributed data processing
Build and maintain data lake and lakehouse architectures on Azure cloud platform
Write efficient SQL and PySpark code for data transformation and analysis
Implement Lakeflow Declarative Pipelines for streamlined data workflows
Perform performance tuning and optimization of data processing systems
Manage version control using Git and implement CI/CD pipelines for data solutions
Collaborate with data scientists, analysts, and stakeholders to understand data requirements
Ensure data quality, security, and governance standards are met across all pipelines
Mentor junior team members and contribute to technical best practices
Required Skills & Experience :
Mandatory Requirements :
Azure Data Factory (ADF) : Strong hands-on experience in designing and managing data pipelines
Azure Databricks : Proficiency in building and optimizing distributed data processing solutions
Apache Spark : In-depth knowledge of Spark architecture and distributed computing concepts
Lakeflow Declarative Pipelines : Practical experience with declarative pipeline development
SQL : Expert-level proficiency in writing complex queries and data manipulation
PySpark : Strong programming skills in PySpark for data transformation
Azure Cloud Platform : Hands-on experience with Azure data services and cloud infrastructure
ETL/ELT Concepts : Strong understanding of data integration patterns and methodologies
Data Warehousing Principles : Knowledge of dimensional modeling, schema design, and best practices
Distributed Data Processing : Experience working with large-scale data systems and handling big data challenges
Performance Tuning & Optimization : Ability to identify bottlenecks and optimize data pipeline performance
Data Lake & Lakehouse Architectures : Understanding of modern data architecture patterns and implementation
Git & CI/CD : Experience with version control systems and continuous integration/deployment practices
Strong experience with Azure Data Factory (ADF) and Azure Databricks
Hands-on expertise in Apache Spark and PySpark
Experience with Lakeflow Declarative Pipelines
Strong proficiency in SQL and data modeling
Solid understanding of ETL/ELT processes and data warehousing concepts
Experience building and managing Data Lake/Lakehouse architectures
Hands-on experience with Azure cloud data services
Proven experience processing and optimizing large-scale data workloads
Strong knowledge of performance tuning for Spark/Databricks environments
Experience with Git, version control, and CI/CD implementation
Notice Period Immediate to 30days only .
Skills
Sign in to read and join the discussion.