Senior Data Engineer
Posted Updated
Roles and Responsibilities :
- Design, develop, and maintain large-scale data pipelines using Azure Data Factory (ADF) to extract, transform, and load data from various sources into Azure Data Lake Storage.
- Collaborate with cross-functional teams to gather requirements for ETL processes and design scalable data models for complex datasets.
- Develop SQL queries and Power BI reports to analyze business intelligence insights from the extracted data.
- Troubleshoot issues related to ADF pipeline failures, data quality problems, and performance optimization.
Job Requirements :
- 7+ years of experience in Data Engineering, Data Platforms, Analytics Engineering, or a related technology role.
- Strong hands-on expertise in SQL, Python, PySpark/Spark, ETL/ELT, data modelling, integration, and data warehousing.
- Experience with Azure Data Factory, Azure Synapse Analytics, Azure Data Lake Storage, Azure Databricks, or comparable cloud data technologies.
- Strong understanding of Data Lake, Data Warehouse, Lakehouse, Medallion Architecture, APIs, Git, CI/CD, DevOps, and automated testing.
- Proven project delivery experience covering planning, estimation, scheduling, risk and dependency management, stakeholder engagement, and vendor coordination.
- Strong knowledge of data quality, governance, security, monitoring, performance optimization, and production support.
- Excellent communication, analytical thinking, problem-solving, and stakeholder-management skills
Skills
- Analytics
- API
- Azure
- Azure Data Factory
- Azure Synapse
- Business Intelligence
- CI/CD
- Cloud
- Data Engineering
- Data Lake
- Data Modeling
- Data Pipelines
- Data Quality
- Data Warehousing
- Databricks
- DevOps
- ELT
- ETL
- Git
- Lakehouse
- Medallion Architecture
- Power BI
- PySpark
- Python
- Spark
- SQL
- Stakeholder Management
- Test Automation