Lead Data Engineer
We at Coforge are looking for "Lead Data Engineer" for Dubai, UAE Location:
Detailed Job Description
Key Responsibilities
- Design, develop, and maintain scalable data pipelines and ETL/ELT solutions.
- Build Data Lake, Data Warehouse, BI, and Analytics platforms.
- Convert business requirements into robust technical solutions.
- Ensure data quality, security, governance, and performance optimization.
- Perform code reviews, troubleshooting, and production support.
- Collaborate with cross-functional teams on architecture and data engineering best practices.
- Maintain metadata, data lineage, and data governance standards.
Technical Skills
- Cloud & Big Data: Azure, Cloudera, Hadoop, Spark, Scala, Hive, HBase, Kafka
- Data Engineering: Azure Data Factory (ADF), Databricks, Airflow, Apache NiFi, Oozie, Sqoop
- Storage & Processing: HDFS, ADLS Gen2, Snowflake, Streaming (Spark Streaming)
- Data Integration: SnapLogic, TIBCO
- Databases & BI: SQL, Power BI, MicroStrategy, Elasticsearch
- Version Control & DevOps: Git, Bitbucket, Azure DevOps
- File Formats: AVRO, PARQUET
Desired Profile
- Strong hands-on experience in end-to-end Data Engineering.
- Familiarity with Data Lake, Data Warehouse, BI, and Analytics solutions.
- Exposure to Agile/Waterfall methodologies.
- Excellent problem-solving, communication, and stakeholder management skills.
- Team player with a strong learning mindset.
Skills
- Agile
- Airflow
- Analytics
- Azure
- Azure Data Factory
- Azure DevOps
- Bitbucket
- Cloud
- Data Engineering
- Data Governance
- Data Lake
- Data Lineage
- Data Pipelines
- Data Quality
- Data Warehousing
- Databricks
- DevOps
- Elasticsearch
- ELT
- ETL
- Git
- Hadoop
- HBase
- Hive
- Kafka
- NiFi
- Parquet
- Power BI
- Scala
- Snowflake
- Spark
- SQL
- Stakeholder Management
- Version Control