Senior Data Engineer
Summary
Designs and optimizes scalable data platforms and pipelines using Microsoft Fabric, Azure, Spark, SQL, and Python to enable analytics, data integration, and governance for hybrid work environments.
Location: London, UK
Work mode: Hybrid (3 days weekly from office)
Skills: Data Engineering, Microsoft Fabric, Cloud Data Engineering, Fabric Data Factory, Lakehouse, Data Warehouse, Dataflows Gen2, and OneLake.
We at Coforge are looking for Sr. Data Engineer in London, UK.
Role Summary
Seeking an experienced Senior Data Engineer to design, build, and optimize scalable data platforms and pipelines using Microsoft Fabric, Azure, Spark, SQL, and Python. The role focuses on enabling analytics, data integration, governance, and cloud-based data solutions.
Key Responsibilities
- Design and develop ETL/ELT pipelines using Microsoft Fabric, Spark, and SQL.
- Build batch and real-time data ingestion solutions.
- Develop solutions using Fabric Data Factory, Lakehouse, Data Warehouse, Dataflows Gen2, and OneLake.
- Create and optimize data models for reporting, analytics, and BI.
- Implement data quality, governance, and lineage practices using Microsoft Purview.
- Develop streaming and CDC solutions using Kafka, Event Hub, and Event Streams.
- Optimize data pipelines, queries, and cloud resources for performance and cost efficiency.
- Collaborate with Architects, BI teams, and DevOps teams in Agile environments.
- Implement CI/CD, Git integration, automation, and deployment best practices.
Required Skills
- Strong experience in SQL, Python, Spark, and Data Engineering.
- Hands-on experience with Microsoft Fabric (preferred), Azure Data Factory, Synapse, or Databricks.
- Good knowledge of Lakehouse Architecture, Delta Lake, Data Warehousing, and Data Modeling.
- Experience with APIs, JSON, Parquet, and streaming technologies.
- Strong analytical, troubleshooting, and communication skills.
Experience & Qualifications
- 6-10 years of Data Engineering experience.
- Bachelor's or Master's degree in Computer Science, IT, or related field.
- Microsoft Fabric/Azure Data Engineer certifications preferred.
- Ability to independently deliver complex data solutions.
- Experience implementing architecture standards and best practices.
- Mentoring junior engineers and contributing to technical design discussions.
Microsoft Fabric, OneLake, Data Factory, Lakehouse, Data Warehouse, Spark, SQL, Python, Azure Data Factory, Synapse, Databricks, Kafka, Event Hub, Purview, Power BI, Git, CI/CD.