Senior Data Engineer - West Yorkshire - £80k
Posted 6
views
Summary
Senior Data Engineer designing end-to-end ETL/ELT pipelines in Microsoft Fabric (Dataflow Gen2, Fabric Notebooks, Lakehouse/Medallion architecture) with SQL, Python and Spark, plus CI/CD, data quality and semantic modelling. Hybrid role, 2 days onsite in West Yorkshire, £70-80k.
Senior Data Engineer Microsoft Fabric
Location: West Yorkshire
Working pattern: Hybrid, 2 days per week onsite
Salary: £70-80k
Key responsibilities
- Design and build end to end ETL and ELT solutions using Microsoft Fabric
- Build scalable data pipelines using Dataflow Gen2 and Fabric Notebooks
- Develop and maintain Lakehouse solutions using Medallion architecture
- Work with SQL, Python, Spark SQL and Delta Lake
- Integrate data from APIs, databases, CRM, ERP and third party platforms
- Support both batch and real time data processing
- Develop semantic models and work with Star Schema and Kimball modelling
- Implement CI/CD and automated deployment of data solutions
- Build monitoring, data quality, lineage and observability into data pipelines
- Work closely with Data Architects, BI teams and wider business stakeholders
Essential experience
- Strong commercial experience with Microsoft Fabric
- Hands on experience with Dataflow Gen2, Fabric Notebooks and Lakehouse architecture
- Strong SQL and Python skills
- Experience with Spark and Spark SQL
- Proven experience designing scalable ETL and ELT pipelines
- Data Lake and Delta Lake experience
- Experience integrating data from multiple structured and unstructured sources
- Good understanding of CI/CD, DevOps and modern data engineering practices
- Able to work onsite in West Yorkshire 2 days per week
Experience with Semantic Models, DAX, Kimball modelling, Microsoft Fabric certifications such as DP 700, and data governance would also be beneficial