Data Engineer
Data Engineer (Microsoft Fabric / SQL / PySpark)
Location: Newcastle Upon Tyne | Hybrid
Salary: Competitive + Benefits
Type: Full-time, Permanent
Overview
A leading technology and financial services organisation is looking to appoint an experienced Data Engineer to join its growing Data Engineering function.
This is a hands-on engineering role focused on transforming complex transactional data into scalable, high-quality analytical datasets. You will work extensively with Microsoft Fabric, SQL and PySpark, designing OLAP models, developing data pipelines and creating the data structures that underpin Power BI reporting, analytics and Data Science.
Working closely with Developers, Analytics and Data Science teams, you will play an important role in modernising the organisation's data platform, migrating legacy reporting solutions and ensuring reliable data is available for business-critical decision making.
Key Responsibilities
- Design and develop OLAP data models, including facts and dimensions, from complex OLTP systems
- Build, maintain and optimise data pipelines within Microsoft Fabric
- Develop large-scale data transformations using Spark and PySpark
- Write, optimise and troubleshoot complex T-SQL queries and procedures
- Build ETL and ELT pipelines integrating internal and external data sources
- Migrate legacy reporting data and processes into Microsoft Fabric
- Design data warehouse models to support reporting and analytical requirements
- Create engineered datasets and summary tables for Power BI and Analytics teams
- Develop aggregated datasets to support Data Science and predictive modelling
- Monitor data quality, pipeline performance, reliability and processing efficiency
Requirements
- Strong commercial experience with T-SQL and complex SQL development
- Hands-on experience with Spark and PySpark for data processing and transformation
- Strong understanding of OLTP and OLAP architectures
- Experience designing dimensional data models using facts and dimensions
- Experience with Microsoft Fabric and wider Azure data technologies
- Strong knowledge of ETL and ELT processes and data integration
- Experience developing and optimising production data pipelines
- Strong understanding of data quality, governance and security principles
- Azure Data Lake, Data Warehouse and Blob Storage experience
- Experience supporting Data Science and predictive modelling teams
- Experience migrating legacy data platforms into Microsoft Fabric
What's on Offer
- Competitive salary plus bonus and benefits
- Hybrid working arrangement
- Opportunity to work extensively with Microsoft Fabric, SQL and PySpark
- Work on large-scale data transformation and modernisation projects
- Exposure to Analytics, Power BI and Data Science initiatives
- Excellent technical development and career progression opportunities