Microsoft Fabric Data Engineer

Want to make an impact? We offer that

Due to continued growth, we are hiring a Microsoft Fabric Data Engineer to support the delivery of enterprise data and AI capabilities for our client in the utilities industry. The role will be responsible for the design, development, testing, deployment, operation and optimisation of enterprise data solutions using Microsoft Fabric, providing the data foundations required to support AI, analytics and business intelligence initiatives.

The successful candidate will combine strong hands-on data engineering capability with knowledge of Microsoft Fabric architecture, data governance, security, DevOps, semantic modelling, performance optimisation and production support.

Key Responsibilities:

Microsoft Fabric Data Engineering

• Design, build, test and deploy enterprise data engineering solutions using Microsoft Fabric.

• Develop scalable data ingestion, transformation and orchestration pipelines.

• Implement Fabric Lakehouse and Data Warehouse solutions.

• Develop notebooks and data transformations using PySpark and SQL.

• Implement robust data loading patterns, including full, incremental and change-based processing.

• Develop reusable data engineering patterns and frameworks.

• Support structured and unstructured data requirements associated with AI and analytics use cases.

• Implement appropriate error handling, monitoring, validation and recovery mechanisms.

OneLake and Data Architecture

• Design and implement data solutions using Microsoft OneLake.

• Apply appropriate Lakehouse, Warehouse, Delta and OneLake architecture patterns.

• Develop reusable data products and curated datasets for analytics and AI consumption.

• Implement appropriate Bronze, Silver and Gold data patterns where appropriate.

• Use OneLake shortcuts, mirroring and other Fabric capabilities where appropriate.

• Contribute to enterprise data architecture and Fabric platform design decisions.

Data Modelling and Analytics Enablement

• Design and implement dimensional and analytical data models.

• Develop fact and dimension structures and appropriate semantic layers.

• Support the development and optimisation of Power BI semantic models.

• Implement and optimise Direct Lake solutions where appropriate.

• Work with analytics engineers, BI developers and business stakeholders to deliver trusted analytical datasets.

• Support Power BI and Direct Lake enablement across analytics and AI landscape.

Data Security and Governance

• Implement secure data solutions aligned with security and governance standards.

• Apply appropriate workspace, item, table, column and row-level security controls.

• Support OneLake security and Fabric access-control patterns.

• Work with Cyber Security, DPO & Legal and Architecture teams to ensure appropriate data protection and access controls.

• Implement data quality, metadata, lineage and governance requirements.

• Ensure solutions comply with applicable information governance, privacy and security requirements.

DevOps and CI/CD

• Implement source control and CI/CD practices for Fabric solutions.

• Use Git-based development practices and appropriate branching strategies.

• Support automated deployment across development, test and production environments.

• Implement appropriate code review, testing and release processes.

• Develop automated validation and testing for data pipelines and transformations.

• Support deployment pipelines and environment promotion.

Performance and Cost Optimisation

• Monitor and optimise Fabric workloads and data pipelines.

• Identify and resolve performance bottlenecks across Spark, SQL, Lakehouse, Warehouse and semantic model workloads.

• Optimise data structures, queries, transformations and pipeline execution.

• Monitor Fabric capacity utilisation and identify opportunities for performance and cost optimisation.

• Recommend architectural or engineering changes that improve scalability, resilience and cost efficiency.

Production Support and Service Transition

• Support the transition of Fabric solutions into operational service.

• Monitor production data pipelines and investigate incidents and failures.

• Perform root-cause analysis and implement corrective actions.

• Develop operational documentation, runbooks and support procedures.

• Ensure appropriate monitoring, alerting and recovery mechanisms are implemented.

• Support knowledge transfer to teams and other support functions.

Migration and Modernisation

• Support migration of existing data workloads to Microsoft Fabric where required.

• Assess existing data platforms and recommend appropriate Fabric migration approaches.

• Support migration from technologies such as Azure Data Factory, Azure Synapse, SQL Server, Databricks and traditional data warehouse platforms where applicable.

• Validate migrated workloads and optimise them for Fabric-native capabilities.

AI and Data Enablement

• Provide data engineering capability supporting AI use cases.

• Develop trusted and appropriately governed datasets for AI and analytics consumption.

• Support data integration requirements for AI applications and automation.

• Collaborate with AI Engineers, AI Architects and Data/Analytics specialists.

• Help establish reusable data patterns that accelerate future AI use-case delivery.

Requirements:

• Minimum 5 years’ relevant data engineering experience, including substantial hands-on experience with Microsoft Fabric.

• Strong Microsoft Fabric data engineering skills, including Data Pipelines, Lakehouse, OneLake and Data Warehouse.

• Strong SQL/T-SQL and PySpark skills.

• Experience with data modelling and dimensional modelling, including developing and supporting Power BI semantic models and Direct Lake.

• Experience with Git, CI/CD, data security and access controls.

• Experience with data quality, metadata, lineage, monitoring and production support.

• Experience with performance and capacity optimisation of data solutions.

• Strong technical documentation and knowledge-transfer skills.

Desirable:

• Microsoft Certified: Fabric Data Engineer Associate / DP-700.

• Experience with Azure Data Factory, Azure Synapse and/or Databricks.

• Experience with Kusto Query Language (KQL) and Fabric Real-Time Intelligence.

• Experience with Fabric administration, capacity management and/or platform/solution architecture.

• Power BI development experience.

• Experience with data migration, modernisation and AI/ML data platforms.

• Experience working in regulated, critical infrastructure or similarly governed environments.

Opportunity type: Initial 6-month contract with possibility of extension, Hybrid in Dublin/Cork