Senior Fabric & Databricks Data Engineer
Summary
Designs and builds scalable data pipelines using Microsoft Fabric and Azure Databricks, focusing on API-driven ingestion, cross-platform data integration, and Power BI analytics with Direct Lake.
We are looking for a highly skilled Data Engineer with strong end-to-end experience in Microsoft Fabric and Azure Databricks. The ideal candidate will have deep expertise in data ingestion frameworks, modern data engineering architectures, and the integration of data across Fabric and Databricks environments.
This role will be responsible for designing and implementing scalable data pipelines, ingesting data from multiple sources, mirroring Databricks data into Microsoft Fabric, and enabling analytics through blended Semantic Models and Power BI reports using the Direct Lake format. The position will involve working with a U.S.-based client, so fluent English is mandatory.
Key Responsibilities
- Design, develop, and maintain end-to-end data engineering solutions using Microsoft Fabric and Azure Databricks.
- Build scalable and reliable data ingestion frameworks to integrate data from multiple sources into Fabric and Databricks.
- Develop API-based data ingestion solutions and frameworks within Microsoft Fabric.
- Implement data pipelines and transformation processes using Azure Databricks.
- Work extensively with Databricks Delta Live Tables (DLT), Auto Loader, and Unity Catalog.
- Design and implement data architectures that support data ingestion, transformation, governance, and consumption.
- Mirror Databricks data into Microsoft Fabric to enable integrated analytics and reporting.
- Develop blended Semantic Models combining data from Fabric and Databricks.
- Build and optimize Power BI reports using the Direct Lake format.
- Ensure data pipelines are scalable, performant, reliable, and maintainable.
- Apply best practices around data quality, governance, security, lineage, and access management.
- Collaborate with data architects, BI developers, analysts, and business stakeholders to translate requirements into technical solutions.
Required Skills & Experience
- Strong hands-on experience with Microsoft Fabric Data Engineering.
- Expert-level experience designing and implementing API-based data ingestion frameworks in Microsoft Fabric.
- Strong experience with Azure Databricks Data Engineering.
- Proven experience ingesting and processing data from multiple sources, including APIs and enterprise data platforms.
- Hands-on experience with Databricks Delta Live Tables (DLT).
- Strong knowledge of Databricks Auto Loader.
- Hands-on experience with Unity Catalog and Databricks data governance.
- Experience designing and implementing end-to-end data pipelines.
- Experience integrating and mirroring Databricks data into Microsoft Fabric.
- Strong understanding of Microsoft Fabric Lakehouse architecture.
- Experience building blended Semantic Models in Microsoft Fabric / Power BI.
- Strong understanding of modern cloud data architecture and ELT/ETL patterns.
Preferred Qualifications
- Experience with Azure cloud data services and Microsoft data platforms.
- Experience with data governance, security, lineage, and access controls.
- Experience optimizing large-scale data pipelines and analytical workloads.
- Strong understanding of Delta Lake and medallion architecture.
- Experience working in Agile/Scrum environments.
- Strong problem-solving and communication skills.
Ideal Candidate
The ideal candidate is a senior-level data engineering professional who can independently design and deliver modern data solutions across both Microsoft Fabric and Azure Databricks, with particular strength in API-driven ingestion, Databricks engineering, cross-platform data integration, and Power BI Direct Lake analytics.