Senior Data & AI Engineering Consultant
Description
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Data & AI Engineering Consultant based in the United States.
As a Senior Data & AI Engineering Consultant, you will help organizations transform complex data environments into modern, scalable, and trusted platforms that power analytics and AI innovation. Working across Microsoft Azure and Microsoft Fabric, you will lead client engagements from initial discovery and architecture design through implementation and delivery. You will build robust data pipelines, develop analytics-ready datasets, and establish the governance and security foundations required for enterprise data solutions. Collaborating with business stakeholders, architects, and technical teams, you will translate business needs into practical, cost-effective engineering solutions. You will also help clients adopt modern data practices and prepare their platforms for advanced analytics, business intelligence, and AI-driven applications. This role combines hands-on engineering, technical leadership, and client-facing consulting in a collaborative Microsoft-focused environment.
Accountabilities:
You will lead the design and delivery of modern data engineering solutions, ensuring that client platforms are reliable, secure, governed, scalable, and optimized for analytics and AI workloads.
-
Lead client discovery sessions to understand business objectives, existing data environments, technical constraints, and success metrics, translating findings into actionable delivery plans.
-
Design and recommend data lake, lakehouse, data warehouse, and analytical data store architectures using Microsoft Azure and Microsoft Fabric.
-
Build and oversee data ingestion, transformation, and orchestration pipelines using Microsoft Fabric and Azure services, including Data Factory, Synapse, and Databricks.
-
Integrate data from operational systems, APIs, and third-party sources, transforming raw information into curated, analytics-ready datasets.
-
Develop and maintain analytical data models, schemas, enrichment processes, business rules, and reusable engineering patterns.
-
Enable multiple compute engines, including SQL, Spark/PySpark, Spark SQL, and KQL, to support diverse analytics requirements.
-
Prepare governed data products and reusable datasets for business intelligence, advanced analytics, and AI applications, including retrieval-ready data patterns where appropriate.
-
Establish data quality frameworks covering validation, reconciliation, automated testing, and monitoring, while troubleshooting failures and improving pipeline reliability.
-
Optimize data transformations, query performance, and pipeline execution to meet service-level expectations and cost objectives.
-
Implement logging, operational metrics, monitoring practices, and runbooks, supporting deployments, cutovers, and post-launch stabilization when required.
-
Design and implement security, governance, and compliance controls, including access management, least-privilege principles, data protection, and lineage across Azure and Fabric environments.
-
Facilitate technical workshops and communicate architectural recommendations and trade-offs to client stakeholders, architects, analysts, and BI developers.
-
Produce consulting deliverables such as architecture diagrams, design documentation, implementation guides, and enablement materials to support adoption and effective knowledge transfer.
Requirements:
The ideal candidate combines strong data engineering expertise with experience delivering modern Microsoft-based analytics platforms and the communication skills needed to guide clients through complex technical decisions.
-
Experience designing and delivering modern analytics platforms using lakehouse and/or data warehouse architectures on Microsoft Azure and/or Microsoft Fabric, including client-facing consulting engagements.
-
Strong SQL skills, with experience developing data transformations and dimensional or analytical data models.
-
Hands-on experience with Apache Spark, including PySpark and Spark SQL, and/or KQL for data engineering and analytics workloads.
-
Experience applying software engineering best practices to data solutions, including modular design, source control, automated testing, code reviews, and technical documentation.
-
Proven ability to build reliable data pipelines, including orchestration, scheduling, error handling, and automated monitoring.
-
Knowledge of data quality practices, including validation, reconciliation, testing, incident triage, and root-cause analysis.
-
Working knowledge of Power BI and semantic modeling concepts to support end-to-end analytics delivery. Experience with Power Apps is an advantage.
-
Understanding of data platform security and governance, including access controls, least-privilege principles, compliance requirements, and data lineage.
-
Strong communication and presentation skills, with the ability to explain technical recommendations, align stakeholders, and manage expectations in client-facing environments.
Preferred Qualifications:
-
Direct experience with Microsoft Fabric components, including OneLake, Lakehouse, Warehouse, Data Factory in Fabric, and Real-Time Analytics/KQL databases, as well as platform administration, capacity planning, and standards.
-
Experience with Azure data platform services such as Azure Data Factory, Synapse, Databricks, Azure Data Lake Storage Gen2 (ADLS Gen2), and Key Vault.
-
Working knowledge of Azure architecture fundamentals, including identity, networking, storage, monitoring, resiliency, and cost management.
-
Familiarity with AI enablement patterns, such as preparing governed enterprise data for Azure AI Search, Microsoft Foundry, retrieval-augmented generation (RAG), and agent-based solutions. Deep agent application development experience is not required.
-
Experience implementing CI/CD pipelines and Infrastructure as Code (IaC), including environment strategies and release management for data solutions.
-
Familiarity with semantic modeling and BI enablement practices, including Power BI datasets, self-service analytics, and governance guardrails.
-
Relevant Microsoft certifications are valued, particularly Microsoft Certified: Fabric Data Engineer Associate (DP-700), Microsoft Certified: Power BI Data Analyst Associate (PL-300), and Microsoft Certified: Azure Solutions Architect Expert (AZ-305).
-
Equivalent current Microsoft credentials and experience mentoring or enabling client teams are also welcome.
Benefits:
-
A flexible and collaborative working environment that combines the agility of a smaller company with the stability of an established organization.
-
A family-first company culture that encourages work-life balance and employee well-being.
-
A supportive and innovative team environment focused on professional development, collaboration, and technical excellence.
-
The opportunity to work on modern Microsoft Cloud technologies and impactful data and AI transformation projects.
-
A workplace recognized with a Best Places to Work Award for five consecutive years.
-
An equal-opportunity employer committed to fair and inclusive hiring practices, considering all applicants without discrimination based on protected characteristics.
Skills
- AI
- Ai Enablement
- Analytics
- API
- Azure
- Azure Data Factory
- Business Intelligence
- CI/CD
- Cloud
- Data Engineering
- Data Ingestion
- Data Lake
- Data Lineage
- Data Pipelines
- Data Quality
- Data Warehousing
- Databricks
- Gdpr
- Infrastructure as Code
- KQL
- Lakehouse
- Microsoft Fabric
- Networking
- Power Apps
- Power BI
- PySpark
- RAG
- Root Cause Analysis
- Spark
- SQL
- Test Automation
- Vault
As published by lever
Resume/CV, Full name, Email, Phone, Current location, Current company
Sign in to read and join the discussion.