Senior Data Engineer - Azure

We are seeking an experienced Senior Data Engineer for an initial three-month contract, with strong long-term potential and the opportunity to transition into a permanent position for the right person.

This position will suit a highly capable Data Engineer who combines strong Microsoft Azure and Fabric experience with an understanding of how reliable, governed data is prepared and delivered to analytics, artificial intelligence and agent-based solutions through Microsoft Foundry.

The opportunity:

You will work closely with architects, developers, analysts and business stakeholders to build scalable data pipelines, lakehouse and warehouse solutions, and trusted enterprise data products. A major focus will be integrating structured and unstructured organisational data with Microsoft Foundry to support generative AI, retrieval-augmented generation and intelligent agent solutions.

The initial contract will provide both parties with the opportunity to establish a successful working relationship, with a genuine pathway into an ongoing permanent position.

Key responsibilities:

• Design, develop and maintain scalable data pipelines across Microsoft Azure and Microsoft Fabric.
• Build modern lakehouse, warehouse and data integration solutions using Fabric, OneLake, Azure Data Factory, Azure Data Lake and related Microsoft technologies.
• Develop robust data transformation processes using SQL, Python and PySpark.
• Prepare and optimise structured and unstructured data for analytics, generative AI and Microsoft Foundry solutions.
• Integrate enterprise data with Microsoft Foundry models, agents, knowledge sources and APIs.
• Support retrieval-augmented generation solutions using Azure AI Search, vector indexing, embeddings and enterprise knowledge repositories.
• Implement data quality, lineage, security, monitoring and governance controls.
• Work with Microsoft Purview, Entra ID and role-based access controls to protect sensitive information.
• Establish automated deployment and testing processes using Azure DevOps, Git and CI/CD.
• Troubleshoot performance, reliability and data-quality issues across complex production environments.
• Translate business requirements into practical, scalable and supportable technical solutions.

Required experience:

• Extensive commercial experience as a Data Engineer within Microsoft-based enterprise environments.
• Strong hands‑on experience with Microsoft Fabric, Azure Data Factory, OneLake, Azure Data Lake, Synapse Analytics or Azure Databricks.
• Advanced SQL skills and commercial experience with Python or PySpark.
• Experience designing data pipelines, lakehouse architectures, data warehouses and dimensional models.
• Practical experience with Microsoft Foundry, formerly Azure AI Foundry, or closely related Azure AI services.
• Understanding of generative AI, AI agents, RAG, vector databases, embeddings and enterprise search.
• Experience integrating data platforms with APIs, applications and cloud-based services.
• Strong knowledge of data security, governance, privacy and production support practices.
• Excellent communication skills and the ability to work effectively with technical teams and business stakeholders.

Highly regarded:

• Experience with Microsoft Foundry Agent Service and Foundry IQ.
• Microsoft Fabric, Azure Data Engineering or Azure AI certifications.
• Experience with Power BI, Microsoft 365 Copilot, Power Platform or Microsoft Purview.
• Knowledge of infrastructure as code using Bicep or Terraform.
• Experience delivering AI-enabled data solutions within a large, regulated or complex organisation.

Additional information

• Immediate start
• Short-term contract with high potential for long-term opportunity
• Exciting opportunity for a seasoned developer