Data Engineer
Summary
Design, build, and optimize data pipelines and storage on Microsoft Fabric and Azure (Synapse, Data Factory), develop PySpark workloads, manage data lakes, and integrate AI/Copilot workflows while ensuring data governance and security. London-based role with hybrid working at insurance/advisory firm WTW.
In this role you will design, build, and optimize data pipelines and storage on Microsoft Fabric and Azure platforms to deliver scalable data solutions. You will work with cross-functional teams to incorporate AI tooling and Copilot workflows, driving practical data-driven insights. You will ensure data governance, security, and efficient data access across data lakes and warehousing systems. The role offers hands-on work with cutting-edge cloud and AI technologies to shape impactful data architectures.
Pay / Benefits- 25 days annual leave
- WTW day
- private healthcare
- life insurance
- pension with company match up to 10%
- hybrid working
- Design, build, and optimize data pipelines using Microsoft Fabric, Azure Synapse, Azure Data Factory, and Azure Fabric
- Develop and tune PySpark notebooks for large-scale data workloads
- Implement, troubleshoot, and enhance ETL/ELT workflows in Azure Synapse and Microsoft Fabric
- Manage and organize Data Lakes for fast data access and high performance
- Integrate AI/LLM models, including Copilot agents, into data pipelines to boost insights
- Collaborate with Business teams, Data Experts, AI Engineers, Data Analysts, and domain experts
- Support data security, governance, and compliance within the Azure and Microsoft Fabric ecosystem
- Ingest data from multiple sources such as APIs and handle large data volumes
- Stay updated on emerging cloud, AI, and big data technologies with hands-on applicability
- Hands-on Data Engineering experience with Cloud Data Tools, especially Microsoft Fabric
- Strong programming skills in Python
- Understanding of data warehousing, modeling, and basic data architecture
- Excellent problem-solving and debugging skills
- Genuine enthusiasm for working with data and delivering business value
- Curiosity about how systems are built and operate
- Excellent communication and collaborative mindset
- Experience with Copilot agents and AI tools (highly preferable)
- Azure Data Engineer Associate certification (preferred)
- Experience with real-time streaming solutions (e.g., Azure Stream Analytics, Kafka, Event Hubs)
- Familiarity with Databricks and its integration with Azure Synapse and Microsoft Fabric
- Knowledge of Graph Databases and NoSQL technologies
- Understanding of AI/LLM applications and integration with data pipelines (Copilot workflows)
- collaboration
- clear communication
- curiosity
- Microsoft Fabric
- Azure Synapse
- Azure Data Factory