Senior Data Engineer

Salary: £29,000 - 56,000 per year

Requirements:

• 5+ years in data engineering across cloud-native environments.
• Expert knowledge of Databricks, Delta Lake, Unity Catalog, ADF, Synapse, Event Hub, and wider Azure Data Services.
• Highly proficient in Python and SQL, with experience building production-grade data pipelines.
• Strong understanding of Lakehouse design patterns, Medallion modelling, and performance optimisation.
• Hands-on experience with CI/CD for data, including Azure DevOps, YAML pipelines, Terraform, and Databricks CLI/Repos.
• Solid grasp of data governance, security principles, and operational best practices.

Responsibilities:

• Design, build, and optimise scalable data pipelines using Databricks, Delta Live Tables, Synapse, and Delta Lake.
• Develop ingestion, transformation, and orchestration workflows across diverse sources including APIs, SQL, SFTP, and SaaS platforms.
• Implement strong data quality, lineage, and governance standards aligned to our Medallion Architecture.
• Build and optimise Lakehouse data models to support analytics, reporting, operational workloads, and downstream product teams.
• Develop reusable frameworks for ingestion, validation, schema evolution, and monitoring.
• Ensure all solutions follow Databricks best practices around Unity Catalog, cluster policies, permissions, and environment separation.
• Work with architects to evolve Databricks platform patterns, security, and performance optimisation.
• Implement monitoring, logging, and observability across pipelines and clusters.
• Contribute to standards for versioning, DevOps, CI/CD, and testing for Databricks notebooks, workflows, and SQL assets.
• Support the rollout of Delta Sharing, Unity Catalog governance, and Iceberg/Delta interoperability where required.
• Partner with analysts and data scientists to ensure high-quality, well-modelled data is available for analytics and future AI use cases.
• Assist with productionising ML pipelines where needed, ensuring they run reliably within Databricks.
• Support feature engineering and experiment environments as part of the wider platform.
• Work cross-functionally with product, marketing, operations, and technology teams to understand data needs and accelerate delivery.
• Mentor junior engineers and help foster a culture of engineering best practice.
• Champion DevOps for data, including CI/CD pipelines, testing automation, IaC, and environment management.

Technologies:

• AI
• Azure
• CI/CD
• Cloud
• Databricks
• DevOps
• Support
• Marketing
• Python
• SQL
• Security
• Terraform
• Unity
• GameDev

More:

We are Clarion Events, one of the worlds leading event organisers, producing innovative and market-leading events since 1947. We are a private equity-backed business owned by Blackstone, and our purpose is to deliver exceptional customer outcomes and experiences by making every connection count. We value PASSION, CARE, IMAGINATION, and TRUST, and we are committed to building an inclusive team that reflects a variety of backgrounds, perspectives, and skills. Our benefits include 25 days holiday plus bank holidays, an end-of-year wellbeing shutdown, an extra day of leave for a personal celebration, summer hours in August, charity days, pension, private medical insurance, a health cash plan, wellbeing support, mentoring, a season ticket loan, cycle to work scheme, free on-site gym and shower facilities, free eyesight tests, and an annual flu vaccination.

last updated 33 week of 2026