Senior Data Engineer | GCP | Kafka | BigQuery | Python

A leading FTSE 100 fintech is looking for a Senior Data Engineer to join its global Data Engineering function and contribute to a major transformation of its data platform.

The team is consolidating a complex data estate onto GCP, creating a scalable and trusted platform that supports analytics, compliance, Data Science and increasingly sophisticated AI/ML use cases.

This is a hands-on role suited to an experienced engineer who enjoys solving complex data problems and wants to have a meaningful influence on engineering standards and platform design.

The role

Working within a Data Engineering squad, you will design, build and operate production-grade data pipelines across both batch and real-time environments.

You will work across:

• GCP – BigQuery, Cloud Composer/Airflow, GCS and Cloud Run
• Apache Kafka – event-driven ingestion and real-time streaming
• Python and SQL – building reliable, production-grade data solutions
• dbt – transformation and data modelling
• Medallion architecture – Bronze, Silver and Gold layers
• Data quality and observability – contracts, lineage, monitoring and SLOs

Role responsibilities

• Designing and building scalable batch and real-time data pipelines
• Developing Kafka-based event streams and Airflow-orchestrated workflows
• Building transformations and data models in BigQuery
• Developing and maintaining ingestion integrations and connectors
• Implementing data quality controls, observability and pipeline monitoring
• Contributing to engineering standards, code reviews and CI/CD practices
• Supporting Data Science and AI-ready data initiatives
• Mentoring junior engineers and contributing to the development of the wider engineering team

Role Requirements

• Strong commercial experience building and operating production data pipelines on GCP
• Strong SQL and Python skills
• Solid experience with Apache Kafka or comparable streaming technologies
• Experience with BigQuery, Airflow/Cloud Composer and GCS
• Experience with dbt and modern data transformation approaches
• Understanding of Medallion architecture and data platform best practices
• Experience with data contracts, lineage, observability and pipeline SLOs
• Experience working within a structured or regulated environment

This is an opportunity to work on large-scale data engineering challenges within a global fintech, with the technical scope to make a real impact on the platform and how data is used across the organisation.