Data Platform Senior Engineer (SEL & SWL) x2

Job overview

Can you engineer data differently – and demonstrate the difference it makes?

We are looking for talented Data Platform Engineers to design, build and improve the data pipelines, models and platform services that support analytics across South East and South West London.

This is a hands-on engineering role for someone who combines strong Data Engineering capability with analytical curiosity and a practical approach to solving problems. Reporting to a Data Platform Lead Engineer or Senior Data Platform Engineer, you will help turn complex data and analytical requirements into secure, reliable, scalable and production-quality data solutions.

You will work closely with other Data Platform Engineers, analysts, Data Scientists and technical specialists across Microsoft Fabric, Snowflake and Azure, helping ensure that data moves efficiently from source systems through ingestion and transformation into trusted analytical data products.

Advert

The Opportunity

You will join at an important point in the development of our Data Engineering capability across South East and South West London.

You will have the opportunity to work across Microsoft Fabric, Snowflake and Azure in a complex health and care data environment, developing your expertise in modern approaches to ingestion, orchestration, transformation, data modelling, testing, deployment, monitoring and automation.

This is a role where you can see the effect of your engineering. Working directly with analysts and Data Scientists, you will understand how data is used downstream and help improve the speed, quality and reliability of analytical delivery.

You will contribute to scalable and reusable data products that support Population Health Management, strategic commissioning, planning, performance and decision-making across two major London health systems.

Working for our organisation

In 2025, NHS South East London ICB and NHS South West London ICB decided to enter into a clustering arrangement, meaning we share a chair, chief executive officer and an executive team, but we remain as separate, statutory organisations.

This also means that the ICBs will share some teams, who will be working across both organisations. Some teams will also be working across both geographical areas and integrated care systems.

ICBs are also changing, with a clear focus on strategic commissioning and delivering neighbourhood working.

This is an exciting time to work within our ICBs. We currently have the opportunity both to shape the future of our clustered organisations, and to be part of a radical transformation of care for the local people and communities across south London.

And while our two ICBs are different in some ways, improving lives for the people we serve is at the core of what we are both here to do.

Please note as you can only be employed by one ICB, this job description refers to ‘the ‘ICB’, but your role may cover both ICB organisations, and / or geographical areas.

Equality and diversity

The ICB is committed to providing services and employment to a community with an increasing variety of backgrounds. To do this effectively it is essential that we promote equality and embrace diversity and treat everyone with dignity and respect. This includes a focus on building a just culture.

Detailed job description and main responsibilities

Who are we looking for?

We are looking for an experienced Data Platform Engineer who can design, build, test and maintain modern data pipelines and analytical data models.

You will understand the data lifecycle from ingestion and orchestration through transformation, modelling, validation, deployment, monitoring and optimisation. You will be comfortable investigating problems, improving existing solutions and working with senior engineers to apply appropriate engineering standards.
Your experience is likely to include several of the following:

Microsoft Fabric • Snowflake • Azure • OneLake • Lakehouse architecture • Data Factory • Spark • Python • SQL • ETL/ELT • dimensional modelling • Data Vault • orchestration • Git • Azure DevOps/GitHub • CI/CD • Infrastructure as Code • data quality • MLOps • performance optimisation

You will be pragmatic and collaborative, able to understand the problem before developing the solution and communicate technical concepts clearly to both technical and non-technical colleagues.

Above all, we are interested not simply in what you have built, but what became better because you built it.

What makes the role different?

For us, successful Data Engineering is not simply about moving data from one place to another. We want our engineers to understand how the data they build and maintain is ultimately used.

You will work closely with the analysts and Data Scientists who consume the data, giving you the opportunity to connect engineering decisions directly to analytical delivery.

You will help us move away from unnecessary manual processing and isolated solutions towards automated, reusable, governed and production-quality data products.

The difference you make should be visible: reducing processing time, eliminating repetitive analyst intervention, improving data quality and pipeline reliability, increasing reuse, improving performance or shortening the time needed to develop analytical products.

The role therefore provides an opportunity to develop deep technical expertise while contributing directly to measurable analytical value.

What will you do?

1. Design, build and optimise scalable data pipelines — develop secure and reliable data flows from source systems through ingestion and transformation to curated analytical data products.
2. Develop data orchestration, transformation and integration solutions — build and maintain automated ETL/ELT processes covering ingestion, cleansing, transformation, scheduling, dependency management and exception handling.
3. Build robust analytical data models — translate analytical and business requirements into reusable dimensional, Lakehouse, Data Vault and semantic models.
4. Automate manual and repetitive processes — identify opportunities to reduce unnecessary intervention and improve the productivity of Data Engineering and analytical teams.
5. Apply modern engineering disciplines — use Git-based version control, CI/CD, automated testing, monitoring, documentation, deployment pipelines, change control and Infrastructure as Code.
6. Engineer for data quality, reliability and resilience — implement validation, reconciliation, monitoring, audit, recovery, backfill and exception-handling processes.
7. Develop and optimise solutions across Microsoft Fabric, Snowflake and Azure — contribute to performance, scalability, resilience and efficient resource utilisation.
8. Support advanced Analytics, Data Science and AI — develop the pipelines, data structures and engineering components required for analytical and Machine Learning solutions.
9. Work collaboratively with analysts, Data Scientists and stakeholders — understand downstream requirements and develop data products that solve genuine analytical and organisational problems.
10. Demonstrate the benefit of your engineering — contribute to measurable improvements in processing time, manual intervention, reliability, data quality, reuse and analytical delivery.

Why join us?

You will work within an integrated Intelligence & Insights function supporting South East and South West London, alongside Data Platform Engineers, Data Scientists, Population Health specialists and wider analytical teams working with complex health and care data.

Your engineering will directly support Population Health Management, strategic commissioning, planning, performance and wider analytical delivery. You will be able to see how your technical work translates into practical analytical capability and better decision support.

You will also be joining during an important period of development, with the opportunity to work across Microsoft Fabric, Snowflake and Azure and to contribute to the evolution of modern Data Engineering practices across the function.

If you are an experienced Data Platform Engineer who combines strong technical capability, curiosity, collaboration and a determination to turn good engineering into measurable analytical value, we would like to hear from you.

1. Design, build and optimise scalable data pipelines — develop secure and reliable data flows from source systems through ingestion and transformation to curated analytical data products.
2. Develop data orchestration, transformation and integration solutions — build and maintain automated ETL/ELT processes covering ingestion, cleansing, transformation, scheduling, dependency management and exception handling.
3. Build robust analytical data models — translate analytical and business requirements into reusable dimensional, Lakehouse, Data Vault and semantic models.
4. Automate manual and repetitive processes — identify opportunities to reduce unnecessary intervention and improve the productivity of Data Engineering and analytical teams.
5. Apply modern engineering disciplines — use Git-based version control, CI/CD, automated testing, monitoring, documentation, deployment pipelines, change control and Infrastructure as Code.
6. Engineer for data quality, reliability and resilience — implement validation, reconciliation, monitoring, audit, recovery, backfill and exception-handling processes.
7. Develop and optimise solutions across Microsoft Fabric, Snowflake and Azure — contribute to performance, scalability, resilience and efficient resource utilisation
8. Support advanced Analytics, Data Science and AI — develop the pipelines, data structures and engineering components required for analytical and Machine Learning solutions.
9. Work collaboratively with analysts, Data Scientists and stakeholders — understand downstream requirements and develop data products that solve genuine analytical and organisational problems.
10. Demonstrate the benefit of your engineering — contribute to measurable improvements in processing time, manual intervention, reliability, data quality, reuse and analytical delivery.

Please see attached job description for full details of the main responsbilities.