Senior Data Engineer
Summary
Build and maintain scalable data pipelines using PySpark, Python, and Azure Data Factory to ingest and transform data for reliable analytics and reporting.
A Senior Data Engineer is required to support the development of a modern data platform for a long-term, multi-year funded programme of work.
You will take ownership of building dependable data solutions, improving existing processes and helping ensure data is delivered accurately and efficiently across the organisation.
The Role
- Develop and enhance data ingestion and transformation processes
- Build solutions for both scheduled and real-time data flows
- Improve the reliability, speed and scalability of data workloads
- Monitor live data processes and resolve technical issues
- Apply appropriate testing, documentation and coding standards
- Contribute to technical discussions and solution planning
- Work closely with colleagues across engineering, analytics and the wider business
Core Skills
- PySpark
- Azure Data Factory or comparable orchestration technology
- Python
- SQL
- Kafka, Event Hubs or another streaming platform
Applicants should have several years of commercial data engineering experience as a contractor and a strong understanding of data architecture, transformation patterns and distributed processing.
The role will suit an engineer who can work independently, make sound technical decisions and deliver straightforward solutions that are reliable and easy to support.