Senior Data Engineer
Exciting opportunity for a Senior Data Engineer to own the maintenance, expansion, and performance of Databricks-based data platform. This is a hands-on role, you'll be writing pipeline code, building jobs, and modelling as the main part of your day to day.
Role Responsibilities
Responsibilities will include:
- Designing, building, and operating Databricks Pipelines and Jobs
- Designing and building Delta Lake pipelines
- Owning Databricks deployment bundles, packaging, versioning, and promoting pipeline code and job definitions
- Building and maintaining environment-variable-driven Azure Data Factory pipelines
- Building the data layer that feeds Microsoft Fabric/OneLake
Person Specification
Essential skills will include:
- Hands-on experience building and running Databricks Pipeline/Job structure in a production
- Strong data modelling ability, able to look at raw source data and design sensible, well-structured silver/gold tables
- Strong understanding of medallion (bronze/silver/gold) design patterns, and data architecture right at table level
- Solid Delta Lake knowledge, deletion vectors, time travel, optimization, compatibility considerations
- Experience with Databricks Asset Bundles or equivalent deployment/CI practices for data pipelines
- Practical experience building environment-variable/parameter-driven Azure Data Factory pipelines
- Proven experience of managing large data architectures
- Strong SQL, and comfort in at least one of Python/Scala for pipeline logic
You will be the technical expert for the data solution, to work alongside the engineering lead to manage the data aspects of the digital solution.
FPSG is committed to equal opportunities regardless of gender, race, disability, sexual orientation, religion or belief and age.