Data Engineer
Summary
Data Engineer maintaining and supporting production data pipelines on Databricks and AWS for a large-scale UK government project. Day to day is a mix of L2/L3 production support and data engineering: owning incidents, root cause analysis, fixes and performance troubleshooting using Python, PySpark, SQL, Spark, Delta Lake, S3, IAM and CloudWatch. Permanent hybrid role based in Leeds.
Salary: £60,000 - 65,000 per year
Requirements:- Were looking for strong hands-on experience with Databricks, AWS, Python, PySpark and SQL.
- We need experience with Databricks notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI, and performance troubleshooting.
- Were looking for experience with Spark SQL, Delta Lake, Parquet, Hive metastore tables, schemas, partitions, and the relationship between table metadata and underlying files.
- We need hands-on AWS experience, particularly with S3, IAM and CloudWatch, plus an understanding of role-based access, KMS encryption, and diagnosing data access failures.
- Were looking for a working knowledge of Git, code reviews, CI/CD, and controlled production deployments, including testing, rollback and release validation.
- Experience with AWS Glue, Lambda, Step Functions, Linux, shell scripts, Terraform, GitLab or Jenkins would be advantageous.
- We need you to run and maintain production data pipelines on Databricks and AWS.
- Youll keep scheduled data processing reliable, resolve incidents, apply fixes, and improve data service performance.
- Youll provide a combination of L2/L3 production support and data engineering.
- Youll support and maintain production data pipelines, including incident investigation, safe recovery, root cause analysis and permanent remediation.
- Youll own incidents from investigation through recovery and closure, diagnosing issues in Python, PySpark, SQL, Databricks jobs and AWS integrations.
- Youll provide clear progress updates and escalate issues in a timely manner.
- AWS
- AWS Glue
- Lambda
- CI/CD
- CloudWatch
- Databricks
- Git
- GitLab
- Hive
- IAM
- Support
- Jenkins
- Linux
- Python
- PySpark
- SQL
- Spark
- Terraform
- UX UI Design
- Cloud
More:
Were hiring a Data Engineer for a permanent, full-time role on a large-scale Government project as part of a leading global IT transformation. The position is based in Leeds with a hybrid model, primarily working from home and typically travelling to our Leeds office once or twice a month. The expected start is ASAP in October or November 2026. We offer a salary of £60,000–£65,000, depending on experience, remote/hybrid working, a pension, 22 days holiday plus bank holidays (rising to 25 days), death-in-service cover and a healthcare plan.
last updated 40 week of 2026