Data Engineer
Summary
Hands-on Data Engineer running and maintaining production data pipelines on Databricks and AWS for a large-scale UK Government project, combining L2/L3 production support with engineering tasks. Core stack: Databricks, AWS (S3, IAM, CloudWatch), Python, PySpark, SQL and Delta Lake.
needed for a
permanent
full-time position based in
Leeds
(Hybrid). Must have strong
Databricks
and AWS experience. Hybrid Working Model - primarily working from home with travel to the Leeds office typically once/twice a month. Start ASAP in October/November 2026. A chance to work with a leading global IT transformation on a large-scale Government project: Data Engineer
with strong Databricks, AWS, Python, PySpark and SQL skills. Hands-on
Data Engineer
to run + maintain production data pipelines on Databricks and AWS. Keeping scheduled data processing reliable, resolve incidents, fixes + improving data services performance. The role combines L2/L3 production support and Data Engineering tasks. Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver / executor logs, Spark UI + performance troubleshooting. Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, schemas, partitions and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly S3, IAM and CloudWatch, with an understanding of role-based access, KMS encryption + diagnosing data access failures. Owning incidents from investigation through recovery and closure. Diagnosing issues in Python, PySpark, SQL, Databricks jobs and AWS integrations + providing clear progress updates and timely escalation. Working knowledge of Git, code reviews, CI/CD and controlled production deployments, including testing, rollback and release validation. Advantageous: AWS Glue, Lambda, Step Functions, Linux, Shell Scripts, Terraform, Gitlab, Jenkins. Benefits:
£60,000 - £65,000
(DOE) + Remote/Hybrid Working + Pension + 22 days holiday + BHs (rising to 25 days) + Death in Service + Health Care Plan.
TPBN1_UKTJ