Data Engineer (Databricks)

Salary: £? - ? per year

• Strong hands-on experience with Databricks, PySpark, AWS, SQL, and Python.
• Databricks experience with notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI, and performance troubleshooting.
• Experience with Spark SQL, Delta Lake, Parquet, and Hive metastore tables, including schemas, partitions, and the relationship between table metadata and underlying files.
• Hands-on AWS experience, particularly with S3, IAM, and CloudWatch; understanding of role-based access, KMS encryption, and diagnosing data access failures.
• Working knowledge of Git, code reviews, CI/CD, and controlled production deployments, including testing, rollback, and release validation.
• Advantageous: experience with AWS Glue, Lambda, Step Functions, Linux, shell scripts, Terraform, GitLab, or Jenkins.

• Run and maintain production data pipelines on Databricks and AWS.
• Keep scheduled data processing reliable by resolving incidents, applying fixes, and improving data services performance.
• Support and maintain production data pipelines, including incident investigation, safe recovery, root cause analysis, and permanent remediation.
• Own incidents from investigation through recovery and closure; diagnose issues across Python, PySpark, SQL, Databricks jobs, and AWS integrations.
• Provide clear progress updates and escalate issues in a timely manner.

• AWS
• AWS Glue
• Lambda
• CI/CD
• CloudWatch
• Databricks
• Git
• GitLab
• Hive
• IAM
• Support
• Jenkins
• Linux
• Python
• PySpark
• SQL
• Spark
• Terraform
• UX UI Design
• Cloud

More:

We are seeking a Data Engineer for a large-scale government project as part of a global IT transformation. This is a six-month contract based in Leeds, with a hybrid working model: primarily working from home, with travel to our Leeds office one to two days per month. The role combines L2/L3 production support with data engineering work and is due to start as soon as possible in October 2026. The rate is listed as £ per day, Inside IR35.

last updated 41 week of 2026