Data Engineer (Databricks)
Posted 5
views
Summary
An onsite Data Engineer role at Pride Park, Derby, focused on building and troubleshooting big-data pipelines in Databricks. Day to day involves PySpark/Spark SQL development, Delta Lake and Hive metastore table work, cluster configuration and performance tuning, plus hands-on AWS (S3, IAM, CloudWatch).
Salary: £? - ? per year Requirements: Strong hands-on experience with Databricks, PySpark, AWS, SQL, and Python. Databricks experience with notebooks, scheduled jobs, cluster configuration, driver and executor logs, Spark UI, and performance troubleshooting. Experience with Spark SQL, Delta Lake, Parquet, and Hive metastore tables, including schemas, partitions, and the relationship between table metadata and underlying files. Hands-on AWS experience, particularly with S3, IAM, and CloudWatch; …