Senior Data Engineer
Summary
Senior Data Engineer at Primark designs and delivers scalable data pipelines and lakehouse architecture on a Databricks/Azure platform using PySpark, SQL, and dbt Cloud, including leading a Teradata-to-Databricks migration. The role mentors engineers, sets best practices (CI/CD, testing, monitoring), and partners with analytics, data science, and governance teams.
As Senior Data Engineer at Primark, you will lead the design and delivery of scalable data solutions on a Databricks-based platform, partnering with analytics, data science, and governance teams. You’ll shape lakehouse architectures and reusable data models, driving performance, security, and cost efficiency. You will mentor engineers, promote engineering best practices, and help migrate from Teradata to Databricks, enabling trusted data products for reporting and AI initiatives. You’ll work across engineering, analytics, and business stakeholders to deliver measurable business value and advancements in our Data & Analytics capabilities.
Responsibilities- Design, build and optimise scalable data pipelines and ETL/ELT workflows using Databricks, PySpark, SQL, and dbt Cloud
- Develop and maintain lakehouse architectures and reusable data models ensuring scalability, performance, and security
- Provide technical leadership, establish best practices, and promote automation, testing, deployment, documentation and monitoring (CI/CD)
- Mentor Data Engineers through code reviews and guidance to foster continuous improvement
- Lead platform modernization efforts including Teradata-to-Databricks migration and pipeline remodelling
- Collaborate with Analytics, Data Science, Governance, and business stakeholders to translate requirements into scalable solutions and reusable datasets
- Identify opportunities to improve platform performance, scalability, and cost efficiency and participate in roadmap initiatives
- 5-8 years of advanced data engineering experience
- Hands-on experience with Databricks, PySpark, and advanced SQL
- Azure-based data technologies and lakehouse architecture
- dbt Cloud or dbt Core including model development, testing, deployment workflows, CI/CD and version control
- Strong understanding of data quality, observability, and performance tuning
- Track record delivering data products supporting analytics, reporting, AI, and data science
- Ability to provide technical leadership and collaborate across Engineering, Analytics, Governance, Architecture, and business teams
- Experience coaching engineers and driving continuous improvement
- stakeholder engagement
- mentoring and coaching
- collaboration across cross-functional teams
- Databricks
- PySpark
- SQL