Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Sabenza IT & Recruitment

New

Principal Data Engineer (Databricks)

Posted Updated 16 views

Description

Reporting to the Data & Analytics Lead, the Principal Data Engineer is a hands-on role, combining the building and optimization of production data pipelines with the technical leadership needed to raise engineering standards and develop the people around them.

The role functions as a key interface between business stakeholders and department heads, the wider data and analytics team, and the owners of source systems and data in Finance and IT.



Requirements

KEY RESPONSIBILITIES:
  • Design and build robust, production-grade data pipelines on Databricks, making appropriate use of current platform capabilities such as Spark Declarative Pipelines (SDP).
  • Work “under the hood” to optimise the platform for performance and cost tuning compute, jobs and queries,managing storage and table layout, and keeping platform spend under control.
  • Define and maintain consistent business metrics and a semantic layer (for example, using Metric Views) so that reporting is built on trusted, reusable definitions.
  • Integrate data from core business systems, including SAP S/4HANA, and other sources, using tools such as Fivetran (both SaaS connectors and HVR).
  • Set and uphold engineering standards across the team, including code quality, testing, documentation, CI/CD and data governance.
  • Coach and mentor junior and mid-level engineers, review their work and help them develop.
  • Partner with business stakeholders to understand their needs, shape practical solutions and ensure the platform delivers genuine value.
  • Engage data owners in Finance and IT to ensure data is well understood, valid and fit for purpose, and initiate data quality improvements where required.

REQUIRED SKILLS AND EXPERIENCE:
  • At least three years of hands-on, day-to-day Databricks experience, covering both pipeline development and under-the-hood performance and cost optimisation.
  • Up to date with recent platform developments, such as Spark Declarative Pipelines (SDP) and Metric Views.
  • Strong proficiency in programming languages commonly used in data engineering, such as Python, SQL and Spark.
  • Advanced experience with data manipulation, data modelling, database design and query optimisation.
  • Experience managing or coaching junior developers.
  • A track record of managing and influencing business stakeholders.

KEY COMPETENCIES:
  • Combining deep, hands-on engineering skill with sound judgement about cost, performance and long-term maintainability.
  • Coaching, mentoring and raising the capability of less experienced engineers.
  • Collaborating, communicating confidently and influencing business stakeholders.
  • Breaking down complex technical concepts and explaining them simply to non-technical audiences.
  • Staying current with a fast-moving platform and bringing new capabilities into everyday practice.
  • Taking ownership and driving work independently, from concept through to production.
NICE TO HAVE:
  • Experience working with SAP S/4HANA data sets.
  • Experience with Fivetran, including both SaaS connectors and HVR.
  • Power BI semantic modelling experience.
QUALIFICATIONS:
  • A relevant tertiary qualification in Computer Science, Information Systems, Engineering or a related field, or equivalent practical experience.


Skills

What Principal Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →