Principal Data Engineer

Job Description

\n Responsibilities\n

\n
• Act as the leading technical authority and Subject Matter Expert for Databricks Data Engineering, representing the Engineering team to stakeholders.
\n
• Define and own the Data Architecture for the creation of speciality insurance analytics data products.
\n
• Design Conceptual, Logical, and Physical Data Models for the business area, aligned to broader/standardised data models.
\n
• Establish engineering standards, patterns, and best practices for AI-assisted pipeline development, orchestration, processing, storage, and observability.
\n
• Design, oversee, and contribute to the development of large-scale, fault-tolerant data pipelines and orchestration jobs across medallion layers in Databricks.
\n
• Ensure pipelines are designed to meet SLAs for reliability, latency, cost, and data quality.
\n
• Implement processes for the management of metadata, lineage, and observability in the delivery of high-quality and dependable Data Products.
\n
• Partner with security teams on data privacy and compliance, implementing data governance with particular focus on managing sensitive, business critical data.
\n
• Mentor senior and mid-level engineers and drive Engineering Excellence across AI Assisted Development, Code quality, Testing, and Documentation.
\n
• Lead design reviews and architecture discussions, encouraging collaboration and innovation to promote a culture of continuous improvement.
\n

Requirements\n

\n
• Databricks and AWS Certifications
\n
• BSc/MSc degree in Computer Science, Engineering or a related subject
\n
• Data Management certifications (e.g. DAMA) an advantage
\n
• Extensive experience in Data Engineering roles
\n
• Proven experience leading or mentoring engineering teams
\n
• Experience delivering large-scale cloud-based data initiatives
\n
• Previous experience working within regulated environments such as Insurance highly desirable
\n
• Experience with Databricks Lakehouse architecture preferred
\n

\n

#J-18808-Ljbffr