Lead Data Engineer - Azure Databricks

Job Description :

Role Overview :

We are looking for a Lead Data Engineer to own the design and delivery of modern data platforms built on AWS and Databricks. This role combines deep hands-on engineering with team leadership, covering data ingestion, transformation, and warehousing across structured and unstructured sources.

Key Responsibilities :

- Architect and deliver end-to-end data engineering solutions on AWS and Databricks, spanning ingestion, transformation, and warehousing layers.

- Design scalable pipelines for structured and unstructured data, balancing reliability, performance, and cost efficiency.

- Lead and mentor a team of data engineers, overseeing sprint planning, code quality, and technical delivery.

- Partner with architects and business stakeholders to translate requirements into robust, production-ready designs.

- Establish and maintain standards for data quality, governance, and platform scalability.

What We're Looking For :

Must-Have Skills :

- AWS Services: Hands-on expertise across core AWS services spanning compute, storage, orchestration, and security in production environments.

- Databricks Platform: Deep proficiency in Databricks, including Delta Lake, Unity Catalog, cluster management, and job orchestration at scale.

- Data Engineering Lifecycle: Proven ability to design and operate ingestion, transformation, and warehousing pipelines across structured and unstructured data.

- Team Leadership: Experience leading and mentoring data engineering teams, managing delivery timelines, and driving technical quality.

Good-To-Have Skills :

- Agentic AI Exposure: Working understanding of agentic AI concepts and frameworks, and how they apply within modern data platforms.

- Manufacturing Industry Knowledge: Familiarity with manufacturing industry data landscapes, processes, and common use cases.

- Data Modelling: Understanding of data modelling principles, including dimensional and canonical modelling approaches.