Data Engineer - SME (Databricks) - Hybrid
Summary
Leads data engineering initiatives, migrates systems to Azure Databricks, and guides the team on Python, ETL, and data architecture best practices.
Role Overview:
The Data Engineer SME leads the delivery of data engineering initiatives and serves as the technical subject-matter expert for the unified data platform. This role provides technical guidance to the data engineering team, drives Azure Databricks migration and capability building, and collaborates with architects, business stakeholders, and IT security to shape the future direction of the data platform.
Key Responsibilities
· Act as Lead Data Engineer for major initiatives, ensuring technical requirements are delivered with quality.
· Lead the migration to Azure Databricks and provide technical guidance across the data engineering team.
· Collaborate with Business Analysts and Architects for solution design; ensure end-to-end integrated solutions conform to architecture standards.
· Work with stakeholders to establish the future roadmap of the unified data platform aligned to business needs, data governance, and data privacy.
· Provide upskilling plans to elevate team capability in Azure and Databricks.
· Ensure production incidents are resolved within SLA.
Qualifications
· Minimum 7 years of experience in data engineering is required.
· Minimum 4 years of hands-on experience with Databricks is required.
· Advanced capability in Azure and Databricks; strong knowledge of Python, ETL, and data architecture.
· Experience with Data Lake, SQL, messaging tools (Kafka, RabbitMQ), and API integration (SOAP/REST).
· Familiarity with Data Governance and CI/CD practices.
· Experience with DWH or Oracle GoldenGate is an advantage.