Senior Data Engineer (Native Databricks Preferred)
Posted Updated 5
views
Summary
Design and build scalable data pipelines in Databricks, migrating SAP data to the platform and optimizing analytics workflows using PySpark, Delta Lake, and cloud storage.
Job Description
We are seeking a highly skilled Senior Databricks Engineer / Architect with hands‑on experience in native Databricks environments and a proven track record in SAP to Databricks data migration projects. The ideal candidate will be responsible for designing, implementing, and optimizing scalable data architectures that support advanced analytics, data engineering, and business intelligence initiatives.
Key Responsibilities
- Design and architect scalable data platforms and pipelines within native Databricks environments.
- Lead and execute SAP to Databricks migration projects, ensuring seamless data integration and transformation.
- Collaborate with business stakeholders, data engineers, and analysts to define architecture and data strategy.
- Implement best practices for data governance, security, and performance optimization on the Databricks platform.
- Develop and maintain data models, ETL processes, and automation frameworks for high-performance analytics.
- Guide and mentor technical teams on Databricks architecture, implementation standards, and platform utilization.
- Work closely with cloud teams (Azure, AWS, or GCP) to ensure seamless integration with enterprise data ecosystems.
- Stay current with Databricks feature releases and advancements to continuously enhance platform capabilities.
Requirements
- Databricks Platform Architect Certification (mandatory).
- 3+ years of hands‑on experience working with native Databricks environments.
- Proven experience in migrating data from SAP systems to Databricks.
- Strong expertise in PySpark, SQL, Delta Lake, and Databricks Workflows.
- Solid understanding of data warehousing concepts, data modeling, and ELT/ETL design patterns.
- Experience integrating Databricks with cloud data platforms (AWS S3, GCP Storage, or Azure).
- Knowledge of data governance, lineage, and security best practices.
- Excellent problem‑solving, analytical, and communication skills.
- Experience in leading or architecting enterprise data solutions.