Databricks Data Engineer
Summary
Lead Data Engineer designing and optimizing enterprise data platforms on Databricks and AWS: building scalable ETL/ELT pipelines with PySpark, Python, and SQL, implementing Delta Lake medallion architecture, governing data with Unity Catalog, and leading end-to-end delivery with CI/CD. Requires 8+ years experience; Japanese (JLPT N2/N1) preferred.
Lead Data Engineer – Databricks
Location: Japan (Remote)
Employment Type: Full-Time, Permanent (FTE)
Experience: 8+ Years
Company: Celebal Technologies
Role Overview
We are seeking a hands-on Lead Data Engineer to design, build, and optimize enterprise data platforms using Databricks and AWS, delivering scalable, governed, and AI-ready data solutions.
Key Responsibilities
- Design and develop scalable ETL/ELT pipelines using Databricks, PySpark, Python, and SQL.
- Implement Delta Lake and Medallion Architecture (Bronze/Silver/Gold).
- Manage data governance, lineage, and access controls using Unity Catalog.
- Build reusable data models and AI-ready data products.
- Lead end-to-end data engineering delivery, performance optimization, and CI/CD implementation.
Required Skills
- 8+ years of enterprise data engineering experience.
- Strong production experience with Databricks, AWS, PySpark, Python, and Advanced SQL.
- Hands-on expertise in Delta Lake, Unity Catalog, and dimensional data modeling.
- Experience with Git, CI/CD, DataOps, and cloud data platforms.
- Strong technical leadership and stakeholder management skills.
Preferred Skills
- Databricks Genie / AI/BI, semantic layers, and ML/MLOps.
- Manufacturing, Supply Chain, ERP/MES, or Tableau experience.
- Japanese communication skills (JLPT N2/N1 preferred) and business-level English.
Skills
Sign in to read and join the discussion.