Databricks Lakehouse Engineer: Governance & Performance
Posted Updated
Software International Corporation (M) Sdn Bhd Databricks Lakehouse Engineer: Governance & Performance
Description
Software International Corporation (M) Sdn Bhd in Malaysia is seeking an experienced Data Engineer to architect and deliver scalable data pipelines on Databricks, with strong governance and security via Unity Catalog. You will lead Dalbricks-based data warehouse and lakehouse implementations, optimize PySpark workloads, and drive DevOps practices across CI/CD and GitHub workflows.
You will collaborate with stakeholders across business and technical teams, mentor junior engineers, and participate
Design, develop, and maintain scalable data pipelines and data products using Databricks and PySpark. Build and optimize ETL/ELT solutions for batch and near real-time processing. Design and maintain Data Warehouse and Lakehouse solutions for reporting, analytics, and AI/ML use cases. Develop enterprise data models with Delta Lake and Delta Tables; implement SCDs. Implement data governance with Unity Catalog, RBAC/ABAC, data lineage and auditing. Create CI/CD pipelines for Databricks solutions and integrate with GitHub Actions. Collaborate with stakeholders across business and technical teams; provide mentorship and production support. Bachelor's degree in CS/IT/Engineering/Data Science or related field. 5-8 years in Data Engineering, Data Warehousing, or Big Data tech. Minimum 4+ years hands-on Databricks in enterprise environments. Experience designing enterprise Data Warehouse solutions and modern Lakehouse architectures. Strong PySpark development and optimization. Experience with Unity Catalog, RBAC/ABAC, and data governance.Skills
Sign in to read and join the discussion.