Senior Data Engineer
Posted 3
views
Summary
Senior Data Engineer in Kuala Lumpur building scalable Lakehouse data platforms for global clients using Azure Databricks, PySpark, Azure Data Factory and Snowflake. Day to day: designing pipelines and schemas for AI/analytics workloads, enforcing data quality and governance, and mentoring junior engineers in Agile sprints across APAC.
Job Summary:
Our Esteemed and Reputable Client is expanding its Data Engineering Team in Malaysia. They are looking for Senior Data Engineers specialise in Databricks, Snowflake to build scalable Lakehouse data platforms for global Clients. The Senior Data Engineer will work alongside DevOps, Cloud Engineering and Data Scientists in Agile Sprints to deliver data solutions across APAC.
- Build and maintain scalable data pipelines using Azure Databricks and PySpark
- Design and manage data ingestion and orchestration workflows using Azure Data Factory
- Define how data is stored, organized, and accessed
- Design schemas and ensure systems support scalable AI and analytics workloads
- Build real-time customer analytics platforms, ML-ready data environments and large-scale reporting pipelines
- Establish and enforce best practices for data quality, governance and security within the Databricks environment
- Mentor Junior Data Engineers and contribute to evolving the team's Databricks expertise
- Drive the data architecture and modeling strategy for AI projects
- Implement data anonymization, quality checks, lineage and controls
- Ensure timely delivery of data components aligned with project milestones
Requirements:
- Min 6 years hands on experience Databricks on Azure, AWS, or GCP, including Spark (PySpark/Scala),
- Programming experience in Python/Scala, with experience in CI/CD pipelines
- Prior experience in ETL/ELT design and implementation
- Familiar with Big data technologies: Hadoop and Kafka
- Passion for building scalable and reliable data systems