Lead Data Engineer
Summary
Lead Data Engineer in Kuala Lumpur designing cloud data architectures, data lakes, and warehouses, and building batch/streaming ETL/ELT pipelines integrating enterprise systems like SAP, Salesforce, and CRM platforms. Requires 8-12 years of data engineering experience with SQL, Python/Scala, and cloud platforms such as AWS, Azure, or GCP.
- Design, evolve, and maintain scalable cloud data architectures, data lakes, and enterprise data warehouses.
- Build and support data integration between enterprise systems, including SAP, Salesforce, CRM platforms, and cloud-based applications.
- Lead the development of batch and streaming ETL/ELT pipelines processing high-volume, multi-source data.
- Implement framework solutions for data monitoring, lineage, data masking, encryption, and role-based access controls
- Implement data models and transformation logic in alignment with enterprise data architecture standards.
- Ensure data integrity, accuracy, and consistency across source and target systems.
- Collaborate with Data Architects, Business Analysts, and Solution Architects in designing scalable data solutions.
- Support analytics, reporting, dashboards, and AI/ML initiatives by providing trusted and governed datasets.
- Monitor and troubleshoot data pipelines, integration jobs, and platform performance issues.
- Implement data governance, security, privacy, and compliance requirements within data solutions.
- Maintain technical documentation, data dictionaries, and data lineage artifacts.
- Identify opportunities for automation and continuous improvement in data engineering processes.
Profile
- Bachelor's Degree in Computer Science, Information Technology, Software Engineering, Data Science, or related discipline.
- Minimum 8-12 years of experience in data engineering, data integration, or related fields.
- Strong understanding of data warehousing, data lakes, and modern data architecture principles.
- Experience with ETL/ELT tools and data integration platforms.
- Proficiency in SQL and experience with Python, Scala, or similar programming languages.
- Hands-on experience with cloud data platforms such as AWS, Azure, or Google Cloud.
- Experience integrating enterprise applications such as SAP, Salesforce, CRM, ERP, or similar business systems.
- Knowledge of data modeling concepts, including dimensional and relational modeling.
- Familiarity with data governance, data quality, and information security practices.
- Strong analytical, troubleshooting, and problem-solving skills.
- Excellent communication and stakeholder management capabilities.
- Preferred Qualifications
- Experience with SAP data structures and Salesforce data models.
- Experience in AWS services such as S3, Glue, Redshift, Lambda, or equivalent cloud technologies.
- Exposure to AI/ML data preparation and feature engineering.
- Knowledge of master data management (MDM) and enterprise data governance.
- Professional certifications in cloud, data engineering, or analytics technologies would be an advantage.
Skills
- AI
- Analytics
- Automation
- AWS
- Azure
- Cloud
- CRM
- Data Engineering
- Data Governance
- Data Lake
- Data Lineage
- Data Modeling
- Data Pipelines
- Data Quality
- Data Science
- Data Warehousing
- ELT
- ERP
- ETL
- Feature Engineering
- GCP
- Lambda
- Machine Learning
- Master Data Management
- MDM
- Python
- Redshift
- S3
- Salesforce
- SAP
- Scala
- SQL
- Stakeholder Management