Data Engineer

Employer Highlight:

• A Malaysian ICT professional services and systems integration company established on 22 March 1999.
• 25+ years of experience delivering enterprise ICT and systems integration solutions across Malaysia.
• Exposure to enterprise-scale digital transformation projects, including cloud, cybersecurity, networking, infrastructure, and managed services.

Project/Goal:

• Working in client's office, serving current project in the healthcare industry

Key Responsibilities:

• Design and implement scalable data lakehouse architectures using modern technologies.
• Build and maintain ETL/ELT pipelines for batch and real-time data processing
• Develop data ingestion frameworks from various sources (APIs, databases, files, streams)
• Optimize data storage, partitioning, and query performance in lakehouse environments
• Support machine learning workflows and AI model deployment pipelines
• Design disaster recovery and backup strategies for critical data assets

Skills Requirements:

• Programming: Python, Scala, Java, SQL, Shell scripting
• Big Data: Apache Spark, Hadoop ecosystem, Delta Lake, Apache Iceberg
• Cloud Platforms: AWS (S3, EMR, Glue, Lambda), Azure (ADF, Synapse), GCP (Dataflow, BigQuery)
• Data Platforms: Databricks, Snowflake, Apache Airflow, dbt
• Streaming: Apache Kafka, AWS Kinesis, Azure Event Hubs
• Infrastructure: Terraform, Docker, Kubernetes, Git
• Databases: PostgreSQL, MongoDB, Cassandra, Redis

Good to Have:

• Experience with real-time streaming platforms (Kafka, Kinesis, Pub/Sub)
• Experience with MLOps and ML pipeline orchestration
• Understanding of data privacy regulations (GDPR, Malaysian PDPA)
• Previous experience in financial services, e-commerce, or technology sectors

Work Location:

• Petaling Jaya, onsite

Career Level:

• Mid to Senior level with 4-6 years of experience.