Data Engineer

Salary: RM 5,000 – RM 7,500 per month

About the role

We are looking for an experienced Data Engineer to design, build, and maintain scalable, reliable, and high-performance data platforms. You will own data pipelines end to end, from ingestion and transformation to modeling, quality, and delivery, enabling analytics, business intelligence, and AI/ML initiatives across the organization. The ideal candidate has strong hands-on expertise in Python and SQL, deep knowledge of modern data engineering practices, and the ability to independently architect and deliver data solutions on cloud platforms, with Google Cloud Platform (GCP) preferred.

Key responsibilities

• Design, build, and maintain scalable batch and real-time data pipelines using ETL/ELT patterns.

• Ingest and integrate structured, semi-structured, and unstructured data from diverse sources, including APIs, databases, files, and event streams.

• Implement incremental loading, change data capture (CDC), and idempotent pipeline design.

• Design and optimize data models (dimensional modeling, star/snowflake schemas, Data Vault where relevant) for analytics and reporting.

• Build and manage data warehouses and lakehouses following the Medallion architecture (Bronze, Silver, Gold layers), ideally on BigQuery and Cloud Storage.

• Orchestrate workflows using tools such as Cloud Composer (Apache Airflow), Dagster, Prefect, or other cloud-native schedulers.

• Implement data validation, testing, and quality checks using tools such as Great Expectations, dbt tests, Soda, or Dataplex data quality.

• Monitor, troubleshoot, and optimize pipelines for performance, cost, and reliability.

• Build and manage data solutions on cloud platforms, preferably GCP, using services such as BigQuery, Dataflow, Dataproc, Pub/Sub, Cloud Storage, Cloud Composer, and Cloud Run.

• Partner with analytics, BI, AI/ML, and software engineering teams to deliver data that meets business needs.

About you

• 3+ years of hands-on experience in data engineering.

• Strong proficiency in Python for data processing and pipeline development.

• Advanced SQL skills, including complex joins, window functions, CTEs, query optimization, and performance tuning.

• Proven experience building and maintaining ETL/ELT pipelines in production.

• Strong understanding of data modeling, data warehousing concepts, and database design.

• Experience with relational databases (PostgreSQL, MySQL, SQL Server) and familiarity with NoSQL databases (MongoDB, Cassandra, DynamoDB, Firestore, Bigtable).

• Hands-on experience with at least one cloud platform and its data services; GCP experience is preferred.

• Strong troubleshooting, analytical, and problem-solving skills.

• Bachelor's degree in computer science, Information Technology, Data Science, Engineering, or a related field (or equivalent practical experience).