Data Engineer
Posted Updated
Description
PageGroup Corporation Solutions (Malaysia) Sdn Bhd is seeking a Data Engineer with 4-7+ years of experience to design and maintain scalable data pipelines. The role focuses on data ingestion from APIs and enterprise systems, data reliability, and supporting enterprise data platforms.
The engineer will drive data quality, implement CDC and incremental loads, and collaborate with stakeholders on data contracts while contributing to governance, security, and runbooks.
Design and develop data pipelines to ingest data from APIs, databases, and enterprise systems. Build and maintain data ingestion frameworks for centralized data platforms. Develop solutions for data recovery, replay, and pipeline resilience. Ensure data quality through deduplication, incremental loading, and handling late-arriving data. Create dashboards and monitoring solutions for data pipelines. Implement automated alerts and remediation processes. Troubleshoot production issues and participate in on-call rotations. Define data requirements and data contracts with stakeholders. Maintain technical documentation and runbooks. 4-7+ years of experience in Data Engineering. Strong programming skills in Python and SQL. Experience building data pipelines from APIs and relational databases. Hands-on experience with CDC, incremental loading, data migration and historical backfills. Strong experience with Apache Spark.Skills
Sign in to read and join the discussion.