Data Engineer
Posted Updated
Description
Job Overview
Design, develop and maintain data engineering solutions, including large-scale batch and streaming pipelines, distributed processing and containerized data workloads.
Key Responsibilities
- Design, develop and maintain ingestion, transformation and validation pipelines using Apache Spark, Apache Flink and Apache Kafka.
- Implement distributed processing using Java or Scala and SQL, integrating data sources and S3-compatible object storage.
- Maintain schemas, metadata, data lineage and data-quality controls across processing stages, supporting layered data processing where applicable.
- Deploy and scale workloads on Kubernetes, and implement automated tests, GitLab CI/CD, monitoring and operational troubleshooting.
- Implement checkpointing, restart recovery, retention and lifecycle controls.
- Support high availability, backup, disaster recovery and inter-site synchronization in line with agreed requirements.
Requirements & Skills
- 5–8 years of data engineering experience, including building and deploying large-scale data systems.
- Strong hands-on experience designing batch and streaming pipelines using Apache Spark, Apache Flink and Apache Kafka.
- Proficiency in Java or Scala for distributed data processing, together with SQL.
- Experience deploying and scaling data workloads in containerised Kubernetes environments.
- Experience with S3-compatible object storage, data-quality controls, schema and lineage management, checkpointing and recovery, GitLab and CI/CD practices.
How to Apply
(EA Reg No: 20C0312)
Please email a copy of your detailed resume to pauline@talentsis.com.sg for immediate processing.
Only shortlisted candidates will be notified.
Skills
Sign in to read and join the discussion.