Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

HYPERSCAL SOLUTIONS PTE. LTD.

Senior Data Engineer: Real-Time Pipelines & Kubernetes

Posted Updated

Description

Beyondsoft Consulting (Singapore) Pte. Ltd. is seeking a seasoned Data Engineer to design, build and operate large-scale batch and streaming data pipelines.

You will optimise distributed processing, manage data storage and ensure high availability in Kubernetes-based environments. The role requires 5+ years of data engineering experience with Spark, Flink and Kafka, plus proficiency in Java/Scala and SQL. Familiarity with CI/CD, GAo and S3-compatible storage is expected.

Design, develop and deploy large-scale batch and real-time data processing pipelines. Build and optimise distributed data processing solutions. Deploy, manage and scale data workloads in Kubernetes environments. Manage data storage, quality, schema, metadata and lineage. Implement data checkpointing, recovery, backup and disaster recovery solutions. Ensure data integrity, retention and high availability across distributed storage systems. Maintain CI/CD pipelines and support automated deployment processes. Minimum 5 years of relevant data engineering experience. Hands-on experience with Apache Spark, Apache Flink and Apache Kafka. Proficiency in Java or Scala and SQL. Experience with Kubernetes and S3-compatible object storage. Knowledge of data quality controls, schema management, data lineage, checkpointing and recovery. Familiarity with GitLab and CI/CD practices. Experience with MinIO or equivalent S3-compatible storage solutions. Knowledge of Bronze/Silver/Gold data architecture and data integrity controls. Experience in high availability, backup, disaster recovery and inter-site synchronisation. Experience with Apache Airflow or equivalent workflow orchestration tools. Familiarity with SAFe and safety-critical or mission-critical systems.

Skills

What Senior Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →