Senior Data Engineer: Spark, Cloud Pipelines & Real-Time Data
Summary
Riskdata Consulting is hiring a Senior Data Engineer in Singapore to design, build, and optimize scalable batch and streaming data pipelines using Spark, PySpark, Scala, and Python on Azure Databricks, Data Factory, and AWS EMR across Hadoop/Hive/Snowflake environments, with data modeling, Airflow orchestration, CI/CD, and automated testing.
Riskdata Consulting is seeking a seasoned Data Engineer to design, build and optimize scalable data pipelines for large datasets. You will work with Spark, PySpark, Scala and Python to develop batch and streaming solutions, leveraging Azure Databricks, Data Factory and AWS EMR across Hadoop/Hive/Snowflake environments.
The role emphasizes data modeling, performance-focused SQL, and robust data governance with Airflow orchestration, CI/CD pipelines, and automated testing.