Senior Data Engineer: Spark, Cloud Pipelines & Real-Time Data
Posted Updated
Description
Riskdata Consulting is seeking a seasoned Data Engineer to design, build and optimize scalable data pipelines for large datasets. You will work with Spark, PySpark, Scala and Python to develop batch and streaming solutions, leveraging Azure Databricks, Data Factory and AWS EMR across Hadoop/Hive/Snowflake environments.
The role emphasizes data modeling, performance-focused SQL, and robust data governance with Airflow orchestration, CI/CD pipelines, and automated testing.