Senior Spark Data Engineer - AWS & Data Pipelines
Summary
Design and optimize scalable data pipelines using Apache Spark and AWS for batch and near-real-time processing, ensuring reliable datasets for analytics and ML.
Open Digital Services in Madrid is seeking a Senior Data Engineer to design and optimize scalable data pipelines using Apache Spark and cloud platforms. You will work on batch and near-real-time processing, creating reliable datasets for analytics, ML and reporting, while ensuring data quality and proper documentation.
The role requires strong PySpark/Scala skills, experience with Spark SQL, and hands-on experience with AWS-based data platforms.