Big Data Engineer: Spark, ETL & Data Pipelines
Posted Updated
Description
Talan Spain is seeking a Mid-Level Big Data Developer to join the Client Contribution Platform team in Madrid. You will design, build, and optimize large-scale data processing solutions, ensuring high performance, data quality, and scalability across the platform.
You will collaborate with technical and business stakeholders in an Agile environment, implementing modern data engineering solutions while improving existing processes and following DevOps practices.
Design, develop, and maintain ETL and data transformation processes. Build, optimize, and support Spark-based data pipelines. Develop scalable data processing solutions in a Big Data environment. Integrate data from multiple sources while ensuring data quality and consistency. Analyze, troubleshoot, and resolve data-related issues. Optimize data performance and processing efficiency; collaborate with cross-functional teams to refine requirements. Participate in Agile ceremonies and contribute to continuous improvement initiatives. Follow DevOps and CI/CD best practices throughout the development lifecycle. 3+ years of software development and data engineering experience. Strong SQL knowledge (ANSI SQL, Hive SQL, Spark SQL, or equivalent). Hands-on Apache Spark with Scala/Python/Java. Experience with Spark-based platforms such as Azure Databricks, Amazon EMR, Cloudera, Hive, or similar Big Data environments. Experience with Qlik Sense reporting and dashboard development. Familiarity with Agile methodologies and DevOps practices. Experience with version control and CI/CD tools such as Git, GitHub, SonarQube, Nexus. Strong analytical and problem-solving skills. Ability to work collaboratively within a multidisciplinary team. English level B2+ or above. Career development opportunities Multicultural environment International projects Training opportunitiesSkills
Sign in to read and join the discussion.