Senior Data Engineer - Apache Spark/Scala

Role Overview :

The ideal candidate will have strong experience in distributed data processing, ETL pipelines, and Big Data technologies, with hands-on expertise in Apache Spark and Scala. You will work on designing, developing, and optimizing large-scale data pipelines while collaborating closely with cross-functional engineering teams to build reliable, production-grade data solutions.

Key Responsibilities :

- Design, develop, and maintain scalable ETL pipelines for large-scale data processing.

- Build and optimize distributed data applications using Apache Spark and Scala.

- Develop reliable, high-performance data pipelines for batch and streaming workloads.

- Work with large datasets to ensure data quality, consistency, and performance.

- Collaborate with engineering, product, and analytics teams to deliver robust data solutions.

- Optimize data workflows for scalability, reliability, and cost efficiency.

- Deploy and manage data workloads in cloud and containerized environments.

- Troubleshoot production issues and continuously improve platform performance.

Requirements :

- 5+ years of experience in Data Engineering or Big Data Engineering.

- Strong hands-on experience with Apache Spark and Scala.

- Experience building and maintaining ETL pipelines.

- Familiarity with Google Cloud Storage (GCS).

- Experience with Kubernetes (K8s).

- Strong SQL skills and understanding of distributed data processing.

- Strong debugging, problem-solving, and performance optimization skills.

- Strong communication and collaboration skills.

Good to Have :

- Experience with AWS and cloud-native data services.

- Familiarity with streaming technologies such as Kafka.

- Experience working on large-scale data platforms or AdTech systems.

- Exposure to orchestration tools such as Airflow.