Spark/Scala Data Engineer
Summary
Design, build, and optimize large-scale data pipelines and processing applications using Spark and Scala, working with the Hadoop ecosystem (HDFS, Hive, Impala), ETL/ELT workflows, CI/CD, and Agile practices alongside data scientists and analysts. Full-time role in Chennai requiring 4-10 years of experience.
Spark/Scala Data Engineer
Job Title: Spark/Scala Data Engineer
Experience: 4-10 Years
Location: Chennai
Job Type: Full-Time
Job Summary
We are looking for a highly skilled Spark/Scala Data Engineer with strong experience in Big Data technologies. The ideal candidate will design, develop, and maintain scalable data pipelines and data processing applications using Spark and Scala while working closely with business and analytics teams.
Key Responsibilities
- Design, develop, and optimize large-scale data processing solutions using Spark and Scala.
- Build and maintain ETL/ELT pipelines for structured and unstructured data.
- Develop scalable and high-performance data ingestion frameworks.
- Work with Hadoop ecosystem components such as HDFS, Hive, and Impala.
- Perform data transformation, cleansing, and aggregation for analytics use cases.
- Optimize Spark jobs for performance, scalability, and reliability.
- Collaborate with Data Scientists, Business Analysts, and Architects.
- Implement CI/CD practices and support production deployments.
- Troubleshoot and resolve data pipeline and performance issues.
- Follow Agile development methodologies and participate in sprint activities.