Data Engineer

Job Description: Data Engineer (Scala, Python, EMR Mandatory)

Role: Data Engineer
Experience: 5-10 Years
Location: Bangalore
Employment Type: Full Time

Job Summary

We are seeking a highly skilled Data Engineer with strong expertise in Scala, Python, and Amazon EMR to design, develop, and optimize large-scale data processing solutions. The ideal candidate should have hands-on experience in building robust ETL pipelines, big data processing, and cloud-based data platforms.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Scala and Python.
• Build and optimize batch and real-time data processing solutions using Apache Spark on Amazon EMR.
• Develop ETL/ELT workflows for ingesting, transforming, and loading large volumes of structured and unstructured data.
• Work with AWS services including S3, Glue, Lambda, CloudWatch, and IAM.
• Optimize Spark jobs for performance, scalability, and reliability.
• Implement data quality, monitoring, and governance standards.
• Collaborate with Data Scientists, Analysts, and Business stakeholders to understand data requirements.
• Troubleshoot production issues and provide timely resolutions.
• Ensure adherence to security, compliance, and data management best practices.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
• 5+ years of experience in Data Engineering and Big Data technologies.
• Strong programming skills in Scala and Python.
• Hands-on experience with Amazon EMR and Spark-based data processing.
• Experience with distributed computing and large-scale data platforms.
• Strong SQL development and query optimization skills.
• Experience working with AWS cloud services.