Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Impetus Technologies

NewBe an early applicant

Spark SCALA Developer

Posted Updated

Description

Job Description

We are seeking a highly skilled Spark Scala Engineer with 5-7 years of experience in designing, developing, and maintaining large-scale data processing applications. The ideal candidate should have strong expertise in Apache Spark, Scala, AWS services, and hands-on experience working on data migration projects. Experience with MongoDB schema validation, cloud-native architectures, and data pipeline optimization is highly desirable.

Roles & Responsibilities
Key Responsibilities
  • Design, develop, and optimize scalable batch and real-time data processing solutions using Apache Spark and Scala.
  • Build and maintain robust data ingestion frameworks for processing high-volume data from multiple source systems.
  • Participate in large-scale data migration projects, including data analysis, transformation, validation, and reconciliation activities.
  • Develop and manage ETL/ELT pipelines using AWS services such as EMR, S3, and Step Functions.
  • Analyze and resolve data quality issues, ensure data consistency, and support production troubleshooting.
  • Work closely with business stakeholders, data architects, and cross-functional teams to gather and translate requirements into technical solutions.
  • Implement monitoring, logging, and performance tuning of Spark applications.
  • Perform schema design, validation, and data modeling activities in MongoDB.
  • Conduct root cause analysis for data ingestion and processing issues and implement preventive measures.
  • Ensure adherence to coding standards, security guidelines, and best practices.
Required Skills & QualificationsTechnical Skills
  • 3-6 years of hands-on experience in Scala and Apache Spark development.
  • Strong experience in building ETL/Data Ingestion pipelines.
  • Extensive knowledge of AWS Cloud Services, including:
    • Amazon EMR
    • Amazon S3
    • AWS Step Functions
    • IAM and CloudWatch (preferred)
  • Experience working on data migration projects involving legacy-to-cloud or platform modernization initiatives.
  • Strong SQL skills and experience with large-scale data processing.
  • Hands-on experience with MongoDB, including:
    • Schema design
    • Schema validation
    • Data modeling
    • Query optimization
  • Experience in handling structured and semi-structured datasets (JSON, CSV, Avro, Parquet, etc.).
  • Familiarity with CI/CD pipelines, Git, and Agile development methodologies.
  • l scripting for automating tasks.

Skills

Apply

See also