Data Software Engineer – Spark, Python, Databricks (L2 / L4)

Experience:

• L2: 4–5 Years
  

• L4: 8–12 Years
  

Mode: FTE (Full-Time Employment)

Job Location: Bangalore

Work Mode: Hybrid

Notice Period:

• L2: Immediate to 15 days
  

• L4: Immediate to 15 days
  

Drive Type: F2F

Drive Location: Bangalore

CTC Band:

• L2: Up to 21 LPA
  

• L4: Up to 42 LPA
  

Role Overview

We are hiring Data Software Engineers with strong expertise in Apache Spark, Python, and AWS/Azure Databricks. The ideal candidates will have deep Big Data engineering experience, strong distributed systems knowledge, and the ability to work on complex end-to-end data platforms at scale.

Key Responsibilities

Big Data Engineering

• Design and build distributed data processing systems using Spark and Hadoop.
  

• Develop and optimize Spark applications, ensuring performance and scalability.
  

• Create and manage ETL/ELT pipelines for large-scale data ingestion and transformation.
  

Streaming & Event Processing

• Build and manage real-time streaming systems using Spark Streaming or Storm.
  

• Work with Kafka / RabbitMQ for event-driven ingestion and messaging patterns.
  

Cloud & Databricks Engineering

• Develop & optimize workloads on AWS Databricks or Azure Databricks.
  

• Perform cluster management, job scheduling, performance tuning, and automation.
  

Data Integration & Storage

• Integrate data from diverse sources: RDBMS (Oracle, SQL Server), ERP, file systems.
  

• Work with query engines like Hive and Impala.
  

• Experience with NoSQL stores: HBase, Cassandra, MongoDB.
  

Programming & Scripting

• Strong hands-on coding in Python for data transformations and automations.
  

• Strong SQL skills for data validation, tuning, and complex queries
  

Team Leadership (L4)

• Provide technical leadership and mentoring to junior engineers.
  

• Drive solution design for Big Data platforms end-to-end
  

Ways of Working

• Work in Agile teams, participate in sprint ceremonies and planning.
  

• Collaborate with engineering, data science, and product teams.
  

Required Skills & Expertise (Both L2 & L4)

• Apache Spark – Expert level (core, SQL, streaming)
  

• Python – Strong hands-on
  

• Distributed computing fundamentals
  

• Hadoop ecosystem: Hadoop v2, MapReduce, HDFS, Sqoop
  

• Streaming systems: Spark Streaming / Storm
  

• Messaging: Kafka or RabbitMQ
  

• SQL – Advanced (joins, stored procedures, query optimization)
  

• NoSQL: HBase, Cassandra, MongoDB
  

• ETL frameworks & data pipeline design
  

• Hive / Impala querying
  

• Performance tuning of Spark jobs
  

• AWS or Azure Databricks
  

• Experience working in Agile
  

Experience & Level Mapping

L2 – Mid-Level (4–5 Yrs)

• Skills: Spark, Python, AWS
  

• Notice Period: Immediate – 20 Days
  

• CTC Band: Up to 21 LPA
  

L4 – Senior-Level (8–12 Yrs)

• Skills: Spark, Python, Azure Databricks
  

• Notice Period: 15 Days (Nov joiners) OR Jan joiners
  

• CTC Band: Up to 42 LPA