Senior Big Data Engineer - AWS Platform
Summary
Senior Big Data Engineer (8-12 yrs) in Bangalore designing and optimizing scalable big data pipelines on AWS: building Spark jobs with PySpark/Scala, ETL via AWS Glue and Amazon EMR, and orchestration with Step Functions and Lambda, in a B2C marketing & analytics domain.
Role : Senior Big Data Engineer | Bangalore.
Experience : 8 - 12 Years.
Location : Bangalore.
Domain : B2C Marketing & Analytics.
Technology : AWS | Apache Spark | PySpark / Scala.
We are looking for an experienced Senior Big Data Engineer to design, develop, and optimize scalable big data solutions in an AWS-based environment.
Key Responsibilities :
- Design, develop, and maintain scalable big data solutions and data pipelines on AWS.
- Build and optimize data processing jobs using Apache Spark with PySpark or Scala.
- Develop and manage ETL workflows using AWS Glue and Amazon EMR.
- Orchestrate data workflows using AWS Step Functions and AWS Lambda.
- Perform performance tuning, debugging, troubleshooting, and pipeline optimization.
- Ensure data quality, reliability, scalability, and maintainability.
- Collaborate with cross-functional teams and provide technical guidance to engineers.
- Use Git and Git repositories for version control and collaborative development.
Mandatory Skills :
- 8 - 12 years of relevant experience in Big Data Engineering.
- Strong hands-on experience with AWS.
- Proficiency in PySpark or Scala.
- Experience with Apache Spark, Amazon EMR, AWS Glue, AWS Step Functions, and AWS Lambda.
- Strong data pipeline development, ETL, debugging, and performance optimization skills.
- Working knowledge of Git and version control.
Good-to-Have Skills :
- Databricks and/or Snowflake.
- Additional AWS services and cloud data engineering.
- Data pipeline monitoring and optimization.
- Basic AI/AI-related knowledge.