Deputy Manager | AWS Data Engineering | Hyderabad | Engineering | Data Modernization & Migration

The Team

Deloitte’s Technology & Transformation practice can help you uncover and unlock the value buried deep inside vast amounts of data. Our global network provides strategic guidance and implementation services to help companies manage data from disparate sources and convert it into accurate, actionable information that can support fact-driven decision-making and generate an insight-driven advantage. Our practice addresses the continuum of opportunities in business intelligence & visualization, data management, performance management and next-generation analytics and technologies, including big data, cloud, cognitive and machine learning.

Your work profile

• Analyze, design, develop, implement, and test scalable Big Data applications for batch processing and API-based workloads on AWS Cloud
• Develop and maintain data processing applications using Java, PySpark, Python, Scala, or Kotlin, as applicable
• Build and integrate REST APIs and SOAP web services for data exchange and application integration
• Develop and manage batch applications and API-based microservices using AWS services such as EMR, EC2, ECS, S3, AWS API Gateway, and Step Functions
• Orchestrate data workflows using Airflow and AWS Step Functions
• Work with Big Data technologies, including Apache Spark, Hadoop, HDFS, MapReduce, and Cassandra
• Perform performance tuning and optimization of Spark, Hadoop, EMR, Java, and Python applications
• Develop automated data ingestion and transformation pipelines
• Create, maintain, and optimize CI/CD pipelines using Maven, Gradle, Jenkins, Artifactory, AWS CodeCommit, and CloudFormation
• Follow best practices for coding, testing, deployment, source control, and application performance optimization

Key skills required

• 7 to 9 years of experience in Big Data application analysis, development, implementation, and testing on AWS Cloud.
• Strong hands-on coding experience in Java and PySpark (both are mandatory).
• Experience developing and supporting both batch processing applications and API-based workloads.
• Experience developing and integrating REST APIs and SOAP web services.
• Hands-on experience with AWS services, including: Amazon EMR, EC2, ECS, and S3, AWS API Gateway, AWS Step Functions, Airflow for workflow orchestration
• Experience with Big Data technologies such as Spark, Hadoop, HDFS, MapReduce, and Cassandra.
• Strong understanding of distributed data processing and batch application performance tuning
• Hands-on experience creating and maintaining CI/CD pipelines using Maven, Gradle, Jenkins, Artifactory, AWS CodeCommit, and CloudFormation.

Nice-to-have

• Experience with Snowflake, particularly in developing automated data ingestion and transformation pipelines using Spark
• Knowledge of DevOps practices and source code management tools such as Git, GitHub, Bitbucket, and GitLab
• Familiarity with development tools such as IntelliJ IDEA and PyCharm
• Additional programming experience in Scala, Kotlin, or Python

Education:

• B. Tech / M. Tech; B.E / M.E; B.Sc. / M.Sc.; B. Com / M.com; MBA/PGDM; BCA/MCA; PHD