AWS Data Engineer

Job Description

Responsibilities and JD in brief along with additional criteria to be considered:

• Design, develop and maintain scalable data pipelines on AWS.

• Build ETL/ELT workflows using AWS Glue, Python and PySpark, with AWS S3 as the data-lake storage layer.

• Develop and optimize Databricks workloads and Delta Lake implementations.

• Write complex SQL queries for transformation, validation and performance optimization.

• Design logical and physical data models to support analytics and reporting requirements.

• Implement automated CI/CD pipelines using AWS DevOps services such as CodePipeline, CodeBuild and CodeDeploy.

• Apply Git-based version control and DevOps practices for build, deployment, monitoring and release management.

• Ensure data quality, security, governance, performance tuning and production support. • Collaborate with architects, business stakeholders and downstream consumers in an Agile delivery model.

Any additional things to be checked:

• Strong hands-on AWS data engineering experience, including AWS S3 and AWS Glue.

• Hands-on coding proficiency in Python, PySpark and SQL.

• Experience in Databricks and Delta Lake-based data processing.

• Practical data-modeling experience for analytics and reporting use cases.

• Experience implementing CI/CD pipelines in AWS using CodePipeline, CodeBuild, CodeDeploy or equivalent tools.

• Working knowledge of Git and Infrastructure as Code using Terraform or AWS CloudFormation. • Good communication, problem-solving and client-interaction skills.

Location(s) where this position can work out of- Bangalore / Hyderabad

Desired experience range- 7 to 12 years

*Must have skills - 2 skills which are non-negotiable 1. AWS Glue , S3 2. PySpark

*Desirable skills - 1 skill which is nice to have Databricks with Delta Lake