Data Engineer (AWS)
Summary
Build and maintain scalable data pipelines on AWS using PySpark, Glue, Lambda, and Terraform, while automating deployments with CI/CD.
Key Responsibilities
- Design, develop, and maintain scalable data pipelines using PySpark on AWS.
- Build and orchestrate ETL/ELT workflows using AWS Glue and AWS Step Functions.
- Develop serverless applications and automation using AWS Lambda.
- Write clean, efficient, and maintainable Python/PySpark code following engineering best practices.
- Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
- Implement and maintain CI/CD pipelines to automate code deployment, testing, and infrastructure changes.
- Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency.
- Collaborate with business stakeholders to deliver high-quality data solutions.
- Follow DevOps, security, and coding standards throughout the engagement.
Required Skills
- Bachelor’s degree in computer science or a related discipline.
- With around 3 plus years hands-on experience with PySpark and Python for data engineering.
- Experience developing ETL pipelines using AWS Glue.
- Proficiency with AWS Step Functions for workflow orchestration.
- Experience building serverless solutions using AWS Lambda.
- Hands-on experience with Terraform for Infrastructure as Code (IaC).
- Experience implementing CI/CD pipelines using tools such as GitLab, GitHub Actions, Jenkins, or similar.
- Good understanding of AWS services, including data lakes, IAM, S3, CloudWatch, and monitoring.
Thankyou for your interest!
We respect your privacy andall communication will be treated with confidentiality. If you wish to knowmore about this position or explore other roles, please prepare your updatedprofile and get in touch with our consultant at cristina.malijan@manpower.com.sg
CristinaMalabuyoc Malijan EA License No. 02C3423 Personnel Registration No. R1111547
Please note that your response to this advertisement andcommunications with us pursuant to this advertisement will constitute informedconsent to the collection, use and/or disclosure of personal data byManpowerGroup Singapore for the purpose of carrying out its business, incompliance with the relevant provisions of the Personal Data Protection Act2012. To learn more about ManpowerGroup's Global Privacy Policy, please visithttps://www.manpower.com.sg/privacy-policy