AWS Data Engineer
Posted 4
views
Summary
6-month contract role (view to extend) in Abu Dhabi building and optimizing scalable AWS data ingestion pipelines — Glue, PySpark, S3, Athena, and Lambda — implementing CDC/batch loading patterns, Bronze-layer pipelines, and Terraform/Git automation for a banking/financial-services environment.
Contract Length – 6 Months (view to extend)
Start Date – ASAP
The role will focus on building scalable AWS-based ingestion pipelines, optimizing performance, and ensuring alignment with architecture and data quality standards.
Key Responsibilities:
- Build and maintain data ingestion pipelines using AWS Glue, PySpark, S3, Athena, and Lambda.
- Implement CDC, full load, delta, snapshot, and historical loading patterns.
- Develop and optimize Bronze-layer ingestion pipelines.
- Improve pipeline performance through partitioning, parallelization, and AWS best practices.
- Automate deployment and infrastructure using Git and Terraform.
- Ensure pipelines comply with data contracts, monitoring, and data quality standards.
- Work closely with Data Architects, Data Assurance, and Platform teams.
- Identify opportunities to automate repetitive migration and engineering activities.
Required Skills & Experience:
- 5+ years of experience in Data Engineering.
- Strong hands-on experience with AWS data services, particularly Glue, S3, Athena, and Lambda.
- Strong PySpark and ETL/ELT development experience.
- Good understanding of Lakehouse architecture and Medallion design principles.
- Experience with CDC, batch ingestion, delta loads, and historical data migration.
- Experience with Git, Terraform, CI/CD, and Agile/Jira.
- Banking or financial services experience is an advantage.