AWS Data Engineer
Summary
Data engineer role at a mid-sized consultancy serving public sector clients in London. Day to day involves building and orchestrating data pipelines with Python, SQL, and Apache Airflow, using AWS data services (Redshift, Glue, Lambda, OpenSearch, Step Functions, Batch), containerising with Docker, and working via GitLab CI/CD.
Working with a mid sized consultancy with a lot of public sector engagements.
- Strong programming skills in Python and proficiency in SQL.
- Experience with Apache Airflow for DAG orchestration and monitoring.
- Hands-on experience with Docker for containerisation.
- Proficient in AWS data services:
- Redshift, OpenSearch, Lambda, Glue, Step Functions, Batch.
- Familiarity with CI/CD pipelines and YAML-based configuration (e.g., GitLab CI/CD).
- Proficient in Git and collaborative development using GitLab (or similar).
- Understanding of AWS security best practices, IAM policies, and RBAC.