Lead Data Engineer ID71008
Summary
Lead a team to design and implement scalable OLAP data architectures, ETL pipelines, and real-time data processing on AWS, using Airflow for orchestration and enforcing code quality.
- Define OLAP data architecture
- Define pipeline dependencies and triggering
- Design ETL pipelines
- Design data lake partitioning and file formats
- Drive AWS data stack architecture
- Extract transform validate data
- Govern schema and data types
- Implement near real time processing
- Lead developers and align engineering practices
- Orchestrate batch workflows with Airflow DAGs
- Partner with DevOps on requirements
- Review pull requests and enforce code quality
Perks/Benefits:
- Annual learning budget
- Flexible hours
- Learning budget
- Mentorship
- Remote work
- Supportive culture
- Techtalks
- Wellbeing programs