Data Engineering Lead
Summary
Technical lead for data engineering at Impelsys in Dubai, responsible for designing and optimizing scalable data pipelines and leading/mentoring data engineers. Day-to-day centers on Apache Airflow DAGs, DBT infrastructure and models, Python/SQL, and managing the underlying AWS infrastructure (ECS, Fargate, EC2, EKS).
By using this form you agree with the storage and handling of your data by this website. *
We are looking for an experiencedData EngineeringLeadto provide technical leadership across multiple data engineering projects and squads.
The ideal candidate will have strong hands-on experience withApache Airflow, DBT, Python, SQL, and AWS, along with proven experience designing scalable data pipelines and leading data engineering teams.
You willbe responsible fordeveloping andoptimizingAirflow workflows, managing DBT infrastructure, improving data engineering architecture, and ensuring reliable, scalable, and cost-effective data solutions.
Key Responsibilities
- Provide technical leadership across data acquisition, transformation, and distribution.
- Lead, mentor, and coach Data Engineers, Data Architects, and Database Administrators.
- Design, implement, and manage scalable data pipelines usingApache Airflow.
- Develop, test, andmaintaincomplexAirflow DAGs and custom operators.
- Monitor, troubleshoot, andoptimizeAirflow workflows for performance and reliability.
- Develop, test, and maintainDBT infrastructure and modelsexecuted through Airflow.
- Manage DBT infrastructure upgrades and optimize DBT code for performance and efficiency.
- Integrate DBT workflows with Apache Airflow.
- Ensure data quality, integrity, reliability, and consistency across data pipelines.
- Collaborate with Data Engineers, Data Scientists, Product Owners, and other stakeholders to translate requirements into scalable solutions.
- Recommend improvements to data engineering architecture, frameworks, and technologies.
- Drive adoption of modernETL/ELT, workflow orchestration, and data managementpractices.
- Document workflows, processes, configurations, and technical solutions.
- Troubleshoot production issues and drive continuous improvement.
Required Skills & Experience
- Proven hands-on experience withApache Airflow.
- Strong experience designing andmaintainingcomplex Airflow DAGs.
- Good Knowledge withDBT (Data Build Tool).
- Experience managing DBT infrastructure upgrades and optimizing DBT code.
- StrongSQLand relational database experience.
- Strong understanding ofETL/ELTconcepts and data pipeline architecture.
- Hands-on experience withAWSdata and infrastructure services.
- Experience withECS, Fargate, EC2, and EKS.
- Experience withGitand version control.
- Strong understanding of data quality, monitoring, troubleshooting, and performance optimization.
- Excellent problem-solving, communication, and collaboration skills.
- Proven experience mentoring and leading technical teams.
AWS Infrastructure Responsibilities
- Manage andoptimizeAWS infrastructure supporting Apache Airflow.
- Hands-on experience withAmazon ECS, AWS Fargate, Amazon EC2, and Amazon EKS.
- Ensure Airflow infrastructure is scalable, reliable, secure, and cost-effective.
- Troubleshoot infrastructure, deployment, and performance issues.
- Support cloud infrastructure optimization and automation.
Must Have
- DBT
- Python
- AWS (ECS/Fargate/EC2/EKS)
- SQL
Nice to Have
- ETL/ELT
- Enterprise Data Warehousing
- Git
Nice to Have
- Experience with modern data architecture and cloud-native data platforms.
- Experience with CI/CD and infrastructure-as-code tools such as Terraform.
- Experience with Agile/Scrum methodologies.