Lead/Architect Data Engineer
About The Role
In this role, you will contribute to validating and refining the approach for migrating Apache Airflow workflows from Astronomer to AWS Managed Workflows for Apache Airflow (MWAA). Combining hands‑on Data Engineering with an architectural mindset, you will execute a pilot migration, identify technical constraints and dependencies, and provide recommendations that will help shape the approach for further migration activities.
About The Role
In this role, you will contribute to validating and refining the approach for migrating Apache Airflow workflows from Astronomer to AWS Managed Workflows for Apache Airflow (MWAA). Combining hands‑on Data Engineering with an architectural mindset, you will execute a pilot migration, identify technical constraints and dependencies, and provide recommendations that will help shape the approach for further migration activities.
Responsibilities
- Execute the pilot migration of selected Apache Airflow DAGs from Astronomer to AWS MWAA
- Validate the proposed migration approach through hands‑on implementation, testing, and technical analysis
- Identify and document technical constraints, dependencies, compatibility considerations, and limitations affecting the migration
- Troubleshoot and resolve technical issues identified during the pilot migration
- Evaluate migration challenges and propose practical workarounds or alternative technical approaches where required
- Document unresolved issues and dependencies to support subsequent analysis and remediation
- Assess pilot outcomes and contribute to refining the migration approach for further migration activities
- Collaborate with technical stakeholders to communicate findings, risks, and recommended solutions
- Provide technical recommendations and implementation guidance based on lessons learned during the pilot
Requirements
- 5+ years of experience in Data Engineering, including experience designing and supporting production data workflows
- Strong hands‑on experience with Apache Airflow, including DAG development, configuration, troubleshooting, and workflow orchestration
- Practical experience with AWS and AWS Managed Workflows for Apache Airflow (MWAA)
- Experience with migration or modernization of data orchestration platforms and workflows
- Strong troubleshooting and analytical skills with the ability to identify technical constraints, dependencies, and compatibility issues
- Ability to evaluate technical approaches and recommend practical solutions based on implementation findings
- Experience working independently on technically complex initiatives while collaborating closely with engineering and architecture stakeholders
- Strong documentation and communication skills for presenting technical findings, risks, and recommendations
- Upper‑intermediate or higher level of English
SoftServe is an equal opportunity employer. Qualified applicants will receive consideration regardless of race, color, ancestry, ethnicity, national origin, religion, sex, sexual orientation, gender identity or expression, age, citizenship, disability, health condition, marital or family status, veteran status, or any other characteristic protected by applicable law.