Airflow Engineer
- Analyze existing Cybermation job schedules, dependency patterns, runtime parameters, execution frequency, owners, success/failure behavior, and operational support expectations.
- Review existing shell scripts and API/cURL invocation logic to determine the appropriate Airflow DAG design, task structure, error handling, and retry configuration.
- Design, develop, and maintain Apache Airflow DAGs to orchestrate shell scripts, API calls, and dependent jobs with appropriate scheduling and operational controls.
- Configure Airflow schedules, task dependencies, retries, logging, alerting hooks, variables, connections, parameters, and environment-specific configuration as per client standards.
- Perform dry runs, unit testing, schedule validation, API response validation, failure/retry validation, and production-readiness checks for migrated jobs.
- Create migration mapping, validation evidence, deployment notes, operational runbooks, and support handover documentation.
- Coordinate with client SMEs, application owners, infrastructure teams, and release teams to support deployment, issue triage, initial run monitoring, and hypercare.
- Analyze existing Cybermation job schedules, dependency patterns, runtime parameters, execution frequency, owners, success/failure behavior, and operational support expectations.
- Review existing shell scripts and API/cURL invocation logic to determine the appropriate Airflow DAG design, task structure, error handling, and retry configuration.
- Design, develop, and maintain Apache Airflow DAGs to orchestrate shell scripts, API calls, and dependent jobs with appropriate scheduling and operational controls.
- Configure Airflow schedules, task dependencies, retries, logging, alerting hooks, variables, connections, parameters, and environment-specific configuration as per client standards.
- Perform dry runs, unit testing, schedule validation, API response validation, failure/retry validation, and production-readiness checks for migrated jobs.
- Create migration mapping, validation evidence, deployment notes, operational runbooks, and support handover documentation.
- Coordinate with client SMEs, application owners, infrastructure teams, and release teams to support deployment, issue triage, initial run monitoring, and hypercare.
- Bachelor’s degree or equivalent experience in Computer Science, Information Technology, Engineering, or a related discipline.
- 4+ years of overall engineering experience with hands-on Airflow, Python, and Linux/shell exposure.
- Strong practical understanding of job scheduling, batch/workflow orchestration, production readiness, and operational support processes.