Staff Data Engineer
Posted Updated 33
views
Summary
Staff Data Engineer building and maintaining dbt transformation models and complex Snowflake SQL pipelines, developing Apache Airflow DAGs on AWS ECS, and contributing Python/AWS Lambda-based ingestion. Role is based in New South Wales, Australia, and centers on AWS data infrastructure and CI/CD with GitHub Actions.
Responsibilities
- Building and maintaining dbt transformation models across multiple pipeline layers — from raw ingestion through to governed, consumer-ready datasets
- Writing Snowflake SQL for complex transformations: type casting, deduplication, incremental load logic, surrogate key generation
- Developing and debugging Apache Airflow DAGs on AWS ECS
- Contributing to AWS Lambda-based ingestion( Python) — file validation, control file generation, S3 event handling
- Maintaining source-to-target mapping documentation and aligning models to business requirements and data specs
- Supporting onboarding of downstream consumers via data sharing mechanisms and event-driven notifications
- Participating in code reviews, writing YAML-based data quality tests, and owning tickets from design through to production
Tech stack required
- Strong SQL
- Proficiency with dbt Core (Must have)
- Apache Iceberg Table format (Nice to have)
- Apache Airflow
- AWS -Working Knowledge of S3, Lambda, MWAA, ECS, Glue Catalogs, Lake Formation, Lambda, SQS, EventBridge
- Experience with fixed-format or structured financial data files
- CICD with Github Actions.
Confidentialityisassured,andonlyshortlistedcandidateswillbenotifiedforinterviews
Skills
Sign in to read and join the discussion.