Data Engineer-Nsw
Posted
Summary
Senior data engineer (9-14 years) building enterprise-scale batch and real-time data platforms on AWS in Sydney — designing ingestion frameworks, streaming pipelines with Kinesis/Kafka, and orchestrated transformations using Python, PySpark, Airflow, Redshift and SageMaker Unified Studio.
We are looking for an experienced Data Engineer (9-14 Years) to join a high-performing team delivering enterprise-scale data platforms and real-time data solutions on AWS.
Must Have Skills
- Python & PySpark
- Apache Airflow (MWAA)
- SQL
- Amazon Redshift
- AWS SageMaker Unified Studio
- AWS Kinesis / Kafka (Real-Time Streaming)
Key Responsibilities
- Design and develop scalable batch and real-time data ingestion frameworks.
- Build and optimize data pipelines using AWS services and modern data engineering practices.
- Develop low-latency streaming solutions using Kinesis, Kafka, or similar technologies.
- Implement data transformations, orchestration workflows, and monitoring solutions.
- Work with Data Warehouses, Data Lakes, and Lakehouse architectures.
- Collaborate with cross-functional teams to deliver robust and scalable data solutions.
Required Experience
- Strong expertise in AWS, Python, PySpark, Airflow, and Redshift.
- Hands-on experience with SageMaker Unified Studio.
- Experience with relational and NoSQL databases (Oracle, MongoDB, DynamoDB, Snowflake, Teradata, etc.).
- Proven experience building real-time and batch data pipelines.
- Knowledge of CI/CD, Git, Infrastructure as Code (Terraform/CloudFormation).
- Data Modelling experience is highly desirable.
Data Engineer-NSW Sydney, New South Wales, Australia