Senior Data Engineer
Summary
Senior Data Engineer designing and building scalable end-to-end data pipelines on AWS using Glue, PySpark, Python, S3, Athena, Redshift, and Airflow.
Senior Talent Acquisition Specialist @ Kaizen Global Technologies
Eligibility: Must have valid Australian work rights (Citizen/PR/Valid Visa holders).
Key Responsibilities
- Design and build scalable end-to-end data pipelines using AWS services.
- Develop and optimise ETL/ELT workflows using AWS Glue, PySpark, Python, and Lambda.
- Build efficient data processing solutions integrated with S3, Athena, Redshift, and RDS.
- Orchestrate and schedule data pipelines using Apache Airflow or AWS Managed Airflow.
- Implement data quality checks, schema validation, and monitoring to ensure reliable data delivery.
- Experience in data engineering within large-scale, data-intensive environments.
- Strong hands‑on expertise in AWS services including Glue, S3, Lambda, Athena, Redshift, and IAM.
- Advanced proficiency in PySpark, Python, and distributed data processing frameworks.
- Practical experience using Airflow for orchestration and workflow management.
- Solid understanding of data modelling, partitioning strategies, performance tuning, and CI/CD practices.
Seniority Level
Mid‑Senior level
Employment Type
Full‑time
Job Function
Information Technology
Industries
IT Services and IT Consulting