Senior Python Data Engineer — Spark & Databricks (Remote)
Posted Updated
Description
Halo Labs in Australia is seeking a hands-on Data Engineer to design, develop, and maintain reusable PySpark pipelines and standardised data engineering patterns across the SaaS platform. You will architect and implement scalable ETL/ELT processes on Azure and AWS, build data models, ensure data security and quality, and collaborate with software engineers to embed data services into the product.
This remote-first role offers a learning budget and wellbeing support, plus opportunities to travel
Design, develop, and maintain PySpark pipelines across the SaaS platform. Architect and implement scalable ETL/ELT across Azure and AWS. Build batch and near real-time ingestion pipelines. Design secure data integration patterns ensuring data integrity and compliance. Collaborate with Software Engineers to embed data services into the product architecture. Contribute to CI/CD of data workloads and IaC practices. Bachelor's degree in a relevant field. Strong Python, SQL, and PySpark skills. Hands-on experience with Azure Databricks and Data Factory. Experience building production-grade data pipelines in SaaS environments.Skills
Sign in to read and join the discussion.