Senior Data Engineer
Summary
Senior Data Engineer in Sydney who designs, builds, and operates scalable, secure Azure data platforms, writing optimized PySpark/Spark SQL pipelines and complex SQL to power analytics, ML, and real-time insights, while mentoring engineers and driving best practices.
Job Description:
Must Have Skills:
Strong hands‑on experience with Azure Data Lake Gen2, Synapse, Databricks
Solid experience in PySpark / Spark SQL, Databricks performance tuning &
job optimization
Proficient in Complex SQL (window functions, CTEs, execution plans)
Strong understanding in Mentoring junior and mid‑level engineers,
Communicating trade‑offs to architects and product teams
Experience working with Azure DevOps or GitHub Actions |
Nice to Have Skills: Experience
with Streaming & Real‑Time Data Processing on Designing low‑latency
pipelines.
Exposure to Data Governance & Metadata Management in Data
classification & sensitivity labels
Familiarity with Advanced Python Engineering Practices on Performance‑aware
Python design
Experience working in Analytics & BI Awareness
Knowledge of Cloud Cost Optimization & FinOps |
Detailed Job Description The Senior Data Engineer will be responsible for designing, building, and
operating scalable, secure, and high‑performance data platforms. The role
involves critical data pipelines and architectures that power analytics,
reporting, and advanced data use cases such as machine learning and real‑time
insights. The candidate will collaborate closely with analytics,
data science, product, and business teams while mentoring engineers and
driving engineering best practices across the data organization. The role combines technical
leadership, architecture ownership, and business impact. |