Senior Data Engineer
Summary
Designs and builds scalable data pipelines and architectures using Azure Data Lake, Synapse, and Databricks to power analytics and real-time insights for enterprise clients.
XPT Software Australia Pty Ltd | Contract
Sydney, Australia | Posted on 07/16/2026
- XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company
- XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.
- We have 120+technocrats in Australia working at our clientlocations.
- XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.
- We have served100+ clients globally, fulfilling their onsite-offshoreneeds.
Job Description
Must Have Skills:
- Strong hands‑on experiencewith Azure Data Lake Gen2, Synapse, Databricks
- Solid experience in PySpark / Spark SQL, Databricks performance tuning & joboptimization
- Proficient in Complex SQL (window functions, CTEs, execution plans)
- Strong understanding in Mentoring junior and mid‑level engineers, Communicating trade‑offs to architects and product teams
- Experience working with Azure DevOps or GitHub Actions
Nice to Have Skills:
- Experience with Streaming & Real‑Time Data Processing on Designing low‑latencypipelines.
- Exposure to Data Governance & Metadata Management in Data classification & sensitivity labels
- Familiarity with Advanced Python Engineering Practices onPerformance‑aware Python design
- Experience working in Analytics & BI Awareness
- Knowledge of Cloud Cost Optimization & FinOps
Detailed Job Description
The Senior Data Engineer will be responsible for designing, building, and operating scalable, secure, and high‑performance data platforms. The role involves critical data pipelines and architectures that power analytics, reporting, and advanced data use cases such as machine learning and real‑time insights.
The candidate will collaborate closely with analytics, data science, product, and business teams while mentoring engineers and driving engineering best practices across the data organization.
The role combines technical leadership, architecture ownership, and business impact.
Minimum Years of Experience: 6 years
Certifications Needed: No
Top 3 responsibilities you would expect the Subcon to shoulder and execute
- Design & Own Scalable Data Architecture: Design batch and streaming data architectures, choose the right storage, compute, and processing patterns, ensure scalability, reliability, fault tolerance, and cost efficiency.
- Build Reliable, High‑Quality Data Pipelines: Develop and maintain ETL/ELT pipelines at scale, handle incremental loads, CDC, and failure recovery, implement data quality checks and monitoring.
- Technical Leadership & Collaboration: Review code, data models, and designs; translate business requirements into technical solutions; communicate trade‑offs to stakeholders and leadership.