Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

XPT Software

New

Data Engineer

Posted 42 views

Summary

Senior Data Engineer designing, building and optimising scalable batch/streaming pipelines and lakehouse solutions for enterprise clients. Core stack: Azure Databricks, PySpark, Python, SQL, Delta Lake, ADLS Gen2 and Azure Data Factory, with CI/CD via Azure DevOps plus pipeline tuning, governance and mentoring duties.

  • XPT SoftwareAustralia PTY Ltd, incorporated in 2016, is a Software Services company
  • XPT works with topclients across Australia in Banking, Insurance, Telecom,Retail, Energy, Mining and Manufacturingdomains.
  • We have 120+technocrats in Australia working at our clientlocations.
  • XPT SoftwareAustralia is part of group companies which has globalpresence across India & Europe.
  • We have served100+ clients globally, fulfilling their onsite-offshoreneeds.

Role Overview

We are looking for a Senior Data Engineerwith strong hands-on experience in Azure Databricks and the Microsoft Azuredata ecosystem. The role will design, build, optimise, and support scalabledata pipelines and lakehouse solutions that enable analytics, reporting, datascience, and enterprise decision-making.

What You'll Work On

  • Design,build, and optimise scalable batch and streaming data pipelines using AzureDatabricks, PySpark, Python, SQL, and Delta Lake.
  • Developreusable ingestion, transformation, validation, and publishing frameworks forlarge-scale enterprise datasets.
  • Implementmedallion or layered lakehouse patterns across raw, curated, andconsumption-ready data zones.
  • IntegrateAzure Databricks with Azure Data Lake Storage Gen2, Azure Data Factory, AzureKey Vault, Azure DevOps, and related Azure services.
  • Developproduction-grade Databricks notebooks, jobs, workflows, libraries, and clusterconfigurations.
  • Createcomplex SQL and PySpark transformations, data-quality checks, reconciliationlogic, and error-handling routines.
  • OptimiseSpark workloads, storage layouts, partitioning, file sizes, caching, and queryperformance while controlling platform cost.
  • ImplementDelta Lake capabilities such as ACID transactions, schema enforcement, schemaevolution, time travel, and merge operations.
  • Buildautomated CI/CD pipelines for code promotion, configuration management,testing, and deployment across environments.
  • Applydata security and governance controls using role-based access, secretsmanagement, data masking, auditability, and controlled access patterns.
  • Monitordata pipelines, troubleshoot failures, perform root-cause analysis, and deliversustainable production fixes.
  • Partnerwith Architects, Business Analysts, Data Analysts, Product Owners, and businessstakeholders to convert requirements into reliable data products.
  • Providetechnical leadership, code reviews, engineering standards, and mentoring toother data engineers.

Key Skills and Experience

  • 8+years of experience in data engineering, data integration, ETL/ELT, orlarge-scale data platform delivery.
  • Stronghands-on experience with Azure Databricks, including notebooks, jobs orworkflows, cluster configuration, debugging, and performance tuning.
  • Advancedproficiency in Python, PySpark, and SQL for production-grade data engineering.
  • Strongexperience with Delta Lake and lakehouse architecture, including reliableincremental processing and data-quality controls.
  • Practicalexperience with Azure Data Lake Storage Gen2 and Azure Data Factory.
  • Experienceimplementing CI/CD using Azure DevOps, Git, branching strategies, automatedtesting, and environment-based deployment controls.
  • Strongunderstanding of dimensional modelling, data warehousing, data lake design,metadata, lineage, partitioning, and file formats such as Parquet.
  • Experiencewith orchestration, monitoring, alerting, operational support, incidentanalysis, and service-level expectations.
  • Understandingof Azure identity, Key Vault, managed identities, networking, privateendpoints, and secure data-platform patterns.
  • Strongcommunication, documentation, problem-solving, code-review, and stakeholdercollaboration skills.

Good to Have

  • Experiencewith Microsoft Purview, Unity Catalog, Azure Synapse Analytics, Event Hubs, orStructured Streaming.
  • Knowledgeof infrastructure as code using Terraform or Bicep.
  • Experiencewith dbt, Airflow, Kafka, Power BI, or machine-learning data pipelines.
  • Experiencein telecommunications or another high-volume enterprise environment.

Skills

What Senior Data Engineering jobs ask for — and how much of it you have →
Apply

See also

Data Engineering jobs by country — openings, pay and top skills →