Data Engineer - Databricks
Summary
12-month contract Data Engineer placed with a major financial services/insurance organisation in Sydney or Brisbane (hybrid). Day to day: building PySpark/SQL pipelines in Databricks, dbt and Delta Lake transformations across Bronze/Silver/Gold layers, and migrating legacy platforms (including SAS) to a cloud lakehouse.
Location: Sydney / Brisbane (Hybrid)
Contract: 12 Months
Industry: Financial Services / Insurance
Data Engineer – Databricks
The Opportunity
INGRITY is seeking an experienced Data Engineer to join a major financial services organisation undergoing a significant data modernisation program.
You'll work within a modern Databricks environment, building scalable data pipelines, transforming complex datasets and supporting the migration of legacy data platforms to a cloud-based lakehouse architecture.
Key Responsibilities
- Design, develop and optimise data pipelines using Databricks, Python/PySpark and SQL.
- Build and maintain data transformations using dbt and Delta Lake.
- Develop scalable data models across Bronze, Silver and Gold layers.
- Support data migration from legacy platforms, including SAS.
- Implement data quality checks, testing and performance optimisation.
- Collaborate with data engineers, analysts and business stakeholders.
Skills & Experience
- Strong hands-on experience with Databricks, PySpark and SQL.
- Experience with dbt, Delta Lake and lakehouse architecture.
- Proven experience developing and maintaining ETL/ELT pipelines.
- Experience with cloud platforms, ideally Azure or AWS.
- Understanding of CI/CD, data testing and production support.
- Experience in insurance or financial services highly regarded.
- Exposure to AI-assisted development tools such as GitHub Copilot or Databricks Genie advantageous.
Why Join?
- Work on a large-scale data transformation program.
- Modern Databricks and cloud technology environment.
- Collaborative engineering team with hybrid flexibility.
- Opportunity to contribute to AI-enabled data engineering practices.