Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

BeachHead

NewBe an early applicant

Databricks Spark Data Engineer

Posted

Summary

Six-month contract building enterprise-grade data pipelines on Databricks and Apache Spark for a top financial/banking client in Toronto: batch and streaming ETL with Auto Loader and Structured Streaming, Unity Catalog governance, Delta Lake, SCD/CDC, and performance tuning via partitioning and liquid clustering.

Build scalable data solutions as a Databricks Developer using Apache Spark and advanced data engineering practices with a top financial client. This six-month contract focuses on enterprise-grade pipelines.

Design, develop, and optimise Databricks solutions across large-scale datasets and distributed computing environments. Build batch and streaming ETL with Auto Loader and Spark Structured Streaming. Apply Unity Catalog, Lakehouse Federation, SCD methodologies, and change data capture while collaborating with stakeholders on practical solution design.

Key Responsibilities:

  • Design and optimise Databricks and Apache Spark data pipelines
  • Build batch and streaming ETL using Auto Loader
  • Maintain catalogs, schemas, tables, views, functions, and volumes
  • Implement SCD 1, SCD 2, and change data capture
  • Apply partitioning and liquid clustering for performance

Requirements:

  • Hands-on Databricks and Apache Spark development experience
  • Experience with Azure Databricks or Delta Lake
  • Strong Unity Catalog and Lakehouse Federation experience
  • Enterprise experience; banking experience preferred
  • Strong communication and stakeholder management skills

Apply Databricks, Spark, ETL, and governance expertise to deliver reliable enterprise data solutions for financial services.

Skills

Apply

See also

Data Engineering jobs by country — openings, pay and top skills →