Senior Databricks Cloud Data Engineer
Summary
Senior data engineer designing Databricks pipelines and enterprise cloud data platforms onsite in Toronto for a provincial ministry. Daily work spans Databricks, Delta Live Tables, Unity Catalog, Azure Data Factory, PySpark, and SQL across Oracle, Azure Data Lake, and Microsoft Fabric, including ETL/ELT automation, CI/CD, and data migration.
Design Databricks pipelines and cloud data platforms onsite in Toronto for the Ministry of Public and Business Service Delivery and Procurement.
This senior assignment requires hands-on expertise with Databricks, Delta Live Tables, Unity Catalog, Azure Data Factory, advanced SQL, and PySpark. You will support enterprise data engineering initiatives across Oracle, Azure Data Lake, Databricks, and Microsoft Fabric. The work includes ETL/ELT automation, CI/CD, data migration, integration, and cloud data service management.
Key Responsibilities:
- Design data pipelines from source systems to end users
- Develop and optimize complex ETL/ELT processes
- Build unified Databricks data platforms with Unity Catalog
- Create Azure Data Factory and Databricks pipelines
- Support migration across Oracle, Azure Data Lake, and Microsoft Fabric
Requirements:
- Advanced SQL, PL/SQL, T-SQL, and query optimization experience
- Hands-on Databricks, Delta Live Tables, and PySpark expertise
- Experience with relational and dimensional data modelling
- CI/CD experience using Azure DevOps Git integration
- Availability to work onsite five days weekly in Toronto
Apply senior Databricks, Azure Data Factory, PySpark, and DataOps expertise to enterprise cloud data engineering in Toronto.