Databricks Data Engineer
Posted 7
views
Summary
A data engineer in Toronto building and optimizing data pipelines on Azure Databricks. Day to day involves ETL/ELT development with PySpark and Spark SQL, managing data on ADLS with Unity Catalog, plus SQL performance tuning and data migration/reconciliation work, including a BigQuery migration on GCP.
- Proven experience with Azure Databricks, including Unity Catalog and Data Lakehouse Architecture.
- Strong proficiency in PySpark and Spark SQL.
- Expertise in ETL/ELT Development.
- Experience with Azure Data Lake Storage (ADLS).
- Familiarity with Git and CI/CD practices.
- SQL Optimization and Performance Tuning skills.
- Experience in Data Migration and Reconciliation Frameworks.
- GCP BigQuery Migration Experience is essential.