Python Databricks Data Engineer
Who recruits here Pro
Up to three recruiters at Electric Mind with their work email, so you can write to a person.
Part of Pro. How it works
Summary
Build and maintain Python/PySpark data pipelines in Databricks — medallion architecture, Unity Catalog SQL queries, Auto CDC and SCD Type 2 merges — supporting a financial services platform across dev, QA, and UAT environments. Permanent or contract role in Toronto, working alongside a senior engineer.
Build Python and PySpark data pipelines at Electric Mind, supporting a financial services platform across development, quality assurance, and user acceptance testing environments.
This developer role is available permanently or by contract. You will maintain Databricks pipelines using SQL, medallion architecture, and curated data models. Working alongside a senior engineer, you will write clean transform code, resolve issues, and learn Databricks internals while supporting reliable platform operations.
Key Responsibilities:
- Write PySpark transforms for silver and gold data layers
- Apply Auto CDC and SCD Type 2 merge patterns
- Query Unity Catalog using SQL warehouses
- Debug pipeline logs, schema drift, and data quality issues
- Maintain tests, scripts, documentation, and GitLab merge requests
Requirements:
- Strong Python fundamentals, including functions, modules, and basic object-oriented programming
- Databricks experience or willingness to learn quickly
- Apache Spark, PySpark, or strong SQL exposure
- Working knowledge of joins, aggregations, and window functions
- Basic Git workflow and methodical debugging skills
Develop Python, PySpark, SQL, and Databricks expertise while improving financial services data pipelines at Electric Mind.
Skills
Sign in to read and join the discussion.