Azure Databricks Data Engineer
Posted Updated
Description
Parkland Corporation is expanding its Enterprise BI team with a BI Developer focused on data engineering in an Azure Databricks environment. You will design, build, test, and support trusted data products and pipelines, leveraging Delta Lake, Unity Catalog, and related Databricks features to deliver reliable analytics-ready data.
The ideal candidate has strong experience in Azure Databricks, Fabric or Synapse, and a solid grounding in SQL and Python.
Design, build, and maintain scalable data pipelines and curated data products in Azure Databricks. Develop data transformations using SQL and Python for reliable, maintainable pipelines. Implement medallion-style data architectures to support reporting and analytics. Utilize Delta Lake, Unity Catalog, and related Databricks features as appropriate. Collaborate with analysts, BI developers, and data owners to translate requirements into data products. Build automated data-quality checks, monitoring, and production support processes. Follow GitHub-based delivery practices including CI/CD and code reviews. Contribute to reusable engineering standards and documentation. 3+ years of experience developing production data pipelines, data transformations, or analytical data products. Strong hands-on data engineering experience with Azure Databricks, Fabric, or Synapse in production pipelines. Strong SQL skills and practical Python or PySpark experience for ingestion, transformation, testing, and automation. Experience with lakehouse architecture, Delta tables, layered data design, data modeling, and performance optimization. Experience orchestrating and monitoring batch or near-real-time data pipelines with dependency management. Knowledge of Git terminology and basic CI/CD concepts; collaboration with code reviews. Experience implementing data-quality checks, reconciliation, logging, monitoring, and production support practices. Understanding of data security, governance, lineage, and metadata management.Skills
Sign in to read and join the discussion.