Technical Specialist (Azure & Databricks DevOps Engineer) - ID: 26-086
Duration: 12 Months
Hours of Work: 35 Hours
Our client is looking for an experienced Azure & Databricks DevOps Engineer to design, automate, secure, and support cloud-based data platforms and DevOps capabilities across Microsoft Azure and Databricks. This role will be responsible for building scalable infrastructure, implementing CI/CD pipelines, managing platform reliability, and enabling data engineering teams to deliver high-quality analytics and data products efficiently.
The ideal candidate has strong hands-on experience with Azure Cloud Services, Azure DevOps or GitHub Actions, Terraform, Databricks Workspace Administration, Unity Catalog, CI/CD automation, Security Controls, Monitoring, and Production Support.
Responsibilities:
- Design, deploy, and manage cloud infrastructure on Microsoft Azure using Infrastructure as Code.
- Build and maintain CI/CD pipelines for data engineering, analytics, and platform services.
- Administer and support Azure Databricks workspaces, clusters, jobs, repos, workflows, and permissions.
- Implement and manage Databricks Unity Catalog, including catalogs, schemas, external locations, storage credentials, and access controls.
- Automate deployment of Databricks notebooks, jobs, workflows, libraries, clusters, policies, and workspace configurations.
- Develop reusable DevOps frameworks and deployment templates for data platform teams.
- Manage Azure services such as:
- Azure Data Lake Storage Gen2
- Azure Key Vault
- Azure Synapse Analytics
- Azure Data Factory
- Azure Monitor / Log Analytics
- Azure Functions
- Implement security best practices including RBAC, managed identities, service principals, secrets management, private endpoints, network security, and least-privilege access.
- Support production operations, incident response, troubleshooting, monitoring, alerting, and performance optimization.
- Collaborate with data engineers, architects, security teams, product owners, and application teams.
- Manage environment promotion across development, UAT, and production environments.
- Support cost monitoring and optimization for Azure and Databricks workloads.
- Create and maintain technical documentation, runbooks, deployment guides, and operational procedures.
Qualifications:
- Proven experience as a DevOps Engineer, Cloud Engineer, and/or Data Platform Engineer
- Strong hands on experience with Microsoft Azure cloud services
- Hands-on experience with Azure Databricks administration and deployment automation
- Experience building CI/CD pipelines using Azure DevOps, GitHub Actions, and/or Jenkins
- Strong Infrastructure as a Code experience using Terraform
- Experience with Databricks Platform Components including Workspaces, Clusters, Jobs/Workflows, Repos, Libraries, Secret Scopes, Cluster Policies, Unity Catalog
- Strong scripting/programming skills using Python, PowerShell, Bash, and/or SQL.
- Understanding of cloud identity and access management, including Azure AD / Microsoft Entra ID, RBAC, service principals, managed identities, and group-based access control.
- Experience with monitoring and logging tools such as Azure Monitor, Log Analytics, Application Insights, or Databricks system tables.
- Familiarity with Git branching strategies, pull requests, code reviews, and release management.
- Strong troubleshooting, communication, and documentation skills.
Technical Specialist (Azure & Databricks DevOps Engineer) - ID: 26-086
Duration: 12 Months
Hours of Work: 35 Hours
Location: Oshawa (2 days/week onsite)
Our client is looking for an experienced Azure & Databricks DevOps Engineer to design, automate, secure, and support cloud-based data platforms and DevOps capabilities across Microsoft Azure and Databricks. This role will be responsible for building scalable infrastructure, implementing CI/CD pipelines, managing platform reliability, and enabling data engineering teams to deliver high-quality analytics and data products efficiently.
The ideal candidate has strong hands-on experience with Azure Cloud Services, Azure DevOps or GitHub Actions, Terraform, Databricks Workspace Administration, Unity Catalog, CI/CD automation, Security Controls, Monitoring, and Production Support.
Responsibilities:
- Design, deploy, and manage cloud infrastructure on Microsoft Azure using Infrastructure as Code.
- Build and maintain CI/CD pipelines for data engineering, analytics, and platform services.
- Administer and support Azure Databricks workspaces, clusters, jobs, repos, workflows, and permissions.
- Implement and manage Databricks Unity Catalog, including catalogs, schemas, external locations, storage credentials, and access controls.
- Automate deployment of Databricks notebooks, jobs, workflows, libraries, clusters, policies, and workspace configurations.
- Develop reusable DevOps frameworks and deployment templates for data platform teams.
- Manage Azure services such as:
- Azure Data Lake Storage Gen2
- Azure Key Vault
- Azure Synapse Analytics
- Azure Data Factory
- Azure Monitor / Log Analytics
- Azure Functions
- Implement security best practices including RBAC, managed identities, service principals, secrets management, private endpoints, network security, and least-privilege access.
- Support production operations, incident response, troubleshooting, monitoring, alerting, and performance optimization.
- Collaborate with data engineers, architects, security teams, product owners, and application teams.
- Manage environment promotion across development, UAT, and production environments.
- Support cost monitoring and optimization for Azure and Databricks workloads.
- Create and maintain technical documentation, runbooks, deployment guides, and operational procedures.
Qualifications:
- Proven experience as a DevOps Engineer, Cloud Engineer, and/or Data Platform Engineer
- Strong hands on experience with Microsoft Azure cloud services
- Hands-on experience with Azure Databricks administration and deployment automation
- Experience building CI/CD pipelines using Azure DevOps, GitHub Actions, and/or Jenkins
- Strong Infrastructure as a Code experience using Terraform
- Experience with Databricks Platform Components including Workspaces, Clusters, Jobs/Workflows, Repos, Libraries, Secret Scopes, Cluster Policies, Unity Catalog
- Strong scripting/programming skills using Python, PowerShell, Bash, and/or SQL.
- Understanding of cloud identity and access management, including Azure AD / Microsoft Entra ID, RBAC, service principals, managed identities, and group-based access control.
- Experience with monitoring and logging tools such as Azure Monitor, Log Analytics, Application Insights, or Databricks system tables.
- Familiarity with Git branching strategies, pull requests, code reviews, and release management.
- Strong troubleshooting, communication, and documentation skills.
AI may be used in evaluating candidates.
This posting is for an existing vacancy.