DataOps / DevOps Engineer
Summary
Build and maintain CI/CD pipelines for Databricks jobs and AI workloads, ensuring secure, automated deployments and operational stability across Dev/Test/Prod environments.
Overview
Own the deployment, promotion, automation, and operational stability of Databricks environments, ensuring secure and reliable CI/CD practices across data and AI workloads.
Role
Job Role: DataOps / DevOps Engineer
Job Location: Abu Dhabi, UAE
Experience: 5+ Years
Role Summary
Own the deployment, promotion, automation, and operational stability of Databricks environments, ensuring secure and reliable CI/CD practices across data and AI workloads.
Key Responsibilities
- Design and maintain CI/CD pipelines for Databricks jobs, notebooks, workflows, and configurations
- Implement pull request governance with mandatory reviews, automated validations, and controlled merges
- Manage environment promotion (Dev, Test, Prod) with approval gates and change controls
- Define workflow orchestration standards including retries, dependencies, and failure handling
- Enable versioning, rollback mechanisms, and release traceability for platform assets
- Standardize secrets and configuration management using secure key management solutions
- Implement monitoring, alerting, and incident management for jobs and workflows
Secondary Responsibilities
- Understand Spark, streaming, and AI workloads to prevent unsafe automation changes
- Integrate data quality and validation checks into CI/CD pipelines
- Collaborate with platform and AI teams during new feature adoption
Required Skills & Experience
- Strong experience with CI/CD implementation in data platform environments
- Hands-on expertise with Databricks Workflows and deployment automation
- Experience with Git-based version control and PR governance
- Knowledge of Azure security, identity, and key management integration
- Experience with monitoring, alerting, and production support practices
- Strong understanding of DevOps best practices for data and AI systems