DataOps / DevOps Engineer

Overview

Own the deployment, promotion, automation, and operational stability of Databricks environments, ensuring secure and reliable CI/CD practices across data and AI workloads.

Role

Job Role: DataOps / DevOps Engineer

Job Location: Abu Dhabi, UAE

Experience: 5+ Years

Role Summary

Own the deployment, promotion, automation, and operational stability of Databricks environments, ensuring secure and reliable CI/CD practices across data and AI workloads.

Key Responsibilities

• Design and maintain CI/CD pipelines for Databricks jobs, notebooks, workflows, and configurations
• Implement pull request governance with mandatory reviews, automated validations, and controlled merges
• Manage environment promotion (Dev, Test, Prod) with approval gates and change controls
• Define workflow orchestration standards including retries, dependencies, and failure handling
• Enable versioning, rollback mechanisms, and release traceability for platform assets
• Standardize secrets and configuration management using secure key management solutions
• Implement monitoring, alerting, and incident management for jobs and workflows

Secondary Responsibilities

• Understand Spark, streaming, and AI workloads to prevent unsafe automation changes
• Integrate data quality and validation checks into CI/CD pipelines
• Collaborate with platform and AI teams during new feature adoption

Required Skills & Experience

• Strong experience with CI/CD implementation in data platform environments
• Hands-on expertise with Databricks Workflows and deployment automation
• Experience with Git-based version control and PR governance
• Knowledge of Azure security, identity, and key management integration
• Experience with monitoring, alerting, and production support practices
• Strong understanding of DevOps best practices for data and AI systems