Senior MLOps/Data Engineer
Posted 5
views
Summary
Senior MLOps/Data Engineer who owns the end-to-end ML lifecycle on Azure and Databricks: orchestrating data/ML pipelines (ingest to deploy and monitoring), standardizing experimentation with MLflow, automating CI/CD, and running observability and incident response for ML models alongside applied scientists. Core stack: Python, PySpark, Databricks, Delta Lake, and Azure.
What you\'ll do
In a nutshell, own the end-to-end ML lifecycle on Azure and Databricks, working with applied scientists to operate reliable models.
- Orchestrate and maintain ML pipelines (ingest → feature engineering → train → evaluate → deploy→monitor→repeat) on Azure + Databricks
- Standardize experimentation using MLflow or similar tools (tracking, artifacts, model registry, stages)
- Automate jobs with Databricks Workflows and CI/CD (GitHub Actions or Azure DevOps)
- Implement data & model observability: freshness/completeness, drift (features/model), training/serving skew, SLA/SLO monitoring
- Ensure security & compliance
- Handle incidents and post-mortems for ML pipelines and serving infrastructure
What you\'ll need
- Excellence in Python software engineering and developing tests
- Fundamental understanding of Machine Learning
- 3+ years in Data Eng/MLOps roles
- Strong PySpark
- Hands-on with Databricks and Delta Lake
- CI/CD for data/ML (Git, PR workflow, automated tests, environment pinning)
- Azure basics
- Monitoring and building dashboards
- Clear communication; operational-excellence mindset (SLA/SLO ownership)
What\'s nice to have
- Unity Catalog experience
- Databricks Feature Store
- Terraform for workspace/clusters/jobs/UC objects
- Telemetry domain exposure
- Optimize PySpark jobs (partitioning, caching, etc.) and cost (autoscaling, spot).