LEAD DATA SCIENCE (ML Ops)

Job Overview

Solves complex problems and help stakeholders make data- driven decisions by leveraging quantitative methods, such as machine learning. It often involves synthesizing large volume of information and extracting signals from data in a programmatic way.

Job Responsibilities

• Architect and evolve the ML platform on Databricks (Unity Catalog, MLflow, Feature Store, Model Serving, Workflows) integrated with AWS services

• Define and enforce standards for CI/CD, testing, model promotion, and reproducibility across teams

• Build automated pipelines for training, validation, deployment, and retraining

• Implement monitoring for model performance, data drift, latency, and cost (Lakehouse Monitoring, CloudWatch, Prometheus/Grafana, or similar)

• Own governance: model lineage, access control, audit trails, and compliance with data and regulatory requirements

• Manage compute and cost optimization (cluster policies, GPU usage, serving autoscaling)

• Mentor engineers, run design and code reviews, and drive incident response and postmortems

• Evaluate and introduce new tooling where it earns its place

Required Skills

Databricks

• Production experience with MLflow (tracking, registry, model serving), Unity Catalog, Delta Lake, and Databricks Workflows/Jobs

• Databricks Asset Bundles or Terraform provider for workspace and job deployment

• Spark/PySpark for scalable data and feature pipelines

• Cluster configuration, policies, and performance tuning

AWS

• Core services: S3, IAM, VPC, EC2, ECR, Lambda, CloudWatch, Secrets Manager, KMS

• Databricks-on-AWS networking and security (PrivateLink, cross-account roles, workspace setup)

• Familiarity with SageMaker, EKS, or API Gateway as complements to Databricks serving

Engineering and DevOps

• Strong Python; clean, tested, production-grade code

• Infrastructure as Code (Terraform strongly preferred)

• CI/CD (GitHub Actions, GitLab CI, Jenkins, or Azure DevOps)

• Docker; Kubernetes is a must

MLOps practices

• Batch and real-time inference patterns

• Drift detection, A/B and shadow deployment, rollback strategies

• Experiment tracking, model versioning, and reproducibility

• LLMOps on Databricks: Mosaic AI, Vector Search, RAG pipelines, evaluation and guardrails

• Feature store design and streaming (Structured Streaming, Must have: Kafka/Kinesis)

• Databricks or AWS certifications (ML Professional, Solutions Architect)

• FinOps experience and regulated-industry or data-residency exposure