MLOPs Lead

Role Summary

The MLOps Lead will build and run the organisation's AI platform, the internal developer platform on which AI teams build, deploy and publish services. The platform serves open-weight models on IndiaAI and on-premise GPU compute, routes calls to approved Indian, cloud and commercial model APIs through a single governed gateway, and runs model and prompt registries and fine-tuning and evaluation pipelines. It integrates with the organisation's DevSecOps and developer platform (Backstage, GitLab CE, Argo CD and Vault). The role requires depth in LLM serving and LLMOps as well as classical MLOps, and leadership of a small platform team.

Educational Qualification

• B.Tech./B.E., M.Tech./M.S. or M.Sc. in Computer Science, Information Technology, AI/ML, Electronics or a related discipline (Must have)
• Certifications (desirable, not mandatory): CKA or CKAD, a cloud ML or DevOps professional certification, HashiCorp Terraform Associate
• Candidates from non-traditional backgrounds with strong verifiable work (platforms built, significant open-source contributions) will be considered in lieu of formal qualification

Experience

• 6+ years in platform, infrastructure or ML engineering (9+ years preferred); minimum 2 years running LLM inference or ML platforms in production
• Demonstrable experience serving open-weight LLMs on GPUs in production, with documented throughput, latency and cost per request
• Experience leading a small platform or infrastructure team
• Experience with Kubernetes in production, infrastructure as code and CI/CD
• Experience building or operating a gateway across several model providers, or an internal developer platform such as Backstage, is a strong plus
• Experience with on-premise GPU clusters or multi-tenant platforms is a plus

Key Responsibilities

Model Serving & GPU Operations

• Serve LLM, embedding, speech and vision models with vLLM, SGLang, TensorRT-LLM or NVIDIA Triton, using quantisation, continuous batching, prefix caching, speculative decoding, LoRA adapter serving and autoscaling, with distributed inference where models exceed one node
• Plan and govern GPU capacity across IndiaAI and on-premise compute through per-team namespaces, quotas and priority classes

AI Gateway & Model Access

• Build and run a single gateway for all model calls (self-hosted, Indian, cloud and commercial), with routing, fallbacks, caching, quotas, API keys, and token and cost metering per project
• Enforce data-classification routing, PII detection and logging policy on calls to external providers, with the Security and AI Safety teams

Model Lifecycle & Developer Platform

• Run model and prompt registries, fine-tuning jobs, automated evaluation gates, canary release and rollback, with provenance and audit trails
• Provide self-service development environments and golden paths, so that an AI engineer can go from a model to a live, governed endpoint without platform-team help

Platform Integration

• Integrate the AI platform with the organisation's developer platform: Backstage catalogue entries and golden-path templates for AI services, GitLab CE pipelines with evaluation and adversarial-test gates, Argo CD deployments, Vault secrets and shared observability

LLMOps & Observability

• Run tracing, dashboards and alerts for latency, errors, GPU utilisation, token usage and cost, and publish monthly cost and performance reports per service and per team

Security, Supply Chain & Reliability

• Manage secrets, network policies and tenant isolation; pin dependencies, generate SBOMs, sign images and verify provenance for every platform component
• Set service-level objectives, on-call and disaster recovery with the DevOps/SRE team

Team Leadership & Standards

• Lead the MLOps engineers, publish platform standards and onboarding guides for AI teams and vendors, and decline changes that threaten platform stability or security

Technical Competencies

• LLM Serving: vLLM, SGLang, TensorRT-LLM, NVIDIA Triton Inference Server; NVIDIA Dynamo or llm-d for distributed inference; quantisation (AWQ, GPTQ, FP8, NVFP4); continuous batching, prefix caching, speculative decoding (such as EAGLE-3); tensor and pipeline parallelism; LoRA adapter serving
• Gateways & Policy: LiteLLM, Kong AI Gateway or Envoy AI Gateway; OpenAI-compatible APIs; Microsoft Presidio; semantic caching
• Kubernetes & GPUs: Kubernetes, Helm, NVIDIA GPU Operator and DCGM; KServe or Ray Serve; Kueue or Volcano; JupyterHub
• Model Lifecycle: MLflow or Weights & Biases; DVC; Hugging Face PEFT and TRL; lm-evaluation-harness, RAGAS, promptfoo
• Developer Platform & CI/CD: Backstage; GitLab CE CI; Argo CD; Terraform or OpenTofu, Ansible
• Observability: Prometheus, Grafana, Loki, OpenTelemetry; Langfuse or Arize Phoenix
• Security & Supply Chain: HashiCorp Vault or OpenBao; Trivy or Grype, Syft, Cosign, SLSA provenance; Kyverno; garak
• Cloud & Providers: AWS Bedrock, Azure AI Foundry or Google Vertex AI; Sarvam and Bhashini APIs
• Programming: Python, and Go or Bash; YAML