MLOPs Engineer

Role Summary

The MLOps Engineer will build and run components of the organisation's AI platform under the MLOps Lead: model-serving deployments on GPUs, AI gateway configuration, CI/CD pipelines with evaluation gates, the model registry, monitoring, and fine-tuning jobs.

Educational Qualification

• B.Tech./B.E., M.Tech./M.S. or M.Sc. in Computer Science, Information Technology, AI/ML or a related discipline (Must have)
• Certifications (desirable, not mandatory): CKA or CKAD, a cloud DevOps certification
• Candidates from non-traditional backgrounds with strong verifiable work will be considered in lieu of formal qualification

Experience

• 3+ years in DevOps, platform or MLOps engineering (5+ years preferred); minimum 1 year deploying and operating ML or LLM models in production
• Demonstrable deployment of at least one ML or LLM model to production by the candidate, with monitoring in place
• Experience with Kubernetes in production, infrastructure as code and CI/CD
• Experience with GPU workloads (drivers, CUDA, GPU Operator) is a strong plus
• Experience serving LLMs with vLLM or SGLang is a plus

Key Responsibilities

Model Deployment & Serving

• Deploy and scale models on Kubernetes with vLLM, SGLang or NVIDIA Triton across development, staging and production, managing versions, canary releases and rollback

CI/CD for AI

• Build GitLab CE pipelines with tests, evaluation gates and adversarial-test gates; package models and services; deploy with Argo CD

Gateway & Registry Operations

• Configure gateway routes, keys, quotas and fallbacks for self-hosted and external providers, and maintain model and prompt registry entries with provenance

Monitoring & Observability

• Maintain dashboards, alerts and runbooks for latency, errors, GPU utilisation, token usage and cost, including LLM tracing

GPU & Training Jobs

• Schedule fine-tuning and batch jobs on shared GPUs, and maintain drivers, base images and environments with the GPU infrastructure engineer

Security, Audit & Support

• Manage secrets and access, pin dependencies and scan images, maintain deployment logs for audit, and take part in on-call and post-incident reviews

Technical Competencies

• Containers & Orchestration: Docker, Kubernetes, Helm; KServe or Ray Serve (desirable)
• Serving: vLLM, SGLang or NVIDIA Triton; ONNX Runtime
• Gateways: LiteLLM, Kong AI Gateway or equivalent
• CI/CD & Infrastructure as Code: GitLab CE CI or GitHub Actions; Argo CD; Terraform or OpenTofu, Ansible
• ML Lifecycle: MLflow, DVC
• Monitoring: Prometheus, Grafana, Loki, OpenTelemetry; Langfuse or Arize Phoenix; NVIDIA DCGM
• GPU: NVIDIA drivers, CUDA, NVIDIA Container Toolkit, GPU Operator
• Security: HashiCorp Vault or OpenBao; Trivy or Grype; Cosign
• Core: Python, Bash, YAML; Linux administration