AI Inference Reliability Engineer
Description
About the role:
You will operate resilient, scalable and observable production AI inference services.
Key responsibilities:
- Deploy inference services using Kubernetes, containers and release automation.
- Implement autoscaling, GPU-aware scheduling, load balancing and traffic management.
- Design high availability, disaster recovery, rollback and automated recovery.
- Monitor GPU utilisation, memory, TTFT, throughput, errors and service health.
- Manage incidents, root-cause analysis and performance degradation.
- Define service-level objectives, error budgets, capacity plans and operational runbooks.
Tech Stack:
Kubernetes, Docker, Helm, Argo CD, Prometheus, Grafana, NVIDIA GPU Operator, Python, Linux, CI/CD, observability.
Expected deliverables:
Reliable inference deployments, monitoring, runbooks and recovery procedures.
Skills
Sign in to read and join the discussion.