Senior LLMOps Engineer
Posted Updated
Description
Senior LLMOps Engineer Social Health Insurance Fund 1,500,000 to 2,000,000 KZT Gross per month Astana / Office The Social Health Insurance Fund is a national organization based in Astana. Build a nationwide AI platform for healthcare on in-house infrastructure equipped with NVIDIA H200 GPUs. The platform is being developed from the ground up, offering an opportunity to influence its architecture. Responsibilities: • Deploy LLMs, embedding models, and reranking models in a closed environment and ensure stable inference. • Optimize inference speed and GPU usage through techniques such as batching, KV caching, and quantization. • Develop infrastructure for RAG and AI agents in collaboration with AI and Data Engineers. • Establish CI/CD, model versioning and rollback, monitoring, access controls, and auditing. Requirements: • Higher education in IT and at least 3 years of experience, including production operation of ML/LLM services. • Experience with Linux, Docker, Kubernetes, and Helm. • Ability to deploy LLMs using vLLM, SGLang, or Triton, with an understanding of inference systems. • Experience with GPU infrastructure, NVIDIA drivers, and CUDA. • Python and Bash skills, CI/CD experience, and familiarity with Prometheus and Grafana.
Skills
Sign in to read and join the discussion.