Senior ML Engineer - LLM Inference & Optimization
Posted Updated
Description
Nebius Group, a leading AI cloud platform, invites a Senior Machine Learning Engineer to own model and endpoint optimization from artifacts through production deployment. You will work on quantization, speculative decoding, KV-cache optimization, and multi-node GPU execution to boost latency, throughput, and memory efficiency.
You will collaborate with kernel and platform engineers to diagnose bottlenecks, evaluate distributed inference designs, and validate improvements under real-world
Own optimization projects for specific model families and endpoints. Compare inference engines and recommend configurations for workloads. Diagnose model-quality and performance regressions in production. Improve LLM/VLM endpoint latency, throughput, and cost per token. Deploy and benchmark engines such as vLLM, TensorRT-LLM, and Triton. Develop workflows for model compression and quantization workflows. Strong engineering skills in Python and PyTorch. Hands-on experience deploying or optimizing LLM/VLM or high-throughput transformer inference. Experience with modern inference stacks such as vLLM, TensorRT-LLM, Triton Inference Server, or KServe. Understanding of transformer bottlenecks like KV cache, attention, memory bandwidth and batching. Competitive compensation Career growth and learning Flexibility and ownership Collaborative culture Impactful AI projects International environmentSkills
Sign in to read and join the discussion.