Senior LLM Inference & Optimization Engineer
Posted Updated
Description
Nebius is seeking a Senior Machine Learning Engineer for the Applied AI team to optimize model artifacts and production deployment, improving latency, throughput, memory use, GPU utilization, and cost per token while maintaining model quality.
You will collaborate with kernel and platform engineers to diagnose serving problems, compare configurations, and validate improvements through reproducible benchmarks and safe, real-world rollouts.
Own optimization projects for model families and serving backends. Compare inference engines and recommend configurations per workload. Diagnose model-quality and performance regressions in production. Improve endpoint latency, throughput, memory use, and cost per token. Deploy, configure, benchmark, and extend inference engines (e.g., Triton, vLLM, SGLang). Develop quantization and compression workflows (quantization-aware training, distillation). Collaborate with kernel and platform engineers to trace bottlenecks across stack. Must have strong Python and PyTorch engineering skills. Hands-on experience deploying or optimizing LLM/VLM or high-throughput transformer inference. Knowledge of modern inference stacks (vLLM, Triton, TensorRT-LLM, etc.). Ability to quantify trade-offs among latency, throughput, quality and cost. Experience designing or optimizing distributed inference system architectures. Excellent communication and collaboration across teams. Health insurance 401(k) plan Parental leave Remote work reimbursement Disability & life insuranceSkills
Sign in to read and join the discussion.