Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

FDE Times

Senior LLM Inference Optimization Engineer

Posted Updated

Description

Nebius is seeking a Senior Machine Learning Engineer to own model and endpoint optimization from artifacts to production deployment on the Applied AI team. You will tackle model internals, quantization, speculative decoding, KV-cache optimization, and multi-node GPU strategies to reduce latency, boost throughput, and lower token成本 while preserving quality.

You will collaborate with kernel and platform engineers to diagnose performance regressions, evaluate engine configurations, and validate

Own optimization projects for specific model families, endpoints, or serving backends. Compare inference engines and configure best-serving setups per workload. Diagnose model-quality and performance regressions during production rollouts. Improve LLM/VLM endpoint latency, throughput, memory efficiency, GPU utilization, and cost per token. Deploy, benchmark, and extend engines (e.g., vLLM, Triton, TensorRT-LLM, Triton Server, CUDA) in production. Develop model-compression workflows including quantization and distillation. Implement speculative decoding, KV-cache optimization, and prefill/decode strategies. Design and tune LLM request routers and scheduling to balance latency and reliability. Scale dense and Mixture-of-Experts inference across multiple GPU nodes with proper parallelism. Strong engineering skills in Python and PyTorch. Hands-on experience deploying or optimizing LLM/VLM or high-throughput transformer inference. Knowledge of modern inference stacks (vLLM, Triton, TensorRT-LLM, SGLang, NVIDIA Dynamo, Ray Serve or KServe). Understanding transformer inference bottlenecks: KV cache, memory bandwidth, batching, and long-context serving. Ability to quantify latency/throughput trade-offs and cost. Experience designing distributed inference system architectures (routing, scheduling, PDD, multi-node inference, or expert parallelism). Strong communication and collaboration across research, kernel, infrastructure, product, and customers. Competitive compensation Career growth and learning Flexibility and ownership Collaborative and innovative culture Impactful AI projects International environment and talented

Skills

What Senior AI Engineering jobs ask for — and how much of it you have →
Apply

See also

AI Engineering jobs by country — openings, pay and top skills →