Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

FDE Times

Senior ML Engineer – LLM Inference Optimization (Remote)

Posted Updated

Description

Nebius is seeking a Senior Machine Learning Engineer on the Applied AI team to own model and endpoint optimization from artifacts to production deployment. You will tackle model internals, quantization, speculative decoding, KV-cache optimization, and benchmarking to improve latency, throughput, memory efficiency, GPU utilization, and cost per token while preserving quality and reliability.

You will collaborate with kernel and platform engineers to diagnose bottlenecks, evaluate engine-level

Own optimization projects for specific model families, endpoints, or serving backends. Compare inference engines and configure serving settings per workload. Diagnose model-quality and performance regressions during production rollouts. Improve LLM/VLM endpoint latency, throughput, memory use, and GPU utilization. Deploy and benchmark inference engines like vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo. Develop production model-compression workflows (quantization, QAT, distillation, low-bit serving, accuracy recovery). Implement speculative decoding, KV-cache optimization, prefix caching, prefill strategies, and disaggregated prefill/decode serving. Design and improve LLM request routers and scheduling to balance utilization while meeting latency targets. Scale dense and mixture-of-experts inference across multiple GPU nodes; tune parallelism and topology. Build reproducible benchmarks measuring TTFT, TPOT, tokens per second per GPU, p95/p99 latency, memory and cost. Collaborate with GPU kernel and platform engineers to trace bottlenecks across model code and cluster layers. Produce design documents, performance reports, rollout plans, and customer-facing explanations. Proficient in Python and PyTorch. Experience deploying or optimizing LLM/VLM inference. Knowledge of modern inference stacks (vLLM, SGLang, TensorRT-LLM, Triton, NVIDIA Dynamo, Ray Serve or KServe). Understand transformer bottlenecks like KV cache, attention, memory bandwidth, batching, and long-context serving. Ability to balance latency, throughput, quality, utilization and cost. Experience with distributed inference architecture (request routing, distributed scheduling, PDD, multi-node inference, expert parallelism). Strong communication and collaboration across teams. Health insurance 401(k) plan Parental leave Remote work reimbursement Disability & life insurance

Skills

What Senior ML / AI jobs ask for — and how much of it you have →
Apply

See also

ML / AI jobs by country — openings, pay and top skills →