Senior Machine Learning Engineer Inference

You will optimize and serve machine learning models for production inference. You will profile and accelerate GPU workloads, improve latency and throughput, containerize models, scale multi-GPU and multi-node serving, and ensure reliable operation in production.

Responsibilities

• Optimize model-serving and inference performance
• Apply quantization, distillation, caching, continuous batching, paged attention, and speculative decoding
• Profile and accelerate workloads on NVIDIA GPUs
• Build and scale multi-GPU and multi-node inference systems
• Containerize models and optimize their production serving
• Ensure reliable production operation for high-concurrency inference

Requirements

• Deep understanding of modern serving frameworks and inference optimization techniques such as vLLM or TRT-LLM
• Hands-on experience with quantization, distillation, caching strategies, continuous batching, paged attention, and speculative decoding
• Proficiency in C++, CUDA, Rust, or highly optimized Python
• Ability to profile code and optimize NVIDIA GPU performance
• Experience with Kubernetes, Ray, load balancing, and distributed inference systems
• Experience handling thousands of concurrent connections
• Evidence of non-trivial systems programming projects, open-source contributions, or technical write-ups
• Ability to own the lifecycle from research model through production serving
• PhD in Computer Science, Physics, or Mathematics, or equivalent practical experience building backend or ML systems

Benefits

• Relocation assistance
• Bonus
• Equity
• Benefits