Research Engineer Infrastructure Inference
Posted Updated
Description
You will design, optimize, and scale infrastructure for large-model inference. You will improve performance, latency, throughput, reliability, and reproducibility; extend distributed inference and serving frameworks; optimize GPU utilization; and document and share technical learnings.
Responsibilities
- Bring AI models into production with researchers and engineers
- Enable high-performance inference for novel architectures
- Design and implement techniques, tools, and architectures that improve inference performance, latency, throughput, and efficiency
- Optimize the codebase and GPU compute fleet
- Extend Kubernetes, Ray, and SLURM orchestration frameworks for distributed inference, evaluation, and large-batch serving
- Establish reliability, observability, and reproducibility standards across the inference stack
- Publish and share learnings through documentation, open-source libraries, or technical reports
Requirements
- Understanding of PyTorch, JAX, and their underlying system architectures
- Experience with inference serving systems optimized for throughput and latency
- Strong engineering, performance, maintainability, and debugging skills
- Ability to work with cross-functional partners and subject matter experts
Benefits
- Health, dental, and vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support
- Visa sponsorship
Skills
Sign in to read and join the discussion.