Member of Technical Staff - Pre-Training Evals Infra (Inference)
Posted Updated
Description
You will own inference performance and correctness for pre-training evaluations, bridging the training framework and inference framework. Your work spans model implementations, GPU execution, distributed serving and inference worker lifecycle. You will work with researchers and engineers to implement research efficiently, establish validation, profile and optimize inference, and improve autoscaling and fleet efficiency.
Responsibilities
- Establish parity between training and inference frameworks by defining numerical acceptance criteria based on activations, logits, and downstream evaluation results
- Optimize inference by profiling CPU and GPU execution, memory use, communication, and scheduling
- Validate that changes improve throughput or cost while preserving correctness
- Improve autoscaling and fleet efficiency by identifying signals for demand and serving capacity
- Work with researchers and engineers to carry improvements into regular use
Requirements
- Track record of improving performance or reliability in complex distributed systems and inference infrastructure
- Understanding of CPU execution, GPU architecture, memory hierarchy, and interconnects
- Understanding of inference including prefill and decode, batching, attention, KV-cache management, and distributed model execution
- Measurement-driven approach to optimization
- Rigor in correctness and validation design
- Ability to take ownership of ambiguous technical problems and communicate findings clearly
Benefits
- Stock options
- Comprehensive medical, dental, vision, and life insurance
- Annual wellness allowance
- Lunch and dinner provided in the office daily
- 22 weeks paid parental leave
- Unlimited paid time off in the U.S. and 30 days in the U.K.
- Visa sponsorship support
- Regular off-sites, happy hours, and team celebrations
Skills
Sign in to read and join the discussion.