Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Reflection

NewBe an early applicant

Member of Technical Staff - Pre-Training Evals Infra (Inference)

Posted Updated

Description

You will own inference performance and correctness for pre-training evaluations, bridging the training framework and inference framework. Your work spans model implementations, GPU execution, distributed serving and inference worker lifecycle. You will work with researchers and engineers to implement research efficiently, establish validation, profile and optimize inference, and improve autoscaling and fleet efficiency.

Responsibilities

  • Establish parity between training and inference frameworks by defining numerical acceptance criteria based on activations, logits, and downstream evaluation results
  • Optimize inference by profiling CPU and GPU execution, memory use, communication, and scheduling
  • Validate that changes improve throughput or cost while preserving correctness
  • Improve autoscaling and fleet efficiency by identifying signals for demand and serving capacity
  • Work with researchers and engineers to carry improvements into regular use

Requirements

  • Track record of improving performance or reliability in complex distributed systems and inference infrastructure
  • Understanding of CPU execution, GPU architecture, memory hierarchy, and interconnects
  • Understanding of inference including prefill and decode, batching, attention, KV-cache management, and distributed model execution
  • Measurement-driven approach to optimization
  • Rigor in correctness and validation design
  • Ability to take ownership of ambiguous technical problems and communicate findings clearly

Benefits

  • Stock options
  • Comprehensive medical, dental, vision, and life insurance
  • Annual wellness allowance
  • Lunch and dinner provided in the office daily
  • 22 weeks paid parental leave
  • Unlimited paid time off in the U.S. and 30 days in the U.K.
  • Visa sponsorship support
  • Regular off-sites, happy hours, and team celebrations

Skills

Apply

See also

Software Engineering jobs by country — openings, pay and top skills →