Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Anthropic

Performance Engineer (Inference Systems)

Posted Updated

Description

  • Anthropic’s inference fleet serves Claude to millions of users across our own products and the world’s largest cloud platforms
  • The stack that makes this possible is deep and tightly coupled: accelerator kernels, model servers, distributed routing, autoscaling, capacity management. Every layer affects the others, often in ways that are hard to see in isolation
  • The Inference System Dynamics team is responsible for understanding that whole system and holding it to a high bar across four dimensions: throughput, latency, reliability, and correctness
  • We measure how the fleet performs against its theoretical performance frontier, run cross-layer investigations to explain the gaps, and own the correctness checks that make sure Claude’s outputs are right, not just fast, across hardware platforms and serving configurations
  • We don’t own the individual components. We instrument and model them, find the highest-leverage opportunities across them, and partner with the owning teams to land the wins
  • You’ll work across all four areas. One week that might mean tracing a tail-latency regression from request timing down through routing and batching into a kernel overhead; the next it might mean tightening a correctness eval so it catches an output regression introduced by a quantization change
  • We’re looking for performance engineers who treat correctness as part of performance
  • Run cross-layer performance investigations across throughput, latency, and reliability, sizing the gap between actual fleet performance and theoretical rooflines, identifying root causes, and quantifying the value of closing them
  • Own and improve the correctness evaluation pipeline that validates model output quality across hardware platforms, numerics, and serving configurations, and lead the investigation when it catches a regression
  • Build the observability, dashboards, and modeling tools that make throughput, latency, cost, reliability, correctness, and their interactions legible across the stack
  • Partner with kernel, serving, routing, autoscaling, and capacity teams to prioritize and land the highest-impact optimizations your analysis surfaces
  • Ruthlessly stack-rank a large surface area of opportunities by impact and effort, and say no to the ones that don’t make the cut
  • Representative Projects:
  • Trace a 350ms latency gap on a new accelerator platform from end-to-end request timing down to a server scheduling overhead, quantify the win, and land the fix directly or with the owning team
  • Redesign the correctness eval gate: determine which signals reliably catch real model-output regressions versus noise, and make it the trusted release criterion across hardware backends
  • Build a FLOPs funnel that breaks down where compute actually goes across the fleet, exposing the gap between achieved throughput and kernel rooflines
  • Root-cause a numerical divergence between two hardware platforms to a specific kernel change, and define the acceptance threshold going forward
  • Model the latency–cost impact of changing batch-sizing and utilization targets, and turn the result into the signal the autoscaler uses in production

Benefits

  • Comprehensive health, dental, and vision insurance for you and your dependents
  • Inclusive fertility benefits via Carrot Fertility
  • 22 weeks of paid parental leave
  • Flexible paid time off and absence policies
  • Mental health support for you and your dependents
  • Competitive salary and equity packages
  • Optional equity donation matching at a 1:1 ratio, up to 25% of your equity grant
  • Retirement plans with competitive matching
  • Life and income protection plans
  • $500/month flexible wellness and time saver stipend
  • Commuter benefits
  • Annual education stipend
  • Home office stipends
  • Relocation support for those moving for Anthropic
  • Daily meals and snacks in the office
  • Genuine interest in correctness as an engineering discipline: numerics, evaluation design, regression detection
  • Hands-on performance engineering experience: profiling, roofline analysis, latency/throughput optimization, and root-cause investigation in complex production systems
  • Ability to communicate quantitative results clearly in writing to influence priorities on teams you don’t manage
  • Proficiency in Python, with the ability to read, instrument, and contribute to large production codebases you didn’t write
  • Solid data analysis skills (e.g. SQL, pandas, or similar) sufficient to turn raw telemetry into clear findings
  • Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience
  • We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed
  • Experience with ML systems, especially training or inference infrastructure or general LLM serving stacks. Direct large-scale inference experience is a strong plus
  • Familiarity with GPU/TPU/accelerator performance concepts (memory bandwidth, kernel overheads, quantization, collective communication). Reasoning about these matters more than having written kernels yourself
  • Experience with reliability engineering for high-throughput services: autoscaling, load balancing, request routing, tail latency
  • Experience with model evaluation or numerical regression-detection pipelines
  • Experience building observability or telemetry for distributed systems
  • Comfortable having impact through influence and evidence rather than direct ownership
  • Deadline to apply: None. Applications will be reviewed on a rolling basis

Skills

Apply

See also