Senior Software Engineer, Inference
Summary
Build and maintain the core infrastructure that powers Claude, focusing on distributed systems, fleet orchestration, and AI hardware integration for efficient LLM inference at scale.
Senior Software Engineer, Inference at Anthropic.
About the role
You will build and maintain the core infrastructure that powers Claude for millions of global users. This role involves managing the full stack from request routing to fleet orchestration across various AI hardware. You will balance the need for compute efficiency with the requirements of researchers developing future models.
Key facts
Location: Dublin, IE
Engagement: Full-time, hybrid (minimum 25% in-office)
Annual Salary: 235,000 - 295,000 EUR
Team: Inference
What you'll do
- Create routing algorithms to manage request distribution across thousands of accelerators.
- Scale compute fleets to meet demand for production and research workloads.
- Develop deployment pipelines for releasing new models.
- Integrate new AI accelerator platforms.
- Build inference features such as prompt caching and structured sampling.
- Support inference for new model architectures.
- Use observability data to optimize performance for production traffic.
- Manage multi-region deployments and global geographic routing.
Requirements
- Significant software engineering experience, specifically with distributed systems.
- Bachelor degree or equivalent combination of education and professional experience in a relevant field.
- Ability to work in a results-oriented environment with a focus on impact.
- Willingness to take on tasks outside of primary responsibilities.
- Interest in machine learning infrastructure and systems.
- Commitment to the societal impact of AI development.
Nice to have
- Experience with high-performance, large-scale distributed systems.
- Background in deploying machine learning systems at scale.
- Familiarity with load balancing, traffic management, or request routing.
- Knowledge of LLM inference optimization, caching, and batching.
- Proficiency with Kubernetes and cloud environments like AWS or GCP.
- Experience with Python or Rust.
Skills & tools
- Distributed systems architecture
- Kubernetes
- AWS and GCP
- Python
- Rust
- Machine learning inference optimization
- Traffic management and load balancing
Practical notes
- Visa sponsorship is available and the company retains immigration counsel to assist, though sponsorship is not guaranteed for every candidate.
- Applications are reviewed on a rolling basis with no fixed deadline.
All recruitment communication will come from @anthropic.com email addresses.