AI Infrastructure Systems Engineer
Posted Updated
Description
- As an AI Infrastructure Engineer at Together, you are responsible for keeping all user-facing services and production systems running smoothly
- Participate in on-call rotation (Pagerduty) to respond to production incidents
- Build and run our infrastructure with Ansible, Terraform, and Kubernetes to enable scaling to a massive number of concurrent users
- Build monitoring systems to ensure the highest quality service for our customers
- Design and implement operational processes (such as deployments and upgrades)
- Debug production issues across all services and levels of the stack
- Identify improvements for the product architecture from the reliability, performance and availability perspectives
- Plan the growth of Together AI’s infrastructure
Benefits
- Competitive health insurance plans
- Income protection & retirement
- Mental health support and services
- Dental and vision insurance
- Pre-tax flexible spending accounts
- AD&D insurance
- 401(k) plan
- Life insurance
- STD & LTD insurance
- Monthly commuting stipend + pre-tax bene
- Flexible time off policy
- Monthly team lunches
- Team-driven celebrations and events
- Strong software engineering skills in Python, Go, or Rust
- Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies
- 3+ years building distributed systems, infrastructure platforms, or large-scale backend software
- Experience building platforms, automation systems, or developer infrastructure
- Strong systems thinking with the ability to understand problems across hardware and software
- A passion for solving complex infrastructure challenges through software
- An automation-first mindset—if a task is repeated, your instinct is to build a system to eliminate it
- GPU infrastructure, CUDA, NCCL, NVLink/NVSwitch
- InfiniBand or RoCE networking
- Bare-metal provisioning and lifecycle management
- Large-scale AI training or inference clusters
- Hardware health monitoring and predictive failure detection
- Distributed storage systems
- AI agents and autonomous infrastructure operations
Skills
Sign in to read and join the discussion.