AI Infrastructure Systems Engineer
Posted Updated
Description
- At Together AI, you’ll build and operate one of the world’s largest GPU fleets used for frontier model training and inference. This isn’t a traditional infrastructure role—we’re looking for engineers who love building systems, automating everything, and solving problems at massive scale
- Design and build fleet automation systems that provision, validate, deploy, upgrade, repair, and retire GPU clusters with minimal human intervention
- Build AI Infrastructure Agents that automate deployment, root-cause failures, incident triage, and autonomous remediation
- Develop Fleet Intelligence platforms that continuously monitor hardware health, firmware, networking, storage, thermals, and workload performance to predict failures before they impact customers
- Build software that maximizes GPU availability, utilization, performance, and reliability across thousands of accelerators
- Create automated validation systems for GPUs, InfiniBand/RoCE fabrics, NVLink/NVSwitch, storage, and distributed AI workloads
- Build internal platforms and developer tools that allow infrastructure to be managed through software—not manual operations
- Continuously improve deployment velocity, reliability, and operational efficiency through automation
- Partner closely with hardware, networking, platform, and AI teams to push the limits of AI infrastructure
Benefits
- Competitive health insurance plans
- Income protection & retirement
- Mental health support and services
- Dental and vision insurance
- Pre-tax flexible spending accounts
- AD&D insurance
- 401(k) plan
- Life insurance
- STD & LTD insurance
- Monthly commuting stipend + pre-tax bene
- Flexible time off policy
- Monthly team lunches
- Team-driven celebrations and events
- Experience with Linux, Kubernetes, Terraform, Ansible, or similar infrastructure technologies
- 3+ years building distributed systems, infrastructure platforms, or large-scale backend software
- Strong software engineering skills in Python, Go, or Rust
- Experience building platforms, automation systems, or developer infrastructure
- Strong systems thinking with the ability to understand problems across hardware and software
- An automation-first mindset—if a task is repeated, your instinct is to build a system to eliminate it
- A passion for solving complex infrastructure challenges through software
- GPU infrastructure, CUDA, NCCL, NVLink/NVSwitch
- Bare-metal provisioning and lifecycle management
- InfiniBand or RoCE networking
- Large-scale AI training or inference clusters
- Hardware health monitoring and predictive failure detection
- Distributed storage systems
- AI agents and autonomous infrastructure operations
Skills
Sign in to read and join the discussion.