Software Engineer, Tinker Infrastructure
Posted Updated
Description
You will help build and scale the distributed systems that power Tinker, a post-training platform. You'll design and operate infrastructure for scheduling, checkpointing, recovery and observability across large GPU clusters, own the reliability of live production systems, lead incident response, and partner with research and ML infrastructure teams. You'll have significant ownership over architectural decisions and mentor other engineers.
Responsibilities
- Design, build, and operate distributed systems underlying Tinker, including job scheduling, resource orchestration, checkpointing, and fault recovery
- Own reliability, latency, and throughput of live production systems
- Build for graceful degradation and fast recovery from node failures, preemptions, and network partitions
- Improve observability with metrics, tracing, and alerting
- Lead incident response, drive root-cause analysis, and implement lasting fixes
- Partner with research and ML infrastructure teams to evolve platform architecture
- Make build-vs-buy and architectural tradeoff calls and mentor other engineers
Requirements
- 7+ years building, running, and scaling distributed systems in production with on-call ownership
- Deep understanding of distributed systems fundamentals: consensus, consistency models, failure detection, scheduling, state management
- Track record of designing reliable systems at scale and resolving production incidents
- Strong systems-level coding in Python, Go, Rust, or C++
Benefits
- Health, dental, and vision benefits
- Unlimited PTO
- Paid parental leave
- Relocation support
- Visa sponsorship
Skills
Sign in to read and join the discussion.