Sr. Software Development Engineer - AI/ML Networking Disaggregated Inference, Annapurna Labs, Annapu
Posted 5
views
Amazon.com Services LLC Sr. Software Development Engineer - AI/ML Networking Disaggregated Inference, Annapurna Labs, Annapu
Summary
Engineer on Amazon's Annapurna Labs team building the networking layer for disaggregated LLM inference — splitting model serving into separate prefill and decode stages so data reaches AI accelerators fast enough to serve large models efficiently. Core focus: high-performance AI/ML networking software.
Job Description Every token a large language model generates depends on data reaching the right accelerator at the right moment. As AI models outgrow any single chip, the network between accelerators becomes the bottleneck that decides how fast — and how affordably — the world's largest models can serve real users. That network layer is what our team builds.\n \n \nWe're looking for an engineer to work at the frontier of disaggregated inference: splitting LLM serving into separate prefill and d…