Staff Applied AI Inference Engineer

• You will spend your time making large language models run faster, cheaper, and more reliably in production
• That means owning the inference stack end to end: profiling where time and cost go, bringing modern optimization techniques into real deployments, and getting deep into the serving code when the defaults are not good enough
• This is core systems and performance work on some of the most demanding models in use today
• The work is applied, not academic
• The optimizations you build land in real customer deployments, each with its own models, traffic patterns, latency targets, and cost constraints
• So while performance is the heart of the role, you will also work directly with customer engineering teams to tailor deployments to their needs, take a workload from an early proof of concept to a fully monitored production service, and make sure the gains you engineer actually show up for the people running the workload
• To set expectations clearly, this is a hands-on engineering role built around coding, profiling, and low-level optimization
• It also carries a customer-facing side, along with elements of product and technical solutions work, because that is where the performance work gets proven
• Bring current inference techniques into production and refine them
• Design and optimize serving architectures, including prefill and decode disaggregation, request routing, and related approaches
• Work down into the serving stack, from frameworks like vLLM and SGLang to the CUDA kernels underneath, profiling and running in-depth analysis to find and fix performance problems
• Adapt and scale optimization methods across many kinds of ML models, with an emphasis on large language models
• Profile and tune deployments against clear targets for latency, throughput, and cost, and keep them dependable under real traffic
• Tailor deployments to each customer’s models and constraints, partnering with their engineering teams to move a workload from an early proof of concept through to a live, well-monitored production service
• Build and support the software and product features around the inference stack in a production setting, using one or more general-purpose languages, with Python preferred given how central it is to ML work
• Experiment quickly: take fuzzy goals, shape them into clear specs and focused proofs of concept, run fast experiments to find what works, and ship well-tested results without delay
• Own delivery end to end, from the first experiment through to the optimization running in production, keeping the underlying performance goals, clear specs, and follow-through front of mind, and drafting features and product requirement documents together with other engineering and product teams
• Work through ambiguity and make sound calls on tradeoffs and tooling, steering away from complexity that is not needed
• Take real pride and ownership in your work, hold yourself accountable, and look for the same from the people around you

Benefits

• Health & wellbeing: Comprehensive health benefits designed to support your overall wellness
• Time away: Paid time off for vacations, family bonding, and unexpected needs
• 401(k) match: Build your financial future with our 401(k) matching program
• Mental wellness: Resources and support for your emotional wellbeing and navigating life’s challenges

• Hands-on experience shipping code in production with one or more general-purpose languages, such as Python or C++, with a strong preference for Python
• Familiarity with methods for optimizing LLMs for high throughput / low latency inference
• Comfort with modern LLM serving frameworks such as vLLM or SGLang, and with profiling and analyzing performance down to the kernel level
• A firm grasp of how GPUs are built and how they behave
• Strong communication skills, particularly when explaining hard technical topics to customers and teammates
• A working knowledge of AI/ML pipelines and the full path of developing and deploying ML models
• Clear interest and hands-on experience with large language models
• A Bachelor’s, Master’s, or Ph.D. in Computer Science, Engineering, Mathematics, or a related field
• A track record of making software systems run faster, especially for large language models
• Experience with CUDA or comparable technologies
• A strong command of software engineering fundamentals, with a record of building and shipping AI/ML inference systems
• Experience with Docker and Kubernetes
• Prior work building or tuning AI/ML projects, particularly in a customer-facing setting