Member of Technical Staff
Summary
On-site San Francisco role on a small AI infrastructure team that builds efficient inference serving for open-source LLMs: productionizing and tuning models, optimizing serving (batching, KV caching, speculative decoding, quantization), writing CUDA/HIP/Triton GPU kernels, and operating heterogeneous GPU clusters.
About the Role
Join a small AI infrastructure team building efficient inference systems for open-source language models. You will improve performance across diverse hardware, helping make production AI serving faster and more cost-effective.
What You'll Do
Bring new open-source models into production with tuning for latency and throughput.
Optimize inference serving, including batching, KV caching, speculative decoding, and quantization.
Write and tune GPU kernels using CUDA, HIP, and Triton across varied accelerator platforms.
Design, deploy, and operate heterogeneous compute clusters across infrastructure vendors.
Run production inference systems, improving reliability, observability, and cost efficiency.
Work with customers to understand needs and help solve infrastructure challenges.
What We're Looking For
Typically 1 to 6 years of experience, with relevant early-career candidates also considered.
Strong systems or infrastructure engineering experience, in addition to any application or full-stack work.
Understanding of computer architecture, memory hierarchy, and operating system internals.
Experience with Python and familiarity with CUDA, HIP, Triton, or other GPU programming tools.
Knowledge of inference optimization techniques such as quantization, batching, KV caching, or speculative decoding.
Experience deploying or operating infrastructure at scale, such as clusters or large GPU fleets, is valuable.
Evidence of taking on challenging systems problems, learning quickly, and using AI coding tools to improve workflows.
Comfort communicating with customers and working across a broad range of technical responsibilities.
Compensation & Benefits
Base salary is $200,000 to $300,000 USD annually. Visa sponsorship is available.
Location
On-site in San Francisco, California, five days per week.
Skills
As published by ashby · 2 questions
Basics
Name, Email, Resume
Short answers (1)
- LinkedIn Profile URL optional
Pick from a list (1)
- Do you have work authorization for the country this role is based in? optional
