Senior Deployment Engineer (LLM Inference & Optimization)
Posted Updated
Description
- We’re looking for a Deployment Engineer who combines hands-on AI engineering, model optimization, and strong customer-facing skills
- You’ll work directly with customers to bring AI models and applications into production, improve their performance, and integrate them with enterprise systems
- You’ll own technical delivery from discovery and benchmarking through deployment and adoption, working closely with customer stakeholders and our product and engineering teams
- This role requires someone who enjoys writing code, experimenting with models, diagnosing performance bottlenecks, and helping customers achieve measurable improvements in model quality, latency, throughput, and cost
- Understand customer requirements: Lead technical discovery to define use cases, workloads, infrastructure constraints, and targets for model quality, latency, throughput, and cost
- Optimize inference: Apply techniques such as quantization, batching, prefix caching, KV-cache optimization, and parallelism to improve performance while preserving output quality
- Adapt models to customer needs: Improve results through prompt optimization, retrieval-augmented generation, and fine-tuning where appropriate. Evaluate the impact of each approach
- Troubleshoot performance: Investigate issues across model execution, GPU utilization, memory consumption, serving infrastructure, and application behavior
- Lead technical customer engagement: Run workshops, present benchmark findings, explain tradeoffs, and guide customers through deployment and optimization decisions
- Build reusable capabilities: Create evaluation tools, benchmark harnesses, deployment configurations, and runbooks. Bring recurring customer needs and performance findings back to product and engineering
- —-What success looks like—-
- You help customers deploy AI solutions that meet agreed quality and performance targets
- Your work produces measurable improvements in latency, throughput, GPU utilization, and cost while maintaining model quality
- Customers understand the tradeoffs, trust your technical guidance, and can operate the solutions you deliver
- Your findings and tools make future deployments faster and more effective
- Experience with model-serving frameworks such as vLLM, SGLang, TensorRT-LLM, or similar technologies
- Experience optimizing embedding models, rerankers, vision models, or multimodal systems
- Experience improving retrieval quality and evaluating end-to-end RAG applications
- 5+ years of experience in software engineering, ML engineering, AI infrastructure, or a related hands-on technical role
- Contributions to open-source model-serving, optimization, or evaluation projects
- Familiarity with GPU-based cloud environments, containers, and monitoring tools used to operate model deployments
- Experience delivering enterprise AI implementations or customer-facing technical engagements
- Experience building evaluations and benchmarks to measure model quality, latency, throughput, and resource efficiency
- The ability to work directly with customers, communicate technical tradeoffs clearly, and translate business requirements into measurable engineering targets
- Experience developing APIs and integrating model endpoints with production applications and customer data systems
- Strong Python programming skills and experience with frameworks such as PyTorch, Hugging Face Transformers, or comparable tools
- Strong ownership and troubleshooting skills, with the ability to move from an ambiguous problem to a tested solution
- An understanding of GPU memory constraints, numerical precision, batching, caching, and parallelism, including how these affect inference performance
- Hands-on experience deploying and optimizing AI models in production, particularly LLMs
- Experience profiling GPU workloads using CUDA, NVIDIA Nsight, or related tools
- Experience with parameter-efficient fine-tuning techniques such as LoRA or QLoRA
- Experience with speculative decoding, multi-GPU inference, or distributed model serving
Skills
Sign in to read and join the discussion.