LLM Serving Ecosystem Engineer NPU Software Stack

You will design and operate CI systems that automatically validate component-version combinations and establish release criteria. You will create efficient tests for serving functionality and output accuracy, manage dependencies and container-image builds, reduce CI duration, and address flaky tests. You will integrate NPU software with vLLM, llm-d, and Kubernetes-based AI platforms, including upstream contributions. You will measure distributed-serving performance, investigate regressions, and build reliable multi-node NPU cluster CI environments and diagnostic systems.

Responsibilities

• Design and operate CI systems to validate component-version combinations and establish release criteria
• Design efficient tests for serving functionality and output accuracy
• Manage dependencies and container-image builds, reduce CI duration, and manage flaky tests
• Integrate NPU software with vLLM, llm-d, and Kubernetes-based AI platforms
• Contribute upstream to open-source serving ecosystems
• Measure distributed-serving performance and analyze regression causes
• Build and operate stable CI environments based on multi-node NPU clusters
• Build diagnostic systems to isolate regressions by component, version, hardware, and configuration

Requirements

• Proficiency with Linux, Python, and shell scripting
• Experience building, deploying, and debugging in container and Kubernetes environments
• Experience building or operating CI/CD pipelines
• Experience isolating reproducible root causes through controlled variables
• Deep understanding of either LLM inference serving or cluster infrastructure, including networking, storage, or virtualization
• Ability to use AI coding agents such as Claude Code and Codex in practical work and validate their outputs
• Experience operating LLM serving frameworks such as vLLM or llm-d, or contributing upstream to open source
• Experience configuring and optimizing distributed LLM serving, including parallelization, KV cache, and PD disaggregation
• Experience building and operating accelerator-based test infrastructure using GPUs or NPUs
• Experience with RDMA networking or distributed storage
• Experience designing reproducible and comparable performance benchmarks
• Experience optimizing CI, improving release procedures, or operating self-hosted runners