Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Cast AI

Senior ML Inference Systems Engineer | Remote & Equity

Posted Updated

Description

Cast AI is seeking a senior ML inference infrastructure engineer to optimize throughput, latency, and cost across our GPU fleet. You will tune loaders, schedulers, and KV caches, ensuring scalable, multi-node inference while maintaining model quality.

You will own the technical direction, work with vLLM/SGLang/TensorRT-LLM stacks, and collaborate across a global, remote-first team. A strong production background is essential.

Push throughput: continuous batching, prefill, drafting models, CUDA-graph capture, scheduler tuning across the engine fleet. Cut latency: profile to find bottlenecks and fix compute, memory, scheduling, or storage issues. Get more out of the KV cache: hierarchical/paged attention, prefix caching, eviction policies, quantized KV. Kill cold starts: weight loaders, JIT and compile caches, shared-filesystem paths. Quantize without regressing quality: INT8/INT4, FP8/FP4 across weights/activations/KV. Scale across nodes: distributed inference, network-aware placement, checkpointing strategies. You've built real ML systems for 5+ years, with depth in inference or training infrastructure. Your Python is production-grade: services, not scripts. You've run vLLM, SGLang, or TensorRT-LLM for real and understand performance. You've measured quantization regressions, not just compression ratios. Multi-GPU and multi-node setups are familiar: tensor/expert parallelism and communication. You instrument before you optimize and can identify real wins from benchmarks. Equity options Remote-first Learning budget Annual hackathon Team events Equipment budget Extra days off

Skills

What Senior Software Engineering jobs ask for — and how much of it you have →
Apply

See also

Software Engineering jobs by country — openings, pay and top skills →