Senior ML Inference Engineer - Remote, Equity
Posted Updated
Description
Cast AI, a Miami-headquartered, globally distributed team, seeks an experienced ML systems engineer to own inference stack optimization. You will improve throughput, latency, and cost per token across multi-GPU clusters, shaping the technical direction with data-driven experiments.
Your work spans production-grade Python services, tuning SGLang, vLLM, TensorRT-LLM, and CUDA tooling, with a focus on scalable, cost-efficient inference on customer workloads.
Push throughput with continuous batching, prefill, and scheduler tuning across the engine fleet. Cut latency by profiling bottlenecks and optimizing compute, memory, storage, and networking. 5+ years building real ML systems for inference or training infrastructure. Production-grade Python services, not scripts. Hands-on with vLLM, SGLang, or TensorRT-LLM. Experience with quantization and KV caches. Multi-GPU and multi-node setups with tensor/expert parallelism. Know where data resides (page caches, O_DIRECT, network FS). Ability to instrument and optimize with measurable gains. Equity options Remote-first global environment Learning budget for professional and个人 Annual hackathon and team eventsSkills
Sign in to read and join the discussion.