Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Cast AI

Senior ML Inference Engineer - Remote, Equity

Posted Updated

Description

Cast AI, a Miami-headquartered, globally distributed team, seeks an experienced ML systems engineer to own inference stack optimization. You will improve throughput, latency, and cost per token across multi-GPU clusters, shaping the technical direction with data-driven experiments.

Your work spans production-grade Python services, tuning SGLang, vLLM, TensorRT-LLM, and CUDA tooling, with a focus on scalable, cost-efficient inference on customer workloads.

Push throughput with continuous batching, prefill, and scheduler tuning across the engine fleet. Cut latency by profiling bottlenecks and optimizing compute, memory, storage, and networking. 5+ years building real ML systems for inference or training infrastructure. Production-grade Python services, not scripts. Hands-on with vLLM, SGLang, or TensorRT-LLM. Experience with quantization and KV caches. Multi-GPU and multi-node setups with tensor/expert parallelism. Know where data resides (page caches, O_DIRECT, network FS). Ability to instrument and optimize with measurable gains. Equity options Remote-first global environment Learning budget for professional and个人 Annual hackathon and team events

Skills

Apply

Other locations (3)

See also