Senior Applied ML Engineer — Real-Time LLM Router
Posted Updated
Description
Cast AI is hiring for an applied ML Platform Engineer to own the routing model end to end. You will define features, objectives, training, calibration, and release cadence, ensuring production-parity benchmarks and robust evaluation with error bars.
You will transform per-request outcomes and benchmark rollouts into training signals, balancing task success with token cost and decoding speed. This is a remote, globally distributed role with strong emphasis on instrumentation, experimentation, and
Own the routing model end to end: features, objective, training, calibration, release cadence. Build and maintain production-parity evaluation with error bars andholdouts. Turn fleet data into training signals while managing data quality. Model the objective from user perspective: task success, token cost, and speed. Run shadow, A/B, and full rollout with drift monitoring and retraining. You have 5+ years in applied ML or ML research with real-time decision experience. Strong Python and SQL at scale; able to interrogate millions of records. Understand model quality, cost, latency trade-offs and how to measure them. Experience with LLM routing, benchmarks, and online experimentation is a plus. Ability to instrument systems and reason about conflicting signals. Remote-first global environment Equity options Learning budget Equipment budgetSkills
Sign in to read and join the discussion.