Staff SRE: AI Gateway & Platform Reliability
Posted Updated
Description
Kraken is building the AI Gateway as part of its Dev Platform. You will own the reliability, availability and performance of a production-critical service used across the company, with clear SLOs, error budgets and on-call practices.
This is a hands-on leadership role for platform engineers who want to shape scalable, secure access to cutting-edge LLM models. You will architect resilience with multi-provider routing, failover and rate limiting, build end-to-end observability for latency and
Own the reliability, availability and performance of the AI Gateway as a production-critical service used across the company, including SLOs, error budgets and on‑call practices. Design the gateway's resilience: multi‑provider routing, failover, retries, rate limiting and graceful degradation when an upstream model provider is slow or unavailable Set the technical direction for gateway architecture, capacity planning and scaling as LLM traffic grows and more workloads move onto the gateway Build observability for LLM traffic, covering latency, errors, token usage and cost, attributed by team and use case Partner with Security, Finance and AI governance stakeholders to put access controls, budgets, audit logging and data‑handling requirements into the platform Lead incident response and blameless post‑incident reviews, and turn what you learn into lasting reliability improvements Automate operations through infrastructure as code, safe deployments and progressive rollouts Mentor engineers and raise the operating standard across Dev Platform Extensive site reliability or platform engineering experience with high-traffic services. Hands-on Kubernetes, major cloud provider and infrastructure as code. Strong SLOs, observability, and incident management. Proficiency in Rust or Python, with production code experience. Experience with API gateways, reverse proxies or traffic management. Experience with Consul and Vault for service discovery and secrets. Knowledge of authentication like OAuth 2.0 / OIDC, JWT, mTLS. Experience with model evaluations and agent harnesses. Familiarity with agent memory systems and vector stores. Staff-level influence and design documents. Balancing reliability, cost and velocity for internal customers.Skills
Sign in to read and join the discussion.