Staff AI Platform SRE: Build Reliable LLM Infra
Posted Updated 5
views
Who recruits here Pro
Up to three recruiters at Manychat with their work email, so you can write to a person.
Part of Pro. How it works
Description
Manychat seeks a seasoned Site Reliability Engineer to own the reliability and performance of our AI infrastructure, including AI Gateway and inference services. You will work with providers like Bedrock and OpenAI, shaping architecture, routing, and guardrails. Build observability, drive cost controls, and lead capacity planning.
You will collaborate closely with the Head of Infrastructure, establishing standards and elevating the entire engineering org with LLM-backed features.
Own reliability and performance of our AI infrastructure: AI Gateway, inference services, and integrations with LLM providers. Design and evolve the AI Gateway: routing, failover between providers, rate limiting, caching, and guardrails. Build observability for AI systems: latency/throughput/error SLOs per model and provider, token-level metrics, quality signals. Drive cost optimization and FinOps for AI workloads: per-feature cost visibility, model right-sizing, caching strategies. Run capacity planning and incident response for inference services; write and improve runbooks and postmortems. Scale AI expertise across the org: set standards, review designs, and coach teams shipping LLM-backed features. 5+ years in SRE / platform / infrastructure engineering, including production ownership at significant scale. Hands-on experience operating LLM-backed systems in production: provider APIs (Bedrock, OpenAI, Anthropic, or similar). Deep cloud-native background: AWS, Kubernetes, Terraform/IaC, CI/CD. Strong observability practice (Prometheus/Grafana, OpenTelemetry, or equivalent) and experience defining SLOs for non-deterministic systems. Proven cost-optimization work: you can show where you cut cloud or inference spend and how you made cost visible. Staff-level influence: you've set technical direction beyond your own team and brought others along Hybrid onboarding Relocation support Health insurance for you and family Professional development budget Flexible benefits budget Hybrid work with flexible time off In-office meals and snacks Sport activities and annual offsites AI tools access from day oneSkills
Sign in to read and join the discussion.