Lead DevOps Engineer
Posted Updated
Description
Responsibilities:
- Own production infrastructure across AWS, GCP, and Azure, including private interconnects, transit and hub-and-spoke topologies, and cross-cloud latency management with explicit p95 budgets.
- Design and operate self-hosted LiveKit and SIP infrastructure at scale, including session-affine, event-driven autoscaling for stateful media servers that can handle rapid traffic growth.
- Build and manage model-serving infrastructure for GPU fleets supporting self-hosted ASR and open-weight LLMs, including inference optimization, prefix caching, sticky-session routing, sub-500ms TTFT targets, and managed API integrations across Vertex AI, Bedrock, and Azure.
- Lead reliability and observability initiatives using OTel-native tracing, latency attribution across telephony, ASR, LLM, and TTS, automated incident response, and self-healing systems.
- Act as incident commander for infrastructure incidents and create practical runbooks for production operations.
- Drive cost engineering with cost-per-minute as an SLO through caching, rightsizing, autoscaling, and workload re-architecture.
- Implement Zero Trust security across clouds, including private endpoints, IAM/RBAC, secrets rotation, and WAF/DDoS protections.
- Operate and maintain controls for SOC 2 and ISO/IEC 27001, and support audits involving banking, telecom, and data residency requirements.
- Provide technical leadership through Terraform-first IaC standards, production-readiness reviews, and mentoring SREs.
- Tackle current challenges, including scaling stateful self-hosted media servers beyond current concurrency limits, migrating LLM inference from managed APIs to self-hosted GPU-based open-weight models without breaking TTFT budgets, optimizing cross-cloud topology for ASR, LLM, and telephony, and building multi-region disaster recovery without doubling spend.
Requirements:
- 6+ years of hands-on cloud infrastructure experience.
- 3+ years operating multiple clouds simultaneously in production, with deep expertise in at least two of AWS, GCP, and Azure.
- Production experience with real-time audio or video systems such as WebRTC, SIP/PSTN, or streaming media, including debugging jitter and related issues.
- Strong networking expertise in VPC/VNet design, load balancing, DNS, NAT, private connectivity, transit gateways, and cross-cloud mesh architectures.
- Experience with Kubernetes at scale across EKS, GKE, or AKS, Helm, and scaling stateful workloads.
- Familiarity with service mesh technologies such as Istio or Linkerd.
- Strong Infrastructure as Code experience with Terraform across multi-account or multi-project environments.
- Production AI/ML serving experience, including GPU allocation and scheduling, inference servers or serverless GPU platforms, and streaming protocols such as WebRTC, WebSocket, and gRPC.
- Experience operating production STT, TTS, and LLM APIs, including streaming integrations, quota management, and multi-vendor failover.
- Strong security fundamentals, including IAM/RBAC, secrets management, encryption, and key rotation.
- Experience with CI/CD using GitHub Actions or GitLab CI, with integrated security scanning.
Nice to Have:
- Experience with LiveKit, pipecat, Twilio, or similar real-time platforms.
- Experience with SIP trunking and PSTN integration.
- Hands-on use of KEDA or other event-driven autoscaling in production.
- MLOps experience, including model versioning and canary or A/B rollouts.
- FinOps experience, including reserved or spot strategy and unit economics reporting.
- Certifications such as AWS Solutions Architect Professional, GCP Professional Cloud Architect, or Azure Solutions Architect Expert.
- Exposure to ISO/IEC 42001:2023.
Skills
- AI
- AKS
- API
- AWS
- Azure
- CI/CD
- Cloud
- DNS
- EKS
- Event Driven Architecture
- FinOps
- GCP
- GitHub
- GitHub Actions
- GitLab
- GKE
- GPU
- gRPC
- Helm
- IAM
- Infrastructure as Code
- Istio
- Kubernetes
- LLM
- Machine Learning
- MLOps
- Networking
- Observability
- RBAC
- Secrets Management
- Serverless
- SOC 2
- Terraform
- Vertex AI
- VPC
- WAF
- WebRTC
- Websockets
- Zero Trust
Sign in to read and join the discussion.