Infra Engineer for AI Inference Platform (K8s, Dynamo)
Posted Updated
Description
Lyceum is seeking a seasoned backend/infrastructure engineer to ensure the AI inference platform remains reliable, secure, and scalable as user demand grows. You will design systems for thousands of concurrent users, implement robust monitoring and auto-recovery, and push for performance improvements across the end-to-end inference path.
You will also evaluate and integrate open-source frameworks like Dynamo, vLLM, and Triton to enhance throughput, latency, and stability, while maintaining
Make Lyceum's AI inference platform reliable, secure, and scalable under heavy load. Architect and implement scalable systems for thousands of concurrent users with proper routing, load balancing, and autoscaling. Build robust monitoring, alerting, and incident response tooling and enable graceful degradation. Profile and optimise the full inference path from request ingestion to response delivery to reduce latency and improve throughput. Evaluate open-source inference frameworks and tooling to improve serving stack performance and stability. 3+ years of backend, infrastructure, or systems engineering experience. Strong proficiency in Go and Python. Experience building or operating a model serving platform or ML platform. Solid understanding of systems performance - profiling, benchmarking, and optimising latency and throughput. Familiarity with observability tooling (Prometheus, Grafana, OpenTelemetry, or similar). Understanding of security fundamentals - network isolation, authentication, encryption, multi-tenant.Skills
Sign in to read and join the discussion.