Backend Engineer, AI Insights
Summary
Backend engineer at an early-stage stealth AI startup building systems that watch LLM traffic in production and turn it into actionable customer insights. You own the full path from data capture to analysis, shipping a Python/Go product packaged with containers and Helm that runs on Kubernetes inside customer cloud or on-prem environments.
About the Company
We are an early-stage, still-in-stealth team building an AI platform that gives organizations control of their AI through specialized models that are faster, more cost-efficient, and more secure than one-size-fits-all alternatives, and that customers can own and run on their own terms.
If you want to build backend systems that measure LLMs in production and run inside customer environments, we would love to talk.
Role Overview
As a Backend Engineer, you will build the product that watches how LLMs run in production and turns that traffic into insights customers can act on. You own the whole path: data capture, processing, analysis and the final finding.
The product runs on Kubernetes inside customer environments, in the cloud or on-premises, with privacy built in.
Core Responsibilities
- Insights: Own the path from captured model traffic to findings customers can trust, proven with benchmarks and tests.
- Product: Build the backend, configuration and licensing that let one product serve different customers, with privacy by design.
- Delivery: Collect data from, package, release and upgrade the product in customer-run and offline environments.
Requirements
- 5+ years of backend software engineering, building and running production services in Python or Go.
- Experience building pipelines that process high-volume event or telemetry data, streaming or batch.
- Experience shipping software that customers run in their own environments, packaged with containers and Helm.
- Working knowledge of Kubernetes and one major cloud, AWS or GCP. You can deploy a service there and read its state.
- Rigor about correctness: automated tests, ground truth, and proof that a metric measures what you claim.
- Clear communication, and ownership of your work in a fast-paced startup.
Nice to Have
- Strong production experience with cloud infrastructure and Kubernetes, including Helm, CI/CD, troubleshooting, infrastructure as code, and GitOps.
- Experience analyzing large datasets in analytical or time-series stores and turning results into reliable metrics.
- Familiarity with LLM serving concepts and performance measurement, including vLLM, TTFT, ITL, KV cache, batching, and benchmarking.
- Experience with observability tools such as Prometheus, Grafana, or OpenTelemetry, and deploying or upgrading software in offline or air-gapped environments.
- Familiarity with privacy and security in data products, or full-stack experience building interfaces for data insights.
Skills
Sign in to read and join the discussion.