Senior Software Engineer (AI Inference & Runtime Platform)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Software Engineer (AI Inference & Runtime Platform) based in the United States.

Join a mission-driven engineering team building advanced AI infrastructure to address complex challenges across critical industries, including energy, sustainability, logistics, and climate technology. In this role, you will own the runtime foundations that power AI workloads, from high-performance model inference to secure execution environments for AI-generated code. You will design and operate scalable systems across Kubernetes clusters, GPU infrastructure, and cloud platforms. Working across Rust, Python, distributed systems, and infrastructure automation, you will help deliver reliable, cost-efficient, and secure AI services. This opportunity is ideal for an engineer who has built and operated production-grade inference or serverless GPU platforms and enjoys solving challenging systems problems. You will also contribute to open-source software and help establish engineering standards as the platform evolves.

You will own the core infrastructure that enables AI models and agent-generated workloads to run securely, efficiently, and reliably in production. Your responsibilities will include:

• Manage the model-serving infrastructure for open-weight large language models, including deployment, configuration, cold-start optimization, model-specific service-level objectives (SLOs), upgrades, and canary releases.

• Operate and improve the Kubernetes infrastructure supporting inference and sandbox workloads, including custom operators, custom resource definitions (CRDs), autoscaling, GPU scheduling and sharing, and node lifecycle management.

• Own the stateful data plane end to end, ensuring reliable operation, documented recovery procedures, and regularly tested restoration processes.

• Develop and maintain the secure agent-sandboxing runtime and host-side control plane, covering lifecycle management, command execution, snapshots, forking, teardown, usage metering, and isolation-boundary security.

• Build and maintain FastAPI control-plane services, infrastructure-as-code configurations using Terraform/OpenTofu and Bicep, and dashboards that support effective operations.

• Provide the infrastructure foundation used by backend engineering teams, collaborating across services and investigating issues that span the platform and application layers.

• Establish and maintain observability, performance, reliability, and operational practices that help teams diagnose problems and make informed decisions.

• Contribute to an open-source-oriented development process through reviewed pull requests, comprehensive documentation, reproducible builds, and releases as components mature.

Requirements:

You are an experienced systems engineer with a strong production background, sound security judgment, and the ability to operate complex infrastructure under real-world conditions. You bring most of the following qualifications:

• At least 5 years of experience delivering production systems using a systems programming language. Strong Rust expertise is valuable, while deep experience in Go, C/C++, or Zig combined with a genuine willingness to work in Rust is also relevant.

• Solid understanding of asynchronous runtimes, memory safety, and low-level debugging, including troubleshooting at the system-call boundary.

• Hands-on experience operating Kubernetes workloads that other teams depend on, including controllers or operators, scheduling, autoscaling, and node lifecycle management.

• Proven ability to threat-model isolation boundaries involving technologies such as namespaces, cgroups, seccomp, and hypervisors, including assessing what an untrusted guest could observe, manipulate, or exhaust.

• Practical knowledge of secure AI-agent execution, including least-privilege access, preventing credentials from entering sandbox environments, maintaining audit trails, and requiring human approval for actions that modify systems of record.

• Experience deploying or operating open-weight LLM inference infrastructure, such as vLLM, SGLang, or comparable serving systems, and packaging models into reliable, metered production endpoints.

• Strong distributed-systems performance skills, including measurement-driven optimization, latency analysis, and the ability to demonstrate improvements with concrete metrics.

• Practical experience with technologies such as Rust and Tokio, Python and FastAPI, Kubernetes operators, controller-runtime, Kubebuilder, CRDs, KEDA, Karpenter, GPU device plugins, or Dynamic Resource Allocation (DRA), along with a willingness to learn unfamiliar components.

• Familiarity with isolation technologies such as Firecracker, Kata Containers, gVisor, or equivalent solutions.

• Experience with secrets management and egress controls, including Vault- or KMS-class systems, as well as stateful technologies such as pgvector, Qdrant, or Neo4j, with strong backup and failover practices.

• Comfort operating across AWS, Azure, GCP, and managed GPU cloud environments, using infrastructure-as-code tools such as Terraform/OpenTofu and Bicep, along with observability tooling such as OpenTelemetry.

• A bachelor's degree is required; a master's degree is a plus.

• Ability to travel twice a year for company summits.

• Current authorization to work full-time in the United States. Employment visa sponsorship or transfer is not available.

• Willingness to complete a written take-home assignment followed by a live, two-hour technical session with the engineering team.

• Applicants may apply to a maximum of two roles at a time. Applying to more than two roles within a six-month period results in automatic disqualification.

Benefits:

• Salary: $140,000–$230,000, with compensation determined by experience, capabilities, and location.

• Equity: Equity participation in an early-stage company.

• Bonus: Eligibility for performance-related bonus compensation.

• Healthcare: Health insurance with meaningful coverage for dependents.

• Flexible time off: Paid time off designed to support flexibility and work-life balance.

• Remote work: Fully remote working culture, with a cluster of teammates based in Seattle.

• Mission-driven work: The opportunity to contribute to AI transformation across critical industries through impactful applications in energy, sustainability, climate, logistics, and related fields.

• Engineering ownership: The opportunity to shape foundational AI infrastructure, work across the stack, and contribute to open-source projects.

• Company summits: Opportunities to connect with teammates in person twice per year.

We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.

We appreciate your interest and wish you the best!

Why Apply Through Jobgether?

Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.

#LI-CL1