Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

JD.COM

NewBe an early applicant

Agent Infra Engineer

Posted 10 views

Description

Position Overview:

We are building a production-grade Agent Infra platform to support enterprise-grade AI Agents as they transition from demos to scalable, observable, governable, and sustainably iterative real-world business systems. This role encompasses the end-to-end infrastructure development spanning model invocation, RAG, context engineering, Agent orchestration, tool execution sandboxes, memory systems, evaluation frameworks, and observability/cost governance. You won't just be building applications on top of LangChain or LangGraph; you will participate in architecting a comprehensive Agent Runtime and AI Infra platform that empowers business teams to develop and run production-grade Agents safely, stably, and efficiently.


Responsibilities

Architecture & Engineering: Lead the architectural design and engineering implementation of the Agent Infra platform, covering core modules such as the model gateway, Agent Runtime, workflow orchestration, tool execution, sandboxing, state memory, evaluation, and observability.

Unified Model Invocation Layer: Build a unified model invocation layer including OpenAI-compatible APIs, model routing, fallbacks, rate limiting and quotas, token metering, caching, model version management, and cost attribution.

RAG & Knowledge Layer: Design and optimize the RAG data and knowledge layer, covering document parsing, chunking, embedding, vector search, re-ranking, access control filtering, knowledge base updating, recall evaluation, and retrieval pipeline tuning.

Task Decomposition & Orchestration: Design complex task decomposition, state machine transitions, multi-agent collaboration, checkpoint resumption, asynchronous task scheduling, and long-running task recovery mechanisms based on open-source or proprietary frameworks.

State & Memory Systems: Design Agent state and memory systems supporting short-term session state, long-term user memory, task memory, permission isolation, memory writing strategies, memory retrieval, and expiration governance.

Evaluation & Quality System: Build the Agent evaluation and quality framework, including offline evaluation datasets, Agent-as-Judge, task success rate, hallucination rate, format compliance rate, tool invocation accuracy, regression testing, release gating, and manual sampling workflows.

AI Observability & Operations: Establish the AI observability and operations framework by integrating traces, logs, metrics, costs, call chains, prompt versions, model versions, and tool execution records to support issue troubleshooting, quality analysis, and cost optimization.

Cross-functional Collaboration: Collaborate closely with algorithms, platform engineering, business R&D, product, and security teams to integrate Agent capabilities into real-world business scenarios while continuously enhancing system stability, response latency, maintainability, and developer experience.


Requirements

Education & Experience: Bachelor’s degree or above in Computer Science or a related field, with solid experience in software engineering, distributed systems, backend services, or infrastructure R&D.

Programming Skills: Proficient in at least one backend development language among Python, Go, Java, and TypeScript, with strong capabilities in engineering abstraction, service governance, and troubleshooting.

LLM Development: Familiar with the LLM application development lifecycle and at least one Agent or workflow framework.

RAG Technologies: Familiar with RAG-related technologies, including document parsing, splitting strategies, vector databases, hybrid search, re-ranking, recall evaluation, access control filtering, and incremental knowledge base updates.

Observability: Familiar with observability systems and trace/metrics/logs platforms such as OpenTelemetry, Prometheus, Grafana, Jaeger, LangSmith, Langfuse, Helicone, or equivalent systems.

Security & Governance: Possess a strong security and governance mindset, understanding engineering requirements for key management, tenant isolation, access control, data masking, audit logging, prompt injection defense, and tool invocation boundary control.

Passion for Infra: Highly enthusiastic about the AI Agent infrastructure domain, with the ability to decompose problems under uncertainty, rapidly validate ideas, and solidify platform capabilities rather than just building isolated demos.


Preferred Qualifications

Production Experience: Hands-on experience in deploying production-grade AI Agents, intelligent customer service bots, code assistants, data analysis agents, enterprise knowledge bases, AI copilots, or automated workflow platforms.

Platform Building Experience: Experience building model gateways, LLM gateways, inference services, RAG platforms, prompt management platforms, Agent Runtimes, or AI observability platforms.

Ecosystem Familiarity: Familiar with MCP, E2B, Modal, Playwright, Puppeteer, RPA, code sandboxing, browser automation, or enterprise SaaS/internal system integrations.

Data & Retrieval Systems: Familiar with data and retrieval systems such as Milvus, Qdrant, Weaviate, pgvector, Elasticsearch, OpenSearch, Redis, and PostgreSQL.

Compute & Scheduling: Familiar with GPU scheduling, Ray, Kueue, Volcano, Slurm, Serverless GPU, inference elastic scaling, or model deployment optimization.

AI Evaluation: Experience building AI evaluation frameworks, including evaluation datasets, golden sets, LLM-as-Judge, A/B testing, canary releases, quality regression, and online feedback closed-loops.

FinOps & Cost Governance: Experience in FinOps or cost governance, capable of token, GPU, embedding, re-ranking, vector database, and log cost attribution broken down by model, application, user, and task dimensions.

Community Contributions: Open-source project contributions, technical blogging, or prior platform architecture design experience related to Agent Infra / AI Infra will be given priority.

Fluent in both Chinese and English.

Skills

Apply

See also