Staff Machine Learning Engineer
Posted Updated
Description
- The Agentic Engineering organization at ServiceNow is the customer-obsessed engineering group that builds a conversational AI experience that turns enterprise intent into completed work. We advance how enterprise AI reasons, remembers, and executes
- The Agent Orchestration team — the team you’ll join — owns the execution core: the agent harness, orchestration runtime, multi-agent coordination, memory management, and the evaluation frameworks that ensure agents behave correctly in production. Every autonomous action Otto promises depends on what this team ships
- By joining our team, you’ll be at the forefront of our AI transformation journey, backed by the global scale of ServiceNow and the agility of a high-growth environment. We are looking for world-class talent to help us extend agentic AI to every employee across every corner of the business
- As a Staff AI Engineer, you will own significant parts of the agent harness — the infrastructure layer that enables AI agents to reason over real enterprise data, take action across workflows, and run safely at Fortune 500 scale
- Harness engineering: Design and build the agent execution harness — the orchestration layer that routes inputs, manages context, invokes tools, handles retries, and surfaces execution state across multi-step agentic workflows
- Reliability at scale: Own the runtime’s fault tolerance, latency, and throughput; design for enterprise workflows that cannot fail silently or unpredictably
- Observability: Instrument the harness with tracing, cost attribution, and latency visibility so the team can reason about agent behavior in production and catch failures before customers do
- Prompt infrastructure: Build prompt management systems — versioning, templating, and systematic evaluation — that keep agent behavior stable across model updates and configuration changes
- Eval engineering: Design and own evaluation frameworks (unit evals, integration evals, production monitors) that measure agent quality, catch regressions, and drive data-informed decisions
- LLM integration: Integrate with and abstract over frontier LLMs, managing model routing, fallback strategies, cost, and latency trade - offs in production
- Technical leadership: Set technical standards through architecture decisions, code reviews, and coaching — particularly on agentic design patterns and production AI discipline
- System boundary design: Define where agent logic lives — what’s a tool call, a sub-agent, a hardcoded path, or a human escalation — and establish those design standards across the team
Benefits
- Generous family leave
- Matched donations
- Annual learning stipends
- Flexible PTO
- Competitive retirement plan
- Paid volunteer time
- Hands-on experience shipping generative AI products — not just integrating LLM APIs or building prototypes, but owning AI-powered features that production users depend on
- Solid depth in how large language models work: failure modes, context constraints, and how prompt design shapes model behavior at scale
- Cost and efficiency awareness at the system level: experience reasoning about model routing, inference cost, and latency tradeoffs in production
- 7+ years building production software systems with a strong track record on reliability, performance, and scalability
- Python, Kubeflow,Experience with SageMaker, Vertex AI or AzureML, SQL
- Comfort operating in fast-moving, ambiguous, startup-like AI product environments
- A real track record in eval engineering — not just familiarity, but a portfolio of evaluation suites designed, shipped, and used to drive quality decisions in production AI systems
- Practical prompt engineering experience: systematically designing, versioning, and evaluating prompts across model updates or A/B evaluation cycles
- Strong software engineering fundamentals: distributed systems, API design, and testing discipline
- Experience with multi-agent coordination patterns (A2A, MCP)
- Familiarity with agent frameworks (LangChain, LlamaIndex, or similar)
- Familiarity with cloud-native infrastructure, service observability, logging, monitoring, reliability engineering, and production troubleshooting
- Experience with AI observability tooling (tracing, cost tracking, LLM-specific monitoring)
- Prior experience shipping AI systems in enterprise software
Skills
Sign in to read and join the discussion.