Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Equinix

NewBe an early applicant

AI Engineer (Trust & Evaluation)

Posted Updated

Description

  • Designs, develops and deploys the evaluation and trust capabilities of Equinix’s GenAI platform
  • Builds platform features that let teams evaluate their agents at scale: golden datasets, synthetic test data, scoring with LLM-as-judge, and monitoring of agent quality in production
  • Builds and tests the trust controls that agents run on, including agent identity, access, guardrails and data boundaries
  • Traces agent failures to their root cause and fixes them in the platform
  • Works in a team with senior engineers and with product, platform and security partners to turn defined problems into robust, secure, observable and performant solutions
  • Applies solid knowledge of software engineering practice and of LLM and agent concepts (prompting, retrieval, tool and function calling, orchestration, MCP)
  • Understands evaluation methods (code-based checks, LLM-as-judge, human review, golden datasets, regression suites) and evaluation statistics (sampling, variance across runs, agreement between reviewers)
  • Understands AI security risks such as prompt injection and data leakage, and identity concepts such as OAuth and per-user access
  • Follows established standards and suggests improvements
  • Works on problems of defined scope where analysis requires evaluation of identifiable factors
  • Traces an agent failure to its cause (retrieval, tool call, prompt, model, data or infrastructure) using logs and traces, and follows it until it is fixed
  • Judges whether a metric reflects real quality
  • Escalates unusual or high-impact issues
  • Work is reviewed at defined checkpoints
  • Works with AI engineers, product managers, forward-deployed engineers, platform and security teams
  • Turns the needs of agent builders into platform features
  • Explains results, defects and risks to partners who are not familiar with the subject
  • Asks for guidance early and shares knowledge with the team
  • Build platform features for evaluation: dataset creation and versioning, golden datasets, synthetic test data and reusable metric templates
  • Build scoring for agents: code-based checks, LLM-as-judge, tool-call accuracy and grounding checks
  • Build human-review features that compare judge scores against human reviewers, and recalibrate when agreement is low
  • Run evaluations at scale: before release, on every change to a prompt, model or tool, and on agents in production
  • Build evaluation playbooks so agent builders can evaluate their agents without specialist help
  • Keep results traceable to the agent, model, prompt and dataset versions tested, with a record of who approved each release
  • Build and test the platform’s trust controls: agent identity, per-user access to connected systems, tool permissions, guardrails and data boundaries
  • Run adversarial tests and red-teaming (prompt injection, out-of-scope requests, data-boundary tests), and turn findings into fixes and regression tests
  • Work with security and governance teams to turn policy requirements into automated checks
  • Investigate agent failures and performance problems end to end, and follow them until the cause is fixed
  • Build test and performance tooling: end-to-end agent tests, load and latency tests, and failure diagnostics
  • Turn production failures and user feedback into regression tests, so the same defect does not return
  • Write tested, reviewed and documented code, and contribute to shared libraries and tooling
  • Test and build to security, data-privacy and responsible-AI requirements
  • Bachelor’s degree in computer science, engineering, data science, mathematics or a related field, or equivalent experience
  • Ability to write SQL and analyze results
  • Working knowledge of LLM applications (prompting, retrieval, agents)
  • Hands-on experience building, not only using, evaluations or test systems for ML or LLM applications, including reading failures from logs or traces
  • 1–4 years in AI/ML or software engineering (internships and applied research count)
  • Working knowledge of evaluation statistics (sampling, confidence intervals, variance across runs) and of measuring agreement between reviewers
  • Clear written and verbal communication
  • Proficiency in Python, with solid habits: version control, testing, code review. Experience building backend services or platform features
  • Experience building evaluation or observability features used by many teams or at scale
  • Experience with LLM evaluation and observability tools, and tracing standards such as OpenTelemetry
  • Experience with red-teaming or adversarial testing of LLM systems
  • Experience with identity and access (OAuth 2.0, OIDC, delegated access)
  • Experience with performance, load or test engineering
  • Experience with agent frameworks, MCP or A2A
  • Experience with Google Cloud (Vertex AI, GKE) and Kubernetes
  • Familiarity with AI risk frameworks (NIST AI RMF, ISO 42001)
  • MS in a related field

Skills

Apply

See also

AI Engineering jobs by country — openings, pay and top skills →