Machine Learning Quality Engineer
Posted Updated
- The Corporate Engineering AI team is the central enablement and platform delivery function for LSEG’s internal agentic AI ecosystem
- The team’s mission is to scale safe, high‑quality AI capabilities across the enterprise by providing shared platforms, patterns, governance, and delivery support
- CE AI owns and operates core AI platforms including LSEG AI Assist, the Question Answering Service (QAS), and the Internal MCP Gateway
- Rather than delivering individual business use cases end‑to‑end, the team enables product engineering groups across LSEG to expose knowledge, data, and actions to AI agents in a consistent, governed, and repeatable way
- The team operates a Central MCP Delivery model: building critical MCP tools and services “for” product teams where required, while simultaneously defining standards, patterns, and platform capabilities that allow teams to progressively move towards self‑service contribution
- This programme delivers an LSEG‑owned, production‑grade agentic AI platform with MCP as its extensibility layer
- The scope of work includes:
- Building and operating LSEG AI Assist, an in‑house agentic experience capable of reasoning, planning, and tool‑calling
- Operating QAS, the enterprise RAG and search layer used to ground agent responses in approved data sources
- Delivering a production Internal MCP Gateway providing discovery, security, policy enforcement, observability, and lifecycle management for MCP tools and Skills
- Designing and building MCP servers and Skills that expose internal and vendor systems safely to agents
- Establishing evaluation, quality control, and governance mechanisms so MCP tools and Skills can be promoted through PTB/PTO and operated with confidence at scale
- The programme follows a “build for” model today, with a strong emphasis on defining the future product and platform experience, patterns, and contribution pathways that will enable federated scale over time
- Quality Engineers – Responsibilities & Skills:
- Define and implement evaluation frameworks for MCP tools and Skills covering correctness, safety, and regression impact
- Build and maintain automated test pipelines for agentic behaviours, including tool invocation and multi‑step workflows
- Evaluate and mitigate agentic failure modes such as hallucination, tool misuse, invalid inputs, and latency amplification
- Produce testing evidence required for Permit to Build (PTB) and Permit to Operate (PTO)
- Partner with ML Engineers to embed testability and evaluation hooks into MCP servers and Skills
- Help define the long‑term quality and governance model for federated MCP contributions across LSEG
- Strong Python experience for test harnesses and automation
- Experience with LLM and RAG evaluation frameworks or custom evaluation pipelines
- Test automation expertise covering unit, integration, and regression testing
- Understanding of agentic system risks and failure modes
- Ability to assess solutions against governance, security, and audit expectations
- Experience working in regulated or highly governed engineering environments