AI System Analyst
Description
For our international client, we are looking for two AI System Analyst to join the project and support the deployment and localisation of an AI-powered platform across multiple global markets.
ABOUT THE ROLE
A global consumer brand has launched a GenAI-powered front-end agent in the UK — a conversational interface that lets customers discover and explore products through natural language. The platform is now rolling out across Europe: Germany, Czech Republic, Romania, and Poland in parallel.
As an AI Business Analyst, you will be one of the people who makes each market deployment actually work. You will log into AWS, query databases, pull and review content files, investigate why the AI is responding the way it does, adjust prompts and configurations, and validate that the agent behaves correctly — linguistically and factually — for each market.
This is not a developer role, but it is a technical one. You will be comfortable opening a terminal, reading JSON, and iterating on a prompt until it produces the right result. The content sets you work with are in the range of dozens of articles — manageable enough to read and understand deeply, complex enough to require real analytical care.
WHAT YOU'LL BE DOING
- Working on market configuration as per local business and localization requirement
- Reading and analysing content sets (articles, product descriptions, target market requirements) to understand what the AI is working from
- Identifying problems in AI responses — factual errors, tone issues, language quality hallucinations, accuracy and adequacy issues and tracing them to root cause
- Writing, testing, and refining prompts to correct AI behaviour across markets
- Configuring and validating market-specific AI settings before and after each country launch
- Documenting issues, configurations, and outcomes clearly for the wider team
- Collaborating with developers, the GenAI Architect, and local market stakeholders
TECH ENVIRONMENT
- AWS S3 primarily via console, some CLI
- JSON reading and interpreting configuration and content files
- Python — basic familiarity helpful but not required; you will not be writing application code
- LLM tooling — working directly with prompt interfaces and reviewing model outputs
- RAG - basic understanding
REQUIREMENTS
- Experience in some of the following areas: prompt engineering, LLM evaluation or testing, chatbot configuration, preparing multilingual synthetic data, knowledge management, creating/fine-tuning guardrails
- Comfortable logging into AWS console and working with S3 to access and download data
- Confident reading and interpreting JSON
- Experience writing and iteratively testing prompts for LLM-based systems
- Analytical mindset — able to diagnose why an AI response is wrong and propose a fix
- Strong communication skills - both verbal and written (clear in documenting findings, issues, configurations and participating in calls with market stakeholders)
- Handling user requests and technical issues via ticketing systems
- Comfortable working in a highly multinational, cross-cultural team environment
- English at B2 or above
NICE TO HAVE
- Working knowledge of how LLMs behave — tokenisation, context windows, temperature, hallucination patterns
- Experience in a business analyst, data analyst, or operations analyst role
- European language skills — especially Germanic (German, Dutch), Slavic (Polish, Czech, Slovak, Bulgarian), Latin (French, Spanish, Italian, Romanian, Portuguese) or Greek, Hungarian, Japanese, Arabic, Indonesian, Malay or Filipino
- Preparing and delivering brief presentations
- Having lived or worked in a non-native language environment — first-hand understanding of cross-lingual complexity is a real advantage
- Background in call centre operations or customer service environments
Skills
Sign in to read and join the discussion.