Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

EPAM Systems

AI Solution Architect & Platform Lead

Posted Updated

Description

EPAM Systems in Turkey seeks an AI Solution/Principal Engineer to lead evaluation of a sovereign, multi-tenant AI platform and its applied products operating in Arabic and English. You will own how we measure and improve non-deterministic AI across assistants, retrieval, tools, and infrastructure, shaping products with trustworthy, evidence-based results.

Responsibilities include building evaluation harnesses, dataset versioning, online drift detection, and CI/CD integrations to gate quality

Define evaluation strategy: what gets measured, layers, methodology, and feed product decisions. Build evaluation harnesses, golden-set management, dataset versioning, automated grading, regression detection and reporting. Ensure grading trustworthiness through judge model selection and calibration. Assess retrieval and agents for grounding, tool-use validation, multi-step reasoning and failure recovery. Create Arabic golden sets and judges calibrated for Arabic; execute online evaluation and drift detection. Integrate evaluation and test automation as CI/CD quality gates. 5+ years building evaluation or test infrastructure and shared libraries. Leadership experience of at least 1 year. Expertise in AI evaluation spanning output quality, retrieval and grounding. Proficiency in Python; intermediate in TypeScript. Statistics knowledge for non-deterministic systems. Private health insurance Career development Mentoring programs Learning platform access English courses

Skills

What Lead Architecture jobs ask for — and how much of it you have →
Apply

See also

Architecture jobs by country — openings, pay and top skills →