Artificial Intelligence Engineer
Posted Updated
Description
Management Solutions International (MSI) seeks a Senior AI Evaluation Engineer to lead evaluation, testing and validation of advanced AI and LLM systems in production contexts. This role emphasizes ensuring accuracy, reliability and safety of AI-powered applications.
You will design evaluation frameworks, benchmarks and automated pipelines, and guide continuous improvement through statistical analysis and controlled experiments. Doha, Qatar-based with senior responsibilities.
Design and implement comprehensive evaluation frameworks for LLMs, Generative AI and RAG-based applications. Develop datasets, test cases, ground-truth benchmarks and scoring rubrics. Build automated evaluation pipelines and statistical validation methodologies. Lead benchmarking and experimental validation to identify performance gaps and improvement opportunities. 7+ years of experience in AI evaluation, testing and validation. Master's degree or higher in Computer Science, Software Engineering or related field. Strong expertise in evaluating AI systems for factual accuracy, reliability and production readiness.Skills
Sign in to read and join the discussion.