AI QA & Evaluation Lead

Jodayn is seeking an AI QA & Evaluation Lead to establish and manage enterprise-grade quality assurance, validation, and evaluation processes for AI-powered systems and intelligent automation platforms.

The successful candidate will be responsible for ensuring the reliability, consistency, security, explainability, and governance alignment of AI-generated outputs and AI-driven workflows. This role is critical in maintaining enterprise trust, operational quality, and compliance standards across AI initiatives delivered to government and enterprise clients.

AI Evaluation & Validation

• Design and maintain AI testing and evaluation frameworks
• Develop benchmarking datasets and output validation methodologies
• Evaluate AI-generated responses for consistency, accuracy, and reliability
• Define confidence scoring and quality measurement approaches

AI Quality Assurance

• Implement automated regression testing for AI workflows
• Monitor prompt performance and behavioural consistency
• Detect model drift and output degradation
• Validate multilingual AI outputs including Arabic and English content

Governance & Compliance

• Ensure AI outputs align with enterprise governance standards
• Support auditability and traceability of AI-generated results
• Implement review and approval workflows for AI systems
• Support compliance validation activities for enterprise deployments

Security & Risk Testing

• Conduct AI security and prompt injection testing
• Validate protections against unsafe outputs and misuse scenarios
• Support secure AI deployment and operational practices

Reporting & Continuous Improvement

• Produce quality assurance reports and evaluation summaries
• Recommend improvements to prompts, workflows, and AI models
• Support continuous optimisation of AI solution quality

Education

• Bachelor's degree in Computer Science, Information Systems, Software Engineering, or related field

Experience

• Minimum 5 years experience in enterprise quality assurance or software testing
• Experience with automation testing frameworks
• Experience evaluating AI-powered systems preferred
• Experience in enterprise or regulated environments preferred

Quality Assurance

• Enterprise QA methodologies
• Automation testing frameworks
• API testing
• Regression testing methodologies
• Test case development

AI Evaluation

• Large language model evaluation
• Prompt engineering concepts
• AI benchmarking methodologies
• AI governance and validation concepts

Security & Governance

• Risk assessment concepts
• AI security testing
• Auditability and traceability principles

Preferred Skills

• Experience in enterprise AI initiatives
• Exposure to regulated industries
• Experience with bilingual AI validation
• Familiarity with enterprise governance frameworks

Core Competencies

• Exceptional attention to detail
• Strong analytical thinking
• Structured problem-solving capability
• High quality and governance awareness
• Strong documentation and reporting capability