Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Dicetek

AI Quality & Reliability Engineer (QA/SRE) | Dicetek | UAE

Posted Updated

Description

Dicetek is seeking an experienced AI Quality & Reliability Engineer to lead automated testing, RAG evaluation, and LLM benchmarking in the UAE. You will own end-to-end testing and reliability for distributed AI platforms, integrating PyTest, Terraform, Prometheus, and Grafana across cloud environments.

You will collaborate with AI platform engineers, MLOps and infra teams to ensure low-latency AI workloads, scalable testing, and robust cost controls while advancing CI/CD with GitHub Actions.

Design, develop, and execute automated testing frameworks for AI/ML systems and cloud-native microservices using Python. Evaluate AI model outputs, retrieval-augmented generation pipelines, and LLM benchmarks using DeepEval, Ragas, or LangSmith. Measure and track generation quality metrics including ROUGE, BLEU, and custom evaluation scores. Configure, manage, and monitor real-time reliability dashboards and alerting systems using Prometheus and Grafana. Build and maintain automated CI/CD deployment pipelines and testing workflows utilizing GitHub Actions. Validate and review infrastructure-as-code provisioning templates and modules written in Terraform. Conduct rigorous performance, load, and stress testing for AI APIs using K6 or JMeter. Implement FinOps cost-control measures to identify expensive failing tests and optimize cloud resource usage. Collaborate with AI engineers, MLOps, cybersecurity, and infrastructure teams in the UAE. Bachelor’s degree in Computer Science, Software Engineering, Artificial Intelligence, or a related technical discipline. 4+ years of professional experience in software quality assurance, site reliability engineering, or AI systems validation. High proficiency in Python for AI testing automation, scripting, and framework development (PyTest, Selenium, or Robot Framework). Strong hands-on experience with cloud infrastructure platforms (Azure or AWS). Practical understanding of prompt engineering, RAG evaluation, and LLM benchmarking methodologies.

Skills

Apply

See also

SRE jobs by country — openings, pay and top skills →