Skip to content
Search jobs inside your AI agent. Claude, ChatGPT, Claude Code, Codex, Gemini, Cursor and Grok
Connect

Arabic.AI

Senior QA Engineer - AI Products (6 Months Contract)

Posted Updated 3 views

Who recruits here

Up to three recruiters at Arabic.AI with their work email, so you can write to a person.

View plans

Available on Pro.

Description

Arabic.AI is seeking a Senior QA Engineer to own quality across our AI products, testing LLM-powered features end to end and building automated test and evaluation suites. You will define test strategies for web, API, and backend, evaluate model outputs for accuracy and safety, and drive CI/CD integrated automated UI/API tests.

Collaborating with ML engineers and product managers, you will set acceptance criteria, report defects clearly, and help ensure every release is reliable and safe.

Define test strategies and plans for AI features across web, API, and backend. Evaluate model outputs for accuracy, consistency, bias, and safety. Build evaluation datasets, including edge cases and adversarial prompts. Develop and maintain automated UI, API, and regression tests integrated into CI/CD. Run regression testing on model, prompt, and configuration changes. Test performance, latency, error handling, and fallback behavior. Report defects clearly and provide release-readiness sign-off. Test non-deterministic outputs using scoring rubrics and human evaluation. Proven experience testing AI/ML or LLM-based products (chatbots, agents, etc.). Hands-on scripting in Python and/or TypeScript/JavaScript. Hands-on with UI automation frameworks (Playwright, Cypress, Selenium) and a test framework such as Pytest. API testing with Postman, REST Assured, or similar. CI/CD and version control: GitHub Actions, GitLab CI or Jenkins; Git. Test management: Jira, TestRail, or Zephyr. Excellent written English and clear bug reporting.

Skills

What Senior QA jobs ask for — and how much of it you have →
Apply

See also

QA jobs by country — openings, pay and top skills →