LLM Test Engineer
Posted
Summary
Entry-level QA role testing AI/LLM chat features for a hospitality product in Pune: validating conversation accuracy, hallucinations, safety (prompt injection), multilingual replies, and regression-testing the dashboard, chat widget and booking flows, with test cases and bug logs on GitHub.
Role & responsibilities
- Test AI conversations for accuracy, hallucinations and correct source citation
- Check role-based answers, multilingual replies and out-of-scope handling
- Run safety tests such as prompt-injection attempts
- Compare new prompt and model versions against the previous baseline
- Do manual, smoke and regression testing of the dashboard, chat widget and booking flows
- Write test cases and log clear, reproducible bugs on GitHub
Preferred candidate profile
- Manual testing: test cases, positive/negative and boundary testing, regression, bug life cycle
- LLM testing: hands-on with ChatGPT, Claude or Gemini; understands hallucinations and prompt behaviour
- Prompt engineering basics
- Strong attention to detail and written English
- Clear bug reporting