Hiring For GenAI QA

Job Title: Generative AI QA Engineer

Experience: 1-2 Years (including relevant QA experience)

Domain: Generative AI / LLM / AI Testing

Employment Type: Full-Time

Job Summary

We are looking for an experienced GenAI QA Engineer responsible for testing, validating, and ensuring the quality, reliability, accuracy, and performance of Generative AI applications and Large Language Models (LLMs).

The candidate should have experience in functional testing, API testing, automation, prompt testing, and validating AI-generated responses. Familiarity with RAG architectures, vector databases, AI agents, and LLM evaluation frameworks is highly desirable.

Key Responsibilities

1. GenAI Application Testing

• Test Generative AI applications, chatbots, virtual assistants, and AI-powered solutions.
• Validate LLM-generated responses for accuracy, relevance, consistency, and completeness.
• Test prompt engineering scenarios and prompt-response behavior.
• Validate multi-turn conversations and contextual understanding.

2. LLM Testing & Evaluation

• Evaluate LLM responses for hallucinations, bias, toxicity, and factual correctness.
• Perform prompt injection and jailbreak testing.
• Validate response quality, relevance, groundedness, and context retention.
• Define test datasets and evaluation benchmarks.

3. RAG Testing

• Test Retrieval-Augmented Generation (RAG) applications.
• Validate document ingestion, chunking, embedding, and retrieval accuracy.
• Test semantic search and vector database integration.
• Validate whether generated responses are grounded in retrieved documents.
• Test citation accuracy and source relevance.

4. API & Automation Testing

• Perform API testing using Postman or REST Assured.
• Develop automation scripts using Python or Java.
• Automate GenAI test scenarios and regression test suites.
• Integrate automated tests into CI/CD pipelines.

5. Performance & Security Testing

• Test response latency, throughput, scalability, and token consumption.
• Validate application behavior under concurrent user loads.
• Test security vulnerabilities specific to LLM applications.
• Validate data privacy and sensitive information leakage.

6. Reporting & Collaboration

• Prepare test strategies, test plans, test cases, and test reports.
• Collaborate with developers, data scientists, and AI engineers.
• Identify defects, analyze root causes, and track resolutions.

Experience & Qualifications

• Bachelor's degree in Computer Science, IT, or a related field.
• 1+ years of software testing experience.
• 1+ years of exposure to Generative AI / LLM testing.
• Strong understanding of software testing methodologies.
• Hands-on experience with Python-based testing and API automation.
• Experience testing chatbot or AI-powered applications.

Interested candidates can directly share their resume on below mentioned email ID:

Sanjeevani.supe@1point1.com