QA Engineer – AI Systems
Summary
Remote QA Engineer maintaining the quality of a Salesforce environment (including ERP/SAP integrations) and AI-powered products such as chatbots and agent AI systems. Day to day: designing and executing functional, regression, integration, and UAT tests across web and API layers, and evaluating multi-step AI workflows.
This is a remote position.
Key Responsibilities
-
Evaluate AI agents for accuracy, consistency, and appropriate handling of edge cases and failures
-
Assess multi-step, multi-tool AI workflows to confirm agents behave as intended across an entire task, not just a single response
-
Conduct functional, regression, integration, and user acceptance testing across web and API layers
-
Identify, document, troubleshoot, finding root cause and track defects, working closely with the development team through to resolution
-
Maintain clear and thorough test documentation for both conventional features and AI-driven functionality
-
Support automation testing initiatives where relevant, or help establish them if they do not yet exist
-
Work with product and engineering teams to translate requirements into meaningful test coverage
Requirements
-
1–3 years of professional experience in quality assurance or software testing
-
Experience with automation testing tools such as Selenium, Playwright, or Cypress
-
Experience testing AI chatbots, AI Assistants, or agent AI systems
-
Solid grounding in QA fundamentals, including test case design and defect lifecycle management
-
Ability to test systems where outcomes are not always fixed or fully predictable
-
Strong written communication skills, particularly when explaining AI behavior to non-technical stakeholders
-
Ability to work independently and reliably in a remote setting
-
Familiarity with API testing tools such as Postman
-
Exposure to prompt evaluation or LLM output assessment methods