The Planet Group

Discussion

QA engineer (Testing AI Product)

QA engineer
Location: San Diego, CA
Duration: 2 Months
Pay: $80/hr - $85/hr (DOE)

The Planet Group is seeking for a QA engineer who uses Swift to test an AI product that helps organizations make sense of their data. You'll join a team building solutions where quality is the product. You'll own two testing frontiers at once: UI automation for a native macOS SwiftUI app, and model evaluation for an AI pipeline that reasons over customer data. To succeed, bring creative energy to collaboration, communication, and delivering high-quality results.
You'll work on problems that matter. The software you test protects customer data, keeps critical workflows running, and delivers tools that organizations from schools to hospitals trust every day. We're looking for someone who solves problems, takes initiative, and cares about the people who use what we build.

Key Responsibilities:

  • Design, develop, and maintain automated test suites in Swift using Swift Testing and XCUITest.
  • Build UI automation for the macOS SwiftUI app, wiring stable accessibility identifiers and dependency-injected test seams for reliable coverage.
  • Author and own model evaluations: curate golden-question datasets and CSV fixtures, define scoring criteria, and run the evaluation harness across different model backends.
  • Score model outputs with deterministic matchers and LLM-as-judge rubrics (correctness, hallucination, concision), and track evaluation-score regressions across commits.
  • Validate model quality, latency, and determinism, including structured model outputs and intent-classification accuracy.
  • Validate REST API endpoints and SSE streams, and verify JWT/OAuth (OIDC) authentication flows.
  • Investigate and triage concurrency issues using Thread Sanitizer and strict concurrency checking across actor and streaming paths.
  • Execute long-running stability and soak tests to validate the reliability of streaming and background work.
  • Build and maintain CI test execution integrated with Jenkins.
  • Collaborate with developers to define testability requirements early, and participate in sprint planning, backlog grooming, and release readiness reviews.

Minimum Qualifications:

  • Computer Science / Software Engineering degree or equivalent software engineering experience
  • Hands-on experience testing with Behavior-Driven (BDD) or Test-Driven Development (TDD), with analytical thinking to build clear, concise, and comprehensive test scenarios
Preferred Qualifications
  • Practical experience with Swift and Apple test frameworks, Swift Testing (@Test/@Suite/#expect) and XCUITest, building UI automation with stable accessibility identifiers and dependency-injected test seams. Comfortable with command-line build and test tooling and Jenkins CI integration.
  • Experience building evaluation harnesses for AI features: authoring golden-question datasets and CSV fixtures, scoring model outputs with deterministic matchers and LLM-as-judge rubrics (correctness, hallucination, concision), and tracking evaluation-score regressions across commits.
  • Ability to validate model quality, latency, and determinism, comparing model outputs across different model backends, and verifying structured outputs and intent-classification accuracy.
  • Strong problem-solving and analytical skills, with the ability to investigate, debug, and triage issues in complex multi-stage, multi-process macOS applications independently and cooperatively.
  • Experience testing actor-based Swift concurrency, including Sendable and actor-isolation correctness, data-race detection with Thread Sanitizer (TSan) as a complement to Swift 6 strict concurrency, and long-running stability and soak testing of streaming paths.
  • Strong understanding of RESTful API and Server-Sent Events (SSE) service testing, with experience validating JWT/OAuth (OIDC) authentication flows.
  • Familiarity with performance validation on macOS: inference latency, token-budget, memory, and resource-constraint testing for AI model workloads.
  • Familiarity with the Model Context Protocol (MCP) or similar tool-use frameworks for validating tool invocation and data retrieval.
  • Experience with Agile and Scrum methodologies and rapid releases, plus strong communication and cross-functional collaboration skills.
Show