Automation QA Lead
Description
QA Lead – Performance & Load Testing (AI-DLC)
Key responsibilities
Performance and scalability engineering
Define the performance test strategy for the 2x, 3x and 5x targets: workload model, test types (load, stress, soak, spike, scale-out) and pass criteria.
Build realistic workload models from current claims patterns: nightly batch, EDI intake (837 claims, 277CA acknowledgements), real-time API calls and event traffic.
Design and run load tests on a production-sized Azure environment, and prove horizontal scale-out to 5x by adding resources rather than changing code.
Find and fix bottlenecks with the engineering pods across Go services, PostgreSQL, Kafka and the infrastructure.
Set performance budgets per service (throughput, latency, batch duration, error rate) and enforce them in CI/CD.
Publish load-test results as a go / no-go input to each production cutover.
QA architecture across the pods
Own test standards, the automation framework and QA oversight across all six delivery pods.
Lead the Validation pod (2 Automation QA engineers).
Build automated tests from Gherkin acceptance scenarios, including API, integration, end-to-end and regression suites.
Design the parity tests that compare Prism's claim decisions with the legacy Que system, aiming for 99.5% decision match.
Support parallel runs, reconciliation and UAT defect triage around each cutover.
AI-driven delivery (AI-DLC)
Use AI coding agents (GitHub Copilot and similar) to generate test cases, load scripts and synthetic test data from specifications.
Set review standards and quality gates for AI-generated tests and code, so AI output is checked before it is trusted.
Contribute to AI-DLC retrospectives: tune prompts, agents and the sizing model, and keep AI tool spend within budget.
Data and compliance
Work only with synthetic, non-production data offshore. No protected health information (PHI) reaches offshore staff or AI tools.
Define synthetic data volumes and profiles that are realistic enough for 5x load tests and repeatable for every release.
Experience
12+ years in QA, with 5+ years as a performance or QA architect on high-volume transaction systems (claims, payments, banking or similar).
Hands-on load testing with k6 (preferred), JMeter, Gatling or Locust, including scripting, distributed load generation and results analysis.
A track record of capacity and scale-out testing: has designed and run tests that proved a system at several times its current volume, and can show the results.
Workload modelling from production patterns, covering both batch windows and real-time traffic.
Performance analysis across the stack: APM and observability (Grafana, Prometheus, Datadog, Dynatrace or Azure Monitor), PostgreSQL query plans and tuning, Kafka throughput and consumer lag.
Performance testing of containerised, cloud-native services on Azure or another major cloud, including autoscaling behavior.
Test automation architecture: framework design, API testing, BDD with Gherkin, and test gates in CI/CD pipelines (GitHub Actions or Azure DevOps).
Daily use of AI coding assistants (GitHub Copilot or similar) for test design, script generation or test data, with clear examples of reviewing and correcting AI output.
Experience leading QA engineers and setting standards across several agile teams.
Clear written and spoken English for daily work with US-based client stakeholders.
Good to have
US healthcare claims domain: claim lifecycle, payer submission, EDI X12 837 and 277CA, HIPAA.
Legacy-to-cloud modernization, especially strangler-pattern migrations with parallel runs and data reconciliation.
Parity or shadow testing that compares a new system's outputs with a legacy system's.
Synthetic test data generation at volume.
Performance testing of Go services.
Chaos or resilience testing (failover, recovery, Kafka partition rebalancing).
Security testing in CI/CD, such as SAST with SonarCloud.
Certifications such as ISTQB Advanced (Test Analyst or Technical Test Analyst) or a cloud certification such as Azure Administrator or Developer.