QA Manager, Software Engineering

At Penguin Solutions (Nasdaq: PENG) – The AI Factory Platform Company – we’re building a team of innovators who thrive on collaboration, creativity, and the opportunity to help shape the future of AI. As part of the AI technology revolution, our teams design, build, deploy, and manage AI factories for enterprises, sovereign AI initiatives, and neocloud providers worldwide.

Headquartered in Silicon Valley, California, Penguin Solutions operates globally through a network of R&D, manufacturing, and sales locations. For nearly three decades, we have operated at the intersection of memory and AI/HPC infrastructure. That engineering expertise positions us to power the next generation of AI workloads, from training to inference and agentic AI at scale.

Penguin Solutions brings together differentiated infrastructure software, advanced memory, compute systems, end-to-end services, and industry-leading partner solutions in a full-stack AI factory platform designed to help customers deploy and scale AI workloads with speed and precision.

At Penguin Solutions, we value ideas over hierarchy and believe in servant leadership, where leaders enable teams to do their best work. We empower employees to take ownership, drive innovation, and grow through challenging work, continuous learning, and exposure to advanced AI tools and technologies. With flexibility where it matters and a strong focus on outcomes, Penguin Solutions is a place to do your best work, grow your career, and make a meaningful impact.

Job Overview

We are seeking a Manager, Quality Engineering to lead quality engineering for ClusterWareAI, a critical component of the Penguin Solutions AI Factory Platform.

ClusterWareAI enables automated provisioning, orchestration, monitoring, and lifecycle management of large-scale AI infrastructure. In this role, you will define and drive the quality strategy for a complex distributed platform spanning GPU infrastructure, compute, networking, storage, Kubernetes, workload orchestration, and AI-enabled operations.

This is a hands-on technical leadership role. You will lead and develop a Quality Engineering team while partnering closely with Software Engineering, Product Management, Platform Engineering, DevOps, Support, and Systems Integration teams to build quality into every stage of the product lifecycle.

The ideal candidate brings a strong product-engineering mindset, deep experience with test automation and distributed systems, and a passion for delivering reliable cloud and infrastructure software at scale.

Responsibilities

• Own the end-to-end Quality Engineering strategy for ClusterWareAI across functional, integration, system, regression, performance, scalability, reliability, security, and resilience testing.
• Lead, mentor, and grow a high-performing QE team while establishing strong technical, automation, and execution practices.
• Drive a shift-left, automation-first quality culture, partnering with Engineering, Product, Platform, DevOps, Support, and Systems Integration throughout the product lifecycle.
• Design scalable test automation frameworks across APIs, UI, CLI, control-plane services, infrastructure integrations, and end-to-end workflows, integrated with CI/CD quality gates.
• Define test strategies for GPU infrastructure, bare metal, Kubernetes, networking, storage, workload orchestration, and distributed systems at scale.
• Validate infrastructure lifecycle workflows including provisioning, configuration, upgrades, rollback, deprovisioning, monitoring, health management, and automated remediation.
• Drive performance, scalability, HA, fault-injection, longevity, and failure/recovery testing across large-scale distributed environments.
• Define validation for multi-tenancy, tenant isolation, security, access controls, resource governance, and workload separation.
• Establish quality and release-readiness metrics covering automation, escaped defects, test reliability, performance, scalability, and product risk.
• Partner with Support and Engineering on customer issues, root-cause analysis, and converting field failures into automated regression coverage.
• Evaluate and adopt AI-assisted quality engineering approaches to improve test generation, failure analysis, defect triage, and engineering productivity.

AI / Agentic Quality Engineering

• Own the AI Quality Engineering strategy for ClusterWareAI's AI Platform capabilities.
• Build automated AI evaluation frameworks and benchmark suites covering accuracy, groundedness, consistency, reliability, latency, cost, and task success.
• Establish LLM and agent evaluation methodologies using golden datasets, scenario-based testing, synthetic data, LLM-as-a-Judge, rubric-based evaluation, and human evaluation.
• Test RAG, tool calling, multi-step agents, planning/execution, agent collaboration, memory, and infrastructure-aware reasoning.
• Develop AI regression pipelines to detect degradation caused by changes to models, prompts, agents, tools, knowledge sources, and orchestration logic.
• Drive AI safety and adversarial testing, including hallucination, prompt injection, unauthorized tool use, policy violations, data leakage, and unsafe infrastructure actions.
• Validate agent guardrails, policy enforcement, approvals, authorization boundaries, tenant isolation, auditability, and human-in-the-loop controls.
• Establish production AI quality metrics including task success rate, hallucination rate, tool-call accuracy, agent failure rate, escalation rate, latency, and cost efficiency.
• Develop scenario-based autonomous infrastructure testing to validate that agents correctly observe, reason, plan, obtain approval, execute authorized actions, verify outcomes, and recover safely from failures.

Qualifications

• Bachelor's degree in Computer Science, Engineering, or equivalent practical experience.
• 12+ years in Quality Engineering, Software Engineering, Test Automation, or related engineering roles, with 3+ years leading QE teams.
• Proven experience owning quality for a commercial software, cloud, infrastructure, or distributed systems platform.
• Strong programming and automation skills, preferably Python, with experience building scalable automation frameworks.
• Strong experience testing APIs, distributed systems, cloud platforms, Kubernetes, infrastructure software, and complex enterprise applications.
• Experience with CI/CD, performance, scalability, reliability, HA, security, and failure/recovery testing.
• Strong Linux, debugging, observability, log analysis, and distributed-systems troubleshooting skills.
• Demonstrated ability to provide technical leadership, coaching, and team development.
• Hands-on experience with AI/LLM quality engineering, AI evaluation, or testing AI-powered applications.
• Experience testing LLM applications, RAG, AI agents, multi-agent systems, or agentic workflows.
• Experience with AI evaluation methodologies including golden datasets, benchmark testing, LLM-as-a-Judge, RAG evaluation, adversarial testing, and red teaming.
• Experience with frameworks such as DeepEval, Ragas, Promptfoo, LangSmith/LangChain evaluation, NVIDIA NeMo Guardrails, or equivalent.
• Understanding of AI failure modes including hallucination, prompt injection, non-determinism, tool-use errors, context failures, model regression, and unsafe agent behavior.
• Experience integrating AI evaluation and regression testing into CI/CD pipelines.

Preferred Qualifications

• Experience with AI infrastructure, GPU cloud/neocloud platforms, or large-scale cloud infrastructure.
• Experience with Kubernetes, container orchestration, and multi-tenant cloud platforms.
• Experience with GPU infrastructure, including GPU lifecycle, drivers, firmware, health monitoring, and heterogeneous accelerator environments.
• Experience with high-performance networking technologies such as InfiniBand, RDMA/RoCE, high-speed Ethernet, and distributed storage.
• Experience with bare-metal provisioning, infrastructure lifecycle management, and fleet-scale automation.
• Experience with workload orchestration and scheduling technologies such as Kubernetes and Slurm.
• Experience with infrastructure automation and Infrastructure as Code technologies such as Ansible.
• Experience with cloud platform security, tenant isolation, identity/access controls, and resource governance.
• Experience with observability, SRE practices, service-level objectives, high availability, fault tolerance, and automated remediation.
• Experience testing AI/LLM-powered capabilities and developing repeatable AI evaluation frameworks.
• Background in enterprise infrastructure, cloud platforms, systems management software, or High Performance Computing environments.

Location

Hybrid – Bangalore, India

Travel

As needed