Operations Engineer
Description
Role summary
Provides first- and second-line technical support for a large-scale AI/HPC platform. Investigates alerts and user issues, restores service using established diagnostic methods and approved procedures, and carries out day-to-day operations across Linux servers, Kubernetes, GPU compute, networking and storage.
Reports to the Technical Lead, with day-to-day technical guidance from the Senior Engineers on shift. Resolves issues within agreed competence and change authority, escalates complex faults with clear evidence, and helps improve team runbooks and automation.
Key responsibilities
• Respond to monitoring alerts, incidents and service requests. Assess impact and urgency, establish what has failed and keep accurate records of symptoms, investigation and actions taken.
• Diagnose and resolve first- and second-line issues across Linux hosts, containers and Kubernetes workloads. Investigate service failures, resource pressure, unhealthy nodes, failed pods and image-pull errors using logs, events and approved diagnostic tools.
• Support workload submission, approved container environments and project storage access. Investigate common scheduling, quota, permission and runtime issues, escalating changes to allocation or security policy.
• Monitor GPU availability and health, and collect driver, device and workload diagnostics. Identify affected nodes and follow approved procedures to isolate faults and validate recovery.
• Perform authorised health checks, node draining, patching and reprovisioning using approved runbooks and automation. Validate results before returning capacity to service.
Essential skills and experience
• Hands-on experience supporting Linux infrastructure in a production environment, with the ability to investigate and resolve routine operational faults independently within agreed procedures.
• Practical Linux administration: command-line diagnostics, services and processes, logs, users and permissions, packages, filesystems, CPU and memory usage, and host networking.
• Working knowledge of containers and Kubernetes operations. Able to inspect pods, events, logs, resource requests and node status, and investigate common workload failures.
• Networking and storage fundamentals, including TCP/IP, DNS, routing, ports, NFS or comparable shared storage, mount points and access permissions. Able to distinguish connectivity, capacity and permission problems.
• Structured troubleshooting: establish scope, gather evidence, test a likely cause and verify the outcome. Recognises when an action could disrupt other users and when senior assistance is required.
Skills
Sign in to read and join the discussion.