Senior HPC DevOps Lead for CORA: ML & GPU on AWS
Posted Updated
Description
EPAM Systems seeks a Senior CORA HPC DevOps Engineer to lead scaling, reliability, and automation of the CORA platform for AWS HealthOmics. You will guide scientists in using the CORA UI and optimize reserved vs on-demand capacity for large-scale ML workloads.
You will maintain CI/CD pipelines, address job scheduling issues, and collaborate with engineering teams to monitor GPU utilization and performance across multi-node clusters using MPI, NCCL, and cloud HPC tools.
Guide scientists and data teams in CORA UI to run self-service workloads. Advise users and manage capacity across reserved vs on-demand compute to optimize cost. Sustain automated CI/CD pipelines for infrastructure provisioning and platform deployments. Provide Tier-2/3 support for job scheduling failures and resource quotas. Collaborate with engineering teams to improve platform documentation. Monitor GPU utilization and refine multi-node scheduling to reduce idle time. 3+ years of experience in HPC DevOps or similar roles. Experience with MPI, OpenMP, and multi-node GPU communication. Experience managing AWS GPU instances (P-series, G-series) and ML pipelines. Expertise in capacity planning with AWS Capacity Blocks, ODCRs, and Service Quota management. Proficient in deploying containerized HPC environments (Apptainer/Singularity, Docker, Enroot). Understanding of I/O bottlenecks with Lustre, GPFS, or BeeGFS. Skilled in profiling with NVIDIA Nsight. Experience deploying/scaling HPC workloads on cloud using EFA, ParallelCluster, FSx for Lustre. English proficiency at least B1+. Continuous upskilling Private health insurance Referral bonusesSkills
Sign in to read and join the discussion.