Senior Platform Engineer, Monitoring & Telemetry

Senior Platform Engineer, Monitoring & Telemetry at Silicon Quantum Computing, Sydney. A senior-level full time position in the Software team, listed under software engineering.

About the role

We are hiring a Platform Engineer to own monitoring and telemetry in Platform & Infrastructure. The scope is everything the team runs: the Kubernetes clusters from small local sites to the central cluster, Ceph storage, core network services, the HPC and simulation environments, the quantum runtime cluster and its FPGA mesh, our AWS footprint, and the cluster that ships with each quantum computer.

The workloads are not a web estate. An FPGA synthesis run takes hours and holds a licence while it does. A simulation job may need to be reproducible years later. The realtime control plane cares about microsecond jitter rather than request percentiles. A physicist will want to correlate a device measurement with a calibration run and a cluster event, so telemetry from unrelated systems has to be joinable.

Compute is on-premise and finite. Utilisation numbers decide where the next hardware spend goes and which team is under-served, so they end up in procurement and scheduling decisions. You own the platform and set the practice: the conventions other teams instrument against, and alerting an on-call engineer acts on without checking it twice.

Based at our Sydney facility, you will work alongside the platform and infrastructure engineers who run the estate, and with the research teams instrumenting their own work. This is a role for someone who wants observability to carry real decisions, and who would rather retire a noisy alert than tune it out.

Essential

• 5+ years in platform, infrastructure or site reliability engineering, with direct ownership of an observability stack
• Prometheus and Grafana at scale, including long-term storage with Thanos, Mimir, VictoriaMetrics or an equivalent
• Log pipelines in production: Loki, OpenSearch or ELK, with a shipper such as Vector or Fluent Bit
• OpenTelemetry, and distributed tracing in a real system rather than a demo

Responsibilities

• Own the monitoring and telemetry platform end to end: metrics, logs, traces and alerting across clusters, storage, network and cloud
• Instrument the platform services the team runs, including Kubernetes, Ceph, the CI/CD platform, GitHub Enterprise, Artifactory and Vault
• Make utilisation and capacity visible and trustworthy, so scheduling decisions and hardware purchases rest on measurement
• Design alerting that names a specific action and keeps noise low, and retire alerts that no longer prompt one
• Support the quantum runtime cluster's observability needs, including high-rate telemetry from the FPGA mesh, without disturbing the realtime path
• Extend monitoring to every cluster we build, one per quantum computer, including air-gapped sites where telemetry cannot leave the building and has to be useful to whoever is standing next to it
• Handle the data engineering of observability at scale: cardinality, sampling, retention and the cost of keeping it
• Define and measure service levels for internal platform services, and report against them
• Build the on-call tooling and runbooks, and improve them after every incident
• Help other teams instrument their own systems, and provide the libraries, conventions and defaults that make that easy

Nice to Have

• Ceph monitoring and troubleshooting at the cluster level
• Network telemetry: SNMP, flow data, switch and fabric monitoring, DHCP and DNS visibility
• HPC and batch scheduler observability, including Slurm job accounting
• GPU telemetry, such as DCGM, and accelerator utilisation reporting
• eBPF-based tooling for host and kernel visibility
• Hardware and out-of-band monitoring: IPMI, Redfish, environmental and power telemetry
• Realtime or hard-deadline systems, where jitter matters more than throughput
• Laboratory or scientific instrument telemetry, and joining it to infrastructure data

#J-18808-Ljbffr