Senior Observability Infrastructure Engineer
Summary
Build and operate observability infrastructure using Go/Python, Kubernetes, and telemetry stacks (Elasticsearch, Prometheus, OpenTelemetry) to ensure reliability and scalability for global payments systems.
You will build and operate observability infrastructure across on-premise and Kubernetes environments. You will design logging and metrics architecture, manage hybrid infrastructure, automate operations with Go or Python, optimize telemetry systems at scale, improve CI pipelines, and develop reliability and self-service solutions.
Responsibilities
- Design and implement logging and metrics architecture
- Redesign infrastructure for global regions, data isolation, and regulatory compliance
- Manage the lifecycle of over 1,500 servers
- Write Go or Python automation
- Build self-healing systems
- Improve CI pipelines
- Optimize distributed tracing and logging pipelines
- Tune Elasticsearch clusters
- Optimize Prometheus and VictoriaMetrics storage
- Maintain OpenTelemetry performance
- Participate in on-call rotations
- Upgrade the observability stack
- Implement automated guardrails and quota management
- Design safer API access patterns
Requirements
- 10+ years of experience in observability or platform/infrastructure engineering
- Experience operating telemetry data stores at scale
- Linux kernel-level experience
- Production Kubernetes experience
- Experience with bare-metal and virtualized hardware
- Proficiency in Go or Python
- Experience with infrastructure as code
- Experience with multi-tenant isolation and quota or cost governance approaches
- Familiarity with regulated environments