Senior Software Engineer - Distributed Systems
Posted Updated
Description
We are building the database and cold-storage tier for observability data at petabyte scale, with a focus on making telemetry data cost-efficient to store and fast to query.
Our technology stack is centered around Rust and Apache DataFusion, and we are looking for a Senior Software Engineer who enjoys solving hard problems across distributed systems, database engines, query execution, and storage.
You will have the opportunity to shape the architecture of a critical data platform from the ground up and work on systems that ingest and query logs, metrics, and traces.
Roles & Responsibilities:
- Design and build scalable and reliable systems for storing and querying observability data.
- Build database and storage components using Rust and Apache DataFusion.
- Work across the query engine and storage layers, including query planning, execution, optimization, and data access.
- Design distributed systems for high throughput, low latency, fault tolerance, and horizontal scalability.
- Work with columnar and analytical data formats such as Apache Arrow and Parquet.
- Optimize query performance across CPU, memory, I/O, and network boundaries.
- Design systems for efficient cold-tier storage, balancing storage cost, query performance, and data retention requirements.
- Build production-grade systems with strong observability, reliability, and operational characteristics.
- Participate in architecture and design reviews and make pragmatic technology trade-offs.
What we are looking for:
- 5+ years of software engineering experience, with significant time spent on backend or infrastructure systems.
- At least 3 years of hands-on Rust programming, including production systems you have shipped and operated. This is a hard requirement.
- Strong understanding of distributed systems and database fundamentals.
- Hands-on experience with one or more of: Query engines, Databases, Distributed storage, Data processing systems, Search/indexing systems, Analytics platforms
- Strong understanding of concurrency, parallelism, memory management, and networking.
- Strong understanding of data structures, algorithms, and system design.
- Experience with Kubernetes - deploying, operating, and debugging stateful workloads in production.
- Strong debugging and performance-profiling skills.
- Experience taking systems from design through to reliable operation at scale.
Good to Have:
- Experience with Apache DataFusion or other query engines such as Spark, Trino, DuckDB, ClickHouse, or Velox.
- Experience contributing to or working with open-source database/query-engine projects.
- Experience with Go.
- Experience with observability systems and telemetry data, particularly logs, metrics, and traces.
- Experience with time-series or analytical databases.
- Familiarity with Apache Arrow, Parquet, Delta Lake, or Iceberg.
- Experience with cloud object storage such as S3, GCS, or Azure Blob Storage, including compaction, indexing, and tiered storage over object stores.
- Experience with cloud-native infrastructure beyond Kubernetes itself - operators, service meshes, and cloud-native storage and networking.
Skills
Sign in to read and join the discussion.