Senior Data Engineer
You will own the OLAP analytics layer from raw event ingestion through API-serving views, including materialized views, query planning, refresh schedules, and cost optimization. You will partner on high-write OLTP tables, improving partitioning, indexing, triggers, autovacuum, and read-path latency. You will define Kafka schema contracts, evolve S3 data lake layouts, develop validation tooling, and design scalable event-sourced data models for analytics such as PnL and position tracking. You will also coordinate schema contracts with upstream and downstream consumers.
Responsibilities
- Own the OLAP analytics layer from raw event ingestion through API-serving views
- Design materialized views, refresh cadences, dictionary catalogs, query plans, and cost optimizations
- Partner on high-write OLTP serving tables, including partitioning, indexing, triggers, autovacuum tuning, and bloat monitoring
- Define Kafka topic schema contracts and evolve the S3 lake layout
- Contribute to parity-validation tooling for data correctness during migrations
- Design event-sourced data models and derivative analytics for PnL, positions, and cohorts
- Coordinate schema contracts with upstream teams and downstream consumers
Requirements
- 5+ years of data engineering experience on production systems serving users at scale
- Deep knowledge of OLTP and OLAP split architectures
- Columnar warehouse expertise, preferably ClickHouse
- Data lake experience with Parquet, Iceberg or Delta/Hudi, compaction strategies, and S3 layouts
- Streaming pipeline experience with Kafka, delivery semantics, backpressure, consumer groups, and schema evolution
- Strong data modeling knowledge, including star and snowflake schemas, SCD, CDC, and idempotent event sourcing
- PostgreSQL experience at scale, including partitioning, indexing, autovacuum, query planning, and replication
- SQL fluency at warehouse scale
- Distributed systems reasoning, including consistency, ordering, replay, and mutable state
Benefits
- Equity
- Unlimited PTO
- Health coverage
- Vision coverage
- Dental coverage
- 401k match
- New MacBook Pro
- Display and accessories