Senior Data Platform Engineer - OLAP
Summary
Build and scale Adyen’s real-time OLAP and big-data infrastructure using Druid, Kafka, Java, and Python to power analytics and ML workloads.
Adyen
Data Platform Engineers at Adyen build the foundational layer of tooling and processes for our on‑premise Big Data Platforms. These tools support a wide range of services and developers.
You’ll collaborate with data engineers, ML scientists, and DataOps Engineers to develop and operate large‑scale big data platforms, including a 500+ node cluster with 25k+ CPUs, 240+ TB of RAM, and 23+ PB of storage.
Your Focus Areas
- The OLAP Ecosystem: Scaling and optimizing our real‑time analytical powerhouses (Druid, FastAPI, Java, Kafka).
- The Data Engineering Ecosystem: Designing and building data contracts, validation frameworks, optimization tooling, and robust Kafka ingestion pipelines.
What You’ll Do
- Build innovative new tools from scratch while continuously improving existing infrastructure.
- Develop and maintain high‑performance, distributed big data platforms.
- Work deeply with databases, file systems, and compute engines.
- Create tools that enhance metadata collection, data lineage, and platform‑wide data quality.
- Optimize continuous release/deployment environments within a large‑scale private cloud.
- Develop stream processing applications to reduce time‑to‑insight for analysts and data scientists.
Who You Are
- Python or Java expert with 6+ years of experience in object‑oriented programming.
- Strong communicator who bridges teams (analysts, data scientists) and dependencies (infrastructure, security).
- Proven experience building and maintaining distributed compute/data systems (e.g., Spark, Trino, Druid, ClickHouse).
- Experience with real‑time and batch pipelines (Kafka, Spark Streaming), focusing on user‑friendliness and reliability.
- Knowledge of DevOps pipelines and data modeling tooling.