Senior Data Engineer - GCP Lakehouse, Airflow & Spark
Posted Updated
Description
Robusta Studio is seeking an experienced Senior Data Engineer to lead the design, development, and scaling of a cloud-native Data Lakehouse on Google Cloud Platform. You will build robust ELT/ETL pipelines and establish data modeling standards to empower analytics, ML, and BI.
Lead lakehouse architecture, implement pipelines with Airflow and dbt, and leverage Apache Iceberg for transactional consistency and time-travel.
Lakehouse Architecture & Development: design, deploy and maintain a scalable GCP-based Lakehouse ecosystem using Iceberg. Pipeline Engineering: build and optimize high-throughput ETL/ELT data pipelines with Python and Airflow. Big Data Processing: implement large-scale distributed data processing workloads with Spark. Data Warehousing & Analytics: leverage BigQuery for high-performance queries and cost optimization. Transformation & Quality: write modular transformations using dbt with automated tests and lineage tracking. Data Modeling: lead dimensional and semantic modeling for analytical workloads. Performance Tuning & SQL Mastery: optimize complex SQL for low latency and cost efficiency. Collaboration: work with data scientists, analytics teams and product managers to deliver data products. 3+ years of hands-on experience designing and implementing enterprise-grade data platforms and pipelines. Cloud Platform: Proven production experience with Google Cloud Platform core data services (BigQuery, Cloud Storage, Dataflow/Dataproc, IAM). Lakehouse & Table Formats: Hands-on experience with Apache Iceberg. Transformation & Orchestration: Proficiency in dbt and Apache Airflow for pipeline transformations and workflow orchestration. Big Data & Compute: Strong working knowledge of Apache Spark for distributed data processing.Skills
Sign in to read and join the discussion.