Senior Data Engineer: Lakehouse, Spark & GCP
Posted Updated
Description
robusta seeks a Senior Data Engineer to design and implement an enterprise-grade Lakehouse on Google Cloud Platform, using BigQuery, Iceberg, Spark, and related technologies. You will build scalable pipelines and collaborate with data scientists and product teams to deliver production-ready data products.
The role emphasizes CI/CD for data pipelines, data quality, and strong SQL-based analytics across large datasets in a fast-paced environment.
Lakehouse Architecture & Development: Design, deploy, and maintain a scalable GCP-based Lakehouse ecosystem leveraging Apache Iceberg for transactional consistency and time-travel. Pipeline Engineering: Build and optimize ETL/ELT data pipelines using Python and Airflow. Big Data Processing: Implement large-scale distributed data processing with Apache Spark. Data Warehousing & Analytics: Leverage BigQuery for high-performance analytical queries and data serving. Transformation & Quality: Develop modular transformation code using dbt with automated data testing and lineage. Data Modeling: Lead dimensional modeling (Star/Snowflake) and Lakehouse medallion architectures. Performance Tuning & SQL Mastery: Write and optimize complex SQL queries for low latency. Collaboration: Work with data scientists, analytics, and product managers to deliver robust data products. 3+ years of hands-on experience designing and implementing enterprise-grade data platforms and pipelines. GCP core data services experience (BigQuery, Cloud Storage, Dataflow/Dataproc, IAM). Deep understanding of lakehouse concepts and hands-on Apache Iceberg experience. Advanced proficiency in dbt for pipeline transformations and Airflow for workflow orchestration. Strong SQL skills and data warehousing knowledge for high-performance analytics.Skills
Sign in to read and join the discussion.