Senior Data Engineer - (GCP & Lakehouse Architecture) - On Site - Octopus by RTG
Who recruits here
Up to three recruiters at robusta with their work email, so you can write to a person.
Available on Pro.
Description
We are seeking an experienced Senior Data Engineer to lead the design, development, and scaling of our next-generation, cloud-native Data Lakehouse platform on Google Cloud Platform (GCP). In this role, you will be instrumental in modernizing our data infrastructure, building robust ELT/ETL pipelines, and establishing enterprise-grade data modeling standards to empower analytics, machine learning, and business intelligence. If you are passionate about modern open-source table formats, high-performance distributed computing, and data reliability, we would love to have you on the team.
Key Responsibilities
• Lakehouse Architecture & Development: Design, deploy, and maintain a scalable GCP-based Lakehouse ecosystem leveraging Apache Iceberg for transactional consistency, time-travel, and efficient storage management.
• Pipeline Engineering: Architect, build, and optimize high-throughput ETL/ELT data pipelines using Python and Apache Airflow for orchestrating complex dependency workflows.
• Big Data Processing: Implement large-scale distributed data processing workloads and transformation pipelines using Apache Spark.
• Data Warehousing & Analytics: Leverage BigQuery for high-performance analytical queries, cost optimization, partition/cluster management, and serving structured data to downstream consumers.
• Transformation & Quality: Write clean, modular, and testable transformation code using dbt (Data Build Tool), embedding automated data testing, documentation,and lineage tracking.
•Data Modeling: Lead dimensional and semantic data modeling efforts (Star/Snowflake schemas, One-Big-Table, and Lakehouse medallion architectures) to ensure high query performance and seamless business usability.
• Performance Tuning & SQL Mastery: Write, optimize, and troubleshoot complex SQL queries to ensure low latency and cost-efficient processing across large datasets.
• Collaboration: Partner closely with data scientists, analytics teams, and product managers to understand data requirements and deliver robust, production-ready data products.
Requirements
•3+ years of hands-on experience designing and implementing enterprise-grade data platforms and pipelines.
• Cloud Platform: Proven production experience with Google Cloud Platform (GCP) core data services (BigQuery, Cloud Storage, Dataflow/Dataproc, IAM).
• Lakehouse & Table Formats: Deep understanding of modern lakehouse concepts and hands-on experience with Apache Iceberg.
• Transformation & Orchestration: Advanced proficiency in dbt for pipeline transformations and Apache Airflow for workflow orchestration.
• Big Data & Compute: Strong working knowledge of Apache Spark for distributed data processing.
• Programming & Querying:
o Advanced Python programming skills for data engineering tasks and custom operator development.
o Expert-level SQL writing and performance tuning capabilities.
• Data Modeling: Demonstrated expertise in dimensional modeling, data warehousing concepts, and schema design for analytical workloads.
• Methodologies: Strong grasp of modern ETL/ELT design patterns, CI/CD for data pipelines, data governance, and data quality best practices.
Skills
- Airflow
- Analytics
- Big Data
- BigQuery
- Business Intelligence
- CI/CD
- Cloud
- Cloud Native
- Data Engineering
- Data Governance
- Data Modeling
- Data Pipelines
- Data Quality
- Data Warehousing
- Dataproc
- dbt
- Design Patterns
- Dimensional Modeling
- Distributed Computing
- ELT
- ETL
- GCP
- IAM
- Iceberg
- Lakehouse
- Machine Learning
- Python
- Snowflake
- Spark
- SQL
- Workflow Orchestration
As published by workable · 7 questions
First name, Last name, Email, Headline, Phone, Address, Photo, Education, Experience, Summary, Resume, Cover letter
- How long is your notice period?
- Please add 1 or 2 employment references and their relationship optional
- Current Net Salary
- Expected Net Salary
- How many years of hands-on Data Engineering experience do you have, specifically working with Google Cloud Platform (GCP) services such as BigQuery, Cloud Storage, and Dataflow/Dataproc?
- Do you have hands-on production experience designing or implementing Data Lakehouse architectures using Apache Iceberg?
- Which of the following technologies have you used in production environments: Apache Airflow, dbt, Apache Spark, Python, and advanced SQL?
Sign in to read and join the discussion.