GCP Data Engineer
Description
Job Description
We are looking for an experienced GCP Data Engineer with strong hands-on expertise in designing, developing, and supporting scalable data solutions on Google Cloud Platform (GCP).
Key Responsibilities
-
Design and develop scalable ETL/ELT data pipelines on GCP.
-
Build batch and streaming data solutions using BigQuery, Cloud Storage, Dataflow, Pub/Sub, and Dataproc.
-
Develop data transformation and processing solutions using Python, SQL, and Apache Spark.
-
Design and optimize BigQuery datasets, tables, partitioning, clustering, and complex SQL queries.
-
Build ingestion frameworks to integrate data from databases, APIs, files, SaaS platforms, and event streams.
-
Implement workflow orchestration using Cloud Composer / Apache Airflow.
-
Develop robust data quality, reconciliation, logging, monitoring, and exception-handling frameworks.
-
Implement incremental data processing, CDC, and high-volume data ingestion patterns.
-
Work with CI/CD pipelines and infrastructure automation for deploying GCP data solutions.
-
Implement appropriate IAM, security, encryption, and data governance controls.
-
Troubleshoot performance and production issues across data pipelines.
-
Collaborate with data architects, analysts, business stakeholders, and downstream reporting teams.
Skills
Sign in to read and join the discussion.