GCP Data Engineer - S

Key Responsibilities

• Design, develop, and maintain scalable data pipelines and data processing frameworks on GCP.
• Build and optimize ETL/ELT workflows to ingest, transform, and load data from multiple sources.
• Develop and manage enterprise-scale data lakes, data warehouses, and lakehouse architectures.
• Implement batch and real-time data processing solutions.
• Ensure data quality, integrity, security, and governance across the data lifecycle.
• Work closely with data architects, analytics teams, data scientists, and business stakeholders to deliver reliable data solutions.
• Optimize BigQuery environments for performance and cost efficiency.
• Monitor data pipelines, troubleshoot issues, and ensure SLA compliance.
• Automate deployment and operational processes using DevOps and Infrastructure-as-Code practices.
• Support AI/ML initiatives by building and maintaining data platforms for analytics and model training.

Key Technical Responsibilities

GCP Data Services

• BigQuery
• Cloud Storage
• Dataflow
• Dataproc
• Pub/Sub
• BigLake
• Dataplex
• Cloud Composer
• Cloud SQL
• Bigtable
• Spanner

Data Engineering

• Data Pipeline Development
• ETL / ELT Processing
• Data Warehousing
• Data Lake Architecture
• Data Modeling
• Batch Processing
• Stream Processing
• Data Migration
• Workflow Orchestration

Big Data Technologies

• Apache Spark
• Apache Beam
• Hadoop Ecosystem
• Kafka
• Dataflow Streaming
• Dataproc Clusters