GCP Data Engineer

Data Engineering & Development

• Design, develop, and maintain data pipelines on GCP.
• Build scalable ETL/ELT solutions for ingesting, transforming, and loading data from multiple source systems.
• Develop batch and real-time data processing solutions.
• Optimize data workflows for performance, reliability, and scalability.

GCP Platform Development

• Work with GCP services including:
  • BigQuery
  • Dataflow
  • Dataproc
  • Pub/Sub
  • Cloud Storage
  • Cloud Composer (Airflow)
  • Cloud Functions
• Implement cloud-native data architectures and best practices.

Data Warehousing & Analytics

• Design and maintain data warehouse solutions in BigQuery.
• Develop data models to support reporting and analytics requirements.
• Ensure data integrity, quality, and consistency across platforms.

Production Support & Monitoring

• Support production environments and troubleshoot data pipeline issues.
• Monitor scheduled jobs and resolve failures in a timely manner.
• Perform root cause analysis and implement permanent fixes.
• Ensure adherence to SLA and operational requirements.

Collaboration

• Work closely with Data Architects, Business Analysts, Product Owners, and Reporting teams.
• Participate in Agile ceremonies including sprint planning, daily stand-ups, and retrospectives.
• Contribute to technical design discussions and code reviews.

Required Technical Skills

GCP Technologies

• BigQuery
• Dataflow
• Dataproc
• Pub/Sub
• Cloud Storage
• Cloud Composer (Airflow)
• Cloud Functions

Programming

• Python
• SQL (Advanced)

Data Engineering

• ETL/ELT Development
• Data Modeling
• Data Warehousing Concepts
• Data Quality and Validation
• Performance Tuning

Databases

• SQL Server
• Oracle
• PostgreSQL
• MySQL

Version Control & CI/CD

• Git
• Jenkins / GitHub Actions
• DevOps concepts

Preferred Skills

• PySpark / Apache Spark
• Kafka
• Terraform
• Snowflake
• Looker
• Tableau
• Power BI
• Data Governance concepts