GCP Data Engineering Consultant
Posted 9
views
Summary
Careernet is hiring a GCP Data Engineering Consultant in Bengaluru to design, build, and maintain scalable batch and real-time ETL/ELT pipelines on Google Cloud, optimizing BigQuery and ensuring pipeline reliability. Core stack: Python, PySpark, Airflow, Pub/Sub, BigQuery, SQL, with Terraform and dbt as nice-to-haves.
Key Skills: Pub/Sub, Pyspark, Bigquery, Airflow, Python, GCP, ETL, Terraform, Data Warehousing, SQL, DBT, Data Modelling
Roles and Responsibilities:
- Design, build, and maintain scalable batch and real-time ETL/ELT pipelines using Python, PySpark, Airflow, Pub/Sub, and GCP services.
- Develop and optimize BigQuery data processing solutions, including SQL queries, partitioning, clustering, and cost-efficient execution.
- Build reliable data ingestion and transformation workflows for analytical and operational data sources.
- Implement data validation, quality controls, error handling, lineage, and governance measures across data pipelines.
- Troubleshoot production data pipeline issues, perform root-cause analysis, and improve reliability, throughput, and operational stability.
Skills Required:
- 4 - 7 years of relevant experience in data engineering, cloud data platforms, or ETL development.
- Strong hands-on experience with Google Cloud Platform data engineering solutions, ETL development, Python, PySpark, and advanced SQL.
- Practical experience developing data solutions with BigQuery, Airflow, Pub/Sub, and related GCP data services.
- Ability to design scalable batch and streaming data architectures for high-volume workloads.
- Experience with data pipeline testing, validation, monitoring, troubleshooting, and production support.
Good to Have:
- Experience with data modelling, Terraform, dbt, and data warehousing concepts.
Education: Any Graduation or Any Post Graduation.