Gcp Data Engineer
Summary
Data engineer in Chennai designing and maintaining scalable data pipelines on GCP — building ETL/ELT workflows with BigQuery, Dataflow/Dataproc, Python/PySpark and SQL, orchestrating with Airflow/Cloud Composer, and automating infrastructure with Terraform.
Role & responsibilities
- Design, develop and maintain scalable data pipelines on GCP.
- Build ETL/ELT workflows using BigQuery, Dataflow and Dataproc.
- Develop data processing solutions using Python, PySpark and SQL.
- Manage workflow orchestration using Apache Airflow / Cloud Composer.
- Work with GCS, Pub/Sub and other GCP services.
- Implement infrastructure automation using Terraform.
- Optimize BigQuery queries and data processing performance.
- Work on data integration, data modelling, validation and quality checks.
- Collaborate with cross-functional teams to deliver analytics and data products.
Preferred candidate profile
Candidates with 48 years of experience in Data Engineering and strong hands-on experience in Google Cloud Platform (GCP). Must have practical experience in BigQuery, Dataflow/Dataproc, Python, PySpark, SQL, Airflow/Cloud Composer and Terraform. Should have experience developing scalable ETL/ELT pipelines, handling large datasets, data integration and cloud-based data processing. Good knowledge of GCS, Pub/Sub and data modelling is expected. Candidates with experience in Cloud Run, Data Fusion, Cloud SQL, PostgreSQL and CI/CD tools will be preferred.