Data Engineer Platform and Tools – Google Cloud Platform (GCP)
Summary
Designs and maintains scalable GCP data pipelines and warehouses using BigQuery, Dataflow, and Airflow to ensure reliable data processing and analytics.
We are looking for a Data Engineer with hands-on experience in Google Cloud Platform (GCP) to design, build, and maintain scalable data pipelines and data warehouse solutions. The ideal candidate should have expertise in ETL/ELT development, cloud-native data processing, and workflow orchestration, while ensuring data quality, performance, and reliability.
Key Responsibilities
- Design and develop batch and streaming data pipelines on GCP.
- Build and optimize ETL/ELT workflows using BigQuery, Dataflow, and Cloud Storage.
- Develop and maintain data orchestration workflows using Cloud Composer (Airflow).
- Optimize SQL queries and BigQuery performance.
- Monitor, troubleshoot, and support production data pipelines.
- Collaborate with cross-functional teams to deliver scalable data solutions.
Required Skills
- Cloud: Google Cloud Platform (GCP)
- GCP Services: BigQuery, Dataflow, Pub/Sub, Cloud Storage, Cloud Composer (Airflow)
- Programming: Python, SQL, Shell Scripting
- Data Engineering: ETL/ELT, Data Warehousing, Data Modeling
- Version Control & CI/CD: Git, Cloud Build/Jenkins
- Monitoring: Cloud Logging, Cloud Monitoring
- Terraform, Docker
- Kubernetes
- Oracle Cloud optional
- Optional - OIC, ODT, golden gate, AIDP