GCP Data Engineer
Summary
Data Engineer building and optimizing cloud data pipelines on GCP (BigQuery, Cloud Run, Cloud Functions) using Python, SQL, DBT, and Terraform for a retail client project. Day to day involves ETL development, query optimization, data integrity/security, and working with technical and non-technical stakeholders.
We are seeking a skilled Data Engineer to join our team for an exciting project with our retail client. The ideal candidate will possess strong expertise in Google Cloud Platform (GCP), SQL, and Python. A comprehensive understanding of ETL processes, especially within cloud environments, is essential. This position offers the opportunity to work closely with both technical and non-technical stakeholders, fostering collaboration and innovative solutions in data engineering.
Main Responsibilities
As a Data Engineer, you will be responsible for:
- Designing and implementing robust data pipelines.
- Utilizing cloud services for efficient data processing and storage.
- Collaborating with cross-functional teams to gather requirements.
- Optimizing SQL queries for performance and scalability.
- Developing and maintaining ETL processes with industry-standard tools.
- Ensuring data integrity and security standards are met.
- Documenting technical specifications and workflows.
Key Requirements
The ideal candidate must have:
- Proficiency in GCP services (BigQuery, Cloud Run, Cloud Functions, etc.).
- Expertise in Python programming.
- Experience with DBT and Terraform.
- Familiarity with at least one business intelligence (BI) tool.
Additional skills
- Basic understanding of Azure.
- Experience with Informatica or other ETL tools.