Data Engineer
Posted 2
views
Summary
Design, develop, and maintain large-scale data pipelines and warehousing solutions on Google Cloud Platform using BigQuery, Cloud Dataflow, Cloud Composer, and Cloud Storage, while owning data governance, quality, and GCP cost optimization as a data engineer or lead.
Job Title: Google Data Engineer / Lead
Experience: 5-15 years
Job Summary: An experienced Google Data Engineer / Lead to design, develop, and maintain large-scale data processing systems on Google Cloud Platform (GCP). The ideal candidate should have a strong background in data engineering, data warehousing, and cloud-based data platforms.
Responsibilities:
- Data Pipeline Development: Design, develop, and maintain data pipelines using Google Cloud Dataflow, Cloud Composer, and BigQuery.
- Data Warehousing: Work with data architects and business stakeholders to design and implement data warehousing solutions using BigQuery and Cloud Storage.
- Data Governance: Develop and maintain data governance policies, procedures, and standards for GCP.
- Collaboration: Work with cross-functional teams to identify and prioritize project requirements, provide technical guidance, and ensure data quality.
- Cost Optimization: Optimize GCP costs and ensure data processing efficiency.
Required Skills:
- Google Cloud Platform: In-depth knowledge of GCP, including Cloud Dataflow, BigQuery, Cloud Storage, and Cloud Composer.
- Data Engineering: Strong understanding of data engineering concepts, including data modeling, ETL, and data warehousing.
- Expertise in Cloud storage, Dataproc, Cloud Functions, Messaging/Streaming (pub-sub)
- Programming: Proficiency in programming languages, including Python, Java, or SQL.
- Data Governance: Familiarity with data governance frameworks, including data quality, data security, and data compliance.
- Communication: Excellent communication and collaboration skills.
Preferred Skills:
- Google Cloud Certifications: Relevant certifications, such as Google Cloud Certified - Professional Data Engineer or Google Cloud Certified - Enterprise Data Engineer.
- Big Data: Experience with big data technologies, including Hadoop, Spark, or NoSQL databases.
- Machine Learning: Experience with machine learning frameworks, including TensorFlow or Scikit-learn.
- Cloud Experience*: Experience with other cloud-based data platforms, including AWS or Azure