DATA ENGINEER (GCP)
Posted Updated
Description
Data Engineer (GCP)
Summary
We are seeking a skilled Data Engineer to join our IT Services team, where you will play a pivotal role in architecting and maintaining robust data infrastructure on the Google Cloud Platform. This position is critical for transforming raw data into actionable insights, ensuring high availability, and supporting advanced analytics initiatives. The ideal candidate will leverage their extensive background to design scalable data pipelines and optimize data storage solutions, directly contributing to the organization's strategic data goals.
Responsibilities
- Design, build, and maintain scalable data warehouses and implement comprehensive data modeling strategies to support business intelligence needs.
- Manage version control systems and oversee Continuous Integration/Continuous Deployment (CI/CD) pipelines to ensure efficient code delivery and system stability.
- Develop and optimize complex data processing workflows using Google Dataflow and Apache Beam for real-time and batch processing.
- Execute data quality assurance protocols and lead initiatives related to cloud migration projects to ensure seamless transitions and data integrity.
- Architect solutions utilizing BigQuery for analytics and Dataproc for distributed data processing to handle large-scale datasets efficiently.
- Orchestrate complex scheduling and workflow dependencies using Cloud Composer (Apache Airflow) to automate data operations.
- Implement event-driven architectures using Pub/Sub and manage data lifecycle policies within Cloud Storage to ensure secure and accessible data repositories.
Requirements
Requirements:
- Possess a professional background of 5 to 7 years in data engineering, with a strong focus on the Google Cloud ecosystem.
- Demonstrate expert proficiency in Data Warehousing concepts and advanced Data Modeling techniques.
- Show hands-on experience with Git/GitHub for version control and the implementation of CI/CD practices.
- Have a proven track record of working with Google Dataflow, Apache Beam, BigQuery, and Dataproc.
- Be skilled in applying data quality standards and executing cloud migration strategies.
- Possess practical knowledge of Cloud Composer (Apache Airflow) for workflow orchestration.
- Understand the integration of Pub/Sub and Cloud Storage within a modern data architecture.
- Be prepared to work within a compensation range of ₹11-13 LPA.