Google Cloud Platform Architect - Big Data Architecture

Job Description :

We are looking for an experienced GCP Architect with strong expertise in Big Data architecture, cloud-native data engineering, and distributed computing systems.

The ideal candidate will design, develop, and optimize scalable data solutions on Google Cloud Platform while leading architectural decisions and mentoring junior engineers.

Primary Roles & Responsibilities :

- Design, select, and integrate Big Data tools and frameworks to support business and technical requirements

- Develop and maintain robust data pipelines implementing ETL processes

- Monitor pipeline performance and recommend infrastructure optimizations and improvements

- Translate complex business and technical requirements into scalable architectural designs

- Evaluate and implement alternative data storage and processing solutions for optimal performance and cost efficiency

- Lead architectural discussions and provide technical direction across data engineering initiatives

- Mentor junior team members by conducting technical training sessions and reviewing deliverables

- Ensure best practices in data engineering, scalability, reliability, and security are followed

Skills & Qualifications:

- 14+ years of overall experience with at least 3+ years in Architecture roles

- Strong understanding of Data Warehousing and Data Modeling techniques

- Hands-on experience with distributed computing systems:Hadoop v2, MapReduce, HDFS

- Strong expertise in Google Cloud Platform (GCP) :Airflow / Cloud Composer

1. Cloud Data Fusion

2. Cloud Dataflow

3. Cloud Dataproc

4. BigQuery

- Experience in building stream-processing systems using tools like Apache Spark Streaming or Apache Storm

- Strong knowledge of Big Data querying tools such as Hive, Pig, and Impala

- Proficiency in Apache Spark, SQL, and Linux environments

- Experience with ETL frameworks and tools such as:Apache NiFi, Flume, DBT

1. Apache NiFi

2. Flume

3. DBT

- Experience with messaging systems such as Kafka or RabbitMQ

- Strong understanding of Lambda Architecture, including trade-offs and implementation considerations

- Excellent analytical, debugging, and problem-solving skills