Senior Big Data Engineer (GCP / Python / PySpark /Streaming)

Required Experience & Qualifications

Bachelor's degree in Computer Science, Engineering, Information Technology, or equivalent practical experience.

6+ years of experience in Data Engineering, Big Data Engineering, or related disciplines.

Strong experience with Python for data engineering and pipeline development.

Hands-on experience with Apache Airflow for workflow orchestration (required).

Advanced experience with PySpark, Spark SQL, and distributed data processing.

Experience with Scala in data engineering environments.

Strong knowledge of relational and NoSQL database technologies.

Experience writing and optimizing complex SQL queries.

Hands-on experience with Apache Kafka and event-driven architectures.

Experience building and supporting real-time data pipelines using Spark Streaming.

Strong experience with Google Cloud Platform (GCP), specifically:

BigQuery

Dataproc

Experience integrating RESTful APIs into data pipelines and data workflows.

Experience implementing CI/CD practices using GitLab CI.

Strong understanding of data modeling, data quality, scalability, and performance optimization.

Excellent verbal communication and technical problem-solving skills.

Ability to explain design decisions and implementation approaches in a collaborative setting.