Data Engineer-GCP
Summary
Data engineer in Sydney designing and maintaining batch and streaming ETL/ELT pipelines on Google Cloud Platform. Core stack is BigQuery, Dataflow, Pub/Sub, Cloud Storage, Python, SQL, and Apache Beam or Spark, with a focus on data warehouse design and data modeling.
Job Details:
Must Have Skills: Strong
GCP Data Services Expertise - Hands-on experience
with BigQuery, Dataflow, Pub/Sub, Cloud Storage, Ability to design scalable
data solutions using native GCP tools. Experience in Data Pipeline Development (Batch & Streaming) - Building and maintaining ETL/ELT pipelines, Experience with real-time
streaming and batch processing. Strong Programming & Query Skills - Proficiency
in Python and SQL, Experience with data processing frameworks like Apache
Beam or Spark Understanding of Data Modeling & Warehousing - Strong
understanding of data modeling (star/snowflake schemas), Experience designing
and optimizing data warehouses (BigQuery) |
Nice to Have Skills: Experience in Multi-Cloud Exposure : Understanding of cross-cloud data integration and migration, Experience
working with AWS or Azure alongside GCP Knowledge of Streaming & Messaging Frameworks : Experience with Kafka or advanced Pub/Sub patterns, Knowledge of
real-time data architectures Basic Understanding of Containerization & Orchestration : Hands-on with Docker and Kubernetes (GKE),
Ability to deploy and manage data workloads in containerized environments Data
Governance & Security : Familiarity with data
lineage, cataloging, and governance tools, Understanding of IAM roles,
data security, and compliance in GCP. |
