SENIOR DATA ENGINEER-GCP
Summary
Designs and builds scalable GCP data pipelines and warehouses using BigQuery, Dataflow, and Airflow to enable data-driven decisions.
Job Description – Senior Data Engineer (GCP)Location: Sydney
We are looking for a highly experienced Senior Data Engineer with strong expertise in Google Cloud Platform (GCP) to design, build, and optimize scalable data pipelines and architecture. The role will focus on enabling data-driven decision-making by delivering high-quality, reliable, and efficient data solutions across the organization.
Key Responsibilities
- Design and implement scalable and robust data pipelines using GCP services
- Build and maintain data lakes, data warehouses, and ETL/ELT workflows
- Work with large-scale structured and unstructured datasets
- Optimize data processing pipelines for performance, cost, and reliability
- Collaborate with data scientists, analysts, and business stakeholders to deliver data solutions
- Develop and enforce data quality, governance, and security standards
- Automate data workflows and orchestration using tools like Apache Airflow / Cloud Composer
- Ensure high availability and fault tolerance of data platforms
- Monitor and troubleshoot data pipelines and production issues
- Mentor junior engineers and contribute to best practices
Must Have Skills & Experience
- 10+ years of experience in data engineering or related roles
- Strong hands-on experience with Google Cloud Platform (GCP) services such as: BigQueryDataflowPub/SubCloud StorageDataprocCloud Composer
- Expertise in Python, SQL, and distributed data processing frameworks (e.g., Apache Beam, Spark)
- Experience designing real-time and batch data pipelines
- Strong understanding of data modeling, data warehousing, and ETL/ELT concepts
- Experience with CI/CD pipelines and DevOps practices
- Familiarity with data governance, data security, and compliance frameworks
- Proven ability to work with large datasets in a cloud environment
- Strong problem-solving and performance optimization skills
Nice to Have
- Experience with multi-cloud environments (AWS/Azure)
- Knowledge of containerization (Docker, Kubernetes)
- Experience with streaming frameworks (Kafka, Pub/Sub streaming)
- Exposure to Machine Learning pipelines (Vertex AI)
- Certifications such as: Google Professional Data Engineer
- Experience in domain-specific industries (telecom, finance, healthcare, etc.)
- Familiarity with data cataloging and governance tools
Key Competencies
- Strong analytical and problem-solving skills
- Excellent communication and stakeholder management
- Ability to translate business requirements into technical solutions
- Leadership and mentoring capabilities
- Focus on scalability, reliability, and performance