Data Engineer

Role: Data Engineer II – GCP Data Platform

Objective of the Role

We are looking for a hands-on Data Engineer II with strong Google Cloud Platform experience to build scalable data pipelines, transformations and reusable data-platform capabilities.

You will work across Data Engineering and Data Platform Engineering, independently delivering production-grade data products while contributing reusable components and engineering patterns that can be adopted across multiple teams and use cases.

The role requires strong hands-on development skills and the ability to take a solution from design through implementation, testing, deployment and production support.

You Will

• Design, develop and maintain production-grade data pipelines on Google Cloud Platform.
• Build scalable batch and incremental data-processing solutions.
• Develop complex transformations using BigQuery, SQL and Dataform / dbt.
• Build and maintain data-processing workflows using Cloud Composer / Apache Airflow.
• Develop reusable transformation components, orchestration patterns, libraries and templates.
• Implement metadata-driven and configuration-driven processing where appropriate.
• Build distributed data-processing solutions using Dataflow / Apache Beam and/or Dataproc / Spark.
• Design data models for analytical and downstream data-product requirements.
• Implement incremental and idempotent processing patterns.
• Define and maintain schemas and data contracts.
• Implement automated data-quality checks, validation and reconciliation.
• Capture and integrate metadata and lineage into data-processing workflows.
• Optimise BigQuery queries and pipelines for performance and cloud cost.
• Implement automated testing and integrate data workloads with CI/CD pipelines.
• Build monitoring and operational controls for production pipelines.
• Troubleshoot production issues and perform root-cause analysis.
• Contribute to reusable data-platform capabilities and engineering standards.
• Collaborate with Data Engineers, Platform Engineers, DevOps, Architects and business stakeholders.

You Must Have

• 4+ years of hands-on Data Engineering experience building production data solutions.
• Strong practical experience with Google Cloud Platform (GCP).
• Strong hands-on experience with BigQuery.
• Advanced SQL skills including complex joins, CTEs, window functions and query tuning.
• Strong Python development skills for data processing, automation and testing.
• Experience developing production ETL / ELT pipelines.
• Hands-on experience with Cloud Composer / Apache Airflow.
• Experience with Dataform and/or dbt.
• Experience designing batch and incremental processing pipelines.
• Experience with Dataflow / Apache Beam or Dataproc / Spark / PySpark.
• Strong understanding of data modelling, including normalization, denormalization and dimensional modelling.
• Experience working with large-scale datasets.
• Experience with incremental and idempotent pipeline patterns.
• Understanding of data contracts and schema evolution.
• Experience implementing data-quality validation and reconciliation.
• Understanding of metadata and data lineage.
• Experience with Git, automated testing and CI/CD.
• Experience troubleshooting and supporting production data pipelines.
• Ability to independently design solutions rather than only implement predefined specifications.

Technical Skills

Cloud & Storage

• Google Cloud Platform
• BigQuery
• Google Cloud Storage

Transformation & Orchestration

• Advanced SQL
• Dataform / dbt
• Cloud Composer / Apache Airflow

Data Processing

• Dataflow / Apache Beam
• Dataproc / Spark / PySpark

Programming & Engineering

• Python
• Git
• Automated testing
• CI/CD
• Monitoring and troubleshooting

Data Engineering Capabilities

• ETL / ELT
• Batch processing
• Incremental and idempotent pipelines
• Data modelling
• Data contracts
• Data quality and reconciliation
• Metadata and lineage
• Query performance optimisation
• Cloud-cost optimisation
• Reusable data-platform components

Good to Have

• Experience with Apache Iceberg or modern lakehouse table formats.
• Experience with streaming or event-driven processing.
• Practical understanding of Data Product / Data Mesh principles.
• Experience with metadata-driven or configuration-driven processing.
• Experience with Change Data Capture.
• Familiarity with Terraform / Infrastructure as Code.
• Experience building reusable components used by multiple engineering teams.

Strong Interpersonal Skills

• Strong ownership mindset and ability to take engineering work through production.
• Strong analytical, debugging and problem-solving skills.
• Ability to communicate technical decisions clearly.
• Comfortable participating in design and code reviews.
• Ability to collaborate with engineering and business stakeholders.
• Ability to work independently while seeking guidance for complex architectural decisions.
• Comfortable working within distributed and multicultural teams.