Senior Data Engineer (Google Cloud Platform Focus)
Posted
Summary
Senior Data Engineer designing and building scalable, cloud-native data platforms and near-real-time pipelines on Google Cloud Platform. Day to day involves distributed batch/streaming processing (Pub/Sub, Dataflow), Cloud Run and GKE workloads, and performance/cost optimisation using BigQuery, dbt, Python and SQL.
About the Role
We are seeking a highly experienced Senior Data Engineer to design, build, and optimise scalable cloud-native data platforms and near-real-time data pipelines on Google Cloud Platform (GCP). The ideal candidate has deep expertise in highly parallelised distributed processing solutions.
Â
Key Responsibilities
- Design scalable GCP data pipelines.
- Build near-real-time and batch processing solutions.
- Develop highly parallelised distributed pipelines.
- Implement Cloud Run and GKE workloads.
- Design streaming architectures using Pub/Sub and Dataflow.
- Optimise performance, scalability and cost.
Qualifications
- Relevant degree. Google Cloud Professional Data Engineer certification advantageous.
Â
Required Skills Experience
- 8+ years data engineering experience. Expertise in BigQuery, Cloud Storage, Dataflow, Cloud Composer, Cloud Run, GKE, Dataproc, Pub/Sub, dbt, Python and SQL. Strong experience with distributed processing, event-driven architectures and near-real-time pipelines.
Â
Complementary Skills
- Azure, AWS, Power BI, Looker, Apache Spark, Terraform, microservices and event-driven architectures.
Â