data engineer banking platforms
Posted 1
view
Summary
Data engineer building batch and streaming data pipelines on Google Cloud Platform (BigQuery, Dataflow, Composer, DBT) for a banking-sector data platform project. Day to day involves Python/SQL development, automating deployments with Terraform and GitHub, and testing/monitoring solutions, with office visits 1-2 times per week in Warsaw on a B2B hourly contract.
Описание
The company provides IT services and consulting, with this project focused on banking and the development of innovative data platforms in Google Cloud Platform.
Задачи
- Develop and maintain data processing solutions in the Google Cloud Platform (GCP) environment;
- Create batch and streaming data pipelines using BigQuery, DBT, Dataflow, Composer, and Cloud Storage;
- Program in Python and SQL;
- Build and optimize data structures supporting analytics and reporting;
- Automate deployments using Terraform and GitHub repositories;
- Test, monitor, and maintain developed solutions;
- Collaborate with analysts, architects, and project teams to deliver business requirements.
Требования
- Experience designing and building cloud solutions on Google Cloud Platform (GCP);
- Very good knowledge of Python and SQL;
- Knowledge of Dataproc, Composer (Airflow), Cloud Storage, and BigQuery;
- Experience working with GitHub code repositories;
- Experience managing application artifacts using Google Artifact Registry;
- Nice to have: knowledge of DBT, knowledge of Terraform.
Условия
Rate: do approx. 158 zł/h net + VAT, B2B; Office visits 1-2 times per week in Warsaw.