GCP Data Engineer
Description
About Position:
We are seeking an experienced GCP Data Engineer to support the migration of a large-scale wholesale banking data mart from on-premises Teradata to Google BigQuery. The ideal candidate will possess strong expertise in Google Cloud Platform, BigQuery, Python, and cloud-native data engineering practices. You will be responsible for designing scalable data pipelines, optimizing data models, enabling secure and compliant data migration, and delivering high-performance analytics solutions within the GCP ecosystem.
- Role: GCP Data Engineer - BigQuery Migration
- Location: Hyderabad, Bengaluru
- Experience: 5 to 10 Years
- Job Type: Full-Time Employment
What You'll Do:
- Design and develop batch and event-driven data pipelines on Google Cloud Platform.
- Build scalable enterprise data warehouses and data marts using BigQuery.
- Lead migration activities from Teradata to BigQuery, including schema conversion, SQL transformation, and validation.
- Develop and maintain workflow orchestration solutions using Cloud Composer (Apache Airflow).
- Implement event-driven architectures using Pub/Sub and Cloud Functions.
- Develop data ingestion frameworks to load data from multiple enterprise source systems into Cloud Storage and BigQuery.
- Optimize BigQuery workloads through partitioning, clustering, materialized views, and query tuning.
- Design and implement data models that support analytics, reporting, and regulatory requirements.
- Develop Python-based utilities and frameworks for automation, data transformation, monitoring, and reconciliation.
- Manage and secure GCP resources using IAM roles, service accounts, and security best practices.
- Implement operational monitoring, logging, alerting, and troubleshooting processes.
- Collaborate with analytics, reporting, business, and technology teams to understand data requirements and deliver solutions.
- Ensure compliance with enterprise security, governance, audit, and regulatory standards.
- Support production deployments, post-migration stabilization, and platform optimization initiatives.
Expertise You'll Bring:
- Bachelor's Degree in Computer Science, Information Technology, Engineering, MCA, BCA, or related discipline.
- 5 to 10 years of experience in Data Engineering, Data Warehousing, or Cloud Platform Engineering.
- Minimum 3+ years of hands-on experience working with Google Cloud Platform (GCP).
- Proven experience building enterprise data platforms using BigQuery.
- Strong hands-on programming experience in Python.
- Strong SQL expertise with experience in complex query development and optimization.
- Experience building and supporting data pipelines using Cloud Composer (Apache Airflow).
- Experience implementing event-driven architectures using Pub/Sub and Cloud Functions.
- Hands-on experience with Cloud Storage, IAM, Service Accounts, and other core GCP services.
- Experience in data migration projects involving Teradata, on-premises platforms, or legacy data warehouses.
- Strong knowledge of data warehousing, dimensional modeling, ETL/ELT frameworks, and data integration concepts.
- Experience implementing cost governance, chargeback reporting, and cloud operational monitoring solutions.
- Understanding of data lineage, metadata management, and data quality frameworks.
- Experience working with Banking, Financial Services, Risk Management, or Regulatory Reporting environments.
- Strong analytical, troubleshooting, and problem-solving capabilities.
- Experience working within Agile delivery environments.
- Excellent communication and stakeholder management skills.
Benefits:
- Competitive salary and benefits package
- Culture focused on talent development with quarterly growth opportunities and company-sponsored higher education and certifications
- Opportunity to work with cutting-edge technologies
- Employee engagement initiatives such as project parties, flexible work hours, and Long Service awards
- Annual health check-ups
- Insurance coverage: group term life, personal accident, and Mediclaim hospitalization for self, spouse, two children, and parents
Values-Driven, People-Centric & Inclusive Work Environment:
Persistent is dedicated to fostering diversity and inclusion in the workplace. We invite applications from all qualified individuals, including those with disabilities, and regardless of gender or gender preference. We welcome diverse candidates from all backgrounds.
- We support hybrid work and flexible hours to fit diverse lifestyles.
- Our office is accessibility-friendly, with ergonomic setups and assistive technologies to support employees with physical disabilities.
- If you are a person with disabilities and have specific requirements, please inform us during the application process or at any time during your employment
Let's unleash your full potential at Persistent - persistent.com/careers
"Persistent is an Equal Opportunity Employer and prohibits discrimination and harassment of any kind."
Skills
- Accessibility
- Agile
- Airflow
- Analytics
- Automation
- BigQuery
- Cloud
- Cloud Composer
- Cloud Native
- Data Engineering
- Data Ingestion
- Data Lineage
- Data Pipelines
- Data Quality
- Data Warehousing
- Dimensional Modeling
- ELT
- ETL
- Event Driven Architecture
- GCP
- IAM
- Metadata Management
- Python
- SQL
- Stakeholder Management
- Teradata
- Workflow Orchestration