Lead Data Engineer
Summary
Lead Data Engineer on a hybrid contract in Toronto, providing end-to-end production support for data ingestion and processing pipelines on GCP (Pub/Sub, Dataflow/Apache Beam, Datastream, Cloud SQL PostgreSQL), managing CDC pipelines with Debezium, handling incidents and RCA, and serving as technical lead for AMS support.
Job Description
Job Description
Role Title: Lead Data Engineer
Location: Toronto- Hybrid
Contract
" 8 years of experience in supporting and managing data engineering pipelines in production environments
" Strong experience in Change Data Capture (CDC) using tools like Debezium and handling file-based ingestion (CSV)
" Hands-on experience with GCP technologies including Pub/Sub, Dataflow (Apache Beam), and Datastream
" Experience working with Cloud SQL (PostgreSQL) environments
" Experience in managing staging and curated data layers for downstream consumption
" Strong understanding of production monitoring, alerting, and incident management processes
" Experience in API-based integrations within data pipelines
" Strong programming experience in Java for data processing and support activities
Roles & Responsibilities
" Provide end-to-end production support for data ingestion and processing pipelines
" Monitor and manage CDC pipelines and batch ingestion processes for multiple source systems
" Handle incidents, perform root cause analysis (RCA), and implement preventive fixes
" Work with GCP Pub/Sub topics, subscriptions, DLQs, and alerting mechanisms for pipeline monitoring
" Support data processing layers including staging and transformed datasets
" Perform production deployments and coordinate testing across lower environments
" Troubleshoot and resolve data, performance, and pipeline failures within SLAs
" Serve as technical lead for AMS support, coordinating day-to-day operations
" Own incident communication, escalation, and stakeholder updates
" Maintain operational runbooks and provide guidance to L2/L3 support teams
" Identify recurring issues and drive continuous improvements for system stability and reliability
Skills
Sign in to read and join the discussion.