Data Engineering Team Lead
Summary
Lead a team building and operating batch and real-time data pipelines on GCP, ensuring data quality, reliability, and alignment with business needs.
You will lead delivery of batch and real-time data pipelines within a GCP consolidation programme. You will coach and manage data engineers, establish data-quality and engineering standards, translate stakeholder needs into deliverables, run sprint cadences, manage risks, and report reliability and delivery metrics.
Responsibilities
- Lead delivery of batch and real-time data pipelines
- Manage squad commitments end to end
- Coach, develop, and performance-manage data engineers
- Embed data contracts, observability, and SLA monitoring
- Translate stakeholder requirements into engineering deliverables
- Run sprint cadences and manage delivery commitments
- Escalate delivery risks early
- Report pipeline SLOs, data freshness, and engineering metrics
Requirements
- GCP data-pipeline design and operations
- BigQuery
- Cloud Composer
- Airflow
- Google Cloud Storage
- Apache Kafka
- Real-time streaming architecture
- Data-engineering leadership or technical mentoring
- Code review
- Design review
- Data contract
- Pipeline observability
- SLA and SLO definition
- Automated alerting and remediation
- SQL
- dbt
- Data ingestion
- Cross-functional communication
- Financial-services regulation
- Sprint cadence
- Change control
- Escalation
Benefits
- Extra time off for volunteering and community work