Senior Data Engineer
Summary
Senior data engineer/consultant role supporting banking data delivery programs in Hong Kong: driving GCP data migration and governance work — BigQuery-based validation and reconciliation, GCS/ETL-ELT pipelines with Airflow, Dataflow or Dataproc — plus SIT/UAT coordination, cutover planning and stakeholder management.
Data Engineering, Data Analytics, Data Delivery, Data Governance, Data Migration, Data Quality, Data Lineage, Data Catalogue, SQL, BigQuery, Google Cloud Storage (GCS), Google Cloud Platform (GCP), ETL/ELT, Batch Ingestion, Data Pipelines, Cloud Composer / Airflow, Dataflow or Dataproc, Data Reconciliation, SIT/UAT, Cutover Planning, JIRA / Azure DevOps / Confluence, Power BI / Tableau / Looker
Desired Competencies (Technical/Behavioral Competency) Must-Have- 6-12+ years of experience in data, technology delivery, platform integration, governance or migration initiatives, preferably within banking or financial services.
- Hands-on delivery exposure to GCP data platforms, with strong BigQuery experience for SQL-based validation, reconciliation and data investigation.
- Strong working knowledge of Google Cloud Storage (GCS), lake/storage concepts, file-based feeds, ETL/ELT, batch ingestion, transformation, scheduling and monitoring.
- Practical exposure to at least one GCP pipeline or orchestration component, such as Cloud Composer / Airflow, Dataflow or Dataproc / Spark.
- Experience supporting backlog execution, user stories, acceptance criteria, backlog refinement, release planning and dependency tracking.
- Strong understanding of data governance execution, including critical data elements, glossary/data dictionary, metadata, lineage, data quality rules, audit evidence and traceability.
- Technically credible with engineers and architects, with the ability to challenge, clarify, document decisions and translate requirements into delivery-ready stories and controls.
- Experience coordinating SIT/UAT, defect triage, dress rehearsals, cutover runbooks, hypercare and post-release verification.
- Basic understanding of cloud environments, access/IAM concepts and operational readiness is required.
- Exposure to the Hadoop ecosystem, including Hive, HDFS, Spark, YARN, Impala and related delivery considerations.
- Familiarity with Microsoft Fabric concepts and components, such as OneLake, pipelines, Lakehouse/Warehouse, semantic models and governance integration.
- Exposure to data governance or catalogue tools such as Collibra, Alation or Purview; Python for lightweight data checks or automation; and BI tools such as Power BI, Tableau or Looker.
- Awareness of Hong Kong regulatory expectations and PDPO considerations is an advantage.
- Support the Senior Technical Product Owner / Data Delivery Lead in delivering Hong Kong market-related data changes across backlog execution, governance controls, migration readiness, testing, cutover and stakeholder cadence.
- Translate business and data requirements into delivery-ready user stories, acceptance criteria, decision logs, controls and traceable implementation actions.
- Coordinate data governance deliverables, including critical data elements, glossary/data dictionary, metadata, lineage, data quality rules, audit evidence and traceability.
- Support migration and reconciliation through source-to-target mapping, profiling, validation rules, reconciliation controls, readiness dashboards and sign-off packs.
- Organise SIT/UAT, defect triage, dress rehearsals, cutover runbooks, hypercare and post-release verification while coordinating working-level stakeholders across Business, Operations, IT, Risk/Compliance and vendors.
Skills
- Acceptance Criteria
- Airflow
- Analytics
- Automation
- Azure
- Azure DevOps
- BigQuery
- Cloud
- Confluence
- Data Analytics
- Data Engineering
- Data Governance
- Data Lineage
- Data Pipelines
- Data Quality
- DevOps
- ELT
- ETL
- GCP
- Hadoop
- Hive
- IAM
- Jira
- Lakehouse
- Looker
- Microsoft Fabric
- Power BI
- Python
- Spark
- SQL
- Tableau
- User Stories