Senior Data Engineer

Key Responsibilities

1. Define and own enterprise data warehouse (EDW) architecture across Bronze, Silver and Gold layers.
2. Design and develop scalable, metadata-driven ETL/ELT frameworks and reusable engineering accelerators.
3. Lead large-scale data integration and pipeline development across Teradata, Hadoop/Cloudera and Spark platforms.
4. Establish engineering standards for data modelling, SCD, CDC, historical data management and data recovery.
5. Implement data lineage, data quality, reconciliation, auditability and governance controls.
6. Optimize complex SQL queries, Spark workloads, MPP architecture, partitioning and performance.
7. Develop enterprise banking data solutions, payment harmonisation and business data products.
8. Automate data engineering deployment and infrastructure using Terraform and CI/CD pipelines.
9. Lead technical design reviews, code reviews and architecture decisions to ensure scalable solutions.
10. Mentor engineering teams and collaborate with architects, business analysts, testing, DevOps and operations teams.

Requirements

1. Bachelor's degree in Computer Science, Information Technology, Engineering or a related discipline.
2. Minimum 10+ years of experience in Data Engineering, including at least 5 years in enterprise data warehouse architecture and development.
3. Strong hands-on experience with Teradata or other MPP databases such as Netezza, Greenplum, Exadata or IBM Db2 Warehouse.
4. Strong understanding of enterprise EDW architecture, including Bronze, Silver and Gold layered architecture.
5. Hands-on experience with Informatica and enterprise data integration frameworks.
6. Strong technical knowledge of Hadoop, Cloudera, Spark, Hive, Impala and advanced SQL.
7. Strong programming skills in Python and/or Java/Scala, with experience developing reusable engineering frameworks.
8. Experience in metadata-driven ETL automation, data modelling, SCD, CDC, data quality and reconciliation frameworks.
9. Banking or financial services domain experience, preferably with Teradata FSLDM, IBM BDW or equivalent banking data models.
10. Exposure to Databricks, Kafka, Airflow, Terraform, Kubernetes, Git and CI/CD pipelines will be an added advantage.