Lead Data Engineer (Teradata/Hadoop)
Posted Updated
Summary
Lead Data Engineer designing and building large-scale batch, streaming, and API data pipelines on Teradata, Hadoop/Cloudera, and cloud for regulated enterprise banking data. The role covers SQL/Spark performance tuning, Terraform-based IaC with CI/CD, and mentoring junior engineers.
About the Role
We are seeking a Senior Data Engineer with deep expertise in enterprise data platforms, banking domains, and large-scale data delivery. In this role, you will lead the design and development of robust data pipelines across Teradata, Hadoop/Cloudera, and cloud ecosystems while mentoring engineering talent and driving technical excellence.
What you'll do Architecture & Development: Lead the design and implementation of scalable batch, streaming, and API-based data pipelines, historical-data processing frameworks, SCD/CDC mechanisms, and semantic-layer data products. Performance & Optimisation: Optimise complex SQL queries, Spark workloads, partition strategies, and storage frameworks for large-volume processing within regulated enterprise banking environments. Infrastructure & Automation: Implement infrastructure-as-code using Terraform and establish automated CI/CD pipelines, containerised deployments, and rigorous testing standards. Leadership & Collaboration: Drive code reviews, enforce engineering standards, mentor junior engineers, and partner closely with architects, product teams, DevOps, and business stakeholders.
What You Will Need to Succeed Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline At least 10 years of data-engineering experience, with technical leadership in Teradata and Hadoop/big-data delivery. Preferred Certifications: Teradata, Cloudera, Databricks, Cloud Data Engineering, or Terraform Core Technologies: Teradata, Hadoop/Cloudera, Spark, Hive, Impala, HBase, SQL, Python, Scala/Java, Kafka, Airflow, and Denodo. Infrastructure & DevOps: Terraform, Kubernetes/OpenShift, Git, and CI/CD automation tools. Domain Expertise: Enterprise banking data (Customer, Payments, Risk, Finance, AML, and Regulatory Reporting) with a strong focus on data quality, reconciliation, lineage, and historical reconstruction. Core Competencies: Demonstrated technical leadership, disciplined delivery in regulated settings, exceptional communication skills, and accountability for production-ready outcomes.
We are seeking a Senior Data Engineer with deep expertise in enterprise data platforms, banking domains, and large-scale data delivery. In this role, you will lead the design and development of robust data pipelines across Teradata, Hadoop/Cloudera, and cloud ecosystems while mentoring engineering talent and driving technical excellence.
What you'll do Architecture & Development: Lead the design and implementation of scalable batch, streaming, and API-based data pipelines, historical-data processing frameworks, SCD/CDC mechanisms, and semantic-layer data products. Performance & Optimisation: Optimise complex SQL queries, Spark workloads, partition strategies, and storage frameworks for large-volume processing within regulated enterprise banking environments. Infrastructure & Automation: Implement infrastructure-as-code using Terraform and establish automated CI/CD pipelines, containerised deployments, and rigorous testing standards. Leadership & Collaboration: Drive code reviews, enforce engineering standards, mentor junior engineers, and partner closely with architects, product teams, DevOps, and business stakeholders.
What You Will Need to Succeed Bachelor's degree in Computer Science, Engineering, Information Technology, or a related discipline At least 10 years of data-engineering experience, with technical leadership in Teradata and Hadoop/big-data delivery. Preferred Certifications: Teradata, Cloudera, Databricks, Cloud Data Engineering, or Terraform Core Technologies: Teradata, Hadoop/Cloudera, Spark, Hive, Impala, HBase, SQL, Python, Scala/Java, Kafka, Airflow, and Denodo. Infrastructure & DevOps: Terraform, Kubernetes/OpenShift, Git, and CI/CD automation tools. Domain Expertise: Enterprise banking data (Customer, Payments, Risk, Finance, AML, and Regulatory Reporting) with a strong focus on data quality, reconciliation, lineage, and historical reconstruction. Core Competencies: Demonstrated technical leadership, disciplined delivery in regulated settings, exceptional communication skills, and accountability for production-ready outcomes.