Data Engineer

ROLE

We are seeking a Data Engineer (Manager) to support the development, maintenance, and improvement of data pipelines for enterprise data warehouse and AI knowledge base platforms within a banking environment. Working under the guidance of senior engineers, you will help transform structured and unstructured data into reliable, reusable datasets that support use cases such as risk management, customer engagement, fraud detection, and intelligent automation. This is a hands-on role with strong learning opportunities across cloud, streaming, and AI-enabled data engineering.

This role reports to Executive Director, Data Engineering, Group Data Office.

KEY RESPONSIBILITIES

Batch & Streaming Data Pipeline Development

• Develop and support data pipelines feeding data lake/ data warehouse platforms, such as Cloudera, AWS Redshift, Snowflakes, under senior guidance.

• Support the implementation of data models for reusable analytical datasets and reporting

• Monitor data pipelines against established SLAs and assist in troubleshooting data quality and processing issues

• Develop and maintain batch processing jobs using Spark, SQL, Python, or Java

• Assist with real-time streaming pipelines using Flink or similar tools

• Build and support ingestion pipelines from APIs, GA4, files, and other data sources

• Support messaging and streaming integrations using Pub/Sub or Kafka

• Write clean, tested, and maintainable SQL and Python code for data ingestion and processing

Orchestration & Automation

• Develop and support Airflow DAGs (or similar tools) for scheduled and event-driven workflows

• Follow established orchestration, coding, and operational standards

Cloud Infrastructure & DevOps

• Support data workloads on Cloudera, or AWS, with exposure to Docker, Kubernetes ((or similar tools)), or Cloud Run

• Use and support established CI/CD pipelines

• Develop and support simple REST APIs and backend services using Python and Flask

• Support Redis caching for data services where required

AI Knowledge Base & RAG Support

• Assist in developing vector database pipelines and embedding generation jobs

• Support document processing and chunking workflows for AI knowledge bases and RAG pipelines

• Help test and monitor semantic search and retrieval quality for AI-facing data layers

• undefined

Cross‑functional Collaboration

• Work with senior data engineers, AI teams, and business stakeholders across Risk, Finance, Marketing, and Operations

• Assist in translating business requirements into clear technical tasks

• Participate in code reviews, testing, documentation, and continuous improvement activities

REQUIREMENTS

• Bachelor’s degree in computer science, information systems, engineering, or a related field

• At least 2 years of relevant experience in data engineering, data platforms, software engineering, or related roles.

• Hands-on experience building or supporting data pipelines using SQL and Python in a data warehouse, data lake, or cloud environment.

• Basic knowledge of AI knowledge base concepts, such as vector databases, embeddings, or semantic search, is a plus

• Exposure to RAG components, such as document chunking, embedding generation, or retrieval, is a plus

• Exposure to data layers supporting LLM or AI agent use cases is a plus.

• Experience or interest in banking and financial services is a plus

Technical Stack

• Data Warehouse / Platform: basic experience with Cloudera, Redshift, or similar platforms

• Batch Processing: working knowledge of SQL and Python; exposure to Spark, ETL, or MapReduce

• Streaming: exposure to Flink or another real-time processing engine is good to have

• Orchestration: basic experience with Airflow or equivalent tools

• Cloud & Infrastructure: exposure to GCP or AWS; basic knowledge of Docker, or Cloud Run

• DevOps / DataOps: basic exposure to CI/CD

• Backend & Serving: Python, Flask, and REST APIs;

Additional Preferred Experience

• Eagerness to learn data engineering, real-time streaming, cloud, and low-latency system design

• Exposure to LLM applications, RAG, or AI agent concepts is welcome but not required

• A quality-focused mindset and willingness to treat data as a reusable product

• Good communication, teamwork, problem-solving, and willingness to learn