Data Engineer

ROLE We are seeking a Data Engineer (Manager) to support the development, maintenance, and improvement of data pipelines for enterprise data warehouse and AI knowledge base platforms within a banking environment. Working under the guidance of senior engineers, you will help transform structured and unstructured data into reliable, reusable datasets that support use cases such as risk management, customer engagement, fraud detection, and intelligent automation. This is a hands-on role with strong learning opportunities across cloud, streaming, and AI-enabled data engineering. This role reports to Executive Director, Data Engineering, Group Data Office. KEY RESPONSIBILITIES Batch & Streaming Data Pipeline Development Develop and support data pipelines feeding data lake/ data warehouse platforms, such as Cloudera, AWS Redshift, Snowflakes, under senior guidance.

Support the implementation of data models for reusable analytical datasets and reporting

Monitor data pipelines against established SLAs and assist in troubleshooting data quality and processing issues

Develop and maintain batch processing jobs using Spark, SQL, Python, or Java

Assist with real-time streaming pipelines using Flink or similar tools

Build and support ingestion pipelines from APIs, GA4, files, and other data sources

Support messaging and streaming integrations using Pub/Sub or Kafka

Write clean, tested, and maintainable SQL and Python code for data ingestion and processing

Orchestration & Automation Develop and support Airflow DAGs (or similar tools) for scheduled and event-driven workflows

Follow established orchestration, coding, and operational standards

Cloud Infrastructure & DevOps Support data workloads on Cloudera, or AWS, with exposure to Docker, Kubernetes ((or similar tools)), or Cloud Run

Use and support established CI/CD pipelines

Develop and support simple REST APIs and backend services using Python and Flask

Support Redis caching for data services where required

AI Knowledge Base & RAG Support Assist in developing vector database pipelines and embedding generation jobs

Support document processing and chunking workflows for AI knowledge bases and RAG pipelines

Help test and monitor semantic search and retrieval quality for AI-facing data layers

undefined

Cross‑functional Collaboration Work with senior data engineers, AI teams, and business stakeholders across Risk, Finance, Marketing, and Operations

Assist in translating business requirements into clear technical tasks

Participate in code reviews, testing, documentation, and continuous improvement activities

REQUIREMENTS Bachelor's degree in computer science, information systems, engineering, or a related field

At least 2 years of relevant experience in data engineering, data platforms, software engineering, or related roles.

Hands-on experience building or supporting data pipelines using SQL and Python in a data warehouse, data lake, or cloud environment.

Basic knowledge of AI knowledge base concepts, such as vector databases, embeddings, or semantic search, is a plus

Exposure to RAG components, such as document chunking, embedding generation, or retrieval, is a plus

Exposure to data layers supporting LLM or AI agent use cases is a plus.

Experience or interest in banking and financial services is a plus

Technical Stack Data Warehouse / Platform: basic experience with Cloudera, Redshift, or similar platforms

Batch Processing: working knowledge of SQL and Python exposure to Spark, ETL, or MapReduce

Streaming: exposure to Flink or another real-time processing engine is good to have

Orchestration: basic experience with Airflow or equivalent tools

Cloud & Infrastructure: exposure to GCP or AWS basic knowledge of Docker, or Cloud Run

DevOps / DataOps: basic exposure to CI/CD

Backend & Serving: Python, Flask, and REST APIs

Additional Preferred Experience Eagerness to learn data engineering, real-time streaming, cloud, and low-latency system design

Exposure to LLM applications, RAG, or AI agent concepts is welcome but not required

A quality-focused mindset and willingness to treat data as a reusable product

Good communication, teamwork, problem-solving, and willingness to learn