Senior Data Engineer Python | Hadoop | AI | Big Data
About the Company
We are seeking highly skilled Data Engineers with strong expertise in Python, Hadoop, AI, and Big Data technologies to join a leading banking client's enterprise Data & AI transformation programme.
About the Role The ideal candidate will have extensive experience designing and building enterprise-scale Data Lakes, Data Warehouses, Big Data platforms, and AI-ready data pipelines that support Machine Learning, Generative AI, Advanced Analytics, and Regulatory Reporting. If you enjoy working on large-scale data platforms and AI-driven solutions in the banking industry, we'd love to hear from you. Responsibilities
Design, develop, and maintain scalable enterprise Data Pipelines and ETL/ELT processes. Build high-performance Big Data solutions using Python, Hadoop, Spark, and distributed computing technologies. Develop and optimise Enterprise Data Warehouse, Data Lake, and Lakehouse platforms. Build AI-ready data platforms to support Machine Learning, Generative AI, LLM, and Advanced Analytics use cases. Process and optimise structured, semi-structured, and unstructured datasets. Design reusable data frameworks, APIs, and data integration components. Collaborate closely with Data Scientists, AI Engineers, Data Architects, Business Analysts, and Product Owners. Implement data governance, metadata management, data quality, lineage, and security controls. Optimise data processing performance and troubleshoot production issues. Participate in DevOps, CI/CD, automation, and code reviews.
Qualifications
Bachelor's Degree in Computer Science, Engineering, Information Technology, or a related discipline. Banking or Financial Services experience is mandatory. Proven experience delivering enterprise Data Engineering, AI, Big Data, and Data Warehouse solutions. Strong understanding of enterprise data architecture, data governance, and security. Experience working in Agile/Scrum delivery environments. Excellent analytical, problem-solving, and communication skills.
Required Skills
4–10 years of Data Engineering experience Strong hands-on experience with Python Strong experience with the Hadoop ecosystem (HDFS, Hive, YARN, MapReduce) Apache Spark / PySpark SQL and advanced database development ETL/ELT pipeline development Enterprise Data Warehouse Data Lake / Lakehouse architecture Big Data technologies Linux / Unix Git Preferred Skills Apache Kafka Apache Airflow Databricks Snowflake Delta Lake Docker Kubernetes REST APIs CI/CD pipelines AWS, Azure, or Google Cloud Platform (GCP) MLflow LangChain or LlamaIndex Vector Databases (Pinecone, Chroma, Weaviate, FAISS) MLOps
EA License No. 01C4394 • EA Registration No. R1113321 (Jacob Tijo)
We are seeking highly skilled Data Engineers with strong expertise in Python, Hadoop, AI, and Big Data technologies to join a leading banking client's enterprise Data & AI transformation programme.
About the Role The ideal candidate will have extensive experience designing and building enterprise-scale Data Lakes, Data Warehouses, Big Data platforms, and AI-ready data pipelines that support Machine Learning, Generative AI, Advanced Analytics, and Regulatory Reporting. If you enjoy working on large-scale data platforms and AI-driven solutions in the banking industry, we'd love to hear from you. Responsibilities
Design, develop, and maintain scalable enterprise Data Pipelines and ETL/ELT processes. Build high-performance Big Data solutions using Python, Hadoop, Spark, and distributed computing technologies. Develop and optimise Enterprise Data Warehouse, Data Lake, and Lakehouse platforms. Build AI-ready data platforms to support Machine Learning, Generative AI, LLM, and Advanced Analytics use cases. Process and optimise structured, semi-structured, and unstructured datasets. Design reusable data frameworks, APIs, and data integration components. Collaborate closely with Data Scientists, AI Engineers, Data Architects, Business Analysts, and Product Owners. Implement data governance, metadata management, data quality, lineage, and security controls. Optimise data processing performance and troubleshoot production issues. Participate in DevOps, CI/CD, automation, and code reviews.
Qualifications
Bachelor's Degree in Computer Science, Engineering, Information Technology, or a related discipline. Banking or Financial Services experience is mandatory. Proven experience delivering enterprise Data Engineering, AI, Big Data, and Data Warehouse solutions. Strong understanding of enterprise data architecture, data governance, and security. Experience working in Agile/Scrum delivery environments. Excellent analytical, problem-solving, and communication skills.
Required Skills
4–10 years of Data Engineering experience Strong hands-on experience with Python Strong experience with the Hadoop ecosystem (HDFS, Hive, YARN, MapReduce) Apache Spark / PySpark SQL and advanced database development ETL/ELT pipeline development Enterprise Data Warehouse Data Lake / Lakehouse architecture Big Data technologies Linux / Unix Git Preferred Skills Apache Kafka Apache Airflow Databricks Snowflake Delta Lake Docker Kubernetes REST APIs CI/CD pipelines AWS, Azure, or Google Cloud Platform (GCP) MLflow LangChain or LlamaIndex Vector Databases (Pinecone, Chroma, Weaviate, FAISS) MLOps
EA License No. 01C4394 • EA Registration No. R1113321 (Jacob Tijo)