Data Engineer – Banking (CIB)

Job Summary

BluePearl Consulting is seeking a strong, experienced Data Engineer to join a software and data engineering team within a Corporate and Investment Banking (CIB) environment.

The successful candidate will specialise in designing, developing and maintaining scalable data pipelines, data integration solutions and cloud-based data platforms. The role requires strong SQL and Python engineering skills, ETL/ELT expertise, data modelling capabilities and hands-on cloud data engineering experience, with AWS as the core platform.

The ideal candidate is a proactive self-starter who takes ownership of deliverables, solves problems independently and maintains high standards of data quality, security and engineering discipline.

Key Responsibilities

• Design, develop and maintain scalable ETL/ELT pipelines and data ingestion processes.
  

• Write, optimise and troubleshoot advanced SQL queries, including CTEs and window functions.
  

• Develop reusable Python modules, libraries and automated tests.
  

• Build and maintain relational, dimensional and analytical data models.
  

• Implement data transformations, mappings, validation and reconciliation processes.
  

• Develop and maintain data pipelines using AWS data services.
  

• Implement data orchestration and workflow automation using appropriate tools.
  

• Work with distributed processing technologies such as Spark and PySpark where required.
  

• Support data warehousing, data lakes and lakehouse architectures.
  

• Implement data quality rules, exception handling and source-to-target reconciliation.
  

• Apply data security, access control, encryption and privacy practices.
  

• Implement logging, monitoring, alerting and data observability.
  

• Contribute to Git-based version control, CI/CD and automated deployment pipelines.
  

• Develop and maintain unit, integration, regression and reconciliation tests.
  

• Troubleshoot production incidents, conduct root-cause analysis and support operational recovery.
  

• Maintain technical documentation, data mappings, designs and operational runbooks.
  

• Collaborate with software engineers, data engineers and business stakeholders.
  

Requirements

Non-Negotiable Requirements

• Relevant banking or financial services experience.
  

• Strong hands-on Data Engineering experience.
  

• Advanced SQL engineering, including query optimisation, CTEs, window functions and query plans.
  

• Strong Python engineering, including reusable modules, libraries and testing.
  

• Practical ETL/ELT, data ingestion and transformation experience.
  

• Data modelling experience across relational, dimensional or analytical models.
  

• Cloud data engineering experience, with AWS as the core platform requirement.
  

• Experience with data quality, validation and reconciliation.
  

• Knowledge of data security, access control and privacy.
  

• Familiarity with Git, CI/CD, automated testing and production support.
  

• Strong troubleshooting, problem-solving and technical documentation skills.
  

• Ability to work independently and take ownership of deliverables.
  

• Excellent communication and stakeholder management skills.
  

• Availability to work onsite in Rosebank approximately 3 days per week.
  

• Availability to commence on 1 January 2027.
  

Core Technical Skills

• AWS
  

• Advanced SQL
  

• Python
  

• ETL/ELT
  

• Data ingestion and transformation
  

• Data modelling
  

• Data quality and reconciliation
  

• Data orchestration
  

• Git and CI/CD
  

• Automated testing
  

• Monitoring and production support
  

Role-Critical Technologies

• AWS S3, Glue, Lambda, Athena, Redshift and EMR.
  

• Spark and PySpark.
  

• Kafka and event-driven processing.
  

• Data warehousing, data lakes and lakehouse engineering.
  

• Delta Lake and medallion architecture patterns.
  

• SQS, SNS and EventBridge.
  

• Airflow/MWAA and Step Functions.
  

• Terraform and Infrastructure as Code.
  

• Metadata, data catalogues and lineage.
  

• Performance optimisation, partitioning and query tuning.
  

Desirable Capabilities

• Batch processing and workflow scheduling.
  

• Data governance and data classification.
  

• Cloud cost optimisation and FinOps awareness.
  

• Legacy ETL modernisation and cloud migration.
  

• Data products, interfaces and service-level agreements/objectives.
  

• Data mesh and federated governance.
  

Core Attributes

• Strong banking or financial services awareness.
  

• Self-starter who requires minimal supervision.
  

• Takes ownership of responsibilities and deliverables.
  

• Proactive and analytical problem-solver.
  

• Excellent written and verbal communication.
  

• Strong attention to detail and data quality.
  

• Professional stakeholder engagement.
  

• Smart and professional presentation appropriate to a banking environment.
  

Recruitment priority: A strong, hands-on Data Engineer with banking experience, advanced SQL and Python capability, AWS expertise and a proven ability to deliver reliable data solutions independently.