Senior Data Engineer
Summary
Senior Data Engineer role designing scalable data pipelines and warehouse solutions for a payment aggregator using Python, SQL, and SQL Server/PostgreSQL to integrate transactional and financial data.
(Client name) a leading South African payment aggregator, enabling secure and efficient
bill payments across a wide network of payment channels. We process millions of
transactions monthly and are building a modern data infrastructure to support
financial reporting, fraud detection, AI, and operational analytics. Role Overview:
We are seeking a skilled Senior Data Engineer to join our growing team. You will be
instrumental in designing, building, and maintaining scalable, robust data pipelines
and warehouse solutions that integrate transactional, accounting, and CRM data
across multiple systems including SQL Server, PostgreSQL, Sage One, Xero, and
Dataverse.
- Responsible for the preparation of data products and advanced technical
thought leadership for the organisation.
- Deliver fit-for-purpose data products for consumption by advanced analytics.
- Provide clean, usable data to whomever may require it.
- Design, construct, install, test and maintain highly scalable data
management systems.
- Build high-performance prototypes and proof of concepts. Research
opportunities for data acquisition and new uses for existing data.
- Develop data set processes for data modelling, mining and production.
- Integrate new data management technologies and software engineering
tools into existing structures and create custom software components (e.g.
specialised UDFs) and analytics applications.
- Employ a variety of languages and tools (e.g. scripting languages) to marry
systems together.
- Install and update disaster recovery procedures.
- Collaborate with data architects, modelers and IT team members on project
goals and recommend ways to improve data reliability, efficiency and
quality.
- In conjunction with Data Architect, determine which data management
systems are appropriate. Tech Stack
- Databases & Warehousing:
- SQL server
- PostgreSQL
- Languages: Python, SQL Required Qualifications, Skills & Experience
- Degree in a quantitative field (e.g. Information Systems, Computer Science,
Mathematics, Statistics, or Finance).
- 8+ years relevant experience in data engineering or related roles.
- Proven experience in python-based data engineering projects
- Strong proficiency in SQL and Python for data transformation and automation.
- Experience with data orchestration tools (e.g., Airflow).
- Excellent problem-solving and communication skills.
bill payments across a wide network of payment channels. We process millions of
transactions monthly and are building a modern data infrastructure to support
financial reporting, fraud detection, AI, and operational analytics. Role Overview:
We are seeking a skilled Senior Data Engineer to join our growing team. You will be
instrumental in designing, building, and maintaining scalable, robust data pipelines
and warehouse solutions that integrate transactional, accounting, and CRM data
across multiple systems including SQL Server, PostgreSQL, Sage One, Xero, and
Dataverse.
- Responsible for the preparation of data products and advanced technical
thought leadership for the organisation.
- Deliver fit-for-purpose data products for consumption by advanced analytics.
- Provide clean, usable data to whomever may require it.
- Design, construct, install, test and maintain highly scalable data
management systems.
- Build high-performance prototypes and proof of concepts. Research
opportunities for data acquisition and new uses for existing data.
- Develop data set processes for data modelling, mining and production.
- Integrate new data management technologies and software engineering
tools into existing structures and create custom software components (e.g.
specialised UDFs) and analytics applications.
- Employ a variety of languages and tools (e.g. scripting languages) to marry
systems together.
- Install and update disaster recovery procedures.
- Collaborate with data architects, modelers and IT team members on project
goals and recommend ways to improve data reliability, efficiency and
quality.
- In conjunction with Data Architect, determine which data management
systems are appropriate. Tech Stack
- Databases & Warehousing:
- SQL server
- PostgreSQL
- Languages: Python, SQL Required Qualifications, Skills & Experience
- Degree in a quantitative field (e.g. Information Systems, Computer Science,
Mathematics, Statistics, or Finance).
- 8+ years relevant experience in data engineering or related roles.
- Proven experience in python-based data engineering projects
- Strong proficiency in SQL and Python for data transformation and automation.
- Experience with data orchestration tools (e.g., Airflow).
- Excellent problem-solving and communication skills.