Senior data engineer
Summary
Senior data engineer for a South African payment aggregator (recruited via Optim-G Sourcing), building and maintaining scalable data pipelines and warehouse solutions for financial reporting, fraud detection and analytics. Core stack: SQL Server, PostgreSQL, Python, SQL and Airflow, integrating transactional, accounting and CRM data.
(Client name) a leading South African payment aggregator, enabling secure and efficient
bill payments across a wide network of payment channels. We process millions of
transactions monthly and are building a modern data infrastructure to support
financial reporting, fraud detection, AI, and operational analytics.
We are seeking a skilled Senior Data Engineer to join our growing team. You will be instrumental in designing, building, and maintaining scalable, robust data pipelines
and warehouse solutions that integrate transactional, accounting, and CRM data
across multiple systems including SQL Server, Postgre SQL, Sage One, Xero, and
Dataverse.
thought leadership for the organisation. Deliver fit-for-purpose data products for consumption by advanced analytics. Provide clean, usable data to whomever may require it. Design, construct, install, test and maintain highly scalable data
management systems. Build high-performance prototypes and proof of concepts. Research
opportunities for data acquisition and new uses for existing data. Develop data set processes for data modelling, mining and production. Integrate new data management technologies and software engineering
tools into existing structures and create custom software components (e.g.
specialised UDFs) and analytics applications. Employ a variety of languages and tools (e.g. scripting languages) to marry
systems together. Install and update disaster recovery procedures. Collaborate with data architects, modelers and IT team members on project
goals and recommend ways to improve data reliability, efficiency and
quality. In conjunction with Data Architect, determine which data management
systems are appropriate. Tech Stack Databases & Warehousing: SQL server Postgre SQL Languages: Python, SQL Required Qualifications, Skills & Experience Degree in a quantitative field (e.g. Information Systems, Computer Science,
Mathematics, Statistics, or Finance). 8+ years relevant experience in data engineering or related roles. Proven experience in python-based data engineering projects Strong proficiency in SQL and Python for data transformation and automation. Experience with data orchestration tools (e.g., Airflow). Excellent problem-solving and communication skills.