Data Engineer
A leading financial services organisation is seeking a Data Engineer to join its growing technology team. This role is ideal for a technically strong professional who enjoys designing and optimising data pipelines, leveraging cloud technologies, and contributing to the development of robust data platforms. You will play a key role in building and maintaining modern data solutions while working closely with cross-functional teams to ensure the delivery of reliable, scalable, and high-quality data products. The environment encourages learning, collaboration, and the adoption of best practices across the data engineering landscape. This is an excellent opportunity to be part of a business that combines technology, data, and financial services to create innovative solutions that positively impact consumers across South Africa.
Key Responsibilities
- Build, maintain, and optimise data pipelines and workflows.
- Develop scalable data solutions using Databricks and Apache Spark.
- Design and implement end-to-end data flows and integration processes.
- Contribute to the development and maintenance of CI/CD pipelines.
- Collaborate with developers, analysts, and business stakeholders to deliver high-quality data products.
- Ensure data platforms are reliable, efficient, and aligned with business requirements.
- Support the implementation of modern data architecture principles.
- Continuously evaluate and adopt best practices in data engineering and cloud technologies.
Job Experience and Skills Required
- Degree or diploma in Computer Science, Information Technology, Engineering, Data Science, or a related field.
- 3 to 5 years of experience in Data Engineering.
- Previous experience building and maintaining enterprise data solutions.
- Experience working with Databricks in a production environment.
- Strong background in Apache Spark and distributed computing.
- Experience developing and supporting CI/CD pipelines.
- Previous backend development experience.
- Exposure to cloud-based data platforms and services.
- Advanced Python skills with hands-on PySpark experience.
- Strong SQL development and optimisation experience.
- Experience with Azure cloud services.
- Understanding of data modelling and modern data architecture principles.
- Experience with Microsoft Fabric and Medallion Architecture.
- Knowledge of data integration, transformation, and orchestration processes.
- Familiarity with software engineering best practices and version control.