Fabric Data Engineer — Pipelines, Warehousing & Bi
Summary
On-site data engineer in Centurion (Pretoria) who designs, builds, and maintains scalable data pipelines and ETL/ELT processes to feed machine learning and AI applications, working closely with data scientists and ML engineers. Core stack: SQL, Python/Scala, Spark, Hadoop, Kafka, and cloud data services (AWS, Azure, GCP).
Our client is seeking a meticulous and experienced Data Engineer to build and optimize robust data pipelines for their AI & Emerging Technologies initiatives in Centurion . In this critical role, you will be responsible for ensuring the availability, quality, and accessibility of data required for machine learning models and AI-driven applications. You will work closely with data scientists and ML engineers to architect scalable data solutions, fostering a data-centric culture. This on-site position provides a unique opportunity to work with a dedicated team in a collaborative and innovative environment.
About the Role
Our client is seeking a meticulous and experienced Data Engineer to build and optimize robust data pipelines for their AI & Emerging Technologies initiatives in Centurion . In this critical role, you will be responsible for ensuring the availability, quality, and accessibility of data required for machine learning models and AI-driven applications. You will work closely with data scientists and ML engineers to architect scalable data solutions, fostering a data-centric culture. This on-site position provides a unique opportunity to work with a dedicated team in a collaborative and innovative environment.
Key Responsibilities- Design, construct, install, test, and maintain highly scalable data management systems and pipelines.
- Develop processes and systems to monitor data quality, ensure consistency, and manage data flow.
- Build and manage ETL/ELT processes for large, diverse datasets.
- Optimize data infrastructure for performance, reliability, and cost-effectiveness.
- Collaborate with data scientists and ML engineers to understand their data requirements and provide solutions.
- Implement data security and governance best practices.
- Bachelor's or Master's degree in Computer Science, Engineering, or a related field.
- 3+ years of experience in data engineering, with a focus on building large-scale data pipelines.
- Proficiency in SQL and programming languages like Python or Scala.
- Experience with big data technologies such as Spark, Hadoop, Kafka, and distributed storage systems.
- Hands-on experience with cloud data services (AWS, Azure, GCP).
- Strong understanding of data warehousing concepts and database design.
- Competitive annual salary and performance incentives.
- On-site work environment with access to advanced facilities.
- Comprehensive medical, dental, and vision insurance plans.
- Opportunities for professional development and certifications.
- Collaborative team atmosphere focused on innovation and growth.