Data Engineer - AI Platforms
Summary
Builds and optimizes robust data pipelines and scalable data systems that power the client's AI/ML platforms in Centurion, South Africa. Day to day involves ETL/ELT development, data quality monitoring, storage optimization, and collaborating with data scientists, using SQL, Python/Java, Spark/Hadoop, and cloud data services (AWS, Azure, or GCP).
About the Role
Our client is seeking a dedicated Data Engineer to build and optimize robust data pipelines for their advanced AI platforms in Centurion . You will play a crucial role in ensuring the availability, reliability, and performance of data systems that power our machine learning initiatives. This is an excellent opportunity for a technically proficient individual to work with large-scale datasets and cutting-edge technologies in a collaborative, innovation-driven environment, supporting critical AI development and deployment processes.
Key Responsibilities
- Design, construct, install, test, and maintain highly scalable data management systems and platforms.
- Develop and optimize ETL/ELT processes to efficiently ingest, transform, and prepare data for AI/ML model training and analysis.
- Implement data quality checks and monitoring systems to ensure accuracy and integrity of data.
- Collaborate with data scientists and ML engineers to understand their data needs and provide solutions.
- Manage and optimize data storage solutions, ensuring cost-effectiveness and performance.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 3+ years of experience as a Data Engineer or in a similar role focused on data infrastructure.
- Proficiency in SQL and experience with relational and NoSQL databases.
- Strong programming skills in Python or Java, and experience with big data technologies (e.g., Spark, Hadoop).
- Familiarity with cloud data services (AWS, Azure, or GCP) and data warehousing concepts.
Benefits
- Attractive annual salary with performance bonuses.
- Comprehensive medical aid and retirement fund contributions.
- Dedicated budget for professional development and certifications.
- A stimulating work environment in Centurion with opportunities for career advancement.
- Access to modern infrastructure and cutting-edge data tools.