Data Engineer - AI/ML Platforms
Summary
Builds and maintains scalable data pipelines, ETL/ELT processes, and data warehousing/lake infrastructure that feed AI and machine learning model training and deployment, working closely with data scientists and ML engineers. Core stack: SQL, Python/Scala, Spark/Hadoop, and cloud data services (AWS/Azure/GCP). On-site in Paarl, South Africa.
About the Role
Our client is seeking a highly skilled Data Engineer to build and maintain robust data pipelines and infrastructure supporting their AI and Machine Learning initiatives in Paarl . This role is critical in ensuring that data is accessible, reliable, and optimized for AI model training and deployment. You will work closely with data scientists and ML engineers to understand their data needs and provide high-quality data solutions. This on-site position offers the opportunity to work with cutting-edge data technologies and contribute significantly to the company's data-driven strategy within a collaborative environment.
Key Responsibilities
- Design, build, and maintain scalable and reliable data pipelines for AI/ML workloads.
- Develop ETL/ELT processes to ingest, transform, and store data from various sources.
- Optimize data infrastructure for performance, efficiency, and cost-effectiveness.
- Collaborate with data scientists and ML engineers to provide clean, structured data for model development.
- Implement data quality checks and monitoring to ensure data accuracy and integrity.
- Manage and maintain data warehousing solutions and data lakes.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related quantitative field.
- 3+ years of experience in data engineering, with a focus on big data technologies.
- Proficiency in SQL and experience with data warehousing concepts.
- Experience with cloud platforms (AWS, Azure, GCP) and their data services.
- Strong programming skills in Python or Scala.
- Familiarity with big data frameworks like Spark, Hadoop, and related tools.
- Understanding of the challenges and requirements for AI/ML data pipelines.
Benefits
- Competitive salary and comprehensive benefits package.
- Opportunity to work with modern data stack and AI technologies.
- Professional development and training opportunities.
- Be part of a key team supporting AI advancements in Paarl .
- Work in a collaborative and innovative company culture.