Data Engineer - AI Platform
Summary
Data engineer designing, building, and maintaining scalable ETL/ELT pipelines, data models, and cloud data infrastructure (AWS/Azure/GCP) to supply high-quality data for the client's AI platform and ML models in Pretoria. Requires 3+ years in data engineering with SQL, Python/Java, and big data tools like Spark and Kafka.
About the Role
Our client is looking for a skilled Data Engineer to build and maintain robust data pipelines for their advanced AI platform in Pretoria . You will be responsible for designing, constructing, installing, testing, and maintaining highly scalable data management systems. Your work will be crucial in ensuring that high-quality data is available for machine learning models and AI applications. This role offers a fantastic opportunity to work with large datasets, modern data technologies, and contribute significantly to the company's data-driven innovation strategy within the bustling capital city of Gauteng .
Key Responsibilities
- Design, build, and maintain scalable and reliable ETL/ELT pipelines for ingesting and processing diverse data sources.
- Develop and optimize data models and database structures to support AI and machine learning workloads.
- Implement data quality checks and monitoring systems to ensure data integrity and accuracy.
- Collaborate with AI/ML engineers and data scientists to understand their data requirements and provide necessary data solutions.
- Manage and optimize data infrastructure on cloud platforms (e.g., AWS, Azure, GCP).
- Ensure data security and compliance with relevant regulations.
Requirements
- Bachelor's degree in Computer Science, Engineering, or a related field.
- 3+ years of experience in data engineering, with a focus on building data pipelines for AI/ML applications.
- Strong proficiency in SQL and programming languages like Python or Java.
- Experience with big data technologies such as Spark, Hadoop, Kafka, and distributed databases.
- Hands-on experience with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP BigQuery).
- Familiarity with data warehousing concepts and data modeling best practices.
Benefits
- Competitive salary and performance incentives.
- Comprehensive medical, dental, and vision insurance.
- Pension fund and other retirement benefits.
- Opportunities for career advancement and skill development in Pretoria .
- A collaborative work environment focused on innovation and data excellence.