Data Engineer - Ai Platform
Summary
Builds and maintains scalable ETL/ELT pipelines, data models, and cloud data infrastructure (AWS/Azure/GCP) for a client's AI platform in Pretoria, ensuring high-quality data feeds machine learning models. Core stack: SQL, Python/Java, Spark/Kafka, big data and cloud data services.
Our client is looking for a skilled Data Engineer to build and maintain robust data pipelines for their advanced AI platform in Pretoria. You will be responsible for designing, constructing, installing, testing, and maintaining highly scalable data management systems. Your work will be crucial in ensuring that high-quality data is available for machine learning models and AI applications. This role offers a fantastic opportunity to work with large datasets, modern data technologies, and contribute significantly to the company's data-driven innovation strategy within the bustling capital city of Gauteng.
Key Responsibilities Design, build, and maintain scalable and reliable ETL/ELT pipelines for ingesting and processing diverse data sources. Develop and optimize data models and database structures to support AI and machine learning workloads. Implement data quality checks and monitoring systems to ensure data integrity and accuracy. Collaborate with AI/ML engineers and data scientists to understand their data requirements and provide necessary data solutions. Manage and optimize data infrastructure on cloud platforms (e.g., AWS, Azure, GCP). Ensure data security and compliance with relevant regulations. Requirements Bachelor's degree in Computer Science, Engineering, or a related field. 3+ years of experience in data engineering, with a focus on building data pipelines for AI/ML applications. Strong proficiency in SQL and programming languages like Python or Java. Experience with big data technologies such as Spark, Hadoop, Kafka, and distributed databases. Hands-on experience with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Lake, Synapse; GCP Big Query). Familiarity with data warehousing concepts and data modeling best practices. Benefits Competitive salary and performance incentives. Comprehensive medical, dental, and vision insurance. Pension fund and other retirement benefits. Opportunities for career advancement and skill development in Pretoria. A collaborative work environment focused on innovation and data excellence.