Data engineer - ai platforms
Summary
Builds and maintains data pipelines, warehouses, and data lake infrastructure powering a client's AI/ML models and analytics in Rustenburg, working closely with data scientists. Core stack: SQL, Python, Airflow, Spark/Hadoop, and AWS or Azure cloud data services; hired via staffing firm Placements24.
Our client is seeking a skilled Data Engineer to build and maintain robust data infrastructure for their AI initiatives in Rustenburg. This role is crucial for ensuring the availability, quality, and accessibility of data that powers their advanced machine learning models and analytics platforms. You will work closely with data scientists and ML engineers to design efficient data pipelines and storage solutions. This position offers a chance to make a significant impact on the company's data strategy and contribute to the scaling of its AI capabilities.
Key Responsibilities Design, build, and maintain scalable and reliable data pipelines for AI/ML workloads. Develop and manage data warehousing solutions and data lake architectures. Implement data quality checks and ensure data integrity and consistency. Optimize data processing performance and cost efficiency. Collaborate with cross-functional teams to understand data needs and provide solutions. Requirements Bachelor's degree in Computer Science, Engineering, or a related field. 3+ years of experience in data engineering, with a focus on big data technologies. Proficiency in SQL, Python, and data pipeline orchestration tools (e.g., Airflow). Experience with distributed data processing frameworks like Spark or Hadoop. Familiarity with cloud data services (e.g., AWS S3, Redshift, Glue; Azure Data Factory, Synapse). Benefits Competitive salary and annual bonus opportunities. Comprehensive health, disability, and life insurance. Pension fund contribution and retirement planning support. Professional development programs and training opportunities. A collaborative and innovative work environment in Rustenburg.