Software/ Data Engineer
Posted Updated 4
views
Summary
Builds and maintains data pipelines and analytics for a mining-industry big-data platform, focusing on ingestion, processing, and storage to enable descriptive, diagnostic, predictive, and prescriptive analytics.
Responsible for the requirements, design, development, testing and documentation of new and legacy data ingestion, integration, processing, and storage software for a big data analytics platform for the mining industry which requires a focus on collecting, processing, contextualising, and managing data assets thus turning it into information, and finally leveraging this information to address the 4 pillars of analytics - descriptive, diagnostic, predictive, and prescriptive.
- 2-5 years experience with data lake and warehousing solutions
- 2-5 years experience with either Python, Java, C++, C#, Sparql, SQL databases
- 2-5 years experience with either Apache Kafka (producers/consumers/connectors), Apache NiFi, Apache Spark, Apache ActiveMQ, MQTT, Modbus, REST API
