AI Data Engineer: Build AI-Ready Pipelines & RAG Systems
Summary
Designs scalable data pipelines and AI-ready systems using Databricks, Spark, and Python to enable retrieval-augmented generation (RAG) and AI capabilities.
Compare the Market, in collaboration with iSelect, is seeking an AI Data Engineer to advance our modern data and AI platform and securely enable AI-enabled capabilities for the business. You will design scalable data pipelines, AI-ready data products, and retrieval systems across batch, streaming and event-driven architectures.
You’ll bring hands-on experience with Databricks, Spark, PySpark, Python and SQL, plus knowledge of DataOps, MLOps, and governance.