Remote Spark Big Data Engineer for AI Training
Summary
Design and optimize Spark-based AI training pipelines using Scala, PySpark, and Spark SQL for scalable data processing and efficient joins.
ixolabs.ai is seeking a Big Data Engineer (Spark) to help design and optimize Spark-based AI training pipelines. You will write and review Spark code across Scala, PySpark, and Spark SQL, focusing on scalable data processing, partitioning, and efficient joins.
Ideal candidates have extensive Spark production experience, strong knowledge of Spark internals, and hands-on with modern table formats and cloud platforms. Flexible, remote, contractor engagement with 10–25 hours per week is offered.