Data Engineer
NewBe an early applicantSummary
A data engineer supporting Spark-based data ingestion, ETL development, and integration between Spark and Elasticsearch/OpenSearch. The role involves working with Quantexa architects, contributing to Spark execution decisions, and preparing demonstration datasets.
Responsibilities
- Participate in technical workshops with Quantexa architects.
- Support decisions around Spark execution approaches.
- Support data ingestion and ETL development activities.
- Assist with loading and preparing demonstration datasets.
- Support integration between Spark processing and Elasticsearch/OpenSearch.
Requirements
- Strong experience in Apache Spark.
- Experience in ETL development, data integration, data transformation, and data modelling.
- Experience with Elasticsearch / OpenSearch is preferred.
- Proficiency in Python and/or Scala is preferred.