Real-Time Data Engineer: Kafka, Spark & Cloud Platforms
Summary
Design and implement real-time data streaming solutions using Apache Kafka and Spark Structured Streaming, with Python/Scala development on Databricks in a cloud environment (Azure or AWS). Day-to-day work involves building ingestion pipelines, managing messaging systems like Kafka/MSK, and troubleshooting Spark workloads.
Aptonet is seeking an experienced data engineering professional to design and implement high-velocity data streaming solutions using Apache Kafka and Spark Streaming. You will develop real-time processing with Spark Structured Streaming, troubleshoot Spark workloads, and work with Python/Scala on Databricks in a cloud environment.
Responsibilities include building ingestion pipelines, deploying data platforms on Azure or AWS, managing messaging systems like Kafka/MSK, and handling data formats