Databricks Data Engineer: Delta Lake, Spark & Streaming
Summary
Design and build scalable data pipelines using PySpark, Spark SQL, Kafka/Event Hub, and Delta Lake to deliver clean, analytics-ready data for cybersecurity insights.
DNS Info Ltd in Ireland is building scalable data pipelines to provide curated cyber data for analytics. You will design and implement end-to-end ingestion, streaming, and storage layers, focusing on reliability and cost efficiency.
The role requires strong PySpark/Spark SQL skills, experience with streaming platforms like Kafka or Event Hub, and hands-on work with Delta Lake to create Bronze-Silver-Gold layers for analytics-ready data.