Senior Databricks Engineer - Spark, Kafka, Stream
Posted Updated
Description
About the Role
Success in this position means crafting high-quality data pipelines that empower decision-making and drive business insights through efficient data processing.
What You'll Do
- Architect and develop robust data pipelines leveraging Databricks and Apache Spark.
- Collaborate with stakeholders to capture data requirements and convert them into actionable technical solutions.
- Enhance current workflow processes for better performance and reliability.
- Diagnose and fix any issues within the data pipelines to maintain data accuracy and accessibility.
- Create thorough documentation outlining the data processes and workflows for future reference.
- Keep abreast of emerging trends in big data technologies and best practices in data engineering.
What We're Looking For
- Deep expertise in Apache Spark and Databricks.
- Proven experience with Kafka for handling real-time data streams.
- Strong SQL skills for effective data querying and manipulation.
- Proficient programming capabilities in Python.
- Familiarity with Snowflake as a solution for data warehousing.
- Sharp analytical skills with meticulous attention to detail.
- Experience working in collaborative environments with cross-functional teams.
Skills
Sign in to read and join the discussion.