Senior Databricks Big Data Engineer (Azure, Delta Lake)
Summary
SoftServe is looking for a Senior Data Engineer to develop and optimize scalable batch and streaming ETL pipelines using the Databricks Lakehouse platform on Azure. The role involves designing data processing solutions with PySpark, Python, and SQL while managing cluster configurations and data governance via Unity Catalog.
SoftServe is seeking an experienced Data Engineer to build scalable data solutions on the Databricks Lakehouse platform in Azure. You will work on batch and streaming ETL pipelines and collaborate with stakeholders to deliver production-ready solutions.
You will design and optimize data processing with PySpark, Python, and SQL, manage Databricks clusters, and implement governance with Unity Catalog. This role emphasizes collaboration and continuous improvement within the team.