data engineer data platform transformation

Описание

The company is a European technology company undergoing a major data platform transformation. Its data platform is being built and modernised with Databricks at the centre of the architecture.

Задачи

• Design and build scalable data pipelines within Databricks
• Develop production-grade workloads using PySpark, Python and SQL
• Work extensively with Delta Lake and Lakehouse architecture
• Support the migration of existing workloads into Databricks
• Optimise Spark workloads and improve pipeline performance
• Build reliable ETL/ELT processes across large datasets
• Work with Data Scientists, Analytics Engineers and Cloud Engineers
• Implement data quality, testing and monitoring
• Contribute to the wider architecture and technical direction of the platform

Требования

• 5+ Years of experience in Data Engineering
• Strong hands-on Databricks experience
• Excellent PySpark, Python and SQL skills
• Strong understanding of Delta Lake and Lakehouse architecture
• Experience building and optimising production data pipelines
• Experience working with Azure, AWS or GCP
• Experience working within large-scale data environments
• Ability to operate independently and hit the ground running

Условия

Initial 6-month freelance contract.