Data Engineer - Databricks
Summary
Senior Databricks Data Engineer on an initial 6-month freelance contract with a European technology company, building and modernising its data platform: designing scalable pipelines with PySpark, Python and SQL on Delta Lake/Lakehouse, migrating workloads to Databricks, and optimising Spark performance alongside data scientists and cloud engineers.
We’re working with a European technology company undergoing a major data platform transformation and they’re looking for a senior Databricks Data Engineer to join the team on an initial 6-month freelance contract.
Solliciteren op deze functie is eenvoudig. Scroll naar beneden en klik op \'Solliciteren\' om in aanmerking te komen voor deze positie.
You’ll be coming into an environment where the data platform is being actively built and modernised, with Databricks at the centre of the architecture.
What you'll be working on
- Designing and building scalable data pipelines within Databricks
- Developing production-grade workloads using PySpark, Python and SQL
- Working extensively with Delta Lake and Lakehouse architecture
- Supporting the migration of existing workloads into Databricks
- Optimising Spark workloads and improving pipeline performance
- Working with Data Scientists, Analytics Engineers and Cloud Engineers
- Implementing data quality, testing and monitoring
- Contributing to the wider architecture and technical direction of the platform
We're looking for someone who has
- 5+ years of experience in Data Engineering
- Strong hands-on Databricks xgiwjmb experience
- Strong understanding of Delta Lake and Lakehouse architecture
- Experience building and optimising production data pipelines
- Experience working with Azure, AWS or GCP
- Experience working within large-scale data environments
- The ability to operate independently and hit the ground running