Senior Data Engineer (Databricks)
About the role
We are looking for a Senior Data Engineer with strong Databricks experience to build and optimize modern data platforms. You will develop scalable ETL/ELT pipelines, process large datasets and contribute to the evolution of cloud-native Data Engineering solutions.
About Spyrosoft
Spyrosoft is an authentic, cutting-edge software engineering company, established in 2016. In 2021 and 2022, we were among the fastest growing technology companies in Europe, according to the Financial Times. We were founded by a group of tech experts with established backgrounds in software engineering, who created an ‘engineer-to-engineer’ workplace, powered by enthusiasm, fairness and authentic relationships. Having a unique offering, which bridge the gap between technology and business, we specialise in technology solutions for industry 4.0, automotive, geospatial, healthcare & life sciences, employee experience & education and financial services industries.
- 5+ years of experience in Data Engineering
- 2+ years of commercial experience with Databricks
- Strong knowledge of PySpark and Spark SQL
- Excellent SQL skills
- Strong Python programming skills
- Experience building ETL/ELT pipelines
- Experience working with Delta Lake
- Good understanding of Medallion Architecture
- Experience with cloud environments (Azure, AWS or GCP)
- Experience with Git and CI/CD
- Understanding of data modelling principles
- Good communication skills
- Fluent English - C1
Nice to have:
- Unity Catalog experience
- dbt
- Kafka or Structured Streaming
- Airflow or Databricks Workflows
- Databricks certification
- Experience with performance tuning and cost optimization