Senior Data Engineer (Databricks) @ Spyrosoft
Summary
Build and optimize scalable ETL/ELT pipelines on Databricks, processing large datasets with PySpark and Delta Lake for modern data platforms.
We are looking for a Senior Data Engineer with strong Databricks experience to build and optimize modern data platforms. You will develop scalable ETL/ELT pipelines, process large datasets and contribute to the evolution of cloud-native Data Engineering solutions.
About SpyrosoftSpyrosoft is an authentic, cutting-edge software engineering company, established in 2016. In 2021 and 2022, we were among the fastest growing technology companies in Europe, according to the Financial Times. We were founded by a group of tech experts with established backgrounds in software engineering, who created an ‘engineer-to-engineer’ workplace, powered by enthusiasm, fairness and authentic relationships. Having a unique offering, which bridge the gap between technology and business, we specialise in technology solutions for industry 4.0, automotive, geospatial, healthcare & life sciences, employee experience & education and financial services industries.
- 5+ years of experience in Data Engineering
- 2+ years of commercial experience with Databricks
- Strong knowledge of PySpark and Spark SQL
- Excellent SQL skills
- Strong Python programming skills
- Experience building ETL/ELT pipelines
- Experience working with Delta Lake
- Good understanding of Medallion Architecture
- Experience with cloud environments (Azure, AWS or GCP)
- Experience with Git and CI/CD
- Understanding of data modelling principles
- Good communication skills
- Fluent English - C1
- Unity Catalog experience
- dbt
- Kafka or Structured Streaming
- Airflow or Databricks Workflows
- Databricks certification
- Experience with performance tuning and cost optimization
We are looking for a Senior Data Engineer with strong Databricks experience to build and optimize modern data platforms. You will develop scalable ETL/ELT pipelines, process large datasets and contribute to the evolution of cloud-native Data Engineering solutions.
About SpyrosoftSpyrosoft is an authentic, cutting-edge software engineering company, established in 2016. In 2021 and 2022, we were among the fastest growing technology companies in Europe, according to the Financial Times. We were founded by a group of tech experts with established backgrounds in software engineering, who created an ‘engineer-to-engineer’ workplace, powered by enthusiasm, fairness and authentic relationships. Having a unique offering, which bridge the gap between technology and business, we specialise in technology solutions for industry 4.0, automotive, geospatial, healthcare & life sciences, employee experience & education and financial services industries.
,(Design, develop and maintain scalable ETL/ELT pipelines, Build reliable batch and streaming data processing solutions, Develop data pipelines using Databricks and Apache Spark, Optimize Spark jobs and improve platform performance, Work with Delta Lake and Medallion Architecture, Build and maintain data models for analytics and reporting, Ensure data quality, monitoring and observability, Collaborate with Architects, Data Scientists and Analytics teams, Participate in code reviews and technical improvements, Contribute to CI/CD processes and engineering best practices) Requirements: Data engineering, PySpark, Spark, SQL, Python, ETL, Cloud, Azure, AWS, Git, Communication skills, Databricks, Performance tuning