Lead Data Engineer - Databricks Platform
Posted Updated 8
views
Summary
Lead a data engineering team to build and scale Databricks-based platforms, set standards, and mentor engineers while driving reliability and innovation.
What we achieve together
- As a Lead Data Engineer, you define and drive the technical evolution of our enterprise data platform, ensuring scalability, reliability, maintainability, and cost efficiency.
- You lead our Databricks modernization journey, driving the migration of existing workloads toward Spark Declarative Pipelines and modern engineering patterns while establishing reusable frameworks and best practices.
- Through hands-on development and team collaboration, you design and implement core platform capabilities, review critical code contributions, and set engineering standards across the data ecosystem.
- You continuously optimize the platform by improving Spark performance, cluster utilization, workload efficiency, and overall Databricks costs, using data-driven insights and engineering best practices.
- As a trusted consultant and technical authority, you support engineering teams and stakeholders in solving complex data challenges, evaluating solution approaches, and translating business requirements into robust technical designs.
- By driving efficient delivery processes, you support sprint planning, backlog refinement, effort estimations and deployment practices while ensuring alignment with product goals and stable environments.
- You actively foster innovation by evaluating new technologies, promoting automation and DevOps practices, and identifying as well as reducing technical debt.
What makes us successful together
- Qualification: You have a degree in computer science, data engineering, information systems, or a comparable field.
- Experience: You bring several years of experience in data engineering and have already taken ownership of technical leadership or senior engineering responsibilities.
- Know-how and skills: You possess deep knowledge of Databricks and Apache Spark, including performance tuning, workload management, Delta Lake, and modern data processing architectures. You have strong programming skills in Python and SQL and apply software engineering principles such as testing, CI/CD, code quality, observability, and automation to data platform development.
- Nice to have: Experience with Spark Declarative Pipelines & Declarative Automation Bundles
- Personality: You are a proactive and empathetic professional who clearly communicates complex technical topics, inspires others, and enjoys working collaboratively in an agile environment toward long-running strategic goals.
- Language skills: You communicate confidently in English.
What you can count on
- Flexible work model: adjustable starting times (7:30 – 9:00AM), hybrid work 70/30, and up to 30 days of remote work from abroad from selected countries.
- Health & Wellness support: private medical care (Medicover), sports card subsidy (Multisport), group insurance (Warta), and ergonomic office spaces.
- Time off & balance: additional paid 2 days off and subsidies for recreation.
- Financial perks: up to 30% discount on company products, support for commuting and eyewear.
- Growth & Development: access to internal training, mentoring programs, self-learning platform for languages (eTutor) and international projects.
- Culture & Community: company events, family friendly activities, and open opportunities within Vaillant Group.