Senior Data Engineer — Azure Lakehouse & Databricks Expert
Posted Updated
Summary
Senior data engineer building and optimizing scalable data platforms on Azure Synapse, Databricks, and Lakehouse: ETL/ELT pipelines in Python, SQL, and Apache Spark, plus Delta Live Tables, Unity Catalog, CI/CD, and production support. Hired through recruitment firm DCG for a partner, based in Poland.
As a recruitment company, DCG understands that every business is powered by experienced professionals. Our management style and partnership approach enable us to meet your needs and provide continuous support. Due to our ongoing growth and the large number of recruitment projects we undertake for our partners, we are currently looking for:
Responsibilities:
- Design, develop, and maintain scalable data solutions across Azure Synapse, Azure Databricks, and Data Lakehouse environments
- Implement and optimize ETL/ELT processes for ingesting, transforming, and serving data across multiple business domains
- Develop and maintain data processing solutions using Python, SQL, and Apache Spark
- Support migration and modernization initiatives, including Unity Catalog adoption, platform upgrades, and data platform transformations
- Build and maintain Databricks workflows, Delta Live Tables (DLT), metadata-driven frameworks, and orchestration processes
- Collaborate with architects, analysts, and business stakeholders to translate business requirements into technical solutions
- Ensure data quality, reliability, performance, and security across the data platform
- Troubleshoot production incidents, perform root cause analysis, and implement corrective actions
- Contribute to CI/CD, infrastructure automation, monitoring, logging, and operational best practices
- Optimize data pipelines, storage structures, and processing workloads for efficiency and cost reduction
- Participate in solution and architecture reviews while promoting engineering standards and governance principles
- Support knowledge sharing, mentoring, and technical development within the team
- Collaborate with Data Architects and platform teams to align solutions with enterprise data architecture, Lakehouse principles, and governance requirements
- Support reporting, analytics, and AI/ML initiatives by providing reliable and scalable data foundations
Requirements:
- Strong experience with Apache Spark, Python, and SQL
- Proven experience in building and maintaining ETL/ELT processes
- Practical experience with Databricks, Delta Live Tables (DLT), and Unity Catalog
- Understanding of Lakehouse architecture, data governance, security, compliance, and operational excellence
- Familiarity with CI/CD, Azure DevOps, infrastructure automation, monitoring, and logging
- Strong analytical and problem-solving skills
- Ability to work independently with a high level of ownership and accountability
- Proactive approach to identifying risks, blockers, and improvement opportunities
- Strong communication and stakeholder management skills across technical and business teams
- Experience working in dynamic environments with frequently changing priorities and cross-functional teams
- Ability to make pragmatic decisions and deliver results despite incomplete requirements
- Adaptability, resilience, self-organization, and ability to independently drive initiatives and propose practical solutions
- English proficiency at minimum B2+ level
Offer:
- Private medical care
- Co-financing for the sports card
- Constant support of dedicated consultant