DigiTribe - Data Engineer (Azure Databricks & Application Development)
Summary
Build and maintain scalable data pipelines and applications on Azure Databricks using Python, PySpark, and SQL, while collaborating with cross-functional teams to deliver robust, cost-efficient data solutions.
Data Engineer - Azure Databricks & application development
Position overview
Digitribe is looking for a Data Engineer with deep expertise in Azure and Databricks to design, develop, and maintain scalable data engineering solutions. You will build production-grade ETL/ELT pipelines, develop data processing applications, and contribute to a modern lakehouse architecture. Working cross-functionally with Architecture, Data, AI, Cloud, and CI/CD teams, you will translate business and analytical requirements into robust, reusable, and cost-efficient data solutions.
Key responsibilities
- Design, develop, test, and maintain scalable data engineering solutions on Microsoft Azure and Databricks, using Python, PySpark, Scala, and SQL
- Implement reliable batch and streaming data workflows using Azure Databricks, Delta Lake, and modern data lakehouse patterns
- Develop data processing applications for ingestion, transformation, enrichment, validation, and serving of analytical datasets
- Build CI/CD pipelines for data applications using Azure DevOps YAML pipelines and apply software engineering best practices: modular design, clean code, Version Control, code reviews, and documentation
- Develop reusable libraries, utilities, and frameworks to accelerate data product delivery and support less experienced engineers through coaching and knowledge sharing
- Collaborate with Cloud, DevOps, and Platform teams on deployment, environment configuration, and operational readiness
Skills & qualifications
- Technical skills:
- 5+ years of hands-on experience in data engineering, application development, or cloud-based data platform development
- Strong hands-on development experience with Python, PySpark, Scala, SQL, and ETL/ELT development
- Solid experience with the Azure Databricks ecosystem: notebooks, jobs, Spark clusters, Delta Lake, lakehouse architecture, and performance tuning
- Good knowledge of Azure data services: Azure Data Lake Storage, Azure Data Factory, Azure Key Vault, Azure DevOps, and Azure monitoring/logging capabilities
- Experience with CI/CD pipelines (Azure DevOps YAML), Version Control, automated testing, modular development, and code reviews
- Understanding of cloud security, access management, observability, and cost-aware development practices
- Working knowledge of TerraForm or infrastructure-as-code concepts is a plus
- Preferred qualifications:
- Experience with large-scale distributed data processing using Spark and streaming/real-time ingestion patterns
- Experience building reusable data engineering frameworks, libraries, or shared components
- Knowledge of data quality frameworks, metadata management, lineage, and governance practices
- Experience integrating data pipelines with AI, Machine Learning, or analytics use cases; familiarity with MLOps is an advantage
- Preferred certifications: Microsoft Azure Data Engineer Associate, Azure Developer Associate, Azure Solutions Architect Expert, Databricks Certified Data Engineer Associate/Professional, or Databricks Certified Developer for Apache Spark
- Soft skills:
- Strong communication and documentation skills, with the ability to explain technical data engineering concepts to diverse audiences
- Strong problem-solving mindset with a focus on automation, reliability, and maintainability
- Team-oriented, proactive, and comfortable working in a cross-functional environment
- Continuous improvement mindset and willingness to adopt evolving data platform technologies
Our offer
We provide a stimulating, people‑focused and expertise‑driven environment
- A competitive salary with performance‑based bonuses
- A company car with Fuel/Electric Card
- All necessary tools for your mission: Laptop, Phone, Phone Subscription
- Comprehensive social security and health benefits in Belgium
- Meal Vouchers, Eco cheques
- Hospitalization Insurance & Group Insurance (Retirement)
- Opportunities for professional training and certifications
- Clear Career plan and growth opportunities created together with Manager
- Exciting assignments with clients engaged in innovative IT projects