Azure Databricks Data Engineer
Summary
Builds and optimizes large-scale ETL/ELT data pipelines on Azure Databricks using PySpark, Spark, and SQL, integrating Azure storage and database sources and preparing data for BI dashboards. Core stack includes Azure cloud services, Delta Lake/Lakehouse, Unity Catalog, and GitLab.
Azure Data Engineers with Databricks expertise will build scalable pipelines using SQL, Python, and cloud ETL tools across Azure environments.
This position focuses on designing, implementing, and optimizing large-scale data pipelines for scalability, reliability, and performance. You will transform datasets with PySpark and Spark, integrate Azure data sources, and prepare information for dashboards and BI tools. Collaboration with teams and business stakeholders supports tailored data solutions.
Key Responsibilities:
- Build and maintain scalable ETL and ELT pipelines using Databricks
- Transform large datasets with PySpark, Spark, and SQL
- Integrate Azure Blob Storage, ADLS, and database sources
- Prepare data for dashboards and business intelligence tools
- Collaborate with stakeholders on tailored data solutions
Requirements:
- Strong Databricks expertise across Delta Lake and Lakehouse Architecture
- Proficiency with Azure Cloud Services
- Solid understanding of Spark and PySpark processing
- Experience with relational databases
- Knowledge of Unity Catalog, GitLab, and Asset Bundles
Apply Azure, Databricks, SQL, and PySpark expertise to deliver reliable, governed data pipelines and tailored analytics solutions.