Data Engineer - Azure Databricks
Posted 4
views
Summary
Hands-on data engineer designing, building, and optimizing high-throughput ETL/ELT pipelines on Azure Databricks with Spark (Scala) and SQL, following medallion architecture and Delta Lake best practices, plus real-time integrations, CI/CD, and observability.
Databricks, SQL, Python, Data warehousing concepts
Detailed Job description (Role & Responsibility, Skill and Experience range)
- We are looking for a hands on Developer Programmer to design, build, and optimize high throughput data pipelines on Azure Databricks using Spark (Scala) and SQL
- Design & develop scalable ETL/ELT pipelines in Azure Databricks using Spark (Scala) and SQL, following medallion architecture (Bronze/Silver/Gold) and Delta Lake best practices.
- Proven expertise in designing real time and near real time integrations
- Operationalize workloads: jobs & workflows scheduling, CI/CD with Git (e.g., Azure DevOps), secrets management, and observability.