Azure Databricks Architect - Apache Spark
Posted Updated
Description
Job Description :
Key Responsibilities :
- Design and architect scalable data engineering solutions using Azure Databricks.
- Develop and optimize ETL/ELT pipelines for enterprise data processing.
- Build and maintain data lake/lakehouse solutions using Azure services.
- Develop data transformation workflows using PySpark/Scala, Python, and SQL.
- Implement Delta Lake architecture to support reliable and high-performance data processing.
- Collaborate with business and technical stakeholders to deliver robust data solutions.
- Ensure data quality, governance, security, and performance optimization.
- Implement CI/CD pipelines and follow DevOps best practices for deployment and version control.
Required Skills :
- 12+ years of experience in Data Engineering, Data Architecture, or related roles.
- Strong expertise in Azure Databricks.
- Hands-on experience with Apache Spark (PySpark/Scala).
- Proficiency in Python and SQL.
- Experience with Azure Data Factory (ADF) and Azure Data Lake Storage (ADLS).
- Strong understanding of ETL/ELT, Delta Lake, and Data Modeling.
- Experience with Git, CI/CD, and DevOps practices.
- Excellent analytical, problem-solving, and stakeholder management skills.
Skills
Sign in to read and join the discussion.