Azure Databricks Developer - Python, SQL, Unity Catalog
Posted
LIVEC STAFFING SERVICES PRIVATE LIMITED Azure Databricks Developer - Python, SQL, Unity Catalog
Description
Role Summary
The Azure Databricks Developer plays a critical role in building and maintaining scalable data solutions on the Azure platform. This position focuses on designing data pipelines that support organizational analytics and decision-making.
Responsibilities
- Develop and implement efficient data solutions using Azure Databricks to ensure performance and scalability.
- Collaborate with data scientists and analysts to gather data requirements and deliver effective solutions.
- Use Python and SQL for data manipulation, ensuring the integrity and accuracy of datasets.
- Implement Medallion Architecture principles to effectively organize and manage data layers.
- Manage Delta Lake and Unity Catalog for robust data governance and security measures.
- Create and maintain Databricks Notebooks and Jobs for streamlined data processing and reporting functionalities.
- Utilize CI/CD methodologies with Azure DevOps to enhance deployment efficiency.
- Apply ABAC policies to guarantee data security and compliance with regulations.
- Integrate Git for version control to support collaborative code development efforts.
- Stay informed about emerging trends in data engineering technologies, particularly within Azure services.
Must-Have Skills
- Strong proficiency in Azure Databricks, Python, and SQL.
- Extensive experience with Delta Lake, Unity Catalog, and Medallion Architecture.
- Familiarity with CI/CD practices alongside Azure DevOps tools.
- Knowledgeable about ABAC policies relevant to data governance.
- Solid understanding of Databricks Notebooks, Jobs, and Git integration processes.
Good-to-Have Skills
- Experience with Spark technology is an advantage.
- Familiarity with Lakeflow connections is considered a plus.
Skills
Sign in to read and join the discussion.