Data Engineer
Posted
Description
Project: Build an internal AI-based Data Modeling tool where a Data Architect can ask AI to create a model according to customer standards, review and refine it through a Databricks web application, approve it, and deploy the approved schema into Databricks Unity Catalog.
Requirements: Databricks with pyspark,Unity catalog,GHCP,Data modelling, Databricks Apps,Lakehouse Postgres, Data Architecture knowledge, have worked on Databricks open solution accelerators
Job Summary
- Seeking an experienced Overseas Contractor specializing in Azure Data Architecture with expertise in Databricks Azure SQL and Dimensional Data Modeling and Databricks App to design the solution
- Job Description
- Design and implement robust data architectures using Azure services focusing on scalability and performance Utilize.
- Databricks for big data processing and analytics workloads.
- Develop and maintain Azure SQL databases ensuring data integrity and optimized querying.
- Apply dimensional data modeling techniques to support business intelligence and reporting needs Collaborate with crossfunctional teams to understand data requirements and translate them into technical solutions.
- Ensure best practices in data security governance and compliance are followed Continuously evaluate emerging Azure data technologies to improve existing data platforms
- Should be able to perform data engineering activities in Databricks as well as have good exposure in databricks App
Roles and Responsibilities
- Lead the design and development of endtoend data architecture solutions on Azure
- Build and optimize data pipelines and workflows using Databricks and Azure SQL
- Create and maintain dimensional data models to support analytics and reporting
- Work closely with stakeholders to gather requirements and provide technical guidance
- Monitor data platform performance and troubleshoot issues proactively
- Mentor junior team members on Azure data services and data modeling concepts
- Document architecture designs data flows and technical specifications
- Ensure alignment of data solutions with organizational standards and compliance policies
Skills
Mandatory Skills : Azure SQL, Data Architecture, Databricks, Dimensional Data Modeling
Good to Have Skills : Azure-Data Architecture
Skills
Sign in to read and join the discussion.