Data Engineer - Azure

Responsibilities:

• Design and develop data processing solutions using various ETL tools for data ingestion, cleansing, transformation, and job orchestration in the Azure cloud environment.
• Implement robust data security with role-based access control, data masking, and encryption for sensitive data.
• Monitor and optimise data storage and data processing performance using monitoring tools, along with implementing archive strategies and query tuning.

Requirements:

• Hands-on experience with Synapse Pipelines, Spark Pools, Synapse SQL Pools, and SQL Managed Instance
• 4-6 years of experience in a data engineering role, with at least 3-5 years of hands-on experience in Azure cloud technologies.
• Strong expertise in Azure Synapse Data Pipelines (knowledge of ADF is also valuable).
• Proficiency in PySpark, Spark SQL, and T-SQL.
• Familiarity with Azure Functions App (coded in Python) is a plus.
• Deep understanding of cloud technologies and the ability to deliver high-quality, large-scale projects on time.
• Excellent problem-solving skills and a passion for data engineering in the cloud.