Azure Data Engineer

Job Description: Azure Data Engineer

Key Skills

• Azure
• Python
• PySpark
• Databricks
• Azure Functions
• PostgreSQL / MongoDB
• Data Pipelines
• Large Dataset Processing

Key Responsibilities

• Design, develop, and maintain data transformation pipelines using Azure services and Python.
• Build scalable data pipelines using PySpark and Azure Databricks.
• Develop and maintain complex calculation engines and data processing formulas.
• Efficiently process and manage large datasets.
• Work with databases such as PostgreSQL and MongoDB.
• Collaborate closely with Software Engineers and other technical teams.
• Participate in code reviews, solution design discussions, and software planning.
• Contribute to automation, testing, deployment, and performance optimization processes.
• Ensure data pipelines are scalable, reliable, and maintainable.

Required Experience

• Strong hands-on experience with Azure, Python, PySpark, and Databricks.
• Good experience in developing ETL/data transformation pipelines.
• Experience handling large-scale datasets.
• Hands-on knowledge of PostgreSQL and/or MongoDB.
• Good understanding of software development, testing, and deployment practices.