Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using PySpark on Azure Data Bricks.
• Collaborate with cross-functional teams to gather requirements and design solutions for complex data processing needs.
• Develop high-quality code in Python to extract insights from structured and unstructured data sources.
• Troubleshoot issues related to data quality, performance, and scalability in the pipeline.

Job Requirements :

• 6-9 years of experience in a similar role as a Data Engineer or equivalent experience working with big data technologies like Hadoop/Hive/Pyspark.
• Strong proficiency in Microsoft Azure services including Databricks, Synapse Analytics, Data Factory etc. .
• Experience with SQL programming language for querying relational databases.