Data Engineer

Roles and Responsibilities :

• Design, develop, test, deploy and maintain large-scale data pipelines using AWS Glue, Hive, PySpark and Data Bricks.
• Collaborate with cross-functional teams to gather requirements and deliver high-quality data solutions on time.
• Develop complex SQL queries to optimize database performance and troubleshoot issues in Amazon Redshift databases.
• Ensure scalability, reliability, security and efficiency of the entire data architecture.

Job Requirements :

• 12-16 years of experience in Data Engineering with expertise in AWS Glue, Hive, PySpark and Data Bricks.
• Strong understanding of big data technologies such as Spark Core (PySpark), Hadoop Distributed File System (HDFS) etc.
• Experience working with relational databases like PostgreSQL or MySQL; knowledge of NoSQL databases like MongoDB is a plus.