Senior Data Engineer

We are looking for an experienced Senior Data Engineer with strong expertise in Python, PySpark, SQL, and Unix. The candidate will be responsible for designing, developing, and optimizing scalable data pipelines, handling large datasets, and ensuring high performance and data quality.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using Python and PySpark.
• Write complex SQL queries and optimize database performance.
• Build and enhance ETL/ELT workflows for data extraction, transformation, and loading.
• Process and transform large volumes of structured and unstructured data.
• Use Unix/Linux commands and shell scripting for automation, job scheduling, and monitoring.
• Troubleshoot complex data pipeline issues and improve performance and reliability.
• Ensure data quality, consistency, and integrity across data processing workflows.
• Collaborate with business, analytics, and engineering teams to deliver data solutions.
• Mentor team members, review code, and establish best practices for data engineering.

Required Skills

• 10+ years of experience in data engineering or related roles.
• Strong hands-on expertise in Python and PySpark.
• Advanced SQL skills, including query optimization and performance tuning.
• Strong knowledge of Unix/Linux commands and shell scripting.
• Experience with ETL/ELT processes and large-scale data processing.
• Excellent problem-solving, debugging, and analytical skills.
• Ability to work independently and provide technical guidance to the team.